From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0CD7EC4361B for ; Thu, 17 Dec 2020 18:52:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8985B238EC for ; Thu, 17 Dec 2020 18:52:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8985B238EC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BB9FC6B0068; Thu, 17 Dec 2020 13:52:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B6AA36B006C; Thu, 17 Dec 2020 13:52:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A33866B006E; Thu, 17 Dec 2020 13:52:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0251.hostedemail.com [216.40.44.251]) by kanga.kvack.org (Postfix) with ESMTP id 8F3216B0068 for ; Thu, 17 Dec 2020 13:52:51 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 5282F180AD82F for ; Thu, 17 Dec 2020 18:52:51 +0000 (UTC) X-FDA: 77603670942.23.cow35_2f133e127437 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin23.hostedemail.com (Postfix) with ESMTP id 2CBF137604 for ; Thu, 17 Dec 2020 18:52:51 +0000 (UTC) X-HE-Tag: cow35_2f133e127437 X-Filterd-Recvd-Size: 8546 Received: from mail-qk1-f175.google.com (mail-qk1-f175.google.com [209.85.222.175]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Thu, 17 Dec 2020 18:52:50 +0000 (UTC) Received: by mail-qk1-f175.google.com with SMTP id 186so27390084qkj.3 for ; Thu, 17 Dec 2020 10:52:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=3vDHKEzj0vF17lzh9SYpP31/FhI4ogCEdqejDZ0noAU=; b=EdFqn1VhcOpZMa3J3DFosAQqJr0UlLj2Q1rIqQzWkpAG9UR36YgESwGeCDL61GQsby 4a764EIeuSe/P0aNnH/Dxs8iJo1WUn1lcnOItCvqtGHqbyY9vLhDvhmFdF8vuur9zjjX xsS2eEr09qduALSup64cnWmdEplzUwQijRRV6F+PgHjf21hlrxI95Duk4dkSrH+7/Ozb QFbUMDNgAO5dnh4pOjC1y1EwhWY1yahxoORTL6KUO89qyCkmZ1c+tGOe/NTVvnsU5yqS KKpUaH8Q+u3CChrO/OASeiKpJwbfxJUEh+eu787k5qeewCn2F+ll2wWUMZ9SU8mcYAf1 AYKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3vDHKEzj0vF17lzh9SYpP31/FhI4ogCEdqejDZ0noAU=; b=ZjTVUfKBNDKW4clko5g7h7G9GibBGrHB3iSWADmLFb4uX0+UvG7cbDogYFbxocb/+8 /oMsIw2oufAmolhL58/Mx3XubCnMS/4tBOSyq58Vs7/d1CtOO6Zagl+sOm409V15m0mK +GHa6EaSbMUZz5BwS2Vv2WBLHx1YRJJun1vjJzzEHfoLeghiRlYFQZ5xHytK7W5gkdDp yw3xhyoKCyYJ9mbdT/kHKlPNuLkjLAOJI9iqnQzUJqbX7rNSMG82i5BDsMIj0enzwXHm HERoDLifp8PbZbH2imp9S/s7XpfrCZWwpGodSycwiOMQrXx08aSc9COuCJhsrSeCqGlc ALNA== X-Gm-Message-State: AOAM532MAwg2zJgHv8OHqagkyq56wyT5sQDvz7bq98VZP/HN81xZkDP/ hxXIg4dmuW0FKN8b8AD1v1PkZg== X-Google-Smtp-Source: ABdhPJzxDRkkYxOXRPc2n4xnQCeM3Vw8LarCGXmJR3py3nOwsamiM0kvEZTB8gPYQhmKuvOxOnYZAg== X-Received: by 2002:a37:a241:: with SMTP id l62mr690495qke.482.1608231170066; Thu, 17 Dec 2020 10:52:50 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id m8sm4127434qkn.41.2020.12.17.10.52.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Dec 2020 10:52:49 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, vbabka@suse.cz, mhocko@suse.com, david@redhat.com, osalvador@suse.de, dan.j.williams@intel.com, sashal@kernel.org, tyhicks@linux.microsoft.com, iamjoonsoo.kim@lge.com, mike.kravetz@oracle.com, rostedt@goodmis.org, mingo@redhat.com, jgg@ziepe.ca, peterz@infradead.org, mgorman@suse.de, willy@infradead.org, rientjes@google.com, jhubbard@nvidia.com, linux-doc@vger.kernel.org, ira.weiny@intel.com, linux-kselftest@vger.kernel.org Subject: [PATCH v4 02/10] mm cma: rename PF_MEMALLOC_NOCMA to PF_MEMALLOC_PIN Date: Thu, 17 Dec 2020 13:52:35 -0500 Message-Id: <20201217185243.3288048-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201217185243.3288048-1-pasha.tatashin@soleen.com> References: <20201217185243.3288048-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: PF_MEMALLOC_NOCMA is used ot guarantee that the allocator will not return pages that might belong to CMA region. This is currently used for long term gup to make sure that such pins are not going to be done on any CMA pages. When PF_MEMALLOC_NOCMA has been introduced we haven't realized that it is focusing on CMA pages too much and that there is larger class of pages th= at need the same treatment. MOVABLE zone cannot contain any long term pins a= s well so it makes sense to reuse and redefine this flag for that usecase a= s well. Rename the flag to PF_MEMALLOC_PIN which defines an allocation context which can only get pages suitable for long-term pins. Also re-name: memalloc_nocma_save()/memalloc_nocma_restore to memalloc_pin_save()/memalloc_pin_restore() and make the new functions common. Signed-off-by: Pavel Tatashin Reviewed-by: John Hubbard Acked-by: Michal Hocko --- include/linux/sched.h | 2 +- include/linux/sched/mm.h | 21 +++++---------------- mm/gup.c | 4 ++-- mm/hugetlb.c | 4 ++-- mm/page_alloc.c | 4 ++-- 5 files changed, 12 insertions(+), 23 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index e5ad6d354b7b..e30d5511ccc0 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1576,7 +1576,7 @@ extern struct pid *cad_pid; #define PF_SWAPWRITE 0x00800000 /* Allowed to write to swap */ #define PF_NO_SETAFFINITY 0x04000000 /* Userland is not allowed to meddl= e with cpus_mask */ #define PF_MCE_EARLY 0x08000000 /* Early kill for mce process poli= cy */ -#define PF_MEMALLOC_NOCMA 0x10000000 /* All allocation request will have= _GFP_MOVABLE cleared */ +#define PF_MEMALLOC_PIN 0x10000000 /* Allocation context constrained to= zones which allow long term pinning. */ #define PF_FREEZER_SKIP 0x40000000 /* Freezer should not count it as fr= eezable */ #define PF_SUSPEND_TASK 0x80000000 /* This thread called freeze_pr= ocesses() and should not be frozen */ =20 diff --git a/include/linux/sched/mm.h b/include/linux/sched/mm.h index 1ae08b8462a4..5f4dd3274734 100644 --- a/include/linux/sched/mm.h +++ b/include/linux/sched/mm.h @@ -270,29 +270,18 @@ static inline void memalloc_noreclaim_restore(unsig= ned int flags) current->flags =3D (current->flags & ~PF_MEMALLOC) | flags; } =20 -#ifdef CONFIG_CMA -static inline unsigned int memalloc_nocma_save(void) +static inline unsigned int memalloc_pin_save(void) { - unsigned int flags =3D current->flags & PF_MEMALLOC_NOCMA; + unsigned int flags =3D current->flags & PF_MEMALLOC_PIN; =20 - current->flags |=3D PF_MEMALLOC_NOCMA; + current->flags |=3D PF_MEMALLOC_PIN; return flags; } =20 -static inline void memalloc_nocma_restore(unsigned int flags) +static inline void memalloc_pin_restore(unsigned int flags) { - current->flags =3D (current->flags & ~PF_MEMALLOC_NOCMA) | flags; + current->flags =3D (current->flags & ~PF_MEMALLOC_PIN) | flags; } -#else -static inline unsigned int memalloc_nocma_save(void) -{ - return 0; -} - -static inline void memalloc_nocma_restore(unsigned int flags) -{ -} -#endif =20 #ifdef CONFIG_MEMCG DECLARE_PER_CPU(struct mem_cgroup *, int_active_memcg); diff --git a/mm/gup.c b/mm/gup.c index f2e50cdd7d67..04602e94856b 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1671,7 +1671,7 @@ static long __gup_longterm_locked(struct mm_struct = *mm, long rc; =20 if (gup_flags & FOLL_LONGTERM) - flags =3D memalloc_nocma_save(); + flags =3D memalloc_pin_save(); =20 rc =3D __get_user_pages_locked(mm, start, nr_pages, pages, vmas, NULL, gup_flags); @@ -1680,7 +1680,7 @@ static long __gup_longterm_locked(struct mm_struct = *mm, if (rc > 0) rc =3D check_and_migrate_cma_pages(mm, start, rc, pages, vmas, gup_flags); - memalloc_nocma_restore(flags); + memalloc_pin_restore(flags); } return rc; } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index cbf32d2824fd..3f5ddac5de8a 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1033,10 +1033,10 @@ static void enqueue_huge_page(struct hstate *h, s= truct page *page) static struct page *dequeue_huge_page_node_exact(struct hstate *h, int n= id) { struct page *page; - bool nocma =3D !!(current->flags & PF_MEMALLOC_NOCMA); + bool pin =3D !!(current->flags & PF_MEMALLOC_PIN); =20 list_for_each_entry(page, &h->hugepage_freelists[nid], lru) { - if (nocma && is_migrate_cma_page(page)) + if (pin && is_migrate_cma_page(page)) continue; =20 if (PageHWPoison(page)) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 774542e1483e..ec05396a597b 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3808,8 +3808,8 @@ static inline unsigned int current_alloc_flags(gfp_= t gfp_mask, #ifdef CONFIG_CMA unsigned int pflags =3D current->flags; =20 - if (!(pflags & PF_MEMALLOC_NOCMA) && - gfp_migratetype(gfp_mask) =3D=3D MIGRATE_MOVABLE) + if (!(pflags & PF_MEMALLOC_PIN) && + gfp_migratetype(gfp_mask) =3D=3D MIGRATE_MOVABLE) alloc_flags |=3D ALLOC_CMA; =20 #endif --=20 2.25.1