From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 419EEC4167B for ; Thu, 10 Dec 2020 00:43:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0211E22D5B for ; Thu, 10 Dec 2020 00:43:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0211E22D5B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A03776B006E; Wed, 9 Dec 2020 19:43:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9DD146B0070; Wed, 9 Dec 2020 19:43:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 719F96B0071; Wed, 9 Dec 2020 19:43:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0042.hostedemail.com [216.40.44.42]) by kanga.kvack.org (Postfix) with ESMTP id 4FDE36B006E for ; Wed, 9 Dec 2020 19:43:46 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 1A184181AEF23 for ; Thu, 10 Dec 2020 00:43:46 +0000 (UTC) X-FDA: 77575524852.08.nest41_08058ca273f4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin08.hostedemail.com (Postfix) with ESMTP id F0E3F1819E769 for ; Thu, 10 Dec 2020 00:43:45 +0000 (UTC) X-HE-Tag: nest41_08058ca273f4 X-Filterd-Recvd-Size: 8394 Received: from mail-qk1-f194.google.com (mail-qk1-f194.google.com [209.85.222.194]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Thu, 10 Dec 2020 00:43:45 +0000 (UTC) Received: by mail-qk1-f194.google.com with SMTP id w79so3309528qkb.5 for ; Wed, 09 Dec 2020 16:43:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=58dOOX3CT4RgbZN4Iw/Ztczf8OW6HingBFIIO3HxvKE=; b=QEV+Vv5BZix6tq+rJigcucRn7NtniibHkkvq84g/WGOiHZ+79JPaLLH0XQIJF1YpBn jyk4ywwH8Wd4bXSoWUbrxq22Zdp348uDTDbWjKxjrAP4qqQ7uQkJUsgSxIE2dw2Z6kGy uNK1FSp+nRR4R17lIUddn8W6RFqwj+d9uTiFbnbDGawrK+6TuQDkbA8uEFNCrkRX99O9 Hx6YLXr5cvmMQbFxTVby4sz/s8H1f0tIPwltNr+VkP4l39RTg70mZVTc3GwG1/SKvgDg 59cDntVv6qlbFWagb0yrhikE0LwGD+hvICZVBhETHT09wrSKOZSdz2PGBrAxpfKxOLAC j0ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=58dOOX3CT4RgbZN4Iw/Ztczf8OW6HingBFIIO3HxvKE=; b=gUlmxMCeZVzLp+vDB2txgw8upm6yB69wXpblXvS/JciB/Jgcf7f7NF5CcuzY5L3xGB AOZXcUlpazHhGj4qNja8SwcEQeX6e6mbIvbPOJioUTEDPmOtAL6+mUuGLBnJBLv9yX3r joe+38ga7MZ90MoT8wa4rGNgrb4E6NKbvww6QpgjJN0wyhTl78BhdBJZr+lncIqXMGZs oKJ0lRg3GwM+3LTlsAuaQm7+T5X1DHkCemFXysujywFV66RyrI1btEDbykFxp/6TZiLP 6HCaacBSJbTpfJiMOxpfy8IQgIibHy84W8s9hQhDYnpiDZBfMdk3WNKRb4dWUGppVwEM 3QkA== X-Gm-Message-State: AOAM530Y+PJLoE/9mc6AzFxhdvAWMgaEXPiWWhUyqtrDcXm7fJ8TIe9L QTXMzpkUjUE/ydv/zQsEfYnkYQ== X-Google-Smtp-Source: ABdhPJxwNDyBJwYij2uBaeo20myfhyZEJi/8ui6ywvr6y8tyrE6/82bqHD4mZ/G5y/YfBxOZbPzsJw== X-Received: by 2002:a37:a890:: with SMTP id r138mr6153843qke.164.1607561024825; Wed, 09 Dec 2020 16:43:44 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id y1sm2538745qky.63.2020.12.09.16.43.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Dec 2020 16:43:44 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, vbabka@suse.cz, mhocko@suse.com, david@redhat.com, osalvador@suse.de, dan.j.williams@intel.com, sashal@kernel.org, tyhicks@linux.microsoft.com, iamjoonsoo.kim@lge.com, mike.kravetz@oracle.com, rostedt@goodmis.org, mingo@redhat.com, jgg@ziepe.ca, peterz@infradead.org, mgorman@suse.de, willy@infradead.org, rientjes@google.com, jhubbard@nvidia.com, linux-doc@vger.kernel.org Subject: [PATCH v2 4/8] mm cma: rename PF_MEMALLOC_NOCMA to PF_MEMALLOC_PIN Date: Wed, 9 Dec 2020 19:43:31 -0500 Message-Id: <20201210004335.64634-5-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201210004335.64634-1-pasha.tatashin@soleen.com> References: <20201210004335.64634-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: PF_MEMALLOC_NOCMA is used ot guarantee that the allocator will not return pages that might belong to CMA region. This is currently used for long term gup to make sure that such pins are not going to be done on any CMA pages. When PF_MEMALLOC_NOCMA has been introduced we haven't realized that it is focusing on CMA pages too much and that there is larger class of pages th= at need the same treatment. MOVABLE zone cannot contain any long term pins a= s well so it makes sense to reuse and redefine this flag for that usecase a= s well. Rename the flag to PF_MEMALLOC_PIN which defines an allocation context which can only get pages suitable for long-term pins. Also re-name: memalloc_nocma_save()/memalloc_nocma_restore to memalloc_pin_save()/memalloc_pin_restore() and make the new functions common. Signed-off-by: Pavel Tatashin Reviewed-by: John Hubbard --- include/linux/sched.h | 2 +- include/linux/sched/mm.h | 21 +++++---------------- mm/gup.c | 4 ++-- mm/hugetlb.c | 4 ++-- mm/page_alloc.c | 4 ++-- 5 files changed, 12 insertions(+), 23 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 76cd21fa5501..5c4bd5e1cbd8 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1548,7 +1548,7 @@ extern struct pid *cad_pid; #define PF_SWAPWRITE 0x00800000 /* Allowed to write to swap */ #define PF_NO_SETAFFINITY 0x04000000 /* Userland is not allowed to meddl= e with cpus_mask */ #define PF_MCE_EARLY 0x08000000 /* Early kill for mce process poli= cy */ -#define PF_MEMALLOC_NOCMA 0x10000000 /* All allocation request will have= _GFP_MOVABLE cleared */ +#define PF_MEMALLOC_PIN 0x10000000 /* All allocation request will have = _GFP_MOVABLE cleared */ #define PF_FREEZER_SKIP 0x40000000 /* Freezer should not count it as fr= eezable */ #define PF_SUSPEND_TASK 0x80000000 /* This thread called freeze_pr= ocesses() and should not be frozen */ =20 diff --git a/include/linux/sched/mm.h b/include/linux/sched/mm.h index d5ece7a9a403..a4b5da13d2c6 100644 --- a/include/linux/sched/mm.h +++ b/include/linux/sched/mm.h @@ -254,29 +254,18 @@ static inline void memalloc_noreclaim_restore(unsig= ned int flags) current->flags =3D (current->flags & ~PF_MEMALLOC) | flags; } =20 -#ifdef CONFIG_CMA -static inline unsigned int memalloc_nocma_save(void) +static inline unsigned int memalloc_pin_save(void) { - unsigned int flags =3D current->flags & PF_MEMALLOC_NOCMA; + unsigned int flags =3D current->flags & PF_MEMALLOC_PIN; =20 - current->flags |=3D PF_MEMALLOC_NOCMA; + current->flags |=3D PF_MEMALLOC_PIN; return flags; } =20 -static inline void memalloc_nocma_restore(unsigned int flags) +static inline void memalloc_pin_restore(unsigned int flags) { - current->flags =3D (current->flags & ~PF_MEMALLOC_NOCMA) | flags; + current->flags =3D (current->flags & ~PF_MEMALLOC_PIN) | flags; } -#else -static inline unsigned int memalloc_nocma_save(void) -{ - return 0; -} - -static inline void memalloc_nocma_restore(unsigned int flags) -{ -} -#endif =20 #ifdef CONFIG_MEMCG DECLARE_PER_CPU(struct mem_cgroup *, int_active_memcg); diff --git a/mm/gup.c b/mm/gup.c index 0e2de888a8b0..0eb8a85fb704 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1726,7 +1726,7 @@ static long __gup_longterm_locked(struct mm_struct = *mm, if (!vmas_tmp) return -ENOMEM; } - flags =3D memalloc_nocma_save(); + flags =3D memalloc_pin_save(); } =20 rc =3D __get_user_pages_locked(mm, start, nr_pages, pages, @@ -1749,7 +1749,7 @@ static long __gup_longterm_locked(struct mm_struct = *mm, rc =3D check_and_migrate_cma_pages(mm, start, rc, pages, vmas_tmp, gup_flags); out: - memalloc_nocma_restore(flags); + memalloc_pin_restore(flags); } =20 if (vmas_tmp !=3D vmas) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 37f15c3c24dc..e797b41998ec 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1033,10 +1033,10 @@ static void enqueue_huge_page(struct hstate *h, s= truct page *page) static struct page *dequeue_huge_page_node_exact(struct hstate *h, int n= id) { struct page *page; - bool nocma =3D !!(current->flags & PF_MEMALLOC_NOCMA); + bool pin =3D !!(current->flags & PF_MEMALLOC_PIN); =20 list_for_each_entry(page, &h->hugepage_freelists[nid], lru) { - if (nocma && is_migrate_cma_page(page)) + if (pin && is_migrate_cma_page(page)) continue; =20 if (PageHWPoison(page)) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index eaa227a479e4..2dea5600f308 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3772,8 +3772,8 @@ static inline unsigned int current_alloc_flags(gfp_= t gfp_mask, #ifdef CONFIG_CMA unsigned int pflags =3D current->flags; =20 - if (!(pflags & PF_MEMALLOC_NOCMA) && - gfp_migratetype(gfp_mask) =3D=3D MIGRATE_MOVABLE) + if (!(pflags & PF_MEMALLOC_PIN) && + gfp_migratetype(gfp_mask) =3D=3D MIGRATE_MOVABLE) alloc_flags |=3D ALLOC_CMA; =20 #endif --=20 2.25.1