From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 117ADC46CD2 for ; Tue, 30 Jan 2024 09:55:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 995446B009C; Tue, 30 Jan 2024 04:55:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 91C2B6B009E; Tue, 30 Jan 2024 04:55:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7E3E66B009F; Tue, 30 Jan 2024 04:55:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 6F7A46B009C for ; Tue, 30 Jan 2024 04:55:38 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 38CBE1603FD for ; Tue, 30 Jan 2024 09:55:38 +0000 (UTC) X-FDA: 81735520356.16.DAD57D0 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf11.hostedemail.com (Postfix) with ESMTP id 5F5044000C for ; Tue, 30 Jan 2024 09:55:36 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf11.hostedemail.com: domain of anshuman.khandual@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706608536; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=riPMl/ewsb/HaLq2kq2zgf4U8pcjz2zU9Lq0oHZwu44=; b=Na4Wa73gM/2mk6dD0D9VTVcrdk6taakZtEjTEfLHIlGJXUZfELpFwjyzTRGbFrpvTZ/tbk l5ZCnrBTk8EhitS2/xKA+I2VSlNrYxPs8Nd8g97y6HFnJIkJpGjGwsK3nkh3CSdFEHuXjv xqXbt2EV1eRsuKh0QyybaDw/+LHRWOQ= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf11.hostedemail.com: domain of anshuman.khandual@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706608536; a=rsa-sha256; cv=none; b=rt6UdvKTpe9f9+HvUESBolDqXwMPxDv2iOmG9+ClNptCIOHhN0hAc8WOcCW0UF6FTg85UB +fm372CTfOFSLTsbFCZDGMMFbhMkniOJcj1uhVNEE5xu0q8X4iJbfokuF0eniQiwhCFvCR D1kBBR3ENhylIkWEZfV6mNv1kQK6X8c= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1E839DA7; Tue, 30 Jan 2024 01:56:19 -0800 (PST) Received: from [10.163.41.110] (unknown [10.163.41.110]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5569D3F738; Tue, 30 Jan 2024 01:55:23 -0800 (PST) Message-ID: <1e03aec4-705a-41b6-b258-0b8944d9dc0c@arm.com> Date: Tue, 30 Jan 2024 15:25:20 +0530 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH RFC v3 11/35] mm: Allow an arch to hook into folio allocation when VMA is known Content-Language: en-US To: Alexandru Elisei , catalin.marinas@arm.com, will@kernel.org, oliver.upton@linux.dev, maz@kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, arnd@arndb.de, akpm@linux-foundation.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, mhiramat@kernel.org, rppt@kernel.org, hughd@google.com Cc: pcc@google.com, steven.price@arm.com, vincenzo.frascino@arm.com, david@redhat.com, eugenis@google.com, kcc@google.com, hyesoo.yu@samsung.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org References: <20240125164256.4147-1-alexandru.elisei@arm.com> <20240125164256.4147-12-alexandru.elisei@arm.com> From: Anshuman Khandual In-Reply-To: <20240125164256.4147-12-alexandru.elisei@arm.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 5F5044000C X-Stat-Signature: kzrcnkz3nf5oozrftaz1p4bep3scxhy4 X-HE-Tag: 1706608536-263182 X-HE-Meta: U2FsdGVkX18P0xDCDc3woiDvCG5H8HjkS7FDBRzi8znoeDce49HHdlzh+1P9WKIB5NafFQ/DMj6bHWDCZsMhFK5LMe3haB1pn2DcHVFMnmXME5Pr3QTMnAKEzbxt6Lim0oVfQE28Ki/KbISJrdmfzkLWS9nwE68LPOe+NV8Z706P1OECwzY5yvS/tsCNSknrD4g0nFbsVq9wa5LhyFXZ6LSOG8e1+MO2BL9iciQpr0+GmSAMLKEAO/6o3qlRyMAEjTSizXD/uNv/xSwpPijylGQ9kjkqiFiX8QGHvwcvHVsg1TK9L3Icm9eVmolhj1z6sVPrhgrw+BASCh5COTZjW5GGZppRdJI1jwXLyD0oqo/EnsFUV8YNAJcCAQL8PhVCtUurE7KTOYzZ9nqCZwX1+ftFOrQC/iB4GcaXwwKHzsaFjlf/Dcu1N250zLi2t6GwWSTAI4ctvcm/4eFXwTAOVYRoQXTVjW23JD7DVL9UOgOK1EBBcPBl60qAo1v5ZkasIQmTOYNBhcRWBgnnNcYnEvMxv4/SB7F13KuSXB16Bx+Osdcgq428zk6x/g4pbtsj9FAXn1ii/UZKmszvWWEQ+jm8ofMjY5Ow9Aa1E9T/yoZvGUcu56ERGN/Uu7Tq8t+oqfdwrqWYVacTjvJn5pNazZ9FNrole8YePxAbWWfWZDRuVx8YmerO1akvuvXDdGJcVAPfn5fUoSeibREaeXMbhSeAjDisLxEfExWHB1M4dIG+euoI4WBugU7ARxrSGyKHoBRxKnNiC8SnVUNx3gtayX3HJMQuK1KZGdFFQ7BIWm/O2CPkKlHqSnrD3Ke4YD5fSHpT0IGRKUex0hhXKEic9IGXd9sBpUjmQELr2LkzvMvRMfXpc8ArXyVxKY4v+owI6BXAf42ohF/lZ1pgYS5nGpB6zPDI4wIMK6vOp+HcjPce9gK7JA7YLMcGxuM/PLbKVBex+D42RMX0QLWGBBl jDGFYhpZ jbkg0TPnRb9nuE3i1kwXlS2BQGSZV1fM4ruPfZmD6TWTlh0+bpDiyrd5S0V5JqsyULUORXV4tRSP/hDzyScJsTd51yoDiImAWQI9oHptFJqKz9uZwiyN+KZGILJQzq9cpmBYnaojjJMLxAcbsrGDLNfTiROuFEQCfJv6QUbZ4H2ukRtj4GTxj5JWNtKTMhMJf1AEnNs2MY4TICRg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 1/25/24 22:12, Alexandru Elisei wrote: > arm64 uses VM_HIGH_ARCH_0 and VM_HIGH_ARCH_1 for enabling MTE for a VMA. > When VM_HIGH_ARCH_0, which arm64 renames to VM_MTE, is set for a VMA, and > the gfp flag __GFP_ZERO is present, the __GFP_ZEROTAGS gfp flag also gets > set in vma_alloc_zeroed_movable_folio(). > > Expand this to be more generic by adding an arch hook that modifes the gfp > flags for an allocation when the VMA is known. > > Note that __GFP_ZEROTAGS is ignored by the page allocator unless __GFP_ZERO > is also set; from that point of view, the current behaviour is unchanged, > even though the arm64 flag is set in more places. When arm64 will have > support to reuse the tag storage for data allocation, the uses of the > __GFP_ZEROTAGS flag will be expanded to instruct the page allocator to try > to reserve the corresponding tag storage for the pages being allocated. Right but how will pushing __GFP_ZEROTAGS addition into gfp_t flags further down via a new arch call back i.e arch_calc_vma_gfp() while still maintaining (vma->vm_flags & VM_MTE) conditionality improve the current scenario. Because the page allocator could have still analyzed alloc flags for __GFP_ZEROTAGS for any additional stuff. OR this just adds some new core MM paths to get __GFP_ZEROTAGS which was not the case earlier via this call back. > > The flags returned by arch_calc_vma_gfp() are or'ed with the flags set by > the caller; this has been done to keep an architecture from modifying the > flags already set by the core memory management code; this is similar to > how do_mmap() -> calc_vm_flag_bits() -> arch_calc_vm_flag_bits() has been > implemented. This can be revisited in the future if there's a need to do > so. > > Signed-off-by: Alexandru Elisei > --- > arch/arm64/include/asm/page.h | 5 ++--- > arch/arm64/include/asm/pgtable.h | 3 +++ > arch/arm64/mm/fault.c | 19 ++++++------------- > include/linux/pgtable.h | 7 +++++++ > mm/mempolicy.c | 1 + > mm/shmem.c | 5 ++++- > 6 files changed, 23 insertions(+), 17 deletions(-) > > diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h > index 2312e6ee595f..88bab032a493 100644 > --- a/arch/arm64/include/asm/page.h > +++ b/arch/arm64/include/asm/page.h > @@ -29,9 +29,8 @@ void copy_user_highpage(struct page *to, struct page *from, > void copy_highpage(struct page *to, struct page *from); > #define __HAVE_ARCH_COPY_HIGHPAGE > > -struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, > - unsigned long vaddr); > -#define vma_alloc_zeroed_movable_folio vma_alloc_zeroed_movable_folio > +#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ > + vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr, false) > > void tag_clear_highpage(struct page *to); > #define __HAVE_ARCH_TAG_CLEAR_HIGHPAGE > diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h > index 79ce70fbb751..08f0904dbfc2 100644 > --- a/arch/arm64/include/asm/pgtable.h > +++ b/arch/arm64/include/asm/pgtable.h > @@ -1071,6 +1071,9 @@ static inline void arch_swap_restore(swp_entry_t entry, struct folio *folio) > > #endif /* CONFIG_ARM64_MTE */ > > +#define __HAVE_ARCH_CALC_VMA_GFP > +gfp_t arch_calc_vma_gfp(struct vm_area_struct *vma, gfp_t gfp); > + > /* > * On AArch64, the cache coherency is handled via the set_pte_at() function. > */ > diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c > index 55f6455a8284..4d3f0a870ad8 100644 > --- a/arch/arm64/mm/fault.c > +++ b/arch/arm64/mm/fault.c > @@ -937,22 +937,15 @@ void do_debug_exception(unsigned long addr_if_watchpoint, unsigned long esr, > NOKPROBE_SYMBOL(do_debug_exception); > > /* > - * Used during anonymous page fault handling. > + * If this is called during anonymous page fault handling, and the page is > + * mapped with PROT_MTE, initialise the tags at the point of tag zeroing as this > + * is usually faster than separate DC ZVA and STGM. > */ > -struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, > - unsigned long vaddr) > +gfp_t arch_calc_vma_gfp(struct vm_area_struct *vma, gfp_t gfp) > { > - gfp_t flags = GFP_HIGHUSER_MOVABLE | __GFP_ZERO; > - > - /* > - * If the page is mapped with PROT_MTE, initialise the tags at the > - * point of allocation and page zeroing as this is usually faster than > - * separate DC ZVA and STGM. > - */ > if (vma->vm_flags & VM_MTE) > - flags |= __GFP_ZEROTAGS; > - > - return vma_alloc_folio(flags, 0, vma, vaddr, false); > + return __GFP_ZEROTAGS; > + return 0; > } > > void tag_clear_highpage(struct page *page) > diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h > index c5ddec6b5305..98f81ca08cbe 100644 > --- a/include/linux/pgtable.h > +++ b/include/linux/pgtable.h > @@ -901,6 +901,13 @@ static inline void arch_do_swap_page(struct mm_struct *mm, > } > #endif > > +#ifndef __HAVE_ARCH_CALC_VMA_GFP > +static inline gfp_t arch_calc_vma_gfp(struct vm_area_struct *vma, gfp_t gfp) > +{ > + return 0; > +} > +#endif > + > #ifndef __HAVE_ARCH_FREE_PAGES_PREPARE > static inline void arch_free_pages_prepare(struct page *page, int order) { } > #endif > diff --git a/mm/mempolicy.c b/mm/mempolicy.c > index 10a590ee1c89..f7ef52760b32 100644 > --- a/mm/mempolicy.c > +++ b/mm/mempolicy.c > @@ -2168,6 +2168,7 @@ struct folio *vma_alloc_folio(gfp_t gfp, int order, struct vm_area_struct *vma, > pgoff_t ilx; > struct page *page; > > + gfp |= arch_calc_vma_gfp(vma, gfp); > pol = get_vma_policy(vma, addr, order, &ilx); > page = alloc_pages_mpol(gfp | __GFP_COMP, order, > pol, ilx, numa_node_id()); > diff --git a/mm/shmem.c b/mm/shmem.c > index d7c84ff62186..14427e9982f9 100644 > --- a/mm/shmem.c > +++ b/mm/shmem.c > @@ -1585,7 +1585,7 @@ static struct folio *shmem_swapin_cluster(swp_entry_t swap, gfp_t gfp, > */ > static gfp_t limit_gfp_mask(gfp_t huge_gfp, gfp_t limit_gfp) > { > - gfp_t allowflags = __GFP_IO | __GFP_FS | __GFP_RECLAIM; > + gfp_t allowflags = __GFP_IO | __GFP_FS | __GFP_RECLAIM | __GFP_ZEROTAGS; > gfp_t denyflags = __GFP_NOWARN | __GFP_NORETRY; > gfp_t zoneflags = limit_gfp & GFP_ZONEMASK; > gfp_t result = huge_gfp & ~(allowflags | GFP_ZONEMASK); > @@ -2038,6 +2038,7 @@ static int shmem_get_folio_gfp(struct inode *inode, pgoff_t index, > gfp_t huge_gfp; > > huge_gfp = vma_thp_gfp_mask(vma); > + huge_gfp |= arch_calc_vma_gfp(vma, huge_gfp); > huge_gfp = limit_gfp_mask(huge_gfp, gfp); > folio = shmem_alloc_and_add_folio(huge_gfp, > inode, index, fault_mm, true); > @@ -2214,6 +2215,8 @@ static vm_fault_t shmem_fault(struct vm_fault *vmf) > vm_fault_t ret = 0; > int err; > > + gfp |= arch_calc_vma_gfp(vmf->vma, gfp); > + > /* > * Trinity finds that probing a hole which tmpfs is punching can > * prevent the hole-punch from ever completing: noted in i_private.