From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1237C04FFE for ; Wed, 15 May 2024 03:09:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 547366B028B; Tue, 14 May 2024 23:09:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4F7896B028C; Tue, 14 May 2024 23:09:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3BE626B028D; Tue, 14 May 2024 23:09:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 1E4E46B028B for ; Tue, 14 May 2024 23:09:35 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 88334403B1 for ; Wed, 15 May 2024 03:09:34 +0000 (UTC) X-FDA: 82119149868.11.6DF085D Received: from out30-97.freemail.mail.aliyun.com (out30-97.freemail.mail.aliyun.com [115.124.30.97]) by imf11.hostedemail.com (Postfix) with ESMTP id A2EF84000A for ; Wed, 15 May 2024 03:09:30 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=CpZ4vKxj; spf=pass (imf11.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.97 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715742572; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dLPH3uRBAuSYs9DKZAOz31UzufSUxH5avu2Y4j0wRMM=; b=atDu+SIDTnsynGpcb6LL5sQrVqXDuKt/1qLztu24BualM1PSa2mV359wB3+PQ6IozZ5ngt HkbG84bcbLZunUDcQqLEodc+lSPWcJ91QFarbg/2haV9SpFf7c+rzZMgY37/DCDV78ufT5 TNSGSrmpRyU3YO1s8YIx81hHIYtw9h4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715742572; a=rsa-sha256; cv=none; b=ZAVvYuqmv5uiw1sXuDKRfKmOeTIKCglgiv/tEDqBon4NQMNyBWDwdYgMKxEWDzT4jzs9Ua 37aI/P0s9BD9nX8eWrEWv2Q/dUTs90NTp0L36ph81TwisH74UDwItinWiuAFPOMYYNuxdp H0kXDEbDkJUnYn6lPIi+OXJoGQXe5QI= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=CpZ4vKxj; spf=pass (imf11.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.97 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com DKIM-Signature:v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1715742565; h=Message-ID:Date:MIME-Version:Subject:To:From:Content-Type; bh=dLPH3uRBAuSYs9DKZAOz31UzufSUxH5avu2Y4j0wRMM=; b=CpZ4vKxjTSpc6uehvI5imDlO4Rd0okOmFr+ROjjW/3bzJCRuMIsdvYykF8XXE90R3xKTtQ9VzjjEVzqDiSey5Ewz9HG4FPKpJEIXVzQHBJwCOmJewHrGF4p6MkewhTCAEyUcMNEHzt2qtDKCAPg+epp4zt+BmCUikl8qd8CFg1g= X-Alimail-AntiSpam:AC=PASS;BC=-1|-1;BR=01201311R691e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033032014031;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=13;SR=0;TI=SMTPD_---0W6WSbza_1715742561; Received: from 30.97.56.58(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0W6WSbza_1715742561) by smtp.aliyun-inc.com; Wed, 15 May 2024 11:09:23 +0800 Message-ID: Date: Wed, 15 May 2024 11:09:21 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 5/7] mm: shmem: add mTHP support for anonymous shmem To: Lance Yang Cc: akpm@linux-foundation.org, hughd@google.com, willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: From: Baolin Wang In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: A2EF84000A X-Rspam-User: X-Stat-Signature: 58rredxfufdpw31cm5s81g1ezwdwk8kx X-HE-Tag: 1715742570-877578 X-HE-Meta: U2FsdGVkX18k5xdPeiJNGhM0C5+nD4TtW9HYWTFLulsVcQX4yJuT7oOe/8vGxe069ebZ6i4iGFDvP6TOemuRHY57P2yoj7LJcV3RzlM6PxPxr9esrEvVvg2PT6f4P6zyQ4R6TKOsAxMSfQX2q0YMdBNKJWPpXwSL7qGknJabPAxvlLJCRdSJftnclCZ6Vp3ToHYtQDh29la1zInhhft8k/DoYoG4KQnlzrfg033L9Am0Qp6IV33Qm/+BqTZryWHXJuhFZdlRV0Bqc8iaMjsqrjXSjnr752h5djlpBaYbzhuTEwrcYL2wOnaNuRV65AnVDaqs3hnOsbJSI16pWWZimUoe4YZNZ9KClg1Kwz3+GQrce1gj67xOQEAtJlTlmBdGd8CO8SQ47xm8cuxtByIPGtHBuJ4SlsDF36tltw5A2TTdaI2h8OAKzZxQilnI97MF6uOfbr/qt8y1gSu4BCVyD7DezDKBAt9dgohEI4qGliN0sMPEv2tmCU2o4KMZiDhU6KeBbLSL6kOiNQEHUuaHlqJiKYcHqJfvMxos+bFE362d1wXOcwzcGBXA1OZB/H9it4HmvXHA6HiZXP0J/3ozWwGOdwIEXkWotQWU64UazriqO/QMfN7h+c8iBiAmlpcp1zrfcUb2q7RTNgLwSeUu2yWSTZpNg8KZ9UcbBLKR8gQaJdu4d2pF84sgDoFItELM1vauBbCU7IPZJp8CG+EBAGIui/gy2cE5yulJnDjlnPde2u3nfVJbbgGr7EQPEXaUx2OGFlCppcNj3rOueL+QKN32PeeBfRYRIFl9i3BRjeSuYU9m206+Ap0ySuOi0QQypXeSeQz70itxJ0EAXI7t8LIIxeY/T3EHzZoKyZeAC00rVBFsNBKQE3to5dVyxg3nkdPp97hSNY+mxtgcDLoYUuK9k8CVKFCcrZtK1gZN7ZrcUCXCFXNMgCXIpgM8w3ZTLQi0/9i+n/J7AqCX6wq hWjJdFWn HUUNzD1usjQee9sY+YRuU4XPaeLwxXbo8nYHczA7hvFPGPYb/PcNFdk2EzgUWEHLHK8/YeSlMfDkCQ3FZrs2usf8JfSiUK+81he0/iWSebguraIXzfVn0WrAoHjQGZHf0di9V+B/1hMKxLz3EtvwsI6v032Gzup70CxD6piLHSSE6LLo3GrgZ1ZqgJzPahPHErxSxFqGJaQdM0tmLTaaCRi7jASGdiBjRDuMgyuSf46DYv2E= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/5/14 21:36, Lance Yang wrote: > Hi Baolin, > > On Mon, May 13, 2024 at 1:08 PM Baolin Wang > wrote: >> >> Commit 19eaf44954df adds multi-size THP (mTHP) for anonymous pages, that >> can allow THP to be configured through the sysfs interface located at >> '/sys/kernel/mm/transparent_hugepage/hugepage-XXkb/enabled'. >> >> However, the anonymous share pages will ignore the anonymous mTHP rule >> configured through the sysfs interface, and can only use the PMD-mapped >> THP, that is not reasonable. Users expect to apply the mTHP rule for >> all anonymous pages, including the anonymous share pages, in order to >> enjoy the benefits of mTHP. For example, lower latency than PMD-mapped THP, >> smaller memory bloat than PMD-mapped THP, contiguous PTEs on ARM architecture >> to reduce TLB miss etc. >> >> The primary strategy is similar to supporting anonymous mTHP. Introduce >> a new interface '/mm/transparent_hugepage/hugepage-XXkb/shmem_enabled', >> which can have all the same values as the top-level >> '/sys/kernel/mm/transparent_hugepage/shmem_enabled', with adding a new >> additional "inherit" option. By default all sizes will be set to "never" >> except PMD size, which is set to "inherit". This ensures backward compatibility >> with the anonymous shmem enabled of the top level, meanwhile also allows >> independent control of anonymous shmem enabled for each mTHP. >> >> Signed-off-by: Baolin Wang >> --- >> include/linux/huge_mm.h | 10 +++ >> mm/shmem.c | 179 +++++++++++++++++++++++++++++++++------- >> 2 files changed, 161 insertions(+), 28 deletions(-) >> >> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h >> index 1fce6fee7766..b5339210268d 100644 >> --- a/include/linux/huge_mm.h >> +++ b/include/linux/huge_mm.h >> @@ -583,6 +583,16 @@ static inline bool thp_migration_supported(void) >> { >> return false; >> } >> + >> +static inline int highest_order(unsigned long orders) >> +{ >> + return 0; >> +} >> + >> +static inline int next_order(unsigned long *orders, int prev) >> +{ >> + return 0; >> +} >> #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ >> >> static inline int split_folio_to_list_to_order(struct folio *folio, >> diff --git a/mm/shmem.c b/mm/shmem.c >> index 59cc26d44344..b50ddf013e37 100644 >> --- a/mm/shmem.c >> +++ b/mm/shmem.c >> @@ -1611,6 +1611,106 @@ static gfp_t limit_gfp_mask(gfp_t huge_gfp, gfp_t limit_gfp) >> return result; >> } >> >> +#ifdef CONFIG_TRANSPARENT_HUGEPAGE >> +static unsigned long anon_shmem_allowable_huge_orders(struct inode *inode, >> + struct vm_area_struct *vma, pgoff_t index, >> + bool global_huge) >> +{ >> + unsigned long mask = READ_ONCE(huge_anon_shmem_orders_always); >> + unsigned long within_size_orders = READ_ONCE(huge_anon_shmem_orders_within_size); >> + unsigned long vm_flags = vma->vm_flags; >> + /* >> + * Check all the (large) orders below HPAGE_PMD_ORDER + 1 that >> + * are enabled for this vma. >> + */ >> + unsigned long orders = BIT(PMD_ORDER + 1) - 1; >> + loff_t i_size; >> + int order; >> + >> + if ((vm_flags & VM_NOHUGEPAGE) || >> + test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) >> + return 0; >> + >> + /* If the hardware/firmware marked hugepage support disabled. */ >> + if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_UNSUPPORTED)) >> + return 0; >> + >> + /* >> + * Following the 'deny' semantics of the top level, force the huge >> + * option off from all mounts. >> + */ >> + if (shmem_huge == SHMEM_HUGE_DENY) >> + return 0; >> + /* >> + * Only allow inherit orders if the top-level value is 'force', which >> + * means non-PMD sized THP can not override 'huge' mount option now. >> + */ >> + if (shmem_huge == SHMEM_HUGE_FORCE) >> + return READ_ONCE(huge_anon_shmem_orders_inherit); >> + >> + /* Allow mTHP that will be fully within i_size. */ >> + order = highest_order(within_size_orders); >> + while (within_size_orders) { >> + index = round_up(index + 1, order); >> + i_size = round_up(i_size_read(inode), PAGE_SIZE); >> + if (i_size >> PAGE_SHIFT >= index) { >> + mask |= within_size_orders; >> + break; >> + } >> + >> + order = next_order(&within_size_orders, order); >> + } >> + >> + if (vm_flags & VM_HUGEPAGE) >> + mask |= READ_ONCE(huge_anon_shmem_orders_madvise); >> + >> + if (global_huge) >> + mask |= READ_ONCE(huge_anon_shmem_orders_inherit); >> + >> + return orders & mask; >> +} >> + >> +static unsigned long anon_shmem_suitable_orders(struct inode *inode, struct vm_fault *vmf, >> + struct address_space *mapping, pgoff_t index, >> + unsigned long orders) >> +{ >> + struct vm_area_struct *vma = vmf->vma; >> + unsigned long pages; >> + int order; >> + >> + orders = thp_vma_suitable_orders(vma, vmf->address, orders); >> + if (!orders) >> + return 0; >> + >> + /* Find the highest order that can add into the page cache */ >> + order = highest_order(orders); >> + while (orders) { >> + pages = 1UL << order; >> + index = round_down(index, pages); >> + if (!xa_find(&mapping->i_pages, &index, >> + index + pages - 1, XA_PRESENT)) >> + break; >> + order = next_order(&orders, order); >> + } >> + >> + return orders; >> +} >> +#else >> +static unsigned long anon_shmem_allowable_huge_orders(struct inode *inode, >> + struct vm_area_struct *vma, pgoff_t index, >> + bool global_huge) >> +{ >> + return 0; >> +} >> + >> +static unsigned long anon_shmem_suitable_orders(struct inode *inode, struct vm_fault *vmf, >> + struct address_space *mapping, pgoff_t index, >> + unsigned long orders) >> +{ >> + return 0; >> +} >> +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ >> + >> static struct folio *shmem_alloc_hugefolio(gfp_t gfp, >> struct shmem_inode_info *info, pgoff_t index, int order) >> { >> @@ -1639,38 +1739,55 @@ static struct folio *shmem_alloc_folio(gfp_t gfp, >> return (struct folio *)page; >> } >> >> -static struct folio *shmem_alloc_and_add_folio(gfp_t gfp, >> - struct inode *inode, pgoff_t index, >> - struct mm_struct *fault_mm, bool huge) >> +static struct folio *shmem_alloc_and_add_folio(struct vm_fault *vmf, >> + gfp_t gfp, struct inode *inode, pgoff_t index, >> + struct mm_struct *fault_mm, bool huge, unsigned long orders) > > IMO, it might be cleaner to drop the huge parameter and just set 'orders' as > BIT(HPAGE_PMD_ORDER), then we only do the 'orders' check :) > > Likely: > > if (orders > 0) { > if (vma && vma_is_anon_shmem(vma)) { > ... > } else if (orders & BIT(HPAGE_PMD_ORDER)) { > ... > } > } Yes, looks better. >> { >> struct address_space *mapping = inode->i_mapping; >> struct shmem_inode_info *info = SHMEM_I(inode); >> - struct folio *folio; >> + struct vm_area_struct *vma = vmf ? vmf->vma : NULL; >> + unsigned long suitable_orders; >> + struct folio *folio = NULL; >> long pages; >> - int error; >> + int error, order; >> >> if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) >> huge = false; > > Currently, if THP is disabled, 'huge' will fall back to order-0, but 'orders' > does not, IIUC. How about we make both consistent if THP is disabled? > > if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) { > huge = false; > orders = 0; > } If THP is disabled, the 'orders' must be 0, so no need to reset it.