From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29376C3DA63 for ; Thu, 25 Jul 2024 01:01:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BC86C6B008C; Wed, 24 Jul 2024 21:01:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B78F46B0092; Wed, 24 Jul 2024 21:01:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A40A06B0093; Wed, 24 Jul 2024 21:01:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 84F766B008C for ; Wed, 24 Jul 2024 21:01:42 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 02B981C231A for ; Thu, 25 Jul 2024 01:01:41 +0000 (UTC) X-FDA: 82376472444.10.D1CA736 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf30.hostedemail.com (Postfix) with ESMTP id 6766180011 for ; Thu, 25 Jul 2024 01:01:38 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=none; spf=pass (imf30.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721869276; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3VL6rJ927Le5eHstEZz6Jwxs/l+PF1L5guT/3xnfyhM=; b=3C3XW3xYAtBOm9+idYwCdOnCVjIpcyaZJkn1SJ0OKFT9gn8kHHR0c1MX/IXvjnnAioBvIe FGYK5w5uZWrZjnuUIgPdet0ZB7HiUS+T9YcRt6GY0wL0nZIT8cxgyqqi7cwPOCj2jr4G/1 ZE5RzX0JAvsRzyFWI1m1cj0srx+I8h4= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=none; spf=pass (imf30.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721869276; a=rsa-sha256; cv=none; b=XL0XLpvbhnMEw/mCE2xlfg3pKgpCRwCOvyr0ctPQzNKVde45B9HY+ZRn3u8IIKv8lGa1iX arSaS9wj5bcpH2v2Jiji0DOg9O8H3DdfXnY1ZxhexbIxNc1uW+Z8nJfihBQw/gLf/1jCTQ Y4Gt2Md19E3hZKPzGPIdTpGR7KcpLWg= Received: from mail.maildlp.com (unknown [172.19.88.105]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4WTsyz5QMMz1L9K7; Thu, 25 Jul 2024 09:01:31 +0800 (CST) Received: from dggpemf100008.china.huawei.com (unknown [7.185.36.138]) by mail.maildlp.com (Postfix) with ESMTPS id 9D1BC14011B; Thu, 25 Jul 2024 09:01:34 +0800 (CST) Received: from [10.174.177.243] (10.174.177.243) by dggpemf100008.china.huawei.com (7.185.36.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Thu, 25 Jul 2024 09:01:34 +0800 Message-ID: <60b10c39-3675-4cbc-96d8-2b6f1170464d@huawei.com> Date: Thu, 25 Jul 2024 09:01:34 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3 2/3] memory tiering: introduce folio_use_access_time() check Content-Language: en-US To: Zi Yan , Andrew Morton , CC: David Hildenbrand , "Huang, Ying" , Baolin Wang , Lorenzo Stoakes , References: <20240724130115.793641-1-ziy@nvidia.com> <20240724130115.793641-3-ziy@nvidia.com> From: Kefeng Wang In-Reply-To: <20240724130115.793641-3-ziy@nvidia.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.174.177.243] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemf100008.china.huawei.com (7.185.36.138) X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 6766180011 X-Stat-Signature: tjmy8hb7bs3pdfzmemwra7zw98rji97i X-HE-Tag: 1721869298-359614 X-HE-Meta: U2FsdGVkX1+W79lcMTBuIwi9EUzMYokiOHEDOzpQMDrfUW4d/klJSZ8lWdidSY2R8rR++lDRh+Gnrg2mZiYKnNVPw9yI5iWM3gy6GR5WUhuSXcTsjQZCvwEJRUtL19HhghU1is86+ZGmys5sDAeBwsOtZJqlPkDM8YxEzpbg/iqarY7lCosEiFjZC1ec0ousOHmAIAe1Cnoj7Z61pFGZMidpR5qjERFKtjSB2ZqgEQhTstY92i+5AWInC9EGjYZ25DEqDj/PxdKQSzFS+9q9163aWMC6ShWFBynLri2m1amfxFIarwaRQ4IxkyyWQiHQkb9YsGJdqKEKYe4cuaz0lG3lp45vO1N7pKOg5LRH/2eJ9eSMny24nn7odL+nFLLA+QOEdpDV7/3ISk3C7cP3L5OBGXVozXKUPt4utunVucgLIJhQY0u97RuPxIGKsAfR+SNRhFLOHwQOQMFzRm2OG6RvJkHwAFP63YypNxQ9vPvtdC+8YAKRlXtfiYgCSiiSqbItJdPj6ZpfrF1y3bAESozvPgmlwjdCwPj95qZGN8ZJJO9AHMRQZ6/yK10gUzOEa0ciJOmj1zY3UARjYTyaS0S6c5MjdmCgCv1w7HUNOKhSyBa44790hMKcKbqUwUGkVw27Y2A1arw4XCTS248K+6vTqkVC1YiEn89e5JLxNnuqfRAhYvFQj81+VOW4UKO/FHo7eWoTPh9Ssf8Hta9K+aaO2lVBwqS2RTiRUHDqzxjziky4E7XT70/EK2fDuhYY9DsMylwFb76OzNWHCgnET6Wa7EI9dgdTlqJRGXDWXYY8TSAtf8wgk2vPeVvMJRtgqXeuKmfeE9HHWuTCeij8b/53yt4kwVPobf18geHsgkEQTe1UeKoZ0U8YyLub+tefwdgNM4IWOkopoTq8F3L0yXtpnp0FvAJCfyEAQ481d21xUZsft3tzr+f6OpARcjYK0fXh+XVLScN4IJaTy/n 0j3NLyw9 y1SHFZDZkfd9PwThngpcIF6nVjZdCEeAPeWaE0Ykqp4T0mjxeMwZJk3ZGYjfqbzyjHin/NRm2cbwkXTq6Q7lZAIldIkRhuTxWrJyoHiVLludmEOFhuNZ3tuF9I5M8wBtbWVOe+kXJ73qkLv4eLkB0im/9hLbmAPSUnApdBO1K9ulut9ULD1tkK5+LjPbYdDKDEwyUFtiCslF4EpGv35n6KW+rPW6MXhFL2br5+jEMWrBrKBBs38jV9CziJWsKKD9NDe00pfOT6T/OjRAzkgK1YbwMq33up0kEiPI97tYjW6LFwzNMPP7mD7S5Ew== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/7/24 21:01, Zi Yan wrote: > If memory tiering mode is on and a folio is not in the top tier memory, > folio's cpupid field is repurposed to store page access time. Instead of > an open coded check, use a function to encapsulate the check. > > Signed-off-by: Zi Yan > Reviewed-by: "Huang, Ying" > Acked-by: David Hildenbrand Reviewed-by: Kefeng Wang > --- > include/linux/mm.h | 6 ++++++ > kernel/sched/fair.c | 3 +-- > mm/huge_memory.c | 6 ++---- > mm/memory-tiers.c | 19 +++++++++++++++++++ > mm/memory.c | 3 +-- > mm/mprotect.c | 3 +-- > 6 files changed, 30 insertions(+), 10 deletions(-) > > diff --git a/include/linux/mm.h b/include/linux/mm.h > index 17753a463e01..2c6ccf088c7b 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -1717,6 +1717,8 @@ static inline void vma_set_access_pid_bit(struct vm_area_struct *vma) > __set_bit(pid_bit, &vma->numab_state->pids_active[1]); > } > } > + > +bool folio_use_access_time(struct folio *folio); > #else /* !CONFIG_NUMA_BALANCING */ > static inline int folio_xchg_last_cpupid(struct folio *folio, int cpupid) > { > @@ -1770,6 +1772,10 @@ static inline bool cpupid_match_pid(struct task_struct *task, int cpupid) > static inline void vma_set_access_pid_bit(struct vm_area_struct *vma) > { > } > +static inline bool folio_use_access_time(struct folio *folio) > +{ > + return false; > +} > #endif /* CONFIG_NUMA_BALANCING */ > > #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index 9057584ec06d..416e29b56cc4 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -1840,8 +1840,7 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, > * The pages in slow memory node should be migrated according > * to hot/cold instead of private/shared. > */ > - if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && > - !node_is_toptier(src_nid)) { > + if (folio_use_access_time(folio)) { > struct pglist_data *pgdat; > unsigned long rate_limit; > unsigned int latency, th, def_th; > diff --git a/mm/huge_memory.c b/mm/huge_memory.c > index 15234b2e252e..5c0a6a4e3a6e 100644 > --- a/mm/huge_memory.c > +++ b/mm/huge_memory.c > @@ -1707,8 +1707,7 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) > * For memory tiering mode, cpupid of slow memory page is used > * to record page access time. So use default value. > */ > - if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) || > - node_is_toptier(nid)) > + if (!folio_use_access_time(folio)) > last_cpupid = folio_last_cpupid(folio); > target_nid = numa_migrate_prep(folio, vmf, haddr, nid, &flags); > if (target_nid == NUMA_NO_NODE) > @@ -2061,8 +2060,7 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, > toptier) > goto unlock; > > - if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && > - !toptier) > + if (folio_use_access_time(folio)) > folio_xchg_access_time(folio, > jiffies_to_msecs(jiffies)); > } > diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c > index 4775b3a3dabe..2a642ea86cb2 100644 > --- a/mm/memory-tiers.c > +++ b/mm/memory-tiers.c > @@ -6,6 +6,7 @@ > #include > #include > #include > +#include > > #include "internal.h" > > @@ -50,6 +51,24 @@ static const struct bus_type memory_tier_subsys = { > .dev_name = "memory_tier", > }; > > +#ifdef CONFIG_NUMA_BALANCING > +/** > + * folio_use_access_time - check if a folio reuses cpupid for page access time > + * @folio: folio to check > + * > + * folio's _last_cpupid field is repurposed by memory tiering. In memory > + * tiering mode, cpupid of slow memory folio (not toptier memory) is used to > + * record page access time. > + * > + * Return: the folio _last_cpupid is used to record page access time > + */ > +bool folio_use_access_time(struct folio *folio) > +{ > + return (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) && > + !node_is_toptier(folio_nid(folio)); > +} > +#endif > + > #ifdef CONFIG_MIGRATION > static int top_tier_adistance; > /* > diff --git a/mm/memory.c b/mm/memory.c > index 802d0d8a40f9..833d2cad6eb2 100644 > --- a/mm/memory.c > +++ b/mm/memory.c > @@ -5337,8 +5337,7 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) > * For memory tiering mode, cpupid of slow memory page is used > * to record page access time. So use default value. > */ > - if ((sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) && > - !node_is_toptier(nid)) > + if (folio_use_access_time(folio)) > last_cpupid = (-1 & LAST_CPUPID_MASK); > else > last_cpupid = folio_last_cpupid(folio); > diff --git a/mm/mprotect.c b/mm/mprotect.c > index 222ab434da54..37cf8d249405 100644 > --- a/mm/mprotect.c > +++ b/mm/mprotect.c > @@ -161,8 +161,7 @@ static long change_pte_range(struct mmu_gather *tlb, > if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_NORMAL) && > toptier) > continue; > - if (sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING && > - !toptier) > + if (folio_use_access_time(folio)) > folio_xchg_access_time(folio, > jiffies_to_msecs(jiffies)); > }