From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B41CBC35FF1 for ; Fri, 14 Mar 2025 15:29:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DEFD9280003; Fri, 14 Mar 2025 11:29:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D77C2280001; Fri, 14 Mar 2025 11:29:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF0BB280003; Fri, 14 Mar 2025 11:29:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 9F4C8280001 for ; Fri, 14 Mar 2025 11:29:00 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 09C5BB8F4B for ; Fri, 14 Mar 2025 15:29:01 +0000 (UTC) X-FDA: 83220539682.14.55F2322 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by imf16.hostedemail.com (Postfix) with ESMTP id B95D6180004 for ; Fri, 14 Mar 2025 15:28:58 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf16.hostedemail.com: domain of jonathan.cameron@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=jonathan.cameron@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741966139; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2Cb/mzl6thcTW78i7GhmAr0/+NoRvNzGPPIViOVnCk8=; b=YAKPqyRWg2vbaKHWqaf/2m6wekCnLvNfozBlv/CQqVgCvrJB2CervAT+8+48ToOYr60oud Nv8dBQ+csgmzDRE2TUU+f3s47IKIVep4H4+y4/SOHhwOIvYFeCmSFCXDaTmnIzUW8DxlSQ Q1sGBlCGdsCuXJElKlNtPqrdXTiTvJY= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf16.hostedemail.com: domain of jonathan.cameron@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=jonathan.cameron@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741966139; a=rsa-sha256; cv=none; b=NGdYSTOsN2gHXDfKxM9n8cRO/v9ZhuBVbJDzMgvibuiXsEmOEQV+PXQKa0tpvzn99fv9lZ AJbC6vrRi4FeuRf96OlbPlrmdM+RFSbih061M4TADzEBG/gf7yMv0nGlwZdEfg3x5+iEc3 NsvsXCE+1ifjD+9QoVdXzozPoAovyak= Received: from mail.maildlp.com (unknown [172.18.186.31]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4ZDp7Q3bdWz6H6s3; Fri, 14 Mar 2025 23:24:18 +0800 (CST) Received: from frapeml500008.china.huawei.com (unknown [7.182.85.71]) by mail.maildlp.com (Postfix) with ESMTPS id C41EB14039F; Fri, 14 Mar 2025 23:28:53 +0800 (CST) Received: from localhost (10.203.177.66) by frapeml500008.china.huawei.com (7.182.85.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Fri, 14 Mar 2025 16:28:52 +0100 Date: Fri, 14 Mar 2025 15:28:50 +0000 From: Jonathan Cameron To: Bharata B Rao CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: Re: [RFC PATCH 2/4] mm: kpromoted: Hot page info collection and promotion daemon Message-ID: <20250314152850.00003112@huawei.com> In-Reply-To: <20250306054532.221138-3-bharata@amd.com> References: <20250306054532.221138-1-bharata@amd.com> <20250306054532.221138-3-bharata@amd.com> X-Mailer: Claws Mail 4.3.0 (GTK 3.24.42; x86_64-w64-mingw32) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.203.177.66] X-ClientProxiedBy: lhrpeml500005.china.huawei.com (7.191.163.240) To frapeml500008.china.huawei.com (7.182.85.71) X-Rspam-User: X-Rspamd-Queue-Id: B95D6180004 X-Rspamd-Server: rspam08 X-Stat-Signature: 79a9rpfnpoe8te5yuttgr1u79qkgmaat X-HE-Tag: 1741966138-220675 X-HE-Meta: U2FsdGVkX18DipS8uwHvutGYusuHtLSo57OERHieFagCcUtZ3cHMK8ZoWjZA4H52g3mj8GCrYwF/+WkHmIPxFoFu/8OEqUfq2U6uR0pSIGmgNgPQ9hHSdptDmwOpXPWAhAQrdVnue7nz0lzYSwFN6gkuaj3EWdM3DOUGs/4VeQFGwf3gwakb14tqRgfcf2jDODW8OEhIv0d8wPQJ2+dcmPS5ICLfc5+lazteh2NjXQmJiUWbf/DB4XlT4mKppHVS2+M84/32Ya0GBnL6M9crGik3gnzew47Yl4hn/LIdBcUrn+toq1Hwa0J9nfdELDgXcI9v/m6CJbBNjjySMvF6tras5wkpd1l9r6rEMbh9VEtiY/G1am0CddLOR/c29sd5DQOFkxTP94AdAc+1P7ry+Me7tVUqhbhX4DtZDj+ktJPOtYe+beAv1/2Zae3Npn14DBDjv2vrFrJtRJRVHWVEkqwuhVtCL6xmMDI/wnsoZX3oftI+J6HA5+hwTbKO2qe3U0Wd94alM8M/B5GH8j0MJ+7WT9lN/4bn/D6l8rruhgbaL9aVlwRgyvKYhsTu9QvvtIDl34lu93xIVjty1R5tQsu9y7ni9ccufV4HCUDEgAjxeQvKrN3V72uwebQnFyv0/kUkZy1yyLz1enwmR/ztwbOFG4OWrSOCmu45uOInrsewZGnCTUrbbpHKoxtNj25HnlqcwZY4ohSlmREOwvN1FknYj7rRbPGXl1YTRXnAxo+0HVPXtRnvrdBNjEIl9vTTww+2UoraOeh1ZCrQSNk60yN/qwfkAtuLS9OVxvlXz/bSYOBmszRKU6Zw8WB/+mjqU6cWmShvWa5teje45ZEpahE9GlvyIsEDhTbY2wxHQpO8OVvohQ5icK6bLkE5uZulM2OWB/sZjwohI1sbIVsOH2/T1z4GRMitGG204JJB1fr0TglSyZx5V3emgkMEJVe00j0Wn3v3f1xX1BcNTcN d5FTHMOY a54AhLxl936AlXpmwG+Lq5DrkxKxPHvatLJz2S6+daDwWRcB7Z6X7ogMD3GlN87Eg4wv/apVlg5OxOaLfcAADI3Hs8XPWHtzUskDLOrJ1oZ5X5m6WI+Wg1m/W88l2H3LRXijOJjznizThBr3HhzqjYVaxti19f9fko3rDpV6Fv5jbrIwqXtBzzC+W+pH/t8PiFUnHR3mrO8DhboukkvjzrT+mVtyMHYgzTtp4wxCCbe4DRlEBRKtj9rcJzduUHrq8ky+LQNBv83HbQfa/lXLfKPtOCw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, 6 Mar 2025 11:15:30 +0530 Bharata B Rao wrote: > kpromoted is a kernel daemon that accumulates hot page info > from different sources and tries to promote pages from slow > tiers to top tiers. One instance of this thread runs on each > node that has CPUs. >=20 Firstly, nice work. Much easier to discuss things with an implementation to look at. I'm looking at this with my hardware hotness unit "hammer" in hand :) > Subsystems that generate hot page access info can report that > to kpromoted via this API: >=20 > int kpromoted_record_access(u64 pfn, int nid, int src, > unsigned long time) This perhaps works as an interface for aggregating methods that produce per access events. Any hardware counter solution is going to give you data that is closer to what you used for the promotion decision. We might need to aggregate at different levels. So access counting promotes to a hot list and we can inject other events at that level. The data I have from the CXL HMU is typically after an epoch (period of time) these N pages were accessed more than M times. I can sort of map that to the internal storage you have. Would be good to evaluate approximate trackers on top of access counts. I've no idea if sketches or similar would be efficient enough (they have a bit of a write amplification problem) but they may give good answers with much lower storage cost at the risk of occasionally saying something is hot when it's not. >=20 > @pfn: The PFN of the memory accessed > @nid: The accessing NUMA node ID > @src: The temperature source (subsystem) that generated the > access info > @time: The access time in jiffies >=20 > Some temperature sources may not provide the nid from which > the page was accessed. This is true for sources that use > page table scanning for PTE Accessed bit. Currently the toptier > node to which such pages should be promoted to is hard coded. For those cases (CXL HMU included) maybe we need to consider how to fill in missing node info with at least a vague chance of getting a reasonable target for migration. We can always fall back to random top tier node, or nearest one to where we are coming from (on basis we maybe landed in this node based on a fallback list when the top tier was under memory pressure). =46rom an interface point of view is that a problem for this layer, or for the underlying tracking mechanism? (maybe with some helpers) Also, see later discussion of consistency of hotness tracking and that the best solution for that differs from that to get potential targets. The answer to Is this page consistently hot?" can be approximated with "Was this page once hot and is it not now cold?" Access time is something some measurement techniques will only give you wrt to a measurement was in a window (potentially a long one if you are looking for consistent hotness over minutes). >=20 > Also, the access time provided some sources may at best be > considered approximate. This is especially true for hot pages > detected by PTE A bit scanning. >=20 > kpromoted currently maintains the hot PFN records in hash lists > hashed by PFN value. Each record stores the following info: >=20 > struct page_hotness_info { > unsigned long pfn; >=20 > /* Time when this record was updated last */ > unsigned long last_update; >=20 > /* > * Number of times this page was accessed in the > * current window I'd express here how that window was defined (I read on to answer the question I had here at first!) > */ > int frequency; >=20 > /* Most recent access time */ > unsigned long recency; Put next to the last_update so all the times are together >=20 > /* Most recent access from this node */ > int hot_node; Probably want to relax the most recent part. I'd guess the ideal here would be if this is the node accessing it the most 'recently'. >=20 > struct hlist_node hnode; > }; >=20 > The way in which a page is categorized as hot enough to be > promoted is pretty primitive now. That bit is very hard even if we solve everything else and heavily dependent on workload access pattern stability and migration impact. Maybe for 'very hot' pages a fairly short consistency of hotness period is good enough, but it gets much messier if we care about warm pages. I guess we solve the 'very hot' first though and maybe avoid the phase transition from an application starting to when it is at steady state by considering a wait time for any new userspace process before we consider moving anything? Also worth noting that the mechanism that makes sense to check if a detected hot page is 'stable hot' might use entirely different tracking approach to that used to find it as a candidate. Whether that requires passing data between hotness trackers is an interesting question, or whether there is a natural ordering to trackers. > diff --git a/mm/kpromoted.c b/mm/kpromoted.c > new file mode 100644 > index 000000000000..2a8b8495b6b3 > --- /dev/null > +++ b/mm/kpromoted.c > +static int page_should_be_promoted(struct page_hotness_info *phi) > +{ > + struct page *page =3D pfn_to_online_page(phi->pfn); > + unsigned long now =3D jiffies; > + struct folio *folio; > + > + if (!page || is_zone_device_page(page)) > + return false; > + > + folio =3D page_folio(page); > + if (!folio_test_lru(folio)) { > + count_vm_event(KPROMOTED_MIG_NON_LRU); > + return false; > + } > + if (folio_nid(folio) =3D=3D phi->hot_node) { > + count_vm_event(KPROMOTED_MIG_RIGHT_NODE); > + return false; > + } > + > + /* If the page was hot a while ago, don't promote */ /* If the known record of hotness is old, don't promote */ ? Otherwise this says don't move a page just because it was hot a long time back. Maybe it is still hot and we just don't have an update yet? > + if ((now - phi->last_update) > 2 * msecs_to_jiffies(KPROMOTED_FREQ_WIND= OW)) { > + count_vm_event(KPROMOTED_MIG_COLD_OLD); > + return false; > + } > + > + /* If the page hasn't been accessed enough number of times, don't promo= te */ > + if (phi->frequency < KPRMOTED_FREQ_THRESHOLD) { > + count_vm_event(KPROMOTED_MIG_COLD_NOT_ACCESSED); > + return false; > + } > + return true; > +} > + > +/* > + * Go thro' page hotness information and migrate pages if required. > + * > + * Promoted pages are not longer tracked in the hot list. > + * Cold pages are pruned from the list as well. When we say cold here why did we ever see them? > + * > + * TODO: Batching could be done > + */ > +static void kpromoted_migrate(pg_data_t *pgdat) > +{ > + int nid =3D pgdat->node_id; > + struct page_hotness_info *phi; > + struct hlist_node *tmp; > + int nr_bkts =3D HASH_SIZE(page_hotness_hash); > + int bkt; > + > + for (bkt =3D 0; bkt < nr_bkts; bkt++) { > + mutex_lock(&page_hotness_lock[bkt]); > + hlist_for_each_entry_safe(phi, tmp, &page_hotness_hash[bkt], hnode) { > + if (phi->hot_node !=3D nid) > + continue; > + > + if (page_should_be_promoted(phi)) { > + count_vm_event(KPROMOTED_MIG_CANDIDATE); > + if (!kpromote_page(phi)) { > + count_vm_event(KPROMOTED_MIG_PROMOTED); > + hlist_del_init(&phi->hnode); > + kfree(phi); > + } > + } else { > + /* > + * Not a suitable page or cold page, stop tracking it. > + * TODO: Identify cold pages and drive demotion? Coldness tracking is really different from hotness as we need to track what= we didn't see to get the really cold pages. Maybe there is some hint to be had from the exit of this tracker but I'd definitely not try to tackle both ends with one approach! > + */ > + count_vm_event(KPROMOTED_MIG_DROPPED); > + hlist_del_init(&phi->hnode); > + kfree(phi); > + } > + } > + mutex_unlock(&page_hotness_lock[bkt]); > + } > +} > +/* > + * Called by subsystems that generate page hotness/access information. > + * > + * Records the memory access info for futher action by kpromoted. > + */ > +int kpromoted_record_access(u64 pfn, int nid, int src, unsigned long now) > +{ > + bkt =3D hash_min(pfn, KPROMOTED_HASH_ORDER); > + mutex_lock(&page_hotness_lock[bkt]); > + phi =3D kpromoted_lookup(pfn, bkt, now); > + if (!phi) { > + ret =3D PTR_ERR(phi); > + goto out; > + } > + > + if ((phi->last_update - now) > msecs_to_jiffies(KPROMOTED_FREQ_WINDOW))= { > + /* New window */ > + phi->frequency =3D 1; /* TODO: Factor in the history */ > + phi->last_update =3D now; > + } else { > + phi->frequency++; > + } > + phi->recency =3D now; > + > + /* > + * TODOs: > + * 1. Source nid is hard-coded for some temperature sources Hard coded rather than unknown? I'm curious, what source has that issue? > + * 2. Take action if hot_node changes - may be a shared page? > + * 3. Maintain node info for every access within the window? I guess some sort of saturating counter set might not be too bad. > + */ > + phi->hot_node =3D (nid =3D=3D NUMA_NO_NODE) ? 1 : nid; > + mutex_unlock(&page_hotness_lock[bkt]); > +out: > + return 0; why store ret and not return it? > +} > +