From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8A93C4345F for ; Fri, 12 Apr 2024 08:50:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5E9936B0083; Fri, 12 Apr 2024 04:50:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 599546B0087; Fri, 12 Apr 2024 04:50:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 460556B0088; Fri, 12 Apr 2024 04:50:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 286696B0083 for ; Fri, 12 Apr 2024 04:50:30 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A4CC8140DDE for ; Fri, 12 Apr 2024 08:50:29 +0000 (UTC) X-FDA: 82000258578.15.AE6A440 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.17]) by imf22.hostedemail.com (Postfix) with ESMTP id 03222C0005 for ; Fri, 12 Apr 2024 08:50:26 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=SzgWcSLC; spf=pass (imf22.hostedemail.com: domain of ying.huang@intel.com designates 192.198.163.17 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712911827; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=W/WU7xNjn8Uk5lzJUmpJZIWGUjFmfW6Qan7lPa81Ii4=; b=aQWby9r6zRlGvk8b8sbNx6CAJox1iMsvboOFPr/XIjTrWWptaUnQ2YqAPknoNKZk65ZNLN teq3zNgcccrKTMjoFHFOJkvyCy8oAhP0hi/JMsHpNAsI1r+4mU0VMpAsUkgbWEAOlHDBS0 N43yBi3wKw4sB8dq3lNtyWriibAJ3hw= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=SzgWcSLC; spf=pass (imf22.hostedemail.com: domain of ying.huang@intel.com designates 192.198.163.17 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712911827; a=rsa-sha256; cv=none; b=OaHL3fPyriiGNSD6exuZNjUeHyHf6El/cMvpJ0QyvaHYZtvg7jm7cJ/DV7zuL1FyMw/Xqe 46o5bV/PQqRkKbjMiJbCY7EYbKLA/KSVsPSWwME3BdMCz4H/6QgNazWIAS2aZSgzC2oNfR UeHuWmZBoONr2HRDVAz6IMKq/400T5s= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1712911827; x=1744447827; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version; bh=bf4dIkV9LetyoNL/ouarPVJQ/zgpxdgdmJYWy/jU+ko=; b=SzgWcSLC6lf6OoamUgnTR3yQChSepLaQb396YBJUbKehs2KvYIhA6fMG xjPG0+aaceTaudAkfAhtc/qXyA6/zzM8mF7h47fwgnQreUOzZgzDCSVQN 7/rlTt9e7G8opTmPu9ZXzI7QTaWzeM/3R1fr6nwQDlNU2Ogws21bVB9Zw 2gONifT6Z0lkIjL6kz6bi3AcYCoHzHtRzjlk6F/uSZaqdgzPgdBdXUeLf /KUw3MvN1iXD3Ni05HO9BR0TYXnG8791yJk2e/RDYa+itClsdYJSk/89p aRwJ2034nnx702sHTDe5zwZOScfZFDylcpa67MU4lZx5PW6b/QmcpuoMd w==; X-CSE-ConnectionGUID: z8ROGO7+RH+NzIPnpGbfow== X-CSE-MsgGUID: Y6GtLoa/Qr+4I6wSUb9xHw== X-IronPort-AV: E=McAfee;i="6600,9927,11041"; a="8220019" X-IronPort-AV: E=Sophos;i="6.07,195,1708416000"; d="scan'208";a="8220019" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Apr 2024 01:50:25 -0700 X-CSE-ConnectionGUID: NcyoHsMzR8WZSAfF4z+NKg== X-CSE-MsgGUID: Usq70mwITMKMCYii2RGu6A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,195,1708416000"; d="scan'208";a="52331444" Received: from unknown (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Apr 2024 01:50:22 -0700 From: "Huang, Ying" To: Bharata B Rao Cc: , , , , , , , , Subject: Re: [RFC PATCH 0/2] Hot page promotion optimization for large address space In-Reply-To: <8b073658-268d-4d3e-bd94-3fe95c948bd9@amd.com> (Bharata B. Rao's message of "Fri, 12 Apr 2024 13:46:12 +0530") References: <20240327160237.2355-1-bharata@amd.com> <87il16lxzl.fsf@yhuang6-desk2.ccr.corp.intel.com> <87edbulwom.fsf@yhuang6-desk2.ccr.corp.intel.com> <929b22ca-bb51-4307-855f-9b4ae0a102e3@amd.com> <875xx5lu05.fsf@yhuang6-desk2.ccr.corp.intel.com> <7e373c71-b2dc-4ae4-9746-c840f2a513a5@amd.com> <87o7asfrm1.fsf@yhuang6-desk2.ccr.corp.intel.com> <9ec3b04b-bde8-42ce-be1b-34f7d8e6762d@amd.com> <87il0yet4z.fsf@yhuang6-desk2.ccr.corp.intel.com> <8692d514-d1c8-4fbf-84d7-1ad609480070@amd.com> <87cyqv59bj.fsf@yhuang6-desk2.ccr.corp.intel.com> <8b073658-268d-4d3e-bd94-3fe95c948bd9@amd.com> Date: Fri, 12 Apr 2024 16:48:29 +0800 Message-ID: <874jc755lu.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspamd-Queue-Id: 03222C0005 X-Rspam-User: X-Stat-Signature: oyy9a8o6am6fb1gqj18no1ynq64u1jjm X-Rspamd-Server: rspam01 X-HE-Tag: 1712911826-952827 X-HE-Meta: U2FsdGVkX1/PC0/G2J6VjwenDx6dJGG3Gb1gD+k6sK6/oz9zcx7JrEFNNsOHlkt7gIPsRYdcjGZ3c5MJsF0pFaTTnZDa4wmpsE0VG5dqEt7tlAVNCWg/IJjFgAhPAkMIzZmN37bcrVIe55Cngk402mP5KVs9iZ8IhNIGGHkEO1JfZkvkSENAqy1U++qO2W1umXjG08kMz18dY4bQ9on2n3gs0vP32IdTqnas57uRMw1mpkVQCjAlUEzgC2Ynye1iA0Jp2G96f0KRk1xOOtz2Hmno4HYXBvNxWhzNH3p2Pfosw4+nVSQgTNgbyJ5v7oeL1zBNSzvDz9/386bRjUK7wINxII2CUYeIoWfg9aBRDI2+eED9WigvG60q+MXeAfo4CakliBbXoKhA+EqW64WzpzoajOigYiGrM4O6JH7aSuWKyv+OwBu/YBBsn9K4pAVEaUOdegPdz6r1VZ9YGwJD9W+wlwth0cv89MY5RjkjVYLrO11tBlzq8W55fyidLYPoJtrq3CxyCg8aQhW2wHaM7Lg3IYvVg4snolO2UVCu1NLo7ihaKq175dFty3zhG4H2BRuMVjrJMS1py+gkJ0weZTvm3LAdSoLpO6hAbq7wKC9GtiYwTD0slCprObFVW+L0nXFG6wY3OwYEVpwKiYd7yns2KGMGoqTTn0xqyZ9hPu7lL5TUTnpf9dFyH15WRO75mxgEvgthi6df1G1ZXyMZinzRlwZ/MsFDoSPDSaNdfRZpvAfLp+B099tkBu3ykEEHUoNzBAxICguN3+nJTQ9ZOYvMZTMthm1LNisbm5Z6R4ww/jHx43zU9Ub1KG6WdqdRNcb/0vw9Z3Uhw15PcRqPJYpVo2HnI3AitKwmp6YQ22cM4J8NihLMSGohKWIymDc5787BtXvxt674k9zjNF6aY3wNsNFY69CujKOcPeJz8/8oFugc8TCa/+T6Kk4ZCMATvwyeFeuHk8azmKJxL3e Q+rLc4vi 75QsiJ2Z74/WAw0ilBBAoaptDv8fewxgv+x74pb2vwZ667z0gaohrIgGQ9a8HA/DixIJakWYOGg3btOwqYcTZaqHQb3Gi0ocuVq2ZJmOHSiDpij91GCPhJHyzimFtzxGd3z+vyvuzatCfB9hXu4FSfGsYwLfK6PTUSUamWZ8qzekdVNk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Bharata B Rao writes: > On 12-Apr-24 12:58 PM, Huang, Ying wrote: >> Bharata B Rao writes: >> >>> On 03-Apr-24 2:10 PM, Huang, Ying wrote: >>>>> Here are the numbers for the 192nd chunk: >>>>> >>>>> Each iteration of 262144 random accesses takes around ~10ms >>>>> 512 such iterations are taking ~5s >>>>> numa_scan_seq is 16 when this chunk is accessed. >>>>> And no page promotions were done from this chunk. All the >>>>> time should_numa_migrate_memory() found the NUMA hint fault >>>>> latency to be higher than threshold. >>>>> >>>>> Are these time periods considered too short for the pages >>>>> to be detected as hot and promoted? >>>> >>>> Yes. I think so. This is burst accessing, not repeated accessing. >>>> IIUC, NUMA balancing based promotion only works for repeated accessing >>>> for long time, for example, >100s. >>> >>> Hmm... When a page is accessed 512 times over a period of 5s and it is >>> still not detected as hot. This is understandable if fresh scanning couldn't >>> be done as the accesses were bursty and hence they couldn't be captured via >>> NUMA hint faults. But here the access captured via hint fault is being rejected >>> as not hot because the scanning was done a while back. But I do see the challenge >>> here since we depend on scanning time to obtain the frequency-of-access metric. >> >> Consider some pages that will be accessed once every 1 hour, should we >> consider it hot or not? Will your proposed method deal with that >> correctly? > > The proposed method removes the absolute time as a factor for the decision and instead > relies on the number of hint faults that have occurred since that page was scanned last. > As long as there are enough hint faults happening in that 1 hour (which means a lot many > other accesses have been captured in that 1 hour), that page shouldn't be considered as > hot. You did mention earlier about hint fault rate varying a lot and one thing I haven't > tried yet is to vary the fault threshold based on current or historical fault rate. In your original example, if a lot many other accesses between NUMA balancing page table scanning and 512 page accesses, you cannot identify the page as hot too, right? If the NUMA balancing page table scanning period is much longer than 5s, it's high possible that we cannot distinguish between 1 and 512 page accesses within 5s with your method and the original method. Better discuss the behavior with a more detail example, for example, when the page is scanned, how many pages are accessed, how long between accesses, etc. -- Best Regards, Huang, Ying