From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00ED7C4332F for ; Wed, 1 Nov 2023 09:21:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3F0D98D0036; Wed, 1 Nov 2023 05:21:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3A2778D0001; Wed, 1 Nov 2023 05:21:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 269108D0036; Wed, 1 Nov 2023 05:21:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 17DD88D0001 for ; Wed, 1 Nov 2023 05:21:10 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C38A7A0BF7 for ; Wed, 1 Nov 2023 09:21:09 +0000 (UTC) X-FDA: 81408841458.30.3BE8833 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf24.hostedemail.com (Postfix) with ESMTP id A9144180037 for ; Wed, 1 Nov 2023 09:21:06 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=kqn6sEW4; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=8gcDjwF4; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf24.hostedemail.com: domain of mgorman@suse.de designates 195.135.220.28 as permitted sender) smtp.mailfrom=mgorman@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698830467; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aEA5Bn5qqMBKnK83id7ZAri0LcBHPYglLttHP/O2Csg=; b=IFHBI7Vh9HEUKvQ3Sj9AFues1ERfGIBOn+yg11sfpE2jsGYwbvh9WSJAAgVF+uH4ZaGceV f57YfG0dEem1OQoNtUA6kUW/jCb4YBy9zeALmWsPNBJWn2MqX9CkXO4WRnR/3OKhwSlyXB inawgjIFW7Phn8nFwdmm83idMVfz1BU= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=kqn6sEW4; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=8gcDjwF4; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf24.hostedemail.com: domain of mgorman@suse.de designates 195.135.220.28 as permitted sender) smtp.mailfrom=mgorman@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698830467; a=rsa-sha256; cv=none; b=oNPBHYDVuvjch/OrhiGf1k0IrwNEjhe2Vmis43dobi+UZKfCtxshIoGp8jI8a5p6zL48i9 nOLn2XWKmeEP5HkRsZBaMYZpeSLS/77fRuFZJIQwvoNbQp0/iwH3VGXnszoczLiBm+FHq0 ERtmfXWwHrUxuvoP/b726gHvqb1WHp4= Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 75CDF219E1; Wed, 1 Nov 2023 09:21:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1698830464; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=aEA5Bn5qqMBKnK83id7ZAri0LcBHPYglLttHP/O2Csg=; b=kqn6sEW4MNl8xclDYz/UTR5PJHXE9m3QBCb8fczzYVJ16j9nBIx+r/VLSB45JXd63PfJr4 EcGPxU6p2VbXezcLa5NRcyFjPKoF4/ZEv1WSHp34b8Nnpqgpl/GACfPL36j8scXm31mUuo Ngq7wBsUURdXwjJxYdXvJu6NwRn1cds= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1698830464; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=aEA5Bn5qqMBKnK83id7ZAri0LcBHPYglLttHP/O2Csg=; b=8gcDjwF4nX6QOJUOk75hIrivSgHdImjRRdFrxwoa0yA8sPIseb283OZhQ8Y6lDiLx34vrC c+S0hDyQ46n/WmBg== Received: from suse.de (mgorman.tcp.ovpn2.nue.suse.de [10.163.32.246]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 4A02F2C65A; Wed, 1 Nov 2023 09:21:03 +0000 (UTC) Date: Wed, 1 Nov 2023 09:21:01 +0000 From: Mel Gorman To: Raghavendra K T Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Ingo Molnar , Peter Zijlstra , Andrew Morton , David Hildenbrand , rppt@kernel.org, Juri Lelli , Vincent Guittot , Bharata B Rao , Aithal Srikanth , kernel test robot , Sapkal Swapnil , K Prateek Nayak Subject: Re: [PATCH V1 1/1] sched/numa: Fix mm numa_scan_seq based unconditional scan Message-ID: <20231101092101.i562axvfqjdvw2rx@suse.de> References: <2ea7cbce80ac7c62e90cbfb9653a7972f902439f.1697816692.git.raghavendra.kt@amd.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline In-Reply-To: <2ea7cbce80ac7c62e90cbfb9653a7972f902439f.1697816692.git.raghavendra.kt@amd.com> X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: A9144180037 X-Stat-Signature: qbx5r4smeut3iktooz7qez93qnkj96t1 X-Rspam-User: X-HE-Tag: 1698830466-451689 X-HE-Meta: U2FsdGVkX18WlDvCDoAOCiS/38vakyUsZwTozGYkmU7Yq4mdrbY76Sj4twYRZk1WwaBXsXq0fXvlTHomDQRzP+m86IVYG4LXNIok8nL9bLCtk1rUNhgktqxkB+GoMQq+mcj3A0ZZI92MjApG4tNK0r2AMKnNGl8EqhCgayjU3ffnMe981UWvTI5HbjOddILQPP5h13kazsQA9qd8dL17lRqqypRBT4AEYnM9K22sN9RuL++ks8yWNnsIVRzKiU8CHwC0dLI8zTWJbIvw1iPIHyrLppoM3wwksIj1Gu0ug+AFeyWvrkWWjQuuBmikTDGmTkpB4hdZlvY+E3ySMm0taiQrKUtit1uTtPOtzqmAJW5BII66mWmiw68MeXwTYOakri4H/z3G3WbG4snfYMTRafhGxfe1gP8OWGSekTwgG6gbnVuyYD0GAu1T/+QT4xX8xQtX0kfQ4vf2bNy7u7gn22Hv0/0fYBcwsfF4b0PM7vmdLp4ybUXMZI/AKPYtIIwauKdxA4UkOI0MPx0EuJh2ojo4CHmuVgbca2KflbhrUNKWuoBN+qlIAj/1JYJ23fx7P13V0Zvb5a85MKx4NWUOPsuSTCp8MRt85u4McVIMBWdfDMh8gfn+PM4RAad6i6fTI6hLkXo5Qp0Co2RDEWMZabwiPMrbI7Od485r51nbPg08/FaLi39SzdWRZAiP0F7g6PZGH89k20k1MsyqEU1iV34/9VdEVN22ptm9bxn3Oye+ToZp4F8yocDVSUmyBhNV93v9dDCGdzo+wLzESWGxvSyW9qDlmqHi4teO7//+q3Drge1edKkqnKwLy75VOctKVmgSj4xID+Ii8LYoFpjVoogZeMG3jeemuzqUb5ho6koVieNPwY3k3POkfn/HvLFLD3ZODQZ/yT5jqy2OIsxMK98bC84GzE+m/ewTcylw6ktnTHMDEdEP5FmzoHqFfTPUAGVBh17bsqQZsOGjM82 +sLcnwJh aVIECORtk4hjLry1t/NMJB5Wy3D85uorAGp3jkMf8bMX0MJQhAkxIQQloXLnUrbTkp7zvdRmjwU3U+egcjCWYopnp/KRhdmU+ZwypU+fnuKjUETIy54w4O7oPp6827r4LJWx43bdeMfnYOF+jEchyuMFycjgoiC8LJftOoH9EmSNHmfi+1Vp7+DUeaBoOKEOKhApqT6M4oynS/OmvUqjoxktbK/sKFQ2qlLtmuPK1idjRSvZf0BBahzzSbaN9GA/Reyp2bUKRyX1N+yw21c9M9ndCHLSb+SfQo0ArjAa4VTx7TWl3cFWvCzu708bdxu8ow1R40EimpGQu4lQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Oct 20, 2023 at 09:27:46PM +0530, Raghavendra K T wrote: > Since commit fc137c0ddab2 ("sched/numa: enhance vma scanning logic") > > NUMA Balancing allows updating PTEs to trap NUMA hinting faults if the > task had previously accessed VMA. However unconditional scan of VMAs are > allowed during initial phase of VMA creation until process's > mm numa_scan_seq reaches 2 even though current task had not accessed VMA. > > Rationale: > - Without initial scan subsequent PTE update may never happen. > - Give fair opportunity to all the VMAs to be scanned and subsequently > understand the access pattern of all the VMAs. > > But it has a corner case where, if a VMA is created after some time, > process's mm numa_scan_seq could be already greater than 2. > > For e.g., values of mm numa_scan_seq when VMAs are created by running > mmtest autonuma benchmark briefly looks like: > start_seq=0 : 459 > start_seq=2 : 138 > start_seq=3 : 144 > start_seq=4 : 8 > start_seq=8 : 1 > start_seq=9 : 1 > This results in no unconditional PTE updates for those VMAs created after > some time. > > Fix: > - Note down the initial value of mm numa_scan_seq in per VMA start_seq. > - Allow unconditional scan till start_seq + 2. > > Result: > SUT: AMD EPYC Milan with 2 NUMA nodes 256 cpus. > base kernel: upstream 6.6-rc6 with Mels patches [1] applied. > > kernbench > ========== base patched %gain > Amean elsp-128 165.09 ( 0.00%) 164.78 * 0.19%* > > Duration User 41404.28 41375.08 > Duration System 9862.22 9768.48 > Duration Elapsed 519.87 518.72 > > Ops NUMA PTE updates 1041416.00 831536.00 > Ops NUMA hint faults 263296.00 220966.00 > Ops NUMA pages migrated 258021.00 212769.00 > Ops AutoNUMA cost 1328.67 1114.69 > > autonumabench > > NUMA01_THREADLOCAL > ================== > Amean elsp-NUMA01_THREADLOCAL 81.79 (0.00%) 67.74 * 17.18%* > > Duration User 54832.73 47379.67 > Duration System 75.00 185.75 > Duration Elapsed 576.72 476.09 > > Ops NUMA PTE updates 394429.00 11121044.00 > Ops NUMA hint faults 1001.00 8906404.00 > Ops NUMA pages migrated 288.00 2998694.00 > Ops AutoNUMA cost 7.77 44666.84 > > Link: https://lore.kernel.org/linux-mm/ZSXF3AFZgIld1meX@gmail.com/T/ > > Signed-off-by: Raghavendra K T Acked-by: Mel Gorman -- Mel Gorman SUSE Labs