From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14D61C2BA1A for ; Fri, 21 Jun 2024 09:24:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A10B36B031F; Fri, 21 Jun 2024 05:24:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9C0C06B0322; Fri, 21 Jun 2024 05:24:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 888316B0323; Fri, 21 Jun 2024 05:24:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6822E6B031F for ; Fri, 21 Jun 2024 05:24:48 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 0435340C7F for ; Fri, 21 Jun 2024 09:24:47 +0000 (UTC) X-FDA: 82254361056.04.1669551 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) by imf11.hostedemail.com (Postfix) with ESMTP id D136540017 for ; Fri, 21 Jun 2024 09:24:44 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=F9JDLzyK; spf=pass (imf11.hostedemail.com: domain of ying.huang@intel.com designates 198.175.65.19 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718961874; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eNVwOgRuUFz5kvrUgQsZ3azTqlpVwoDDhoWGXhljdHA=; b=ZWz6bQ1OXUPlw+8C+5FpkM6SO16P3VWXHCSpvbTvYeUsyVeBCob4UfN75Jg8HiKVBDyTC3 Eyx7lRTf6wHW3AqpR3OW2bY2m5MsEXUiCetQYhGU1gvgosWNN1sZ4J/UERMBMZTqKhlObh KSQMnZObR422P0c7q+FhaIHJE5rtwxI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718961874; a=rsa-sha256; cv=none; b=FWKEvXzirVqHZuSL8SWGAND3ugPajFbFHUqSetqPYXa0gZooyN8gD5FRH1XIEM1/vTDPn+ hhmS1SUNrz0axK9sNxdEP4vvTgyqcdwwvdi742S9MLUZn9sT4aVP7L22CwFxNsL9XZNu7V wdKpQeObZErVaFdrTJFWNS/QQwfcz18= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=F9JDLzyK; spf=pass (imf11.hostedemail.com: domain of ying.huang@intel.com designates 198.175.65.19 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1718961885; x=1750497885; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version:content-transfer-encoding; bh=NBWTcpKZIKcW6NL5NChdd9KplU+HhaBpcnqHOd3GkH4=; b=F9JDLzyKcCw+0b7y+VWy42ZiwJIJFtda02LXLRSDGeam7Defj8z3HGd5 ZS3H+w02GeB503vBQP7R0x3mWPSg59jVfgRHYSVBF8wMAvSEq6ocwo/dn AP99hnU+ZBfdDd9afq9Bv9tM0xNlOjETmROd6elRKvfr2jn7lQ8L/G42E 6S3H89K0rjqaf5SJwsxYeBjfHTw+7tQ5mBD3hAsgAkKxwqDdSzwhthbUK 8oHnwcq9cXPq+wV20BfxP9btGYZBxtjPgQxQTAUwcUZtvogJ+aE37p9xg 6s3SJyokuLeqE7zDWrYAWBx7zys3IXImMXJ99icUfybKZySYNPS/KSjzL A==; X-CSE-ConnectionGUID: YrJkrjFVSaClafk4CfQKzw== X-CSE-MsgGUID: yGM9mdeiQq6HTrzfZXBwqQ== X-IronPort-AV: E=McAfee;i="6700,10204,11109"; a="15821221" X-IronPort-AV: E=Sophos;i="6.08,254,1712646000"; d="scan'208";a="15821221" Received: from orviesa007.jf.intel.com ([10.64.159.147]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jun 2024 02:24:43 -0700 X-CSE-ConnectionGUID: 22g+VwFiQBWSlHtaAoNqOg== X-CSE-MsgGUID: f/88ox6tSbeoFtvnlyIrMA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,254,1712646000"; d="scan'208";a="43203306" Received: from unknown (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orviesa007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jun 2024 02:24:40 -0700 From: "Huang, Ying" To: Barry Song <21cnbao@gmail.com> Cc: Ryan Roberts , David Hildenbrand , akpm@linux-foundation.org, shuah@kernel.org, linux-mm@kvack.org, chrisl@kernel.org, hughd@google.com, kaleshsingh@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Barry Song Subject: Re: [PATCH] selftests/mm: Introduce a test program to assess swap entry allocation for thp_swapout In-Reply-To: (Barry Song's message of "Fri, 21 Jun 2024 19:47:02 +1200") References: <20240620002648.75204-1-21cnbao@gmail.com> Date: Fri, 21 Jun 2024 17:22:49 +0800 Message-ID: <87cyoa1wgm.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Stat-Signature: hjdst4ide9z5s1sxq555ohermoaj7bzh X-Rspamd-Queue-Id: D136540017 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1718961884-193595 X-HE-Meta: U2FsdGVkX19Mzi2jlWnzyftAKw5w6iaMSz0azgs8cWsH66ctYfdAonOlusu0+4shT9qn5JO90lyY6KAXFf8M64JnJWnaK6uGKZhR0R/dIXcO9FmVu9J8e0ra9ufab0UOoOyVFPwPDOVOoiMk4eF9LMmosTwgmIzoI1eknl777Ostm2L7bkzmvzuVFAKr6ImtNTNnYyi7MbWvM7o+pDJQsxtsOsz7neL3bbDoBC6YxQQKT357ICgGitTUyubzTmC8MsvjIQGPc8vdoK6T4X+k4gmSfDddOrH0wD2S2rgJJ3/ChtrW2wbmd2XSZquFZBBpV/raYdCHTULbOvfQQebRKOecZdtyE6KDBms1VXI4IFaUFi+Q4rhkxAh7umDN/vcE6h31hL8ofh4OolWWzpDR2jzboAe5Tty+vIhVE/jUFtN+VOJhurnwhZi4c91ecZLVaBfoI/b9jxGQ+pZjhP4Hzn6VUYwQ7pUyv1ZLEz23y+F9zhgbbnZlhOfcsNGxLYSsfsqYdGf01hDqsMVWq2p7K1ByMSiZxXAii3CmScb1/942/jVDhGc3unvz2eHmW63Cd1UvEVWUxIhZV6NiiQNVz0v2xTroIEueBUl/U4pt/yassryZijWTtLLJ0tlgcy+rneEOuXwDmQMwCbu2/RWK/Tpxp5uyV3oRF6DKNwr3OXzfILh0lzN5a9+60iHRAikYYJyDeCjnZioBopxGjTiYs8BBGizUIQ9SMT5OKSpVgRPfCYXIuFWp6c+wnMIdfo+ODTCMgDZQKC+lyFL9d0Ye8j/fMXpgeKg00zc5UMzda0QreKgUL64UPRcHUiW7ZWq3tSH4PVGxSixb88F3i6s9P+lVYQU26Drgf8JpF758CyyoVKX3PdyzgZpztU4pOB8lxlDQZVqUEca3eVwD/N1AIy0ltzV8daT10VSbiWpUTtGIUe7S1QB4J7hJzZCGmf9OLE7O16hHHUlpT8ll3dY GZhnEG/D NSJqyR8YX0sjJJm4qEETkc8numgIzguNXP2/C+/DFfx6D25J/Q5IyzVSWVI8DNRxbwTbMiqSsPGa85kpMGxJdJsEET4DUlldWFQrNLloMlFBeQBhibqntFVi3ZHTO0+Id2Wb4ji4G6BDCWs31Ym3sx8HXQs3M7bpSXKUOizfxYbdAEU2OibLvTsuxisrasmZR1F7gBGIF8ncivIpEwcLFFqxsP9vMfnfOgg0imUap6iI9rM74M6yhmRVY9913PW63d+jhI40vujxP23z4/sBh73zyiCNVVhTRg/YX3LoCB68AodqDQ6TDXAisGBYQReX3GTe3Zv+xTlYeKZ0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Barry Song <21cnbao@gmail.com> writes: > On Fri, Jun 21, 2024 at 7:25=E2=80=AFPM Ryan Roberts wrote: >> >> On 20/06/2024 12:34, David Hildenbrand wrote: >> > On 20.06.24 11:04, Ryan Roberts wrote: >> >> On 20/06/2024 01:26, Barry Song wrote: >> >>> From: Barry Song >> >>> >> >>> Both Ryan and Chris have been utilizing the small test program to aid >> >>> in debugging and identifying issues with swap entry allocation. While >> >>> a real or intricate workload might be more suitable for assessing the >> >>> correctness and effectiveness of the swap allocation policy, a small >> >>> test program presents a simpler means of understanding the problem a= nd >> >>> initially verifying the improvements being made. >> >>> >> >>> Let's endeavor to integrate it into the self-test suite. Although it >> >>> presently only accommodates 64KB and 4KB, I'm optimistic that we can >> >>> expand its capabilities to support multiple sizes and simulate more >> >>> complex systems in the future as required. >> >> >> >> I'll try to summarize the thread with Huang Ying by suggesting this t= est program >> >> is "neccessary but not sufficient" to exhaustively test the mTHP swap= -out path. >> >> I've certainly found it useful and think it would be a valuable addit= ion to the >> >> tree. >> >> >> >> That said, I'm not convinced it is a selftest; IMO a selftest should = provide a >> >> clear pass/fail result against some criteria and must be able to be r= un >> >> automatically by (e.g.) a CI system. >> > >> > Likely we should then consider moving other such performance-related t= hingies >> > out of the selftests? >> >> Yes, that would get my vote. But of the 4 tests you mentioned that use >> clock_gettime(), it looks like transhuge-stress is the only one that doe= sn't >> have a pass/fail result, so is probably the only candidate for moving. >> >> The others either use the times as a timeout and determines failure if t= he >> action didn't occur within the timeout (e.g. ksm_tests.c) or use it to a= dd some >> supplemental performance information to an otherwise functionality-orien= ted test. > > Thank you very much, Ryan. I think you've found a better home for this > tool . I will > send v2, relocating it to tools/mm and adding a function to swap in > either the whole > mTHPs or a portion of mTHPs by "-a"(aligned swapin). > > So basically, we will have > > 1. Use MADV_PAGEPUT for rapid swap-out, putting the swap allocation code = under > high exercise in a short time. > > 2. Use MADV_DONTNEED to simulate the behavior of libc and Java heap in fr= eeing > memory, as well as for munmap, app exits, or OOM killer scenarios. This e= nsures > new mTHP is always generated, released or swapped out, similar to the beh= avior > on a PC or Android phone where many applications are frequently started a= nd > terminated. MADV_DONTNEED 64KB memory, then memset() it, this just simulates the large folio swap-in exactly, which hasn't been merged by upstream. I don't think that it's a good idea to make such kind of trick. > 3. Swap in with or without the "-a" option to observe how fragments > due to swap-in > and the incoming swap-in of large folios will impact swap-out fallback. It's good to create fragmentation with swap-in. Which is more practical and future-proof. And, I believe that we can reduce large folio swap-out fallback rate without the large folio swap-in trick. > And many thanks to Chris for the suggestion on improving it within > selftest, though I > prefer to place it in tools/mm. -- Best Regards, Huang, Ying