From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47976C54FB3 for ; Fri, 30 May 2025 02:11:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BAEB06B0082; Thu, 29 May 2025 22:11:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B5E716B0083; Thu, 29 May 2025 22:11:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A50326B0085; Thu, 29 May 2025 22:11:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 82E0A6B0082 for ; Thu, 29 May 2025 22:11:18 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id E53631D5BFB for ; Fri, 30 May 2025 02:11:17 +0000 (UTC) X-FDA: 83497946994.30.D6A8052 Received: from mail-ej1-f42.google.com (mail-ej1-f42.google.com [209.85.218.42]) by imf12.hostedemail.com (Postfix) with ESMTP id CC27740005 for ; Fri, 30 May 2025 02:11:15 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="b16EW/Q9"; spf=pass (imf12.hostedemail.com: domain of richard.weiyang@gmail.com designates 209.85.218.42 as permitted sender) smtp.mailfrom=richard.weiyang@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748571075; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Me0xm+hLto0th3OO9ynF20JyO/JQ1SmQxt+QOF95X2o=; b=baiIxYTOsW1Ob+ENe1mfq1yhL++hLy98NneI4a8ee2cLpJgSLWA8SNwW3dXc0XelZmaNz0 P6qvJooteCPNNJQtuHFMG+VhOzJjLzXKdBDUvbdHLBCB72cnfQYCTX0RiSGrum9As7UTOY Y8fYq0aDSeNxB2pXILN7RXGRNo2EsqE= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="b16EW/Q9"; spf=pass (imf12.hostedemail.com: domain of richard.weiyang@gmail.com designates 209.85.218.42 as permitted sender) smtp.mailfrom=richard.weiyang@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748571075; a=rsa-sha256; cv=none; b=PPGr+/ZAYwnc/YH0R0uyWCsT4j/VkSUVjnNLOZZzO0QPeLLFtXCFtAFeVDvBdBdVgwRw/r nqCdNCiZ0Jx5Hnz04qtjb2Eyg62YGpQP0IIlFp54WEXPjADvyrWmhbN4/KPiGkJhGhUoi6 RTC8t4il0cN1kOls/+EzowOj9iG0yC4= Received: by mail-ej1-f42.google.com with SMTP id a640c23a62f3a-ac3eb3fdd2eso281244766b.0 for ; Thu, 29 May 2025 19:11:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1748571074; x=1749175874; darn=kvack.org; h=user-agent:in-reply-to:content-disposition:mime-version:references :reply-to:message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=Me0xm+hLto0th3OO9ynF20JyO/JQ1SmQxt+QOF95X2o=; b=b16EW/Q99EK5aq6I9okGX9yoYU9PBGuYJ8ahB8IsXGbLxxPs/QeMG1mWC2BGm3Zbls kD7xtgz6Zb9a/1Rmt4WtiS+Y+8nW7nUmZ52o41vkT5jGUAj/wIEGcwc5KGnhmnwkbWCh wVDn9Pkhb6YdRzwK4LguzwwQGfOWOMLOvDvoJCZIOphd4iQb+DMwj/EfRMsiuOSrT8IL 1x3rOHiagcC1uXqOGtGvJFDfhT9VA0XrfKI9V0l4fBOw5lD0VZ/wPUE9kZUvcqxl0GqU sXZeg/AqIKo9k9UscNjaT1sGGZuxGOs4dERCBQzH1gxRnRoLSCr9q/BuBRDDz+dMwWU4 NO4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748571074; x=1749175874; h=user-agent:in-reply-to:content-disposition:mime-version:references :reply-to:message-id:subject:cc:to:from:date:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Me0xm+hLto0th3OO9ynF20JyO/JQ1SmQxt+QOF95X2o=; b=ZKCxBMjIlFkgjtdoLgoASGyfplwpFJRiQzfRCtNsgkxNaBr0r42CtBpqjArGMww+Mz mfDD7C57U8xjuB/GQBmInjRil38sIWC1JGxplfOVqvj1eEexOvHU679HM8zEFV/5+Cpt tv3gc7z/sc0/rfa3HrlOlPw1qEZffFM2Y0c9W7QyTvP2rewpH7JU111uqWDiNID46VHb x8NUigypfLWMuJQBh6qd8OUhqLP3kih3F9V0AaFlA/aaONiAsE4EUEmrTrLojo6zbxGE VUHQIYvLnWjBSJhZpgDJPyBMz3QsIwVT4iSQGuk1a+s4EqUn6S+rYfZ/TbhcOu66W3wd gBbg== X-Forwarded-Encrypted: i=1; AJvYcCUGYXgFjHcbShwK5CfyRLxHKLTDw2a/RshbWkPdRsdIU65cbfRuR/xFn4O7K5uUplGyA8xCu5YN+A==@kvack.org X-Gm-Message-State: AOJu0YzB7dCBvg+yQxYnw+ivU7HQyUeifYxXd9UaiIMihdP0bjx2Zikm WOL7EKvWaof+yufY12/aSe0Br/xjBNGm2Hc/VXzINEpd4CHMsegohv47 X-Gm-Gg: ASbGncv5RpZCSmcVWuexNWtUvj5nr4ivUQe7m/0rl6pJ3XYL3HmefaTBh/TTsOpRh+T uS+EilDmYrpsBEUX9jz062jv5y/3HihW5JzpJKvxl3VGM8UnR/8jlIydoVFDL+0bhTbutpVbT2m ety4gIR1ckjQ2fY09UySNO0j+VpvbxwO6MocGSJUfeCxyFiuiNxJVvkfb1hEvYfifx/dvevTz5h leK/YaensGIn5yI1hpiyr5ItqiN9/petEIrAwUBdweBPdhPTznRyYTWdEDvYQpK+aV7Nl7qjT3V CXJheak2eJWji9sxLMwvw9EiJYg+kd2Gwyj2gRCGrFmx0O06QIryekt6ud6LsvwPg+OV/7LZ X-Google-Smtp-Source: AGHT+IEWCkQp8Ukfw0PcupRIQEAHMLnXKt38IjxVUFok8MgG2lHc3PWUZg/qKCYfXtV2UltK4dfqMA== X-Received: by 2002:a17:907:97c1:b0:ad5:468d:6ecc with SMTP id a640c23a62f3a-adb32453fcdmr112865266b.59.1748571073945; Thu, 29 May 2025 19:11:13 -0700 (PDT) Received: from localhost ([185.92.221.13]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ada5dd04551sm239512266b.93.2025.05.29.19.11.13 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 29 May 2025 19:11:13 -0700 (PDT) Date: Fri, 30 May 2025 02:11:12 +0000 From: Wei Yang To: David Hildenbrand Cc: Wei Yang , Lorenzo Stoakes , akpm@linux-foundation.org, riel@surriel.com, vbabka@suse.cz, harry.yoo@oracle.com, jannh@google.com, baohua@kernel.org, linux-mm@kvack.org Subject: Re: [RFC Patch 0/5] Make anon_vma operations testable Message-ID: <20250530021112.6b2ldosl3hvhcnvq@master> Reply-To: Wei Yang References: <20250429090639.784-1-richard.weiyang@gmail.com> <8c268ece-71a7-470a-bf8b-71d3e4920977@redhat.com> <20250429235608.yvbcxybkfsmp6ow5@master> <20250514012318.2impqotf65p4ihsc@master> <20250527063428.4ykrdtive2hfnqdp@master> <40dc48f9-94d7-46b0-83ed-d3511789c15a@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <40dc48f9-94d7-46b0-83ed-d3511789c15a@redhat.com> User-Agent: NeoMutt/20170113 (1.7.2) X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: CC27740005 X-Stat-Signature: ao58a4z45i9n1q131sfqyuehb9tzg5yg X-Rspam-User: X-HE-Tag: 1748571075-232943 X-HE-Meta: U2FsdGVkX1/cBYJCj270oZJ2zqi8Ew2llP8iZBX2onJz8NdzSpyzR5J393TGegOVPVFmtO+9QE7bYS30Kd7sPruj7p8fnGizKBi+ejr97aru/t2ybSn7XUZOMUMRA44T98++Zy6EorC/iLGYZoLSnH7P7d/Dz487mw8jC4rahiIWtr+B16van2Ra5uvMdndEVFSItTcoEEQ0QTgZ2xMfMh6tgVXMd80G6Ei4sO0gsyS85DnGq0Rz3HCqvu3umc90tRIGscy3BEsUiEKW8K9ziieUyG4kH+icBurZPj3q5ie7nD3qRkeVSlXZRcSM98rwsFC++SLDG1t29HRoky9EwHIMiADjYjSLBsWPHGqeXnL23n6ye0sPZ3sil2jqVO/ZbKdxfsilN85570SOOVgQn36mH3GqD05ih8o7MnIQGigzEzgwOHK5bRFJT/fJ9m8I+nx4ELbvHnCiKBniGdd9vPyLF5FQFzqMOBkUbzrq0ze02S70sv9oYKe7zUyy5sS+vl0a9O7n77m5vlyW5lC0ex7cSbwf5BrLWsxORu0E9w0d5apcwYvRUOZfQSKjyYTNKUKeTGGgW1MwWr3J2goEKaju5H/6aJZ+RX1S095//8MM29uhLZTEvhF/HQu2FCJ9+8bWAOmBl+IASo12nr8cWBgfvALJTfmmvvFElDd5NRqHsJSrFVtPdcjSaNLpieBuc5MpNXbGh5jMQOmjZploO/vAcSXjKipMlY4VZhecLS0cK49jzpIuFPyaNy9bdPIt7TZO4H+pbEjULYaRo98MavZQPo7OmKmu9cosyMDIrYiioDOu/scqb3RAnmAsDIkSuAiR1oXl+BPSeIsZTut/LDnCeHYIYoeHBbYK4nLzPXDl6XBkoW4lZ0uV2LLCLtiVQgpuBdBSJwjOPXpZMpTsy2P4ygXhMLGGtR+orqMYDypRLGd1y9ayN1lgcyXnn4gzYrVaM88d6qgtA6b1PHR njsXrI2K g10gFJFDGvQrULFbq+0H1fqRWmkX/D1O1Gmjl2nJVmNWaTTmvj05/s0Q5C9z8zI10Wz2Q0bM8Am1tm7RfI+q1y2plU7YGQ3XeLeoyIy3jpLSrN/+UW9f9ta/rZzWkne4Aro+rex0qzqKijvLMtkFI+y6PCz/4SH2dmZW3YqwVYGHMSAFRFOWsfw25Gqhx6QgxRdGJTSB7XbrAfcWz/+GBxtuu+UzozzWveMkVNLzuvI99Qic2SbiOC8tohhiFJ0og9X6Lpgp267qRL236hKnc2lrBBb/c4N1342PXJYDBTCsV9Yt19Cyc8L8xjt+yO+V5bEPhCX1wYX/yG8SrfqX6zZ+I6odKAVv0PRYqg1BEbv9Cp6LfDWZ6DF7ZN7xJ01EipVDCMksho+oIBTPq8ARut4xS3SpZvwhEV/VHfeglOnx31O/vzx4kv5UU0mVg3e5SIwf5Lfjp5YEcO+FaCbl+4cQwHx0XduNQqF8XLgqnwtsttO/yuSVXkkE+slZpyEdf2XPkzqTf0R/w8O+82b2Diw+Tn2K9W5jCVSirM3aFrdm5Xwm7m0FbKpf4VxMKCTemMLWM5G39mVeAqZLZiu84O2lmDiycMLDXOiY9gz0dahOMyR1VbAV9HSkEH2KUroaQQCGCg8fgyV7Ikb/v9dYjzeTSIXLC3nFm/25bozXNrkhwYRUNuu3sKydXz0QTHbGBw5cLQ0bFec5qGAXZuFRiKuJ6fA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, May 27, 2025 at 01:31:47PM +0200, David Hildenbrand wrote: >On 27.05.25 08:34, Wei Yang wrote: >> On Wed, May 14, 2025 at 01:23:18AM +0000, Wei Yang wrote: >> > On Wed, Apr 30, 2025 at 09:47:16AM +0200, David Hildenbrand wrote: >> > [...] >> > > > > > Agreed, skimming over the tests there are some nice diagrams and cases. >> > > > > > >> > > > > > But I would hope that for most of these cases we could test on a higher >> > > > > > level: test our expectations when running real programs that we want to >> > > > > > check, especially when performing internal changes on how we handle anon >> > > > > > memory + rmap. >> > > > > > >> > > > > > E.g., do fork(), then test if we can successfully perform rmap >> > > > > > lookups/updates (e.g., migrate folio to a different numa node etc). >> > > > > > >> > > > > >> > > > > That's a great point! Wei - if you could look at making some self-tests >> > > > > (i.e. that live in tools/testing/selftests/mm) that try to recreate _real_ >> > > > > scenarios that use the rmap like this and assert correct behaviour there, >> > > > > that could be a positive way of moving forward with this. >> > > > > >> > > > >> >> Ping > >Thanks for reminding me and sorry for the late reply. > >> >> > > > I am trying to understand what scenarios you want. >> > > >> > >> > Sorry for the late reply, I handled other things a while. >> > >> > > That is exactly the task to figure out: how can we actually test our rmap >> > > implementation from a higher level. The example regarding fork and migration >> > > is possibly a low-hanging fruit. >> > >> > If my understanding is correct, you suggested two high level way: >> > >> > 1. fork + migrate (move_pages) > >Yes, that should be one way of testing it. We could even get multiple child >processes into place. > >> > >> > > >> > > We might already have the functionality to achieve it, *maybe* we'd even want >> > > some extensions to make it all even easier to test. >> > > >> > > For example, MADV_PAGEOUT is refused on folios that are mapped into multiple >> > > processes. Maybe we'd want the option to *still* page it out, just like >> > > MPOL_MF_MOVE_ALL allows with CAP_SYS_NICE to *still* migrate a folio that is >> > > mapped into multiple processes. >> > > >> > >> > 2. madvise(MADV_PAGEOUT) >> > >> > Not fully get it here. You mean fork + madvise(MADV_PAGEOUT) + migrate ? > >fork + madvise(MADV_PAGEOUT) only. > >Then verify, that it was actually paged out. > >For example, we could have 10 processes mapping the page, then call >madvise(MADV_PAGEOUT) from one process. We can verify in all processes if the >page is actually no longer mapped using /proc/self/pagemap > Hi, David I'd like to clarify some detail here. >We could likely do it with anon pages (swap), shmem pages (swap) and >pagecache pages (file backed). > I could understand the three cases here, but for the (swap) case, it is not clear to me. You mean we force the page to be swapped out before migration? I may not have an idea to ensure that. >... and possibly even with KSM pages! > For KSM, it looks like an general background routine. I guess you want something like: * fork process tree * do KSM and wait for it to finish? * migrate I guess the KSM duration would related to the total memory on the system. The bigger, the longer. While the selftest generally have timeout setting. Not sure it would be suitable for all situation. Would you mind sharing more on KSM case? -- Wei Yang Help you, Help me