From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D84DC54ED1 for ; Wed, 28 May 2025 01:17:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 835DE6B007B; Tue, 27 May 2025 21:17:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7BF146B0082; Tue, 27 May 2025 21:17:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6870E6B0083; Tue, 27 May 2025 21:17:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 497AD6B007B for ; Tue, 27 May 2025 21:17:24 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 9C5DDEB20A for ; Wed, 28 May 2025 01:17:23 +0000 (UTC) X-FDA: 83490553566.09.2C39BDA Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) by imf06.hostedemail.com (Postfix) with ESMTP id 9164E18000A for ; Wed, 28 May 2025 01:17:21 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="BsNRPS/W"; spf=pass (imf06.hostedemail.com: domain of richard.weiyang@gmail.com designates 209.85.218.53 as permitted sender) smtp.mailfrom=richard.weiyang@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748395041; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=L4k0fiUiSTl6PhuQUqqUEDpjRi4wF/AmcnyyGHL3N3M=; b=oocpEe6jUrJ7OO4u9d37SADLNr+ShTgveEG+x2rvvzIDaGveAGwYPlQTtyZpzVd+WfU38J 4dfP56Vo8CdjAwpLFz56V+2LNpZE0vznhNDfDhf+cftNsJFK14vogF+Kbzdz7Uy9byO8Mt gxkl9K/hFgdilLt4doDrhu3xrk0JHNU= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="BsNRPS/W"; spf=pass (imf06.hostedemail.com: domain of richard.weiyang@gmail.com designates 209.85.218.53 as permitted sender) smtp.mailfrom=richard.weiyang@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748395041; a=rsa-sha256; cv=none; b=a4KO2x8haM5POH9o3roYs0TbHKWM5hun3DdztJA8ah54lPwdB/dGOtuab/bKCHGmavbP0N k31JLM2qs26y5WTK98rGUG+qbrbt7ZyFp6BGaEbCSQ4k01uVvXigYeawUuj6PuhOy6FrUc eDP+MEhB4fzkhrKd9xZS83fz4as43Bo= Received: by mail-ej1-f53.google.com with SMTP id a640c23a62f3a-ad88d77314bso270184166b.1 for ; Tue, 27 May 2025 18:17:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1748395040; x=1748999840; darn=kvack.org; h=user-agent:in-reply-to:content-disposition:mime-version:references :reply-to:message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=L4k0fiUiSTl6PhuQUqqUEDpjRi4wF/AmcnyyGHL3N3M=; b=BsNRPS/Wko2O/WNcffS34i0viHTaNAZ3VyRc8XaS4qyepQWyc6qejjM4jRXTY2gqiV jDATr1OImX3ieGxCL0yOzjSzHvDEWV2JwBRydCcpuEV+ie1sDV7GYRWQlR2Li/dNh6MP ZI/ortfRnmLch5UZyAgknssRC4EfCvLlaImurv3LrMEzPhYoPr/Z5xVDv8VdNU/kQJlV 51Ua3TdvW7+O4zLpVjYNf5k/3e9YcRV2gXom5q40ciT1fMuzbPKv5LgN/4BKFBKHnMbk 3vw2Xakk2WJ0AzOjQArizZP6gItvq+hJQnQ3odeIf92+xJ53stWfqICRB1jcAVXNEVug hakQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748395040; x=1748999840; h=user-agent:in-reply-to:content-disposition:mime-version:references :reply-to:message-id:subject:cc:to:from:date:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=L4k0fiUiSTl6PhuQUqqUEDpjRi4wF/AmcnyyGHL3N3M=; b=HQ1PXo2wXUyH1Y2Rt8QkGl8Oyz4QhXyn50pLbOtYfRlqx7VgJk0hsMjLuyxJVtHRbL KbQHDphDQwuG3G1tmlh8JFV33X4a922F8mz7XuokMVFZyZzfjbX3jMpMl7N/th/RpBeO RiBYgEbK8Tw1mPCgpfaZLU5xJbcYSkxgJU2DFb/cMzE3Ju3PJT9QlaxcqRhXh0Hy5/cY DvkKM+HzKQR6eJUV/ulVT++ual+yaACco30+jUJZmYObLnlNBr9upQ9KEXTQOL6Vaj8r UBHCEIpfgA3Z/7pWaEV0Q+1Eo7vWKoZ7tu5coaVTRGxdFLlBkcKkwWLDDqgTjhh2pt3+ 7bUw== X-Forwarded-Encrypted: i=1; AJvYcCWVi8qRW2P/x2gvyk9t3ssDP+lWED/sWcMHX85Wn/CDBgmRw3p6G7p9SUXnMMPowHnnxz9oxop+0A==@kvack.org X-Gm-Message-State: AOJu0Yz3wu2huBtEkK/D9vylJbljE5iPNwN3U7lwTo0x3DfwAC4TDPVT pDHik2fMqg4iFRy9xtc15BQ0bq3V1TVEviDVXnpT2+2wI4uTXssXNwMT X-Gm-Gg: ASbGncus6XzyY7UZRg2jmWT9fKOZn9GH0VKB5dR9BJYR+BaKkJtfpIYGyGYzYXRfCmL 8ZHWknnaj8PDHct8b0XDE5MMZipGIdYekgw8kyFdzZt9AOtyRSZ2WR9WSj/lLDFxMRTxPZFhHJt Z4GR++cBdb8UDg1oHr7gDCN87PwGneI5nYu46GxzYnlODpWdXO+vZRKAIzyXLxdspKjKgICP0Xi Xm/ovxRvx07SLLSB1axcFuRMErw9fWQrmeuzUxkXNspCeg8ga9XFz+jiEhEwtlFMnGBwC3JTA+G Kn2qxP6QOhapi9rMrZaV5NBPyim2YszdgSvnQOT1eZwmAPBda2I= X-Google-Smtp-Source: AGHT+IGoWPhbqQgrPTfN+HahUoqYMCdzw97k6SQMecL+0/kyAXfDwotPc2YSZrHXNklw9nNNqtKDZg== X-Received: by 2002:a17:906:7956:b0:ad8:9ff1:c69c with SMTP id a640c23a62f3a-ad89ff1ca47mr55655366b.38.1748395039717; Tue, 27 May 2025 18:17:19 -0700 (PDT) Received: from localhost ([185.92.221.13]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ad8a19adc2fsm16458566b.12.2025.05.27.18.17.18 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 27 May 2025 18:17:18 -0700 (PDT) Date: Wed, 28 May 2025 01:17:18 +0000 From: Wei Yang To: David Hildenbrand Cc: Wei Yang , Lorenzo Stoakes , akpm@linux-foundation.org, riel@surriel.com, vbabka@suse.cz, harry.yoo@oracle.com, jannh@google.com, baohua@kernel.org, linux-mm@kvack.org Subject: Re: [RFC Patch 0/5] Make anon_vma operations testable Message-ID: <20250528011718.aaeyfcnlavyxjn3j@master> Reply-To: Wei Yang References: <20250429090639.784-1-richard.weiyang@gmail.com> <8c268ece-71a7-470a-bf8b-71d3e4920977@redhat.com> <20250429235608.yvbcxybkfsmp6ow5@master> <20250514012318.2impqotf65p4ihsc@master> <20250527063428.4ykrdtive2hfnqdp@master> <40dc48f9-94d7-46b0-83ed-d3511789c15a@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <40dc48f9-94d7-46b0-83ed-d3511789c15a@redhat.com> User-Agent: NeoMutt/20170113 (1.7.2) X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 9164E18000A X-Stat-Signature: 8cqn143mcxphhstnu7j9f9mns5nbj54h X-Rspam-User: X-HE-Tag: 1748395041-618866 X-HE-Meta: U2FsdGVkX19NfrUO0lwWzesNgsT1qdEwWC5a3B45pG3zVkHAs1mF3jyR5X3qn4PQ9x4c689AugCjE5hXG6NZaa/5xRlHzkbgD0K3xloHGQmT4vKdQpdB3CmtTlHklfz6mvGLO6nLq6t2FOx1z0ppw6RWzWVSRpv+2GD9lZkZBoch2LFJUJYwLa5uzbfdaXFS9Dw7GAUgVyqajoQ4zpxINirhYK+DJgE53Gc/n5XM7xzOJ2K9yysQDlKJUftu/r4FrhhfbxxC6sQBGupQJD1l2TXw0gmty4BXpJhsfarQMv5y74CGLdvtyP672mA4CthuzmS9AuUvhQHa2qD4ZDxAwhFCPl+zBUqqcE6acJdNQnyXHjNO1PYSWDsNuVfRKMfHAWA4LwAfWMQQPwlBQKLez59ExHEsCzkdjRaFXwO7f5SLFoRSYf4pAhg9l6jfLi5SwixHOti5AKPhSL7t9Ve6ze2WxytyPLfzFRYIpBoK+jUbvnBvODdf8xu6bhYI/JWd+sYBDjrO6TT+cg3x/LjVjFt09jLiuxxUiLbF7alAmLLcFWKDKJzJuOqgrtcfVAwZhy8STTcpoQgnXhHYV55fKVwcT7pZTCh7KLh5fIDF/1Dow/lSmY8rR8FxoCBswUYP2+6x8Xj0UMpN2dy1HvbLMujxcee2IMb+BrnB6OwYUMW/3srfDENB37ZUibURFsuFKvLb5ty9Upr4hbDASORkYWh7dhL7gjgMQBpOPVepuV4047KEaI3uxavWAvYjpjiH9cf82/LWwQTZNBm1aL4GQ/TffqVldbQLWZpFvO3kaTvivptn9jUkTaaq9uPUFWr8JdElIQXovPAcqddYmtQmrocbKXGr52O47l+jqltD5Vq2TQ31qPyhRKlyi86U2Mlw72Cf3Cu/k1+a5alQMyQaTgX1p3JZlvAP4QKzVeFMlif+QcOguLfcGk7JSaggT1+RsnbmQkMy2bIeszJJlWY 7P3ZAStj 3ocxpgB32wfK9mCk3NjID55engaEosMAGizhkXp5NpZ4cOSeNqYpRb1hVMTBU+PE6DAwB5BvNP2juc6qdHcT0B2SBTFPEYg2wW2z7MTfCZI6oxnddcj5CQDoKctAFIywao9+X85Ea3n+V9hpup+9uRVDomE7eHjS+nxIPBPm9SfFoarqUy/E1HZIlTRGa8ZfliB8MgtaPwZDv7vGubMwdLCkgFi19CMmx42BXz6USR5jbkK6BBKc8pXHuJGCxyYjFeV9fELsjQbpAia1EjmapAuC8nsS74QR7Jp0325Qeln5qL6WNWWCZhEpJ30MR0qS6BUWN5I+5MWb/kOQ9nlhOvKup5ghTojEk4fNtpdHdPZ+++pm3JJoEZl6ejl8zKj6R6W4i8kv9ZnVI+W9e+u4/xEtDC8tBnoz0c2t6R8peR0FKBcGyN1ioOVYs/VUEfu6jrbwfPwOwd7wFzKIrJrw6i1k5HCnYSk9dlAfe1J9h4zeTjUmjOp0yWz8lI8TBSQ/BYz1xSA0rc+KNzSDQ6oAvT8SAoI8MMPQHGON+0LFhJYwrzbgaARUSJb8zI9jMjeJGjRlT+tKxWAEnhnHd55OdcusMGciwcNHRqz2VfkyW8jd1tltaGbJJ+SZTtet/YhvS4MdkkBqRRbsQwW/0prJ5SHfs/TVVcvy1la3RLSEdyZEZGZtseltK8R9jG17kUTDWI9nH7Dq1RwKqBIGB26WCOHH6SA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, May 27, 2025 at 01:31:47PM +0200, David Hildenbrand wrote: >On 27.05.25 08:34, Wei Yang wrote: >> On Wed, May 14, 2025 at 01:23:18AM +0000, Wei Yang wrote: >> > On Wed, Apr 30, 2025 at 09:47:16AM +0200, David Hildenbrand wrote: >> > [...] >> > > > > > Agreed, skimming over the tests there are some nice diagrams and cases. >> > > > > > >> > > > > > But I would hope that for most of these cases we could test on a higher >> > > > > > level: test our expectations when running real programs that we want to >> > > > > > check, especially when performing internal changes on how we handle anon >> > > > > > memory + rmap. >> > > > > > >> > > > > > E.g., do fork(), then test if we can successfully perform rmap >> > > > > > lookups/updates (e.g., migrate folio to a different numa node etc). >> > > > > > >> > > > > >> > > > > That's a great point! Wei - if you could look at making some self-tests >> > > > > (i.e. that live in tools/testing/selftests/mm) that try to recreate _real_ >> > > > > scenarios that use the rmap like this and assert correct behaviour there, >> > > > > that could be a positive way of moving forward with this. >> > > > > >> > > > >> >> Ping > >Thanks for reminding me and sorry for the late reply. > >> >> > > > I am trying to understand what scenarios you want. >> > > >> > >> > Sorry for the late reply, I handled other things a while. >> > >> > > That is exactly the task to figure out: how can we actually test our rmap >> > > implementation from a higher level. The example regarding fork and migration >> > > is possibly a low-hanging fruit. >> > >> > If my understanding is correct, you suggested two high level way: >> > >> > 1. fork + migrate (move_pages) > >Yes, that should be one way of testing it. We could even get multiple child >processes into place. > >> > >> > > >> > > We might already have the functionality to achieve it, *maybe* we'd even want >> > > some extensions to make it all even easier to test. >> > > >> > > For example, MADV_PAGEOUT is refused on folios that are mapped into multiple >> > > processes. Maybe we'd want the option to *still* page it out, just like >> > > MPOL_MF_MOVE_ALL allows with CAP_SYS_NICE to *still* migrate a folio that is >> > > mapped into multiple processes. >> > > >> > >> > 2. madvise(MADV_PAGEOUT) >> > >> > Not fully get it here. You mean fork + madvise(MADV_PAGEOUT) + migrate ? > >fork + madvise(MADV_PAGEOUT) only. > >Then verify, that it was actually paged out. > >For example, we could have 10 processes mapping the page, then call >madvise(MADV_PAGEOUT) from one process. We can verify in all processes if the >page is actually no longer mapped using /proc/self/pagemap Got it, thanks. Will prepare the test cases. > >We could likely do it with anon pages (swap), shmem pages (swap) and >pagecache pages (file backed). > >... and possibly even with KSM pages! > >> > >> > But we need to enable pageout in this way first. > >Yes, madvise(MADV_PAGEOUT) needs to be extended to allow forcing a pageout >(e.g., a new MADVISE_PAGEOUT_SHARED that only works with with CAP_SYY_ADMIN). > >> > >> > I am not sure why this one is easier way to test. Would you mind sharing more >> > idea on this? > >It might be easier than the migration case, because for migration we >currently need 2 NUMA nodes ... > >> > >> > > Some rmap tests could make sense for both, anon and pagecache folios. >> > > >> > > > >> > > > Something like below? >> > > > >> > > > * fork and migrate a range in child >> > > > * fork/unmap in parent and migrate a range in child >> > > > >> > > > If the operation is successful, then we are good, right? >> > > >> > > Yes. And one can come up with a bunch of similar rmap test cases, like doing >> > > a partial mremap() of a THP, then testing if the rmap walk still works as >> > > expected, pairing the whole thing with for etc. >> > > >> > >> > For both way, we could arrange all those scenarios and also do partial >> > mremap() during it. > >Exactly. > >> > >> > > One "problem" here is that even with MPOL_MF_MOVE_ALL, >> > > move_pages() will not move a folio if it already resides on the target node. >> > > So one always needs two NUMA nodes, which is a bit suboptimal for testing >> > > purposes. >> > > >> > > For testing purposes, it could have been helpful a couple of times already to >> > > just have a way of migrating a folio even if it already resides on the >> > > expected node. >> > > >> > >> > This looks we need a new flag for it? >> > >> > Here is my plan if my understanding is correct. >> > >> > 1. Add test cases for fork + migrate. We may limit it only works on machine >> > with 2 NUMA nodes. >> > 2. Enable move_pages() on local node, then remove the test limitation >> > 3. Enable madvise(MADV_PAGEOUT) with multiple mapping, then add related cases >> > 4. Add mremap() or other cases >> > >> > In general, to verify rmap dose the work correctly, my idea is to >> > >> > * mmap(MAP_SHARED) >>> * write some initial data before fork>> * after fork and migrate, we >write some different data to it >> > * if each process do see the new data, rmap is good. > >With pageout (see above) test, we can just verify using /proc/self/pagemap in >all processes whether the page was paged out (IOW: the rmap was able to >identify all page locations). And that should work with anon/shmem/file/ksm >pages IIUC. > >> > >> > Does it sound good to you? > >Yes, that absolutely goes into the right direction. Having also rmap tests >for KSM as raised above might be a real benefit. > > >-- >Cheers, > >David / dhildenb -- Wei Yang Help you, Help me