From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6291CC27C4F for ; Fri, 21 Jun 2024 11:20:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BA1838D0154; Fri, 21 Jun 2024 07:20:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B51928D00DB; Fri, 21 Jun 2024 07:20:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F1B28D0154; Fri, 21 Jun 2024 07:20:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 7FE108D00DB for ; Fri, 21 Jun 2024 07:20:58 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id ECB591A0DB2 for ; Fri, 21 Jun 2024 11:20:57 +0000 (UTC) X-FDA: 82254653754.14.04D4DBE Received: from mail-qv1-f44.google.com (mail-qv1-f44.google.com [209.85.219.44]) by imf29.hostedemail.com (Postfix) with ESMTP id 1CA7912001D for ; Fri, 21 Jun 2024 11:20:55 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Wk1rr0aI; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf29.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.219.44 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718968847; a=rsa-sha256; cv=none; b=UTN+uSHm+5Eap5FSPI4JoCpAsH7qWkcEGZFio8uQVI8ZuHDzkLrQMeIF4d8/n9ZTCrh8LF pP8mESE5VpdFLqEuHJ3ENmlQOxvE3T3wSUHybA/NKpflWlCTMUN6OME49Y8knsIAKHv4pt V43xb+hG9lyRY7vTHkMK7ABMUfDqnu8= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Wk1rr0aI; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf29.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.219.44 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718968847; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=expKverTQwLFbgLvc5JqaPG/XGjYSuNCZkjywWmUcWc=; b=3HHqyHcB81mcINVqS/NYXuy+mqIcwjDKqhg9G0ZUjoIX9nf23oNkB7GPX+auOq5GWZXEbd sYyLO/ILpdk5kdQus26ejq0BPYkyx8C5XJCdP23s1iDGsM9bHMlu6lYXXgO919z0h7t4SN 0CmJrEs6f8KuJLGVtXVIkF/jVp4oQsE= Received: by mail-qv1-f44.google.com with SMTP id 6a1803df08f44-6b081c675e7so8843526d6.1 for ; Fri, 21 Jun 2024 04:20:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718968855; x=1719573655; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=expKverTQwLFbgLvc5JqaPG/XGjYSuNCZkjywWmUcWc=; b=Wk1rr0aIkAwEsWSDl1ABL2s8svmVZaimmzfgLdOFJiHek7K9T9nodmHtnPTHBUQpm0 a9ykv8qPYRYYBniORA3XCrz1aMQNbNpR7hAzUmn9i3+Daqkk1V7V728rO5mTs9GYKLhL 1Xq2IL+KC9L7x0wnVgUMs8qxKQyTNucDk+o08rZJC9MPSu5rf3E9yQxN3WismIhJlXf/ +i/JaQOUyGaTuPjaQH1Wlf/2PY/QgfGqee2q81ZzlJ5dtsJo1L6T29nwOUc6QfSvc/cZ K4owF6OBsACLjR7E88e6oXCDO/jR1lzvKHVNX3x4Ma/05SJ4fqQiNEif2aq92EdX5VTU P80A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718968855; x=1719573655; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=expKverTQwLFbgLvc5JqaPG/XGjYSuNCZkjywWmUcWc=; b=D7nvFKCjgZSI1bg1yg7gH7mWktPLm+iJZWtD0bJxFAlI0wWYgltiJ+M9U4NAeRdp6y VV9cXDGzFyYZzEoAXtwVfWFgyrUNo4lGvIlJhV8oi6hgnkMAjfSgAJ+MkocgJKqNE0Ie +tCozWLy3lXSN2WKhIA1XRxk+UBHM/xetd6w0owsRXjGLyP011NCiyx9RmJwZAlHrL9O i/Cvj9McaY/cAonNAFlLbmVV23CQYJKf8mFroKHSEZd7qxp7Nkto9C2/H7hjlvppJiwj 3N4+ERd1UcZHDxFemNwByCveJiUzfkPr12Vme3QpUQLgqzGGXboxKy0T+zYtE9Ukxbs7 b6MA== X-Forwarded-Encrypted: i=1; AJvYcCU773BANcPfsIgRZnYOGrIaleFczmHx7V04vNr9vJFiDF4EbbhhnabhuTJ++PVPiQhpAPf3quGNNbZxlCxKDSq6Ts8= X-Gm-Message-State: AOJu0YzbXfi3SqqWE9rlKcc7AvXSya/Che4hDwO6z8EUhzEckjjM5hws izhq1tIiHs2yocs1bKG+zt78MBfAUoQQRJ+iQ/kqODQHQE+WevfUwazGJwUjOWE3WaY9nCITWrE QYDFMtGMLipr3upXM6IsJsnKBVuQ= X-Google-Smtp-Source: AGHT+IGmJ6cnArzxMGquykZMa9FOUQZ2LPltqRLgO4ZbdvrZSjSCd8tc4lPHCPg3LwW9FM4RyDa2LZl9geq/qEjVUUM= X-Received: by 2002:a05:6214:80c:b0:6b0:7864:90ac with SMTP id 6a1803df08f44-6b501e06c38mr90689636d6.11.1718968855094; Fri, 21 Jun 2024 04:20:55 -0700 (PDT) MIME-Version: 1.0 References: <20240620002648.75204-1-21cnbao@gmail.com> In-Reply-To: From: Barry Song <21cnbao@gmail.com> Date: Fri, 21 Jun 2024 19:20:43 +0800 Message-ID: Subject: Re: [PATCH] selftests/mm: Introduce a test program to assess swap entry allocation for thp_swapout To: Chris Li Cc: Ryan Roberts , David Hildenbrand , akpm@linux-foundation.org, shuah@kernel.org, linux-mm@kvack.org, hughd@google.com, kaleshsingh@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, ying.huang@intel.com, linux-kselftest@vger.kernel.org, Barry Song Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 1CA7912001D X-Stat-Signature: k93ox39n8dmub1n6oodetabxh8kg9yrb X-Rspam-User: X-HE-Tag: 1718968855-166748 X-HE-Meta: U2FsdGVkX1+gN6WO+F1JqNoeauIEFbsVrrXhPD1MIc+wvHia8kTHmjYJksFUUJf6XqdFCPm7ogcZY30vZmNiD79+9yj6H5v2eRroYFlhA5sG5BYxcJMW44Y9EcnD/tnfn+vmecyDYBBoxz0+ZxqpHWyMLSPExN++VvgINnLLAcRQaRsI8oYgzKrYC697oiTmchYwWQPsQ0/cXzwY5xK0m5wpfMzydXnu0pmBrk46v/IoylwxSizK+PixaXd1U8j5Q89qmTZArPDPPdvKC+gSD1OsXGjxE1xPDV/3eWEUues0qcv1AOsMp7r4RnqHMAPxQBRnQX5zIvylvL3Zv7WjcOgA/oxeU4DrC7qF7I4f7riQo0QWPmxQQFPlKuPWWku1EIdr3CFiOWCWZMlTD7updYzBrUEOj0t7VdhM2TDdDohogZMr+Gq9emXG3Mnrts2KznRTrNioZJiTf5A4hDaKeikEx5qbdaq1bb1L404q73OJCSy+qkeraMcVXN6lkYU1PTRVMK7jVcpxZJu2q0ux3lhe01gtzn7D8AcrIUyIh1J750C3P7DchFwde02T/0IrrYDAJEkcymMWSw70h9Nrn9S2YWaTrdkeGG1dzpoBLuJkHjlO6XziRsNCdjGM6yYrJrze4rZYnae2C9DuQOQiIsl0SvbmYCvcnHBy1yOQReLCSEMhtJpCiIgfMc6Pooaa/E698nfa+XDkCRO+hfkxI27w+ba2ZQ7RkTahORag0qZ9M5ajN/c5OJsKSnxGPIDHB2siE41VC23UD3x7QEDaJ6dmBcZ7+Dp1mgeh6xqLtDQn2XqdzDkEjVVT2iNKnHdqP3egJ/TorwfKwEOAErDGzRASVDj0iJkl9NP5j2Yp3sFyvoLRbpew23yPp5QDAv31k3W31F27uioQU4s4AXGm3oDQey3eg4KHOkrc6RXbSAqC/YCWa705pcTGgHU7+2vfeXh8kMm4d5F03GSB5Wx 9rtSZsfo lTi/1Bqo82y6LJSoJhofgu2zafBTOEFxtX6e5FdvFD7sRx6lFzbTGVmUF/irsXlZUVs/qdBP4lmOsILpP65YQ0mu/tikd0gWSBM87C2+VuWMs+ctbzgojpYcaNDdp3KAnqkklLlY+gmM18Q9x0IsMcdb3fxq/17g4D0sWcx95HJzaouAk/zQNRKplkCjVJihnCpS/7V9IuZJODEcYTwLNVbqLnsLGzb2Ef6fZScKWO+cAwej2ThYshva/hUcYpytWuzKTle+SxuPcUyDnrupXEgEhYDDcXFqi5miF0vuuEWcsP8uTpfKxCcH5lNSK5CZzkebRKGptiZ0PJ3bgzWDDXdOE8qUnrEcCYE5Rs4yYeMJU/O4x5jk7FAIC0TltcfaUpDCT+F62nVh8EH0dRUpQfgluwS4najWbGpJnzNF3eFbo9yZzCk4xhH5nmedk1dUWnJdEnNwH4PT55UkevFIBAn7yb7tzA0oe83UPxscSDimjSjaV5rpNwbTFx4pQetY2u/UddHXkajdSd+piaNnU46ZGbi4FGBU9SOPX X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Jun 21, 2024 at 4:50=E2=80=AFPM Chris Li wrote: > > On Fri, Jun 21, 2024 at 12:47=E2=80=AFAM Barry Song <21cnbao@gmail.com> w= rote: > > > > On Fri, Jun 21, 2024 at 7:25=E2=80=AFPM Ryan Roberts wrote: > > > > > > On 20/06/2024 12:34, David Hildenbrand wrote: > > > > On 20.06.24 11:04, Ryan Roberts wrote: > > > >> On 20/06/2024 01:26, Barry Song wrote: > > > >>> From: Barry Song > > > >>> > > > >>> Both Ryan and Chris have been utilizing the small test program to= aid > > > >>> in debugging and identifying issues with swap entry allocation. W= hile > > > >>> a real or intricate workload might be more suitable for assessing= the > > > >>> correctness and effectiveness of the swap allocation policy, a sm= all > > > >>> test program presents a simpler means of understanding the proble= m and > > > >>> initially verifying the improvements being made. > > > >>> > > > >>> Let's endeavor to integrate it into the self-test suite. Although= it > > > >>> presently only accommodates 64KB and 4KB, I'm optimistic that we = can > > > >>> expand its capabilities to support multiple sizes and simulate mo= re > > > >>> complex systems in the future as required. > > > >> > > > >> I'll try to summarize the thread with Huang Ying by suggesting thi= s test program > > > >> is "neccessary but not sufficient" to exhaustively test the mTHP s= wap-out path. > > > >> I've certainly found it useful and think it would be a valuable ad= dition to the > > > >> tree. > > > >> > > > >> That said, I'm not convinced it is a selftest; IMO a selftest shou= ld provide a > > > >> clear pass/fail result against some criteria and must be able to b= e run > > > >> automatically by (e.g.) a CI system. > > > > > > > > Likely we should then consider moving other such performance-relate= d thingies > > > > out of the selftests? > > > > > > Yes, that would get my vote. But of the 4 tests you mentioned that us= e > > > clock_gettime(), it looks like transhuge-stress is the only one that = doesn't > > > have a pass/fail result, so is probably the only candidate for moving= . > > > > > > The others either use the times as a timeout and determines failure i= f the > > > action didn't occur within the timeout (e.g. ksm_tests.c) or use it t= o add some > > > supplemental performance information to an otherwise functionality-or= iented test. > > > > Thank you very much, Ryan. I think you've found a better home for this > > tool . I will > > send v2, relocating it to tools/mm and adding a function to swap in > > either the whole > > mTHPs or a portion of mTHPs by "-a"(aligned swapin). > > > > So basically, we will have > > > > 1. Use MADV_PAGEPUT for rapid swap-out, putting the swap allocation cod= e under > > high exercise in a short time. > > > > 2. Use MADV_DONTNEED to simulate the behavior of libc and Java heap in = freeing > > memory, as well as for munmap, app exits, or OOM killer scenarios. This= ensures > > new mTHP is always generated, released or swapped out, similar to the b= ehavior > > on a PC or Android phone where many applications are frequently started= and > > terminated. > > Will this cover the case that the ratio of order 0 and order 4 swap > requests change during LMK, and swapfile is almost full? > > If not, please add that :-) Due to 2, we ensure a certain proportion of mTHP. Similarly, because of 3, we maintain a certain proportion of small folios, as we don't support large folios swap-in, meaning any swap-in will immediately result in small folios. Therefore, with both 2 and 3, we automatically achieve a system containing both mTHP and small folios. Additionally, 1 provides the ability to continuously swap them out. If we set the same sizes for 2 and 3, we'll achieve a 1:1 ratio of large folios to small folios. How about starting with a 1:1 ratio? To meet the requirement that the swapfile is almost full, I can increase the memory to ensure the total size is quite close to zRAM. This way, we give the small folios a chance to perform a slow scan and observe the impact. > > > 3. Swap in with or without the "-a" option to observe how fragments > > due to swap-in > > and the incoming swap-in of large folios will impact swap-out fallback. > > > > And many thanks to Chris for the suggestion on improving it within > > selftest, though I > > prefer to place it in tools/mm. > > I am perfectly fine with that. Looking forward to your V2. > > Chris Thanks Barry