From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F43AC27C53 for ; Thu, 20 Jun 2024 02:04:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B6ED08D0099; Wed, 19 Jun 2024 22:04:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AF4268D0066; Wed, 19 Jun 2024 22:04:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 972CD8D0099; Wed, 19 Jun 2024 22:04:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 6EF698D0066 for ; Wed, 19 Jun 2024 22:04:18 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id EEF451C1F2D for ; Thu, 20 Jun 2024 02:04:17 +0000 (UTC) X-FDA: 82249622154.29.C1591F5 Received: from mail-ua1-f54.google.com (mail-ua1-f54.google.com [209.85.222.54]) by imf12.hostedemail.com (Postfix) with ESMTP id 2CC9840008 for ; Thu, 20 Jun 2024 02:04:16 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BEc5XrTy; spf=pass (imf12.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.54 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718849047; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jhmMCJlWLT6P0x0JEVsA2CFzvtPsQuWVwbgag5ghfwE=; b=iEfADCb3louUH/Atol4jjfPBYq3WDVgagXc7t5MUEQ9hRAEtQ8p8mjvLVkLkhQ6i9kJMNv Te8RcvQn9+M4UM1fnTU/sdy5CIKNFljuwJRboneIVDYlwv9ji5BdCctm9tocd3jt0HKUtb bA0lCOQrVQRLBX+vkIEUDha8qXrVpj8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718849047; a=rsa-sha256; cv=none; b=Dxl4yVWQx8jXOS1rxYxvVv3ngtzPffsOw6sJl8RNURU44eE1mu6YxCpTquWr1IWluLmCzR dgd0MNv5bK5GrDEZY3/LPzgtpyAUpzUAn6zR26zL/A8RPAclcbgRII4KKroGz57O+2yzHE /uVw6zrBjANa3r+1SqwD1fruF2C6QSs= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BEc5XrTy; spf=pass (imf12.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.54 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ua1-f54.google.com with SMTP id a1e0cc1a2514c-80f5518082aso142060241.3 for ; Wed, 19 Jun 2024 19:04:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718849055; x=1719453855; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=jhmMCJlWLT6P0x0JEVsA2CFzvtPsQuWVwbgag5ghfwE=; b=BEc5XrTydas/PZId0TtQRJz9aO82YdNSIqDx5SITplUeFX8tD0m3gxNRB1mTu2Wp89 RP8KCC+BYrOxpQeCRWQqZVWXN4RZw0nLxivhPYcvPh8LzqTVBlxP0wZ3grWitRRNCYc3 iNOmOXSejPSQMNi5xULGTZQxT5L0e6IObsSTRBsuHbL9dBTUeHKjy/JvoENkj1DdfsFS QatZd0VPNpbvNDheemM1ooQTsimDFh8397x+Akt04YgbILmGLlOprwCaSoW4XFkRF5SQ DOsAoEqInaCFQ/iP30NDVwWdKwiJUegtWodrOTkziWemkpFSo9C0tPanwJMiQk/cMwGU JnhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718849055; x=1719453855; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jhmMCJlWLT6P0x0JEVsA2CFzvtPsQuWVwbgag5ghfwE=; b=tyRMgsJxV0tz5gLZnxJgML02aXWHF66J+a2P/p4/BXDsEIh1xAHHdnoWSsqzkqHAF1 9pHuisDM+VS3n5WJDJk75EmzAV0Nv3C9B5EiBS0tf8EMeX+8CPuX9Prh9+D1jvv9eWeY 6pI4/cxmpJ5v5fNoEc4AOfZ3Om3jZJMOHVvp0UgMxS73I9TSBn9BDBO5WCU+534N05yA Vf3LWvw6AaM+sa+kOGYbqdiXT1f63fdHqm7epZZbrv1qWxoo/BGA1w8fgyommLbsY6e3 Am//0no6XHGsZrxoqpP+ExBwTTsvu3i7paUax6e49HaD+lcOwNXSWOa1dBI0+nfmx48s iAVQ== X-Forwarded-Encrypted: i=1; AJvYcCXorovMZn6j3uUZX23Gq+WXeAxb6UTPGmwO5AfPZKJy0B4gc22SeKXMr3Au0ZehsJ+GSYKJ73U3DE/Ysa/rgwycm54= X-Gm-Message-State: AOJu0YxbYFx+vbHFnB+xt4RgbE69Wf41/8x+GNU3XIMOvm5jAPcCKDkK cNvUM3YBPbWLoeSQ3cMFEbjWVmAu48nzH9AXbj6p3YXn4Cd6rQ4B4dufspCosD5WFLVI6K8skU+ EmIWU58rtt2SnEwALWO3fDQffJ8w= X-Google-Smtp-Source: AGHT+IGWMX6GIQMKhSTglWLeOPg1r0KL1VIJVatQqACDfbph/Dm/svF8qriPsm3BUQO0wRsDfIBQ+vY4BH5McdG6U1A= X-Received: by 2002:a05:6102:4c46:b0:48f:1d57:9b with SMTP id ada2fe7eead31-48f1d5700f5mr2656379137.26.1718849054960; Wed, 19 Jun 2024 19:04:14 -0700 (PDT) MIME-Version: 1.0 References: <20240620002648.75204-1-21cnbao@gmail.com> <87zfrg2xce.fsf@yhuang6-desk2.ccr.corp.intel.com> In-Reply-To: <87zfrg2xce.fsf@yhuang6-desk2.ccr.corp.intel.com> From: Barry Song <21cnbao@gmail.com> Date: Thu, 20 Jun 2024 14:04:03 +1200 Message-ID: Subject: Re: [PATCH] selftests/mm: Introduce a test program to assess swap entry allocation for thp_swapout To: "Huang, Ying" Cc: akpm@linux-foundation.org, shuah@kernel.org, linux-mm@kvack.org, ryan.roberts@arm.com, chrisl@kernel.org, david@redhat.com, hughd@google.com, kaleshsingh@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Barry Song Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: wyhtq8o4kptwhq8not4z9kgekguytz1c X-Rspamd-Queue-Id: 2CC9840008 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1718849055-618807 X-HE-Meta: U2FsdGVkX194vXGh2y8flp9BWuSHsbciwrjUDiwf/KV4+vZmXLyB6RuV19UBmNv+PtdgtawY9CtpPNRM9KdrTTCMtfUaZPKZ2KHj9loyqhpi79m56iYV2amSd/BlQTOssSyns76oEdKhHROI4RmuyXCoXg8c7FwiIR/mI2mlARyXg2ig8p+hjtcm5N2E6xyECSJKW0DkBWlATNCW2a/rYft2PhQJulD2w6sP0vtqLU2oOpzOou1AiZaea8klwRQeE9iBHKeUiK/YNHwGowPCqE1ksJ4k5sVSz3D0Zm7VPpm/BWLSiBNuEB1+1CbehZ78xbCW6NceET2wWZUedrSBIfRMuehaD3Iqf2/oQc8/AuUj8xSq789xgtzQW8GARNYfvtY4yQuMQXJgz3LlH7gYRdIBq0wmjB0iJjwhpIHhMfrmlgjV5Au+2pucr3y4VQXOVPmJtSqBwmVul42QevtlPkGftu2JdG1J2xM66QAjm+/O0W/VuZoh/liJ1IS6pjqyh4lQtrQY0fivHwBr23UFi6dRkyse1vAzVejY+Nm+n3/3Xdn1o3FbIa9UckFmml/nwcDTjDEe5y23rSCz+iSmyTBj2cGhdHp/kiq1GnKUDNPHgAcirsDYqnMR19BibzurRYrF2kqc8arKdZ2QjupLD3yK2gAfW0abA7hR/qyMngv6PILvtGOWizHm3DMcApUF7U60p77QxGiJZbMG41IunS2BB8J69j47pja5C1uDusSaygkpLUXcy8VgeTnfpeHMH76ewaxNt9tYtR+m+146EmVh/L0cOi0foAF+zg0Xz80I0A/l9zhWuhlTnDqiBr9pG3l77ixkTGmCX1eadJof/55Z/Dlh09ykd8+wbKFsV/Dhl3qoch8bO6fqhipbNwJ3jOjht5tdD1Hrg859x74ULm1ngGa0quRlmB+Zmmhgh/l7Uee2SETVDjJnWKSdl6lboGqn7vaNTDrelp7lUGa lHyz0q/E 3lOIIH8hJqWL8c1Sqgm4A4jRiVyKWpRI/BSeRsh6JVCF8hNH4eP2jGBmx9fgT857GfQgZPD6NdnVkDNRMG0oVbv86RDSpPb3umoXkNWZrCOlxNPDOiZNA9S6LohD3nT/Iz1vPn6NlCoiGca4X8Z1m4jbOmM3R3amJ0a6Z9qRp8qG7HCHjWImVgKP7GvbAWae1Jh+97lOuVFYSYyi8yUOX7PRQ5aVQHjQG3m2AEZ2j2Rs36WJnWqBi+Dx0JjS7EOHrgeG4LomyLL5zH7Ko54SU9Sx1ZUxHZXlMi9kUoANpjsrRdl7mTGD9hrLgO7TBs9EHa2tHmJFpoBD/3Io8oOMJZ+vzjPY6WYqqN10TEacUQ75RUpcCGaXaS7UGD5yqd3aOxs6CwTwbz90EV6Hi3dBHWocdlpaM8HHs9C4zB8zWTe4WX4uDMMGpSNdYRqM+A+L+XS87BzDgxuqwh81eg81Dniy+PeaXBi1MuA6f X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jun 20, 2024 at 1:55=E2=80=AFPM Huang, Ying = wrote: > > Barry Song <21cnbao@gmail.com> writes: > > > From: Barry Song > > > > Both Ryan and Chris have been utilizing the small test program to aid > > in debugging and identifying issues with swap entry allocation. While > > a real or intricate workload might be more suitable for assessing the > > correctness and effectiveness of the swap allocation policy, a small > > test program presents a simpler means of understanding the problem and > > initially verifying the improvements being made. > > > > Let's endeavor to integrate it into the self-test suite. Although it > > presently only accommodates 64KB and 4KB, I'm optimistic that we can > > expand its capabilities to support multiple sizes and simulate more > > complex systems in the future as required. > > IIUC, this is a performance test program instead of functionality test > program. Does it match the purpose of the kernel selftest? I have a differing perspective. I maintain that the functionality is not functioning as expected. Despite having all the necessary resources for allocation, fai= lure persists, indicating a lack of functionality. > > > Signed-off-by: Barry Song > > --- > > tools/testing/selftests/mm/Makefile | 1 + > > .../selftests/mm/thp_swap_allocator_test.c | 192 ++++++++++++++++++ > > 2 files changed, 193 insertions(+) > > create mode 100644 tools/testing/selftests/mm/thp_swap_allocator_test.= c > > > > diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selfte= sts/mm/Makefile > > index e1aa09ddaa3d..64164ad66835 100644 > > --- a/tools/testing/selftests/mm/Makefile > > +++ b/tools/testing/selftests/mm/Makefile > > @@ -65,6 +65,7 @@ TEST_GEN_FILES +=3D mseal_test > > TEST_GEN_FILES +=3D seal_elf > > TEST_GEN_FILES +=3D on-fault-limit > > TEST_GEN_FILES +=3D pagemap_ioctl > > +TEST_GEN_FILES +=3D thp_swap_allocator_test > > TEST_GEN_FILES +=3D thuge-gen > > TEST_GEN_FILES +=3D transhuge-stress > > TEST_GEN_FILES +=3D uffd-stress > > diff --git a/tools/testing/selftests/mm/thp_swap_allocator_test.c b/too= ls/testing/selftests/mm/thp_swap_allocator_test.c > > new file mode 100644 > > index 000000000000..4443a906d0f8 > > --- /dev/null > > +++ b/tools/testing/selftests/mm/thp_swap_allocator_test.c > > @@ -0,0 +1,192 @@ > > +// SPDX-License-Identifier: GPL-2.0-or-later > > +/* > > + * thp_swap_allocator_test > > + * > > + * The purpose of this test program is helping check if THP swpout > > + * can correctly get swap slots to swap out as a whole instead of > > + * being split. It randomly releases swap entries through madvise > > + * DONTNEED and do swapout on two memory areas: a memory area for > > + * 64KB THP and the other area for small folios. The second memory > > + * can be enabled by "-s". > > + * Before running the program, we need to setup a zRAM or similar > > + * swap device by: > > + * echo lzo > /sys/block/zram0/comp_algorithm > > + * echo 64M > /sys/block/zram0/disksize > > + * echo never > /sys/kernel/mm/transparent_hugepage/hugepages-2048kB/= enabled > > + * echo always > /sys/kernel/mm/transparent_hugepage/hugepages-64kB/e= nabled > > + * mkswap /dev/zram0 > > + * swapon /dev/zram0 > > + * The expected result should be 0% anon swpout fallback ratio w/ or > > + * w/o "-s". > > + * > > + * Author(s): Barry Song > > + */ > > + > > +#define _GNU_SOURCE > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > +#include > > + > > +#define MEMSIZE_MTHP (60 * 1024 * 1024) > > +#define MEMSIZE_SMALLFOLIO (1 * 1024 * 1024) > > +#define ALIGNMENT_MTHP (64 * 1024) > > +#define ALIGNMENT_SMALLFOLIO (4 * 1024) > > +#define TOTAL_DONTNEED_MTHP (16 * 1024 * 1024) > > +#define TOTAL_DONTNEED_SMALLFOLIO (768 * 1024) > > +#define MTHP_FOLIO_SIZE (64 * 1024) > > + > > +#define SWPOUT_PATH \ > > + "/sys/kernel/mm/transparent_hugepage/hugepages-64kB/stats/swpout" > > +#define SWPOUT_FALLBACK_PATH \ > > + "/sys/kernel/mm/transparent_hugepage/hugepages-64kB/stats/swpout_= fallback" > > + > > +static void *aligned_alloc_mem(size_t size, size_t alignment) > > +{ > > + void *mem =3D NULL; > > + > > + if (posix_memalign(&mem, alignment, size) !=3D 0) { > > + perror("posix_memalign"); > > + return NULL; > > + } > > + return mem; > > +} > > + > > +static void random_madvise_dontneed(void *mem, size_t mem_size, > > + size_t align_size, size_t total_dontneed_size) > > +{ > > + size_t num_pages =3D total_dontneed_size / align_size; > > + size_t i; > > + size_t offset; > > + void *addr; > > + > > + for (i =3D 0; i < num_pages; ++i) { > > + offset =3D (rand() % (mem_size / align_size)) * align_siz= e; > > + addr =3D (char *)mem + offset; > > + if (madvise(addr, align_size, MADV_DONTNEED) !=3D 0) > > + perror("madvise dontneed"); > > IIUC, this simulates align_size (generally 64KB) swap-in. That is, it > simulate the effect of large size swap-in when it's not available in > kernel. If we have large size swap-in in kernel in the future, this > becomes unnecessary. > > Additionally, we have not reached the consensus that we should always > swap-in with swapped-out size. So, I suspect that this test may not > reflect real situation in the future. Although it doesn't reflect > current situation too. Disagree again. releasing the whole mTHP swaps is the best case. Even in the best-case scenario, if we fail, it raises concerns for handling potenti= ally more challenging situations. I don't find it hard to incorporate additional features into this test program to simulate more intricate scenarios. > > > + > > + memset(addr, 0x11, align_size); > > + } > > +} > > + > > [snip] > > -- > Best Regards, > Huang, Ying Thanks Barry