From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38102C46467 for ; Wed, 4 Jan 2023 23:21:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B80F28E0002; Wed, 4 Jan 2023 18:21:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B2FF98E0001; Wed, 4 Jan 2023 18:21:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9D08B8E0002; Wed, 4 Jan 2023 18:21:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 8817B8E0001 for ; Wed, 4 Jan 2023 18:21:30 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 5E365C0BE8 for ; Wed, 4 Jan 2023 23:21:30 +0000 (UTC) X-FDA: 80318690340.11.728BF1E Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) by imf30.hostedemail.com (Postfix) with ESMTP id C1BC480010 for ; Wed, 4 Jan 2023 23:21:28 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=VuWjPMk9; spf=pass (imf30.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.128.44 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1672874488; a=rsa-sha256; cv=none; b=Y6YldXjknv+CPzV+uhO1/TqViYuKSrnIsRoJH2HRns7NGA21qnkCYtHLbPX/oPQoRA6pF+ 4pSVmF5T2KSHRbWOvdcsy56yxKaHe3uHj6asy3EL8SkMWASwTT7evDNPdkGwgwc68qU3pN jqazH7RBCk5H4XK/0RMo6n6BsGVK0J4= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=VuWjPMk9; spf=pass (imf30.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.128.44 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1672874488; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=R+w/QWwuvtOmFx+OVR4xG+AfY+zwDbmJijYlotknW6s=; b=XCrm3SqvLs1n2QrUKMjws0kVupIThyznTE1yApudbMI9NH+cv6JKrTe1TOyW38zMwTcs3s nyTxzx6WqWbT9sZnW2JXTA6GZGhKZIQn9vrvkEj082ISB3ppy1ujyckHGHv6L9zWD4UV92 GYbecLU6enbLXK/L6iA8+jjpNNMa0Kw= Received: by mail-wm1-f44.google.com with SMTP id l26so25222354wme.5 for ; Wed, 04 Jan 2023 15:21:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=R+w/QWwuvtOmFx+OVR4xG+AfY+zwDbmJijYlotknW6s=; b=VuWjPMk9shaCeXQ8DhN/P7DjmfsKWloX+W/HghViHxrB4cvZFadupjiPl/TIEPeU7k 6ckW4dfvESWwdFdD4hed1i3F1WyH0MH/2CN2DioAO+nfrwi4l1oVCwoya9l2UqjWTj/M WmqAWINFjmbdnq4PXPlol6N5wwK0wcsFGJZ8OtrvX0QJbKZ6LqRrj2yeQdQmf+Aj2bMc /gtpmhYhUapAAWTMxQCtd6jzDINjbaBQ5KLOBI7MMQAwhIB/KvwMMyyKNISgaeaQJOCA jLWTM/kuGps0MMQ6835MzPCJCeBvTnGQR6cmDw/lFSAI7p3Xjwv8gPHuqpjHeBrYC2HO JN4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=R+w/QWwuvtOmFx+OVR4xG+AfY+zwDbmJijYlotknW6s=; b=YJMTugiLaHqTDdLup6Kebt8feG7eSsfHLmivWPhsiL6v3srkn/AtCdaZKvqo8Y9RbR NzhLuidFFy5fWnlJm5I4TxtyLtB4d8m5+V2BmtUGj1ToSdcuKxGuSKjcZ9ySLkLOCMFG x+NyFU+FdZ0Otpan+TiJfx2KGp7HDp/KB6C3555WfXqPsBMPRHYtfsqpr/doCFKxGS4p cbVYo7Sa+LtzZTp2r3bOEZ+UMHEa7NZm1MV+xHR4Rh0oUs0l53OpPR3wV/UMAL2z+AGQ gsGKlJTjjyk/sNFIF6YB4t873esQNotc5znZKhLyoGiCcRzKhKBYHC2CvZCJTtJDOOh8 oYLA== X-Gm-Message-State: AFqh2kqJDlgXNc3XLgM6XjWRuGAgGSnay4QchH6O7PdVffY1s7MtilAD cv/eyUFlIjMC60wCVCat1CgVEyqBU3NpVRhKVE4= X-Google-Smtp-Source: AMrXdXvzLJzrj3ZSrGoIQ3+3Nul/7vLcBF+MEuPQhr+3sftu+gy/rfCzdj/8DJuLSBXIuqqwEft1n25PFMK22oMEea4= X-Received: by 2002:a05:600c:1c90:b0:3d2:1d51:2476 with SMTP id k16-20020a05600c1c9000b003d21d512476mr2253182wms.77.1672874487318; Wed, 04 Jan 2023 15:21:27 -0800 (PST) MIME-Version: 1.0 References: <20230104231127.2634648-1-nphamcs@gmail.com> In-Reply-To: <20230104231127.2634648-1-nphamcs@gmail.com> From: Nhat Pham Date: Wed, 4 Jan 2023 15:21:16 -0800 Message-ID: Subject: Re: [PATCH v5 0/3] cachestat: a new syscall for page cache state of files To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, bfoster@redhat.com, willy@infradead.org, kernel-team@meta.com Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: C1BC480010 X-Stat-Signature: tcpqqptgcdt78zd5a7b9krbtk78scgzc X-HE-Tag: 1672874488-680883 X-HE-Meta: U2FsdGVkX1/0HZiHO9JnGCmerNzhuQxb+Le6M7+dEmxCASvkWw3PS6GFyO7Tv1gu0liW5Ip5GuoYV3vWfrAd5yQlwofbeVSgawD+8sRcJ7+8AXAXz23M8pAH/RPp84u+KVpsElwmrp+oKoKQB+7eEJShysxHjrXDcQZhJzvv+nNJIntWvf2jz3Sw1iaOQXo4kDL/sDU3MnYwNOskXKnoZhJ0Ks7BHUdNRG2sBt4R5o65fW0j3vBoeNTxscVTOn1uKZCtIGiH/lGMGtrqj38xSMcEZHcU+221BzSTYB72qihnyWHlMb9+ydwyp6w9bM4kEWgURFiIvTc5nUjJjeCSViNClQDc3bKy9gFxDQLm1DXdav3A11CAtKtcwY653cn/rZWLK5sXqFvwOO/9+ZYW5DUEhZ1DL2YY9SQjbAYZ/RrmII8z5Uae2Dx0cLQ0/cnVnSftB+Jb3K/jKG+5lwe5GpcjCnKWnfjXXQlnYYrnSEIkf9rDLIuHMR3SXP+xJHuCGlPEcRsNS52dWRXE3mKz9Gr6MJR74KFlcHA4Gta+faoGmRT4bW3IQvh/FH1ByIH2oXiQ+NQueTklPEAsqeUEejcOpspLhWnoCQerVF9JRuL6tsGUb0JFQRBbHD+ET/fy9jIgFvcPWADW1/1Ip0xrnTKbP0e7fWEt6vqJ+Uk5Z/vWQecpeO4giqmnKm8qkejFviA8MTITY/e+tzrkF/Kb7PcJiaGWnbUcL9Jka/XWpdoLgI6QxQERSKUhQI9Nfpeo3ouGThSslGceFUpHA9h7UvML6BwHNCkS31Jtwc7/5KJezI9dDdILvptKJtp20hofaZ9zIUnePPotAj2/VJ6c1PBYZzLenuxd9U1aN9xBeTin2zjqFDM6KSP/G+evpFW6F3UYQcX6YAgi9xhcU6lRLS9mrwRgXHkO5/qCSgNNyX14w0rKDgppLc2GB5aSoGJ14nTfRP7VfwFtQZ3VWqk sx6bEy79 ZmHfL3xisRH1xdjTTIX+joqhKz7sIS9lBBpfCzYrP9luKJjg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Jan 4, 2023 at 3:11 PM Nhat Pham wrote: > > Changelog: > v5: > * Separate first patch into its own series. > (suggested by Andrew Morton) > * Expose filemap_cachestat() to non-syscall usage > (patch 2) (suggested by Brian Foster). > * Fix some build errors from last version. > (patch 2) > * Explain eviction and recent eviction in the draft man page and > documentation (suggested by Andrew Morton). > (patch 2) > v4: > * Refactor cachestat and move it to mm/filemap.c (patch 3) > (suggested by Brian Foster) > * Remove redundant checks (!folio, access_ok) > (patch 3) (suggested by Matthew Wilcox and Al Viro) > * Fix a bug in handling multipages folio. > (patch 3) (suggested by Matthew Wilcox) > * Add a selftest for shmem files, which can be used to test huge > pages (patch 4) (suggested by Johannes Weiner) > v3: > * Fix some minor formatting issues and build errors. > * Add the new syscall entry to missing architecture syscall tables. > (patch 3). > * Add flags argument for the syscall. (patch 3). > * Clean up the recency refactoring (patch 2) (suggested by Yu Zhao) > * Add the new Kconfig (CONFIG_CACHESTAT) to disable the syscall. > (patch 3) (suggested by Josh Triplett) > v2: > * len == 0 means query to EOF. len < 0 is invalid. > (patch 3) (suggested by Brian Foster) > * Make cachestat extensible by adding the `cstat_size` argument in the > syscall (patch 3) > > There is currently no good way to query the page cache state of large > file sets and directory trees. There is mincore(), but it scales poorly: > the kernel writes out a lot of bitmap data that userspace has to > aggregate, when the user really doesn not care about per-page information > in that case. The user also needs to mmap and unmap each file as it goes > along, which can be quite slow as well. > > This series of patches introduces a new system call, cachestat, that > summarizes the page cache statistics (number of cached pages, dirty > pages, pages marked for writeback, evicted pages etc.) of a file, in a > specified range of bytes. It also include a selftest suite that tests some > typical usage > > This interface is inspired by past discussion and concerns with fincore, > which has a similar design (and as a result, issues) as mincore. > Relevant links: > > https://lkml.indiana.edu/hypermail/linux/kernel/1302.1/04207.html > https://lkml.indiana.edu/hypermail/linux/kernel/1302.1/04209.html > > For comparison with mincore, I ran both syscalls on a 2TB sparse file: > > Using mincore: > real 0m37.510s > user 0m2.934s > sys 0m34.558s > > Using cachestat: > real 0m0.009s > user 0m0.000s > sys 0m0.009s > > This series should be applied on top of: > > workingset: fix confusion around eviction vs refault container > https://lkml.org/lkml/2023/1/4/1066 > > This series consist of 3 patches: > > Nhat Pham (3): > workingset: refactor LRU refault to expose refault recency check > cachestat: implement cachestat syscall > selftests: Add selftests for cachestat > > MAINTAINERS | 7 + > arch/alpha/kernel/syscalls/syscall.tbl | 1 + > arch/arm/tools/syscall.tbl | 1 + > arch/ia64/kernel/syscalls/syscall.tbl | 1 + > arch/m68k/kernel/syscalls/syscall.tbl | 1 + > arch/microblaze/kernel/syscalls/syscall.tbl | 1 + > arch/parisc/kernel/syscalls/syscall.tbl | 1 + > arch/powerpc/kernel/syscalls/syscall.tbl | 1 + > arch/s390/kernel/syscalls/syscall.tbl | 1 + > arch/sh/kernel/syscalls/syscall.tbl | 1 + > arch/sparc/kernel/syscalls/syscall.tbl | 1 + > arch/x86/entry/syscalls/syscall_32.tbl | 1 + > arch/x86/entry/syscalls/syscall_64.tbl | 1 + > arch/xtensa/kernel/syscalls/syscall.tbl | 1 + > include/linux/fs.h | 3 + > include/linux/swap.h | 1 + > include/linux/syscalls.h | 3 + > include/uapi/asm-generic/unistd.h | 5 +- > include/uapi/linux/mman.h | 9 + > init/Kconfig | 10 + > kernel/sys_ni.c | 1 + > mm/filemap.c | 143 ++++++++++ > mm/workingset.c | 129 ++++++--- > tools/testing/selftests/Makefile | 1 + > tools/testing/selftests/cachestat/.gitignore | 2 + > tools/testing/selftests/cachestat/Makefile | 8 + > .../selftests/cachestat/test_cachestat.c | 259 ++++++++++++++++++ > 27 files changed, 555 insertions(+), 39 deletions(-) > create mode 100644 tools/testing/selftests/cachestat/.gitignore > create mode 100644 tools/testing/selftests/cachestat/Makefile > create mode 100644 tools/testing/selftests/cachestat/test_cachestat.c > > > base-commit: 1440f576022887004f719883acb094e7e0dd4944 > prerequisite-patch-id: 171a43d333e1b267ce14188a5beaea2f313787fb > -- > 2.30.2 Oops, I think I accidentally sent out v5 twice :( Please ignore the first set of emails and review this one only, LKML link for convenience: https://lkml.org/lkml/2023/1/4/1095