From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2A87C77B7C for ; Thu, 4 May 2023 18:07:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5767D6B007B; Thu, 4 May 2023 14:07:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 526336B007D; Thu, 4 May 2023 14:07:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 415876B007E; Thu, 4 May 2023 14:07:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from mail-qv1-f50.google.com (mail-qv1-f50.google.com [209.85.219.50]) by kanga.kvack.org (Postfix) with ESMTP id 1F94B6B007B for ; Thu, 4 May 2023 14:07:03 -0400 (EDT) Received: by mail-qv1-f50.google.com with SMTP id 6a1803df08f44-61b5a6865dfso6679476d6.3 for ; Thu, 04 May 2023 11:07:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1683223622; x=1685815622; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=V535/fMfcjeqLiO2/vab+SRrcLiLBAaen1OstfXBeK8=; b=ZmkM8UcRuviiMSd7Q8VM/zaA3XhH1eQ52DzTlNTkbYGYPSMfQT+BcyIRlS6e9FyYf7 2iAdw29CgKH6JHvEq9j7HF0ArgFb0AViqHKpYMRkrSXKs3VNl/vivRBBdeRWU7rnshMt DLviYcsJBnIVtlSJnW/X91bwcbujOCiVjBUhKQXZBV8LgQPDrp0iKz6e6m//U7H2eUK7 d3026LWjbVuYqrsFAlDznctGJl0/SzIBtwcADaUxafkeAwLU/1cTB8SEu+DDIZxoPK/f sWfVWOw9ABvWwk/CcuMDcv7XPp3brZeWZIPyKMTDwP8JAXHj+Q/3rTUKFS2QL84QgKQp k9eA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683223622; x=1685815622; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=V535/fMfcjeqLiO2/vab+SRrcLiLBAaen1OstfXBeK8=; b=Q0+wcSkrvXmenpT/cppyLf0LNScF6ftL3L0HEuABTPbSNnLODrW3RSQTaIeCP/Uyf3 csFW1kco3o2ohwMghcIOqLhdzwB+0C3OaOU5E12VzrsdmAmfoSno7wCTaEkw9yCPPAkW wzkA66IDZloWqDfMBkD/S28PClaR/TEKpwJqVfPDEFSpLKzzjlKJzaw7y/mejVUPQvCT ltpD36BILxYhL1O2mdZWl/6JgYEJyvT6FS4ErlSt649OtbdTFQB/GEPiyhF/rBbxwZL1 RlaYtsoEskOP1li9ofF8BOoFG2MiLiA1XBJ2wYKAm/1TTkKyVv4vsmMyshPi39+/rY4h YbtQ== X-Gm-Message-State: AC+VfDzNugHicu3Lr/88gt6PRMJ4KFUXnSMA/DZKZh5kdviDLu4YnwG1 KIJz6QIrwlSAiQw5ToOpmrip/lL+8Wyy6yrjpW0= X-Google-Smtp-Source: ACHHUZ4diJ2WJC49P78EhjIm9s71+L2giKPw2hIr18KuA4zbtBNSjYIf+qmkj0b5DqNSh8DaUowYRWzsY035Obqzo1k= X-Received: by 2002:ad4:5bcc:0:b0:5ac:fb9a:67a1 with SMTP id t12-20020ad45bcc000000b005acfb9a67a1mr14751083qvt.47.1683223622413; Thu, 04 May 2023 11:07:02 -0700 (PDT) MIME-Version: 1.0 References: <20230503013608.2431726-1-nphamcs@gmail.com> <20230503013608.2431726-3-nphamcs@gmail.com> In-Reply-To: From: Nhat Pham Date: Thu, 4 May 2023 11:06:51 -0700 Message-ID: Subject: Re: [PATCH v13 2/3] cachestat: implement cachestat syscall To: Geert Uytterhoeven Cc: akpm@linux-foundation.org, hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, bfoster@redhat.com, willy@infradead.org, linux-api@vger.kernel.org, kernel-team@meta.com, linux-arch@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, May 4, 2023 at 10:26=E2=80=AFAM Geert Uytterhoeven wrote: > > Hi Nhat, > > On Wed, May 3, 2023 at 3:38=E2=80=AFAM Nhat Pham wrot= e: > > There is currently no good way to query the page cache state of large > > file sets and directory trees. There is mincore(), but it scales poorly= : > > the kernel writes out a lot of bitmap data that userspace has to > > aggregate, when the user really doesn not care about per-page > > information in that case. The user also needs to mmap and unmap each > > file as it goes along, which can be quite slow as well. > > > > Some use cases where this information could come in handy: > > * Allowing database to decide whether to perform an index scan or > > direct table queries based on the in-memory cache state of the > > index. > > * Visibility into the writeback algorithm, for performance issues > > diagnostic. > > * Workload-aware writeback pacing: estimating IO fulfilled by page > > cache (and IO to be done) within a range of a file, allowing for > > more frequent syncing when and where there is IO capacity, and > > batching when there is not. > > * Computing memory usage of large files/directory trees, analogous to > > the du tool for disk usage. > > > > More information about these use cases could be found in the following > > thread: > > > > https://lore.kernel.org/lkml/20230315170934.GA97793@cmpxchg.org/ > > > > This patch implements a new syscall that queries cache state of a file > > and summarizes the number of cached pages, number of dirty pages, numbe= r > > of pages marked for writeback, number of (recently) evicted pages, etc. > > in a given range. Currently, the syscall is only wired in for x86 > > architecture. > > > > NAME > > cachestat - query the page cache statistics of a file. > > > > SYNOPSIS > > #include > > > > struct cachestat_range { > > __u64 off; > > __u64 len; > > }; > > > > struct cachestat { > > __u64 nr_cache; > > __u64 nr_dirty; > > __u64 nr_writeback; > > __u64 nr_evicted; > > __u64 nr_recently_evicted; > > }; > > > > int cachestat(unsigned int fd, struct cachestat_range *cstat_range, > > struct cachestat *cstat, unsigned int flags); > > > > DESCRIPTION > > cachestat() queries the number of cached pages, number of dirty > > pages, number of pages marked for writeback, number of evicted > > pages, number of recently evicted pages, in the bytes range given b= y > > `off` and `len`. > > > > An evicted page is a page that is previously in the page cache but > > has been evicted since. A page is recently evicted if its last > > eviction was recent enough that its reentry to the cache would > > indicate that it is actively being used by the system, and that > > there is memory pressure on the system. > > > > These values are returned in a cachestat struct, whose address is > > given by the `cstat` argument. > > > > The `off` and `len` arguments must be non-negative integers. If > > `len` > 0, the queried range is [`off`, `off` + `len`]. If `len` = =3D=3D > > 0, we will query in the range from `off` to the end of the file. > > > > The `flags` argument is unused for now, but is included for future > > extensibility. User should pass 0 (i.e no flag specified). > > > > Currently, hugetlbfs is not supported. > > > > Because the status of a page can change after cachestat() checks it > > but before it returns to the application, the returned values may > > contain stale information. > > > > RETURN VALUE > > On success, cachestat returns 0. On error, -1 is returned, and errn= o > > is set to indicate the error. > > > > ERRORS > > EFAULT cstat or cstat_args points to an invalid address. > > > > EINVAL invalid flags. > > > > EBADF invalid file descriptor. > > > > EOPNOTSUPP file descriptor is of a hugetlbfs file > > > > Signed-off-by: Nhat Pham > > --- > > arch/x86/entry/syscalls/syscall_32.tbl | 1 + > > arch/x86/entry/syscalls/syscall_64.tbl | 1 + > > This should be wired up on each and every architecture. > Currently we're getting > > :1567:2: warning: #warning syscall cachestat not implemented [= -Wcpp] > > in linux-next for all the missing architectures. Hi Geert, I saw that there are several instances where we have separate patches to wire up a syscall to these architectures, so I was doing something similar. For e.g: ARM: wire up process_vm_writev and process_vm_readv syscalls (e5489847d6fc0ff176048b6e1cf5034507bf703a) MIPS: Hook up process_vm_readv and process_vm_writev system calls. (8ff8584e51d4d3fbe08ede413c4a221223766323) As for these non-x86 architecture wiring patches, I can give it a shot and cross-compile to see if it builds, but I have limited abilities for runtime tests as I don't have machines with these architectures. I would really appreciate it if there are arch people that could help wire it up. (cc-ing linux-arch as well) > > Gr{oetje,eeting}s, > > Geert > > -- > Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m6= 8k.org > > In personal conversations with technical people, I call myself a hacker. = But > when I'm talking to journalists I just say "programmer" or something like= that. > -- Linus Torvalds