From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCCD3C433F5 for ; Wed, 29 Sep 2021 18:31:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 42DC161505 for ; Wed, 29 Sep 2021 18:31:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 42DC161505 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 9BBA794004E; Wed, 29 Sep 2021 14:31:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 943DE94003A; Wed, 29 Sep 2021 14:31:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7E59594004E; Wed, 29 Sep 2021 14:31:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0030.hostedemail.com [216.40.44.30]) by kanga.kvack.org (Postfix) with ESMTP id 6C56394003A for ; Wed, 29 Sep 2021 14:31:29 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 171C8183C7647 for ; Wed, 29 Sep 2021 18:31:29 +0000 (UTC) X-FDA: 78641453898.24.F042ACF Received: from mail-pg1-f180.google.com (mail-pg1-f180.google.com [209.85.215.180]) by imf04.hostedemail.com (Postfix) with ESMTP id B889E500032B for ; Wed, 29 Sep 2021 18:31:28 +0000 (UTC) Received: by mail-pg1-f180.google.com with SMTP id s75so3624893pgs.5 for ; Wed, 29 Sep 2021 11:31:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=EDxWKrkBNuwtscCUnWnrQX4z1oFAsXag7E5OmQf8Q0Y=; b=BPOF37dk+fGPzpRboMeMfINxNYQelW6JxCrXGIklsY6hPBFAJe9fT2GA3DVXyjNtny ipg045baJHE0iOJJHykXUcRSiDc8t1rXzLJC0IuD+wt+Mg1zoWRvSOy9AKwfR2ikq0fD zkQ+aSKC9CSRu6/yJwYkw5S/Fr1/y+aaeTNX2cbUACJK5JQVYn+B2OR1AEJV59d8n6P/ /AM3gZaBU4zt3Uk1LUpxommmIoLQUQlbqJxJ3ErtWEIn2umyXdXbS3yYdIi33vom0KhX kyYLPFR8sJppnhNbUE8KLwtfFO3Spu4FNjkY8RYqBcNYitj0vh8xyDKxOBINt5Nny/9i lXWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=EDxWKrkBNuwtscCUnWnrQX4z1oFAsXag7E5OmQf8Q0Y=; b=ddEPz+ZHbFOoZ7k02YT1ogpdyHI5p74oIDNo9gbNrOxn3N0uTNqZLd5itFTz7G46TE P1wf2r8aI+fL4Em3v2+JIBtXGGRcUkGPo+/mQZ5rAabDUNCjsXzGHUsVG2HmUmDFkp81 mXnStoyWlldExnLk6VU9TXfZ/WtJ0Rjfaf2t5AJDzrHlPljJ8NF/D/2Cr4Upu5UxDFJi 2p27kmX2bv3TjxKV8oVqhUAYGnJU2QOmup3lrZY3+LkZ8v4FkdRcp+LIINf/GD8vJzk0 0QhgMj0NNnlDyte/mRZRJKv24bebBtW8gYfrOmc14WEaLTsGCTgrHLNRP8YYyl578Icg 6CzA== X-Gm-Message-State: AOAM530X1ST+lXhtxOW7FxEpJxydiv0oCDA2RYfM2j4OvwL9mtZJPE15 6zNZ410UjHuplfF4+1simx0= X-Google-Smtp-Source: ABdhPJyCqaL3em1P1e20gZVMndRpFUW6agFVdrsWYmXSByskDXDqy1BM+fIb9uyCzX0XFrjx7X3a0g== X-Received: by 2002:a63:f4b:: with SMTP id 11mr1176984pgp.189.1632940287330; Wed, 29 Sep 2021 11:31:27 -0700 (PDT) Received: from smtpclient.apple (c-24-6-216-183.hsd1.ca.comcast.net. [24.6.216.183]) by smtp.gmail.com with ESMTPSA id z10sm470758pfn.70.2021.09.29.11.31.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Sep 2021 11:31:26 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: [RFC PATCH 0/8] mm/madvise: support process_madvise(MADV_DONTNEED) From: Nadav Amit In-Reply-To: Date: Wed, 29 Sep 2021 11:31:25 -0700 Cc: David Hildenbrand , Andrew Morton , Linux-MM , Linux Kernel Mailing List , Peter Xu , Andrea Arcangeli , Minchan Kim , Colin Cross , Suren Baghdasarya , Mike Rapoport Content-Transfer-Encoding: quoted-printable Message-Id: References: <20210926161259.238054-1-namit@vmware.com> <7ce823c8-cfbf-cc59-9fc7-9aa3a79740c3@redhat.com> <6E8A03DD-175F-4A21-BCD7-383D61344521@gmail.com> <2753a311-4d5f-8bc5-ce6f-10063e3c6167@redhat.com> <0FC3F99A-9F77-484A-899B-EDCBEFBFAC5D@gmail.com> To: Michal Hocko X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: B889E500032B X-Stat-Signature: xftdjbu3ufyoimaoadoignhuptbcm753 Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=BPOF37dk; spf=pass (imf04.hostedemail.com: domain of nadav.amit@gmail.com designates 209.85.215.180 as permitted sender) smtp.mailfrom=nadav.amit@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1632940288-645661 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > On Sep 29, 2021, at 12:52 AM, Michal Hocko wrote: >=20 > On Mon 27-09-21 12:12:46, Nadav Amit wrote: >>=20 >>> On Sep 27, 2021, at 5:16 AM, Michal Hocko wrote: >>>=20 >>> On Mon 27-09-21 05:00:11, Nadav Amit wrote: >>> [...] >>>> The manager is notified on memory regions that it should monitor >>>> (through PTRACE/LD_PRELOAD/explicit-API). It then monitors these = regions >>>> using the remote-userfaultfd that you saw on the second thread. = When it wants >>>> to reclaim (anonymous) memory, it: >>>>=20 >>>> 1. Uses UFFD-WP to protect that memory (and for this matter I got a = vectored >>>> UFFD-WP to do so efficiently, a patch which I did not send yet). >>>> 2. Calls process_vm_readv() to read that memory of that process. >>>> 3. Write it back to =E2=80=9Cswap=E2=80=9D. >>>> 4. Calls process_madvise(MADV_DONTNEED) to zap it. >>>=20 >>> Why cannot you use MADV_PAGEOUT/MADV_COLD for this usecase? >>=20 >> Providing hints to the kernel takes you so far to a certain extent. >> The kernel does not want to (for a good reason) to be completely >> configurable when it comes to reclaim and prefetch policies. Doing >> so from userspace allows you to be fully configurable. >=20 > I am sorry but I do not follow. Your scenario is describing a user > space driven reclaim. Something that MADV_{COLD,PAGEOUT} have been > designed for. What are you missing in the existing functionality? Using MADV_COLD/MADV_PAGEOUT does not allow userspace to control many aspects of paging out memory: 1. Writeback: writeback ahead of time, dynamic clustering, etc. 2. Batching (regardless, MADV_PAGEOUT does pretty bad batching job on non-contiguous memory). 3. No guarantee the page is actually reclaimed (e.g., writeback) and the time it takes place. 4. I/O stack for swapping - you must use kernel I/O stack (FUSE as non-performant as it is cannot be used for swap AFAIK). 5. Other operations (e.g., locking, working set tracking) that might not be necessary or interfere. In addition, the use of MADV_COLD/MADV_PAGEOUT prevents the use of userfaultfd to trap page-faults and react accordingly, so you are also prevented from: 6. Having your own custom prefetching policy in response to #PF. There are additional use-cases I can try to formalize in which MADV_COLD/MADV_PAGEOUT is insufficient. But the main difference is pretty clear, I think: one is a hint that only applied to page reclamation. The other enables the direct control of userspace over (almost) all aspects of paging. As I suggested before, if it is preferred, this can be a UFFD IOCTL instead of process_madvise() behavior, thereby lowering the risk of a misuse. I would emphasize that this feature (i.e.,=20 process_madvise(MADV_DONTNEED) or a similar new UFFD feature) has little to no effect on the kernel robustness, complexity, security or API changes. So the impact on the kernel is negligible.