From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05465C433EF for ; Mon, 27 Sep 2021 12:00:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7ED6860FED for ; Mon, 27 Sep 2021 12:00:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7ED6860FED Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id D71AE900002; Mon, 27 Sep 2021 08:00:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D21106B0072; Mon, 27 Sep 2021 08:00:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC12C900002; Mon, 27 Sep 2021 08:00:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0126.hostedemail.com [216.40.44.126]) by kanga.kvack.org (Postfix) with ESMTP id AC8356B0071 for ; Mon, 27 Sep 2021 08:00:15 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 579E48249980 for ; Mon, 27 Sep 2021 12:00:15 +0000 (UTC) X-FDA: 78633210390.21.7FBC9DD Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by imf15.hostedemail.com (Postfix) with ESMTP id E9ED6D0000A8 for ; Mon, 27 Sep 2021 12:00:14 +0000 (UTC) Received: by mail-pf1-f171.google.com with SMTP id s16so15698648pfk.0 for ; Mon, 27 Sep 2021 05:00:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=bw0oRxKf3JFM9EeUXYvDRBPNH5GnO41KoUgDGV8RG1I=; b=oE63S/xPVqjabQZ6QGQMrjiB351K03/h01HVFPiII7EbrEhQHFYpKIbU5XZAlijqg5 0VAOitmfcMIUiS3kWanFSWgVZlawcSDcXsjKzVK3OkstGZgc4YbxKU10r/8+1E9keeva KMt/ecZNa1RBoltO5DtUblLdzEPu6MBYrt9HmhXASLg91L18p34IQWVPXe7NC7IYVCH3 LJrpSR4JUUt+51gfpqAhior3pGCuekKyldJCBEScwOTOLI4EMHRUntFl74Cv5gVVuJ4U Y+B8COfN5nBy2nL/soM7GOYYGi4yL84JcpE07VdeSuHiM78CsjH6+eXMM7KdzY5WVBzh wQUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=bw0oRxKf3JFM9EeUXYvDRBPNH5GnO41KoUgDGV8RG1I=; b=5J9P/nBaqDRAITRi9qKXRh2QjEeZlRYIwxthOkALkrWY0p6MXOI9qU1oCBWyJ8iOha 7BaYFInod9G0oJTezhVY4fqoqL5n54/FToFBOdrFue5TVoOp9hk78sFuq6EFLOOpOudJ JB7vqtXH3scdX+ThRitK6gfWcYeHQifgGJjruSDG76KbefxS0SVr0q04LmttJpfROtfG 6lbH6MQgU9e4izHiGEK3jtbs1dVM/oDllCfS9aH7Isp2wwsfEc9fFyn0Ee40mOzcaOO0 GNxcHNKimWyCoX5mSHoGo5nP7LMuRk8K6dIrzIexdcSdRdH/0Ynzg6E2Wcr8I7qN/XP/ Uolg== X-Gm-Message-State: AOAM530UXeyTdTVcA9cpOWVRB6/nCnj1FxzIE+pIu4BNf58QdexypOpp SVkC+fuU2B60P0oKFAE7+oc= X-Google-Smtp-Source: ABdhPJwp2tN3e6iOVTPHNSKmY4uRC0cRN2RNE+4sxxGhbCcYYyK26Twb77jiZJiMFQIGh8FoGxcH1g== X-Received: by 2002:aa7:9d02:0:b0:43d:ea96:5882 with SMTP id k2-20020aa79d02000000b0043dea965882mr22835813pfp.23.1632744013449; Mon, 27 Sep 2021 05:00:13 -0700 (PDT) Received: from smtpclient.apple (c-24-6-216-183.hsd1.ca.comcast.net. [24.6.216.183]) by smtp.gmail.com with ESMTPSA id u24sm18403484pfm.81.2021.09.27.05.00.11 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Sep 2021 05:00:13 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: [RFC PATCH 0/8] mm/madvise: support process_madvise(MADV_DONTNEED) From: Nadav Amit In-Reply-To: <2753a311-4d5f-8bc5-ce6f-10063e3c6167@redhat.com> Date: Mon, 27 Sep 2021 05:00:11 -0700 Cc: Andrew Morton , Linux-MM , Linux Kernel Mailing List , Peter Xu , Andrea Arcangeli , Minchan Kim , Colin Cross , Suren Baghdasarya , Mike Rapoport Content-Transfer-Encoding: quoted-printable Message-Id: References: <20210926161259.238054-1-namit@vmware.com> <7ce823c8-cfbf-cc59-9fc7-9aa3a79740c3@redhat.com> <6E8A03DD-175F-4A21-BCD7-383D61344521@gmail.com> <2753a311-4d5f-8bc5-ce6f-10063e3c6167@redhat.com> To: David Hildenbrand X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: E9ED6D0000A8 X-Stat-Signature: xzid9a1qxpcunujo4gbr4qjnsye9y436 Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="oE63S/xP"; spf=pass (imf15.hostedemail.com: domain of nadav.amit@gmail.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=nadav.amit@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1632744014-779399 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > On Sep 27, 2021, at 3:58 AM, David Hildenbrand = wrote: >=20 > On 27.09.21 12:41, Nadav Amit wrote: >>> On Sep 27, 2021, at 2:24 AM, David Hildenbrand = wrote: >>>=20 >>> On 26.09.21 18:12, Nadav Amit wrote: >>>> From: Nadav Amit >>>> The goal of these patches is to add support for >>>> process_madvise(MADV_DONTNEED). Yet, in the process some (arguably) >>>> useful cleanups, a bug fix and performance enhancements are = performed. >>>> The patches try to consolidate the logic across different = behaviors, and >>>> to a certain extent overlap/conflict with an outstanding patch that = does >>>> something similar [1]. This consolidation however is mostly = orthogonal >>>> to the aforementioned one and done in order to clarify what is done = in >>>> respect to locks and TLB for each behavior and to batch these = operations >>>> more efficiently on process_madvise(). >>>> process_madvise(MADV_DONTNEED) is useful for two reasons: (a) it = allows >>>> userfaultfd monitors to unmap memory from monitored processes; and = (b) >>>> it is more efficient than madvise() since it is vectored and = batches TLB >>>> flushes more aggressively. >>>=20 >>> MADV_DONTNEED on MAP_PRIVATE memory is a target-visible operation; = this is very different to all the other process_madvise() calls we = allow, which are merely hints, but the target cannot be broken . I don't = think this is acceptable. >> This is a fair point, which I expected, but did not address properly. >> I guess an additional capability, such as CAP_SYS_PTRACE needs to be >> required in this case. Would that ease your mind? >=20 > I think it would be slightly better, but I'm still missing a clear use = case that justifies messing with the page tables of other processes in = that way, especially with MAP_PRIVATE mappings. Can you maybe elaborate = a bit on a) and b)? >=20 > Especially, why would a) make sense or be required? When would it be a = good idea to zap random pages of a target process, especially with = MAP_PRIVATE? How would the target use case make sure that the target = process doesn't suddenly lose data? I would have assume that you can = really only do something sane with uffd() if 1) the process decided to = give up on some pages (madvise(DONTNEED)) b) the process hasn't touched = these pages yet. >=20 > Can you also comment a bit more on b)? Who cares about that? And would = we suddenly expect users of madvise() to switch to process_madvise() = because it's more effective? It sounds a bit weird to me TBH, but most = probably I am missing details :) Ok, ok, your criticism is fair. I tried to hold back some details in = order to prevent the discussion from digressing. I am going to focus on (a) which = is what I really have in mind. The use-case that I explore is a userspace memory manager with some = level of cooperation of the monitored processes. The manager is notified on memory regions that it should monitor (through PTRACE/LD_PRELOAD/explicit-API). It then monitors these regions using the remote-userfaultfd that you saw on the second thread. When it = wants to reclaim (anonymous) memory, it: 1. Uses UFFD-WP to protect that memory (and for this matter I got a = vectored UFFD-WP to do so efficiently, a patch which I did not send yet). 2. Calls process_vm_readv() to read that memory of that process. 3. Write it back to =E2=80=9Cswap=E2=80=9D. 4. Calls process_madvise(MADV_DONTNEED) to zap it. Once the memory is accessed again, the manager uses UFFD-COPY to bring = it back. This is really work-in-progress, but eventually performance is not = as bad as you would imagine (some patches for efficient use of uffd with iouring are needed for that matter). I am aware that there are some caveats, as zapping the memory does not guarantee that the memory would be freed since it might be pinned for a variety of reasons. That's the reason I mentioned the processes have = "some level of cooperation" with the manager. It is not intended to deal with adversaries or uncommon corner cases (e.g., processes that use UFFD for their own reasons). Putting aside my use-case (which I am sure people would be glad to = criticize), I can imagine debuggers or emulators may also find use for similar = schemes (although I do not have concrete use-cases for them).