From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ADBE1C433EF for ; Wed, 13 Jul 2022 14:01:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 02293940137; Wed, 13 Jul 2022 10:01:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EEEBA940134; Wed, 13 Jul 2022 10:01:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D6936940137; Wed, 13 Jul 2022 10:01:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id C0195940134 for ; Wed, 13 Jul 2022 10:01:10 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 72A6434DBC for ; Wed, 13 Jul 2022 14:01:10 +0000 (UTC) X-FDA: 79682238300.23.267F5F9 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf13.hostedemail.com (Postfix) with ESMTP id 9FAA2200DD for ; Wed, 13 Jul 2022 14:01:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1657720866; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9KGqf/qeI+POrrn3TKUE5pxqC/C4LValvbrjKc6vdrc=; b=PChRLYnq2MSFzjgfE63YYjSQYlsK+8GwoXoC3PcTrCQ+sNnuF+635o0D2CLpIphgHKTXMx f0AJmW9j+lJBRSaJg/FROGIU3Erh3DTkIj18X3otblt2y4DU5rg9juhH60i+khVw5C4bnZ p3WZYrhankdm1mFArwEPC8SYJHGU48o= Received: from mail-ej1-f69.google.com (mail-ej1-f69.google.com [209.85.218.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-13-Ejk4bpY6MWuKNgxYy6xCyQ-1; Wed, 13 Jul 2022 10:01:03 -0400 X-MC-Unique: Ejk4bpY6MWuKNgxYy6xCyQ-1 Received: by mail-ej1-f69.google.com with SMTP id hs16-20020a1709073e9000b0072b73a28465so2377031ejc.17 for ; Wed, 13 Jul 2022 07:01:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent :content-language:to:cc:references:from:organization:subject :in-reply-to:content-transfer-encoding; bh=9KGqf/qeI+POrrn3TKUE5pxqC/C4LValvbrjKc6vdrc=; b=6Bs0ua1qz9s6AUyeg9FdVtZC3TKJfIWGN7wGiWF6yFOcvKsneBrlJ6or29Pa0V4s1/ i/qhVy5R61VGi8uL+DbSiiyJtzgQfrulVJAD5gy5UXhglLtmOwRMJJANCNcvFZpvXVD8 gSXPQ/s8sebv3oMUbTLstaLKhg2Zi/gOkm71EfB+wTBI5B1Tz8TmJVZZbgvbYVnpb77X xGlQspvxflw43hzHh6I7YEfmpPW/+k4YZmm708+LaHEmu0Uk5D6vK1WWzHquAmRatt/J VzQHy83LCrQSIp9kNgpxqLF8wZ0cZBzhD6LOUAm3llwmmsQ/G3FbBTMc2PVwY2XKkNy3 QroA== X-Gm-Message-State: AJIora842tQNcm2ztFpXpy1LQpJXcN7hX5nyqLSLfzEd+FvhTo7H5ciE mznbj9lnCX+bKKJo2OygDChT+wiVsmcB2U5bGyspbwr7MKq0tfXKEXVsvTFyBhP0TLvgnK5t2y6 /IpFpUxnr5Rs= X-Received: by 2002:a05:6402:194d:b0:43a:82da:b0f3 with SMTP id f13-20020a056402194d00b0043a82dab0f3mr5153384edz.104.1657720862081; Wed, 13 Jul 2022 07:01:02 -0700 (PDT) X-Google-Smtp-Source: AGRyM1tfg+NQpq62PNkTi+9zGICpZjxIA1/V35s1b3D7GuzEpMEmnElvJd01BSGXkYV7V6mf4wtNGg== X-Received: by 2002:a05:6402:194d:b0:43a:82da:b0f3 with SMTP id f13-20020a056402194d00b0043a82dab0f3mr5153326edz.104.1657720861765; Wed, 13 Jul 2022 07:01:01 -0700 (PDT) Received: from ?IPV6:2003:cb:c707:5800:5009:e8d0:d95e:544d? (p200300cbc70758005009e8d0d95e544d.dip0.t-ipconnect.de. [2003:cb:c707:5800:5009:e8d0:d95e:544d]) by smtp.gmail.com with ESMTPSA id e2-20020a056402088200b0042dcbc3f302sm7975139edy.36.2022.07.13.07.01.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 13 Jul 2022 07:01:01 -0700 (PDT) Message-ID: <397f3cb2-1351-afcf-cd87-e8f9fb482059@redhat.com> Date: Wed, 13 Jul 2022 16:00:59 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.11.0 To: Khalid Aziz , Andrew Morton , Mike Kravetz Cc: willy@infradead.org, aneesh.kumar@linux.ibm.com, arnd@arndb.de, 21cnbao@gmail.com, corbet@lwn.net, dave.hansen@linux.intel.com, ebiederm@xmission.com, hagen@jauu.net, jack@suse.cz, keescook@chromium.org, kirill@shutemov.name, kucharsk@gmail.com, linkinjeon@kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, longpeng2@huawei.com, luto@kernel.org, markhemm@googlemail.com, pcc@google.com, rppt@kernel.org, sieberf@amazon.com, sjpark@amazon.de, surenb@google.com, tst@schoebel-theuer.de, yzaikin@google.com References: <20220701212403.77ab8139b6e1aca87fae119e@linux-foundation.org> <0864a811-53c8-a87b-a32d-d6f4c7945caa@redhat.com> <357da99d-d096-a790-31d7-ee477e37c705@oracle.com> From: David Hildenbrand Organization: Red Hat Subject: Re: [PATCH v2 0/9] Add support for shared PTEs across processes In-Reply-To: <357da99d-d096-a790-31d7-ee477e37c705@oracle.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657720867; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9KGqf/qeI+POrrn3TKUE5pxqC/C4LValvbrjKc6vdrc=; b=jLXHeDphLr8HzJqrWJjJW7PAJgXJJrFtqrLibUW8+uZZzTYzZHVycP09LdaDGXeWzPXHvN p9FFUmpw9fYtBi53/OMAYjelbVO0mzw/cmOrvt6Arg2U76zceZG03DsnfmXaTBq4XJzOWo tk6FxgKIqTfOVfuUq+3PhKppnM6UP30= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1657720867; a=rsa-sha256; cv=none; b=LxUEGbGAxOYj/CQfdofNcPTHvrsPQjF+EzVfFTVpo55ktj588+NAI2H+Ewptv+EJcqB/MN uDnbzYs7S20mmU7SV0s83pANvZBj5BcJ63PkIOabSyQLqs6vWS2gsXRFkyQQnhDFqf+Le4 7bMpOOmplfFIbysueOn8VX/45Z43mCE= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=PChRLYnq; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf13.hostedemail.com: domain of david@redhat.com has no SPF policy when checking 170.10.129.124) smtp.mailfrom=david@redhat.com X-Stat-Signature: t9m431cqoeaet9taczafps8kwjwyfoc6 X-Rspamd-Queue-Id: 9FAA2200DD X-Rspam-User: Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=PChRLYnq; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf13.hostedemail.com: domain of david@redhat.com has no SPF policy when checking 170.10.129.124) smtp.mailfrom=david@redhat.com X-Rspamd-Server: rspam05 X-HE-Tag: 1657720867-37392 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 08.07.22 21:36, Khalid Aziz wrote: > On 7/8/22 05:47, David Hildenbrand wrote: >> On 02.07.22 06:24, Andrew Morton wrote: >>> On Wed, 29 Jun 2022 16:53:51 -0600 Khalid Aziz wrote: >>> >>>> This patch series implements a mechanism in kernel to allow >>>> userspace processes to opt into sharing PTEs. It adds a new >>>> in-memory filesystem - msharefs. >>> >>> Dumb question: why do we need a new filesystem for this? Is it not >>> feasible to permit PTE sharing for mmaps of tmpfs/xfs/ext4/etc files? >>> >> >> IIRC, the general opinion at LSF/MM was that this approach at hand is >> makes people nervous and I at least am not convinced that we really want >> to have this upstream. > > Hi David, Hi Khalid, > > You are right that sharing page tables across processes feels scary, but at the same time threads already share PTEs and > this just extends that concept to processes. They share a *mm* including a consistent virtual memory layout (VMA list). Page table sharing is just a side product of that. You could even call page tables just an implementation detail to produce that consistent virtual memory layout -- described for that MM via a different data structure. > A number of people have commented on potential usefulness of this concept > and implementation. ... and a lot of people raised concerns. Yes, page table sharing to reduce memory consumption/tlb misses/... is something reasonable to have. But that doesn't require mshare, as hugetlb has proven. The design might be useful for a handful of corner (!) cases, but as the cover letter only talks about memory consumption of page tables, I'll not care about those. Once these corner cases are explained and deemed important, we might want to think of possible alternatives to explore the solution space. > There were concerns raised about being able to make this safe and reliable. > I had agreed to send a > second version of the patch incorporating feedback from last review and LSF/MM, and that is what v2 patch is about. The Okay, most of the changes I saw are related to the user interface, not to any of the actual dirty implementation-detail concerns. And the cover letter is not really clear what's actually happening under the hood and what the (IMHO) weird semantics of the design imply (as can be seen from Andrews reply). > suggestion to extend hugetlb PMD sharing was discussed briefly. Conclusion from that discussion and earlier discussion > on mailing list was hugetlb PMD sharing is built with special case code in too many places in the kernel and it is > better to replace it with something more general purpose than build even more on it. Mike can correct me if I got that > wrong. Yes, I pushed for the removal of that yet-another-hugetlb-special-stuff, and asked the honest question if we can just remove it and replace it by something generic in the future. And as I learned, we most probably cannot rip that out without affecting existing user space. Even replacing it by mshare() would degrade existing user space. So the natural thing to reduce page table consumption (again, what this cover letter talks about) for user space (semi- ?)automatically for MAP_SHARED files is to factor out what hugetlb has, and teach generic MM code to cache and reuse page tables (PTE and PMD tables should be sufficient) where suitable. For reasonably aligned mappings and mapping sizes, it shouldn't be too hard (I know, locking ...), to cache and reuse page tables attached to files -- similar to what hugetlb does, just in a generic way. We might want a mechanism to enable/disable this for specific processes and/or VMAs, but these are minor details. And that could come for free for existing user space, because page tables, and how they are handled, would just be an implementation detail. I'd be really interested into what the major roadblocks/downsides file-based page table sharing has. Because I am not convinced that a mechanism like mshare() -- that has to be explicitly implemented+used by user space -- is required for that. -- Thanks, David / dhildenb