From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8D7AC433F5 for ; Thu, 29 Sep 2022 13:49:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 30E3C8D0002; Thu, 29 Sep 2022 09:49:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2BE868D0001; Thu, 29 Sep 2022 09:49:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 13A0C8D0002; Thu, 29 Sep 2022 09:49:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 01A208D0001 for ; Thu, 29 Sep 2022 09:49:11 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 81A8CAB702 for ; Thu, 29 Sep 2022 13:49:11 +0000 (UTC) X-FDA: 79965254502.14.FA2E3BD Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by imf27.hostedemail.com (Postfix) with ESMTP id 2D9874001C for ; Thu, 29 Sep 2022 13:49:10 +0000 (UTC) Received: by mail-pf1-f171.google.com with SMTP id v186so1489900pfv.11 for ; Thu, 29 Sep 2022 06:49:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date; bh=mo7RD9nR8Jx0PNu4c0lHK+kg2j3k5lpegLOiCypyUG0=; b=PzoqGY4fADF/2W2Kxf0freCb7sEY3NfgSM94FUPSAskhUyySetLXo/K+Yu0GV+FfnP wEysz3ZART1xXLmk6HFKQbAqpFl1XmcgBYrAhzVPCMmPSgM+7W71G5UA9xJpaEVdNAiW 7COSu3HFQncP2FBRG1HQyK78ljR+XPZgRsN0fegXtscy2LsssgkCvCgb4Db9LvEU9tQj XobcDXCcEzaNPqMLaDqU03t5L7BBUBsEhEnbAx5e8ToPOf7uoc9YtchNdWyybRm/4uWz IolnblJ+Ji4UrFkKUkmvhAViUn8YTHZipy9siqJTsvVkgq4cbQi8tQnCmLRWneCNWrJe bJzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date; bh=mo7RD9nR8Jx0PNu4c0lHK+kg2j3k5lpegLOiCypyUG0=; b=dcQN3F/Yek0zsWNJa1m8Vko2rzJJIHrbGZe9k4mBBk7VplUFXOst71UZBTgbN+kEj4 zb1XUSE8CfRE6MKRoodhjBVVB6vsYaLXk5ky3B8JAYJ42C4miIhasOUPsfc6zVz9BPwZ vkvhkia9YPnnn+2haQ55HvllwFCu+YSOsehLrQZA9iM7MVS97RT7l9dSjetZNXOh4aD6 R4BOFgs5AsRoEbhDnU3OM44ZHS1mpcAmCvSQjOaJxgeILXUESOJtIiMx04FZE4jy13Mo PK8eZBqejp46GtF+GM8XKXxRCPowkQRjj88iHMIeLXfPRbItEMlJ7RQeEmjw6cToJgFy dxbA== X-Gm-Message-State: ACrzQf01xohSzQU3kBGweKnDsANKAZdpeVdhBJkUfSSc2BsOuTKWMStJ 2hDWz1sJhayJ60CdoPjw+Kg= X-Google-Smtp-Source: AMsMyM48QS59Fz/R8dWdZ9MoDEIVOyLm0HWq8Gfy2o3siGZbVGCAX7689IjRDcTkNjRNGf4BNi9MnQ== X-Received: by 2002:a63:ec0d:0:b0:439:5dcd:2a3c with SMTP id j13-20020a63ec0d000000b004395dcd2a3cmr3089805pgh.358.1664459350017; Thu, 29 Sep 2022 06:49:10 -0700 (PDT) Received: from strix-laptop (2001-b011-20e0-1b9a-f5f9-665b-0715-9cc1.dynamic-ip6.hinet.net. [2001:b011:20e0:1b9a:f5f9:665b:715:9cc1]) by smtp.gmail.com with ESMTPSA id o19-20020a170902779300b0016ef87334aesm5924641pll.162.2022.09.29.06.49.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Sep 2022 06:49:09 -0700 (PDT) Date: Thu, 29 Sep 2022 21:49:02 +0800 From: Chih-En Lin To: David Hildenbrand Cc: Nadav Amit , Andrew Morton , Qi Zheng , Matthew Wilcox , Christophe Leroy , "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" , Luis Chamberlain , Kees Cook , Iurii Zaikin , Vlastimil Babka , William Kucharski , "Kirill A . Shutemov" , Peter Xu , Suren Baghdasaryan , Arnd Bergmann , Tong Tiangen , Pasha Tatashin , Li kunyu , Anshuman Khandual , Minchan Kim , Yang Shi , Song Liu , Miaohe Lin , Thomas Gleixner , Sebastian Andrzej Siewior , Andy Lutomirski , Fenghua Yu , Dinglan Peng , Pedro Fonseca , Jim Huang , Huichun Feng Subject: Re: [RFC PATCH v2 9/9] mm: Introduce Copy-On-Write PTE table Message-ID: References: <20220927162957.270460-1-shiyn.lin@gmail.com> <20220927162957.270460-10-shiyn.lin@gmail.com> <3D21021E-490F-4FE0-9C75-BB3A46A66A26@vmware.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=PzoqGY4f; spf=pass (imf27.hostedemail.com: domain of shiyn.lin@gmail.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=shiyn.lin@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664459351; a=rsa-sha256; cv=none; b=28RU+DdZ2knSNoxVF0oaxQTQ337k1VAqRo1VxR/+QK2mrGeqK53LJ4GW2Z6qlp/S5r68GK KvAly8X+bkgPPZ3BiqcGZxknX1BXb2ZIezefMHsU4If3Ocr2S7MKbhAIqZ8wlCzNkk+9Oc IKgfB5SRBduAWYSuQSxcR/idkli2C0I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664459351; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mo7RD9nR8Jx0PNu4c0lHK+kg2j3k5lpegLOiCypyUG0=; b=VEbD1UzXaSMTHCXG7wh+RxaSbNPXtC4gDm1fb6VhguWD7lBLPlsiG+qaMqLwiu5aGpTRvD jMFtFAoII1vl8oEE3AonDbjK5YTay1Q1iN1Y49dI2TDCyl9IGBLzBntm+o9GcWudpSQbw7 B5nITcKvYJI2YY84LsNwPVRzhxV/jtA= X-Stat-Signature: mmg8q8kg3m8cow57796k5fsfhxcs1mpg X-Rspamd-Queue-Id: 2D9874001C Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=PzoqGY4f; spf=pass (imf27.hostedemail.com: domain of shiyn.lin@gmail.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=shiyn.lin@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Rspamd-Server: rspam07 X-HE-Tag: 1664459350-340849 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Sep 29, 2022 at 09:38:53PM +0800, Chih-En Lin wrote: > Sorry for replying late. > > On Wed, Sep 28, 2022 at 04:03:19PM +0200, David Hildenbrand wrote: > > On 27.09.22 21:53, Chih-En Lin wrote: > > > On Tue, Sep 27, 2022 at 06:38:05PM +0000, Nadav Amit wrote: > > > > I only skimmed the patches that you sent. The last couple of patches seem a > > > > bit rough and dirty, so I am sorry to say that I skipped them (too many > > > > “TODO” and “XXX” for my taste). > > > > > > > > I am sure other will have better feedback than me. I understand there is a > > > > tradeoff and that this mechanism is mostly for high performance > > > > snapshotting/forking. It would be beneficial to see whether this mechanism > > > > can somehow be combined with existing ones (mshare?). > > > > > > Still thanks for your feedback. :) > > > I'm looking at the PTE refcount and mshare patches. And, maybe it can > > > combine with them in the future. > > > > > > > The code itself can be improved. I found the reasoning about synchronization > > > > and TLB flushes and synchronizations to be lacking, and the code to seem > > > > potentially incorrect. Better comments would help, even if the code is > > > > correct. > > > > > > > > There are additional general questions. For instance, when sharing a > > > > page-table, do you properly update the refcount/mapcount of the mapped > > > > pages? And are there any possible interactions with THP? > > > > > > Since access to those mapped pages will cost a lot of time, and this > > > will make fork() even have more overhead. It will not update the > > > refcount/mapcount of the mapped pages. > > > > Oh no. > > > > So we'd have pages logically mapped into two processes (two page table > > structures), but the refcount/mapcount/PageAnonExclusive would not reflect > > that? > > > > Honestly, I don't think it is upstream material in that hacky form. No, we > > don't need more COW CVEs or more COW over-complications that destabilize the > > whole system. > > > > I know setting the write protection is not enough to prove the security > safe since the previous COW CVEs are related to it. And, if skipping the > accounting to reduce the overhead of fork() is not suitable for upstream > , we can change it. But, I think COW to the table can still be an > upstream material. > > Recently the patches, like refcount for the empty user PTE page table > pages and mshare for the pages shared between the processes require more > PTE entries, showing that the modern system uses a lot of memory for the > page table (especially the PTE table). So, I think the method, COW to > the table, might reduce the memory usage for the side of the multiple > users PTE page table. Sorry, I think I need to explain more about "the multiple users PTE page table". It means that it has more than one user holding the page table and the mapped page that still has the same context. So, we can use COW to reduce the memory usage at first. > > > IMHO, a relaxed form that focuses on only the memory consumption reduction > > could *possibly* be accepted upstream if it's not too invasive or complex. > > During fork(), we'd do exactly what we used to do to PTEs (increment > > mapcount, refcount, trying to clear PageAnonExclusive, map the page R/O, > > duplicate swap entries; all while holding the page table lock), however, > > sharing the prepared page table with the child process using COW after we > > prepared it. > > > > Any (most once we want to *optimize* rmap handling) modification attempts > > require breaking COW -- copying the page table for the faulting process. But > > at that point, the PTEs are already write-protected and properly accounted > > (refcount/mapcount/PageAnonExclusive). > > > > Doing it that way might not require any questionable GUP hacks and swapping, > > MMU notifiers etc. "might just work as expected" because the accounting > > remains unchanged" -- we simply de-duplicate the page table itself we'd have > > after fork and any modification attempts simply replace the mapped copy. > > Agree. > However for GUP hacks, if we want to do the COW to page table, we still > need the hacks in this patch (using the COW_PTE_OWN_EXCLUSIVE flag to > check whether the PTE table is available or not before we do the COW to > the table). Otherwise, it will be more complicated since it might need > to handle situations like while preparing the COW work, it just figuring > out that it needs to duplicate the whole table and roll back (recover > the state and copy it to new table). Hopefully, I'm not wrong here. > > > But devil is in the detail (page table lock, TLB flushing). > > Sure, it might be an overhead in the page fault and needs to be handled > carefully. ;) > > > "will make fork() even have more overhead" is not a good excuse for such > > complexity/hacks -- sure, it will make your benchmark results look better in > > comparison ;) > > ;);) > I think that, even if we do the accounting with the COW page table, it > still has a little bit improve. > > > -- > > Thanks, > > > > David / dhildenb > > > > Thanks, > Chih-En Lin Thanks, Chih-En Lin