From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81BA6C32771 for ; Wed, 28 Sep 2022 14:03:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 873886B0072; Wed, 28 Sep 2022 10:03:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 823966B0073; Wed, 28 Sep 2022 10:03:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6C3F56B0074; Wed, 28 Sep 2022 10:03:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5D59E6B0072 for ; Wed, 28 Sep 2022 10:03:31 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 9F99B416E1 for ; Wed, 28 Sep 2022 14:03:30 +0000 (UTC) X-FDA: 79961661780.06.98BBA8D Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf25.hostedemail.com (Postfix) with ESMTP id D7784A0048 for ; Wed, 28 Sep 2022 14:03:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664373806; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3g42XETLWROK5vKG8OlaJRSjBZ46PuNDTUQkdxQffSI=; b=hr5z0PB45z+7HQ7m+dqiJTo37kNj5KBP/VYm1emPvUv+LYrzbUsZNH3p7J/e0O++2B3M6S CpgHhB3uh0zsUWo2zOVoUQ8odtOD3LEb+w5Eq2uvI8A4qByOPn0Lr3/yq2IoJREDKp7fAF s6WLkLz7xyZKsBQrkCQKmMUa4/GoeTo= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664373809; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3g42XETLWROK5vKG8OlaJRSjBZ46PuNDTUQkdxQffSI=; b=CuLk8XOJ+RoOKgy21LQxIBxCigLxAqXN6e8bvboyvRAXfVCXeuYqt/xU15g0WaS+nCWMti Y17NIsys2mjiRk4O00OjKGvQjKhYmjEvFAPBNFaf13CugO3M7DW7FLwBQ2L39IacIWO/cD GfkDYwTku3F6XQvO3s5cbnvtbdedSSY= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-477-G18npiQtNpW-sNXSHIgdAQ-1; Wed, 28 Sep 2022 10:03:24 -0400 X-MC-Unique: G18npiQtNpW-sNXSHIgdAQ-1 Received: by mail-wm1-f70.google.com with SMTP id l15-20020a05600c4f0f00b003b4bec80edbso7145914wmq.9 for ; Wed, 28 Sep 2022 07:03:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:subject:organization:from :references:cc:to:content-language:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date; bh=3g42XETLWROK5vKG8OlaJRSjBZ46PuNDTUQkdxQffSI=; b=tv+cFQVg5BRuJwMWvs6baNMi+fdh37TUTrfkyUUmtThIWNscLFYZohN57yaSz6plXe /01Y+A+R5b6zn3IIHa12FCbYDq0jaE3WO/mselsM0OzOriS/brGdE4SOo1BrrE4/RAH0 JmymjziExvXE7Hgj2fEjq5aPnkHAfSIEuCdzmV/O3UoZM9dDHS7LLubzaMEUOe4Ffua/ bZ0ZKa7tsgRyu9gbEmyb6YmOVM2cAPtUjcsr4S+ORD4kgnTY8ZjbpWiTUb9E24avGg96 cdmlDEPz40KOUyrWX7OA19NLcQ9zmRbPk/p26mm2dT18aXmKntm0YgKHivFSwLYosi7N TzmA== X-Gm-Message-State: ACrzQf2SjPU8OuUeQ1vcnYaSQ0jKzLPCZazd+6w/J1yIuB51tIRC+tPI ZZxdoUGXNLqwhxl+9f8Hz2b16aVtoeaQhNfCmU+vNQeONSm5ZOAfRzyoSKK+qUtzYXNBK8CGRSk z6wTujjsCvls= X-Received: by 2002:a05:6000:1689:b0:22a:a66d:1f37 with SMTP id y9-20020a056000168900b0022aa66d1f37mr20953962wrd.197.1664373802197; Wed, 28 Sep 2022 07:03:22 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5ELLQED00nR1Z+ZvxqEmG4byd8/bnKeeNrTnLv/m+ZE8ddVj32aihv7u5mlLSmhLo8aHoyjA== X-Received: by 2002:a05:6000:1689:b0:22a:a66d:1f37 with SMTP id y9-20020a056000168900b0022aa66d1f37mr20953914wrd.197.1664373801781; Wed, 28 Sep 2022 07:03:21 -0700 (PDT) Received: from ?IPV6:2003:cb:c704:1100:add9:5f61:6b94:7540? (p200300cbc7041100add95f616b947540.dip0.t-ipconnect.de. [2003:cb:c704:1100:add9:5f61:6b94:7540]) by smtp.gmail.com with ESMTPSA id a20-20020a05600c069400b003a5fa79007fsm1744360wmn.7.2022.09.28.07.03.20 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 28 Sep 2022 07:03:21 -0700 (PDT) Message-ID: Date: Wed, 28 Sep 2022 16:03:19 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.3.0 To: Chih-En Lin , Nadav Amit Cc: Andrew Morton , Qi Zheng , Matthew Wilcox , Christophe Leroy , "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" , Luis Chamberlain , Kees Cook , Iurii Zaikin , Vlastimil Babka , William Kucharski , "Kirill A . Shutemov" , Peter Xu , Suren Baghdasaryan , Arnd Bergmann , Tong Tiangen , Pasha Tatashin , Li kunyu , Anshuman Khandual , Minchan Kim , Yang Shi , Song Liu , Miaohe Lin , Thomas Gleixner , Sebastian Andrzej Siewior , Andy Lutomirski , Fenghua Yu , Dinglan Peng , Pedro Fonseca , Jim Huang , Huichun Feng References: <20220927162957.270460-1-shiyn.lin@gmail.com> <20220927162957.270460-10-shiyn.lin@gmail.com> <3D21021E-490F-4FE0-9C75-BB3A46A66A26@vmware.com> From: David Hildenbrand Organization: Red Hat Subject: Re: [RFC PATCH v2 9/9] mm: Introduce Copy-On-Write PTE table In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664373810; a=rsa-sha256; cv=none; b=niZEli9eWDzoRZcX0+dG+v3z3IonbzaFQRkkGXTfPWM1TNiRGppMQ41IaR5qnQfH+amoKT 3oem2eI8VsWitL/cQMeeY5x/OtmDOtZwfhG8aLYGTWR1iRfm6tIupycAGf7fLUokNU/hQ0 r6Au6+zzAMrtde40bDNxo2E+wCYEl7Y= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hr5z0PB4; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=CuLk8XOJ; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf25.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664373810; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3g42XETLWROK5vKG8OlaJRSjBZ46PuNDTUQkdxQffSI=; b=JAFjhZYhM3ZnA/CcVnG2ZEuSvk6MD3LDofmIZt/ilNkXKFjQzXI3RSQi+wZVOUa4ywuqfX wvBooW+hvmaC9ay2Q7wjQLJaYDh/mwn+S58XI9y1FYsFfcavGhHxq6NAbEoaw7ltUnc/h6 uJN1Nu9cozhEz4DAqHLwGEOp8DpY4WQ= X-Stat-Signature: 5tf5o5yjkb9sy7h58qp4rftqfr6658iy X-Rspamd-Queue-Id: D7784A0048 X-Rspam-User: Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hr5z0PB4; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=CuLk8XOJ; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf25.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com X-Rspamd-Server: rspam11 X-HE-Tag: 1664373809-809843 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 27.09.22 21:53, Chih-En Lin wrote: > On Tue, Sep 27, 2022 at 06:38:05PM +0000, Nadav Amit wrote: >> I only skimmed the patches that you sent. The last couple of patches seem a >> bit rough and dirty, so I am sorry to say that I skipped them (too many >> “TODO” and “XXX” for my taste). >> >> I am sure other will have better feedback than me. I understand there is a >> tradeoff and that this mechanism is mostly for high performance >> snapshotting/forking. It would be beneficial to see whether this mechanism >> can somehow be combined with existing ones (mshare?). > > Still thanks for your feedback. :) > I'm looking at the PTE refcount and mshare patches. And, maybe it can > combine with them in the future. > >> The code itself can be improved. I found the reasoning about synchronization >> and TLB flushes and synchronizations to be lacking, and the code to seem >> potentially incorrect. Better comments would help, even if the code is >> correct. >> >> There are additional general questions. For instance, when sharing a >> page-table, do you properly update the refcount/mapcount of the mapped >> pages? And are there any possible interactions with THP? > > Since access to those mapped pages will cost a lot of time, and this > will make fork() even have more overhead. It will not update the > refcount/mapcount of the mapped pages. Oh no. So we'd have pages logically mapped into two processes (two page table structures), but the refcount/mapcount/PageAnonExclusive would not reflect that? Honestly, I don't think it is upstream material in that hacky form. No, we don't need more COW CVEs or more COW over-complications that destabilize the whole system. IMHO, a relaxed form that focuses on only the memory consumption reduction could *possibly* be accepted upstream if it's not too invasive or complex. During fork(), we'd do exactly what we used to do to PTEs (increment mapcount, refcount, trying to clear PageAnonExclusive, map the page R/O, duplicate swap entries; all while holding the page table lock), however, sharing the prepared page table with the child process using COW after we prepared it. Any (most once we want to *optimize* rmap handling) modification attempts require breaking COW -- copying the page table for the faulting process. But at that point, the PTEs are already write-protected and properly accounted (refcount/mapcount/PageAnonExclusive). Doing it that way might not require any questionable GUP hacks and swapping, MMU notifiers etc. "might just work as expected" because the accounting remains unchanged" -- we simply de-duplicate the page table itself we'd have after fork and any modification attempts simply replace the mapped copy. But devil is in the detail (page table lock, TLB flushing). "will make fork() even have more overhead" is not a good excuse for such complexity/hacks -- sure, it will make your benchmark results look better in comparison ;) -- Thanks, David / dhildenb