From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3290EB64D7 for ; Tue, 20 Jun 2023 23:52:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3036C8D0002; Tue, 20 Jun 2023 19:52:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2B3768D0001; Tue, 20 Jun 2023 19:52:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 12D608D0002; Tue, 20 Jun 2023 19:52:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 01EF88D0001 for ; Tue, 20 Jun 2023 19:52:42 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id B6D45140552 for ; Tue, 20 Jun 2023 23:52:42 +0000 (UTC) X-FDA: 80924778564.20.D598E26 Received: from mail-qv1-f47.google.com (mail-qv1-f47.google.com [209.85.219.47]) by imf06.hostedemail.com (Postfix) with ESMTP id B265F180013 for ; Tue, 20 Jun 2023 23:52:40 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=eHDK8AuL; dmarc=none; spf=pass (imf06.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.219.47 as permitted sender) smtp.mailfrom=jgg@ziepe.ca ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687305160; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Vb4/4UZZyVYXRpnqB4ac9M1vy7WiRZuGJxfCVEQQGnA=; b=ZlpY9D3G2PTBacKBr6vYTHL+6XdYUr3dCkeSEpu/QzpBJv2DQmtZQyyGHePJI2w9XRkCH5 bg4xHLyU+AgxhivWjoSuFuemqif14iUGsCpLfT4KUNqmGFUMoZdzLJKJCqFdoe7tqeit+6 d+PHVlUZlP6TdjblR9hIJlQFfROBuiI= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=eHDK8AuL; dmarc=none; spf=pass (imf06.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.219.47 as permitted sender) smtp.mailfrom=jgg@ziepe.ca ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687305160; a=rsa-sha256; cv=none; b=7V9t5cbsF3BkbugpRk1iGdcsoiOyaT4wonac/9z4qP86PSN68ae1smvZM5eiQlsp9LcAc0 /754R+PPYgM1EA0hQfbC4taABgFAuVc//VX2nx1lq9TsTp5TUvmnlgDWr4tNWZpihQai37 bHvEmpy99NCiZn4WGzpTgaqGZLcXJ60= Received: by mail-qv1-f47.google.com with SMTP id 6a1803df08f44-62ffc10180aso41414286d6.3 for ; Tue, 20 Jun 2023 16:52:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1687305160; x=1689897160; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Vb4/4UZZyVYXRpnqB4ac9M1vy7WiRZuGJxfCVEQQGnA=; b=eHDK8AuLxbgGjPONohhcJU+nFgRy3VUDmlsbmajZr4D1UWA3e8AuMYXaeciUgBkep8 f6bKXam0Jvh8Fg7MhXn+MFgX3auRLfaEe2gd6saoiplTzPSIaF+kI0Qk0wgOmmfVZKHs 6JKEGXzCCBK6/+fpaJ1IjsLnhvPkkSeY2cwgGMmWbML9jCxP4UGazGmT4xxDCwrd5oUv XT0f1oc61zdfM88/KTJtP2UPYpFj8Ale9ODjFUxpR8mMv4Y8u2pkZdAgw3PtmINxiS7a nLJfeWKVJl0lroNewiTzHETr0Qxsvmv2HDsbeeYrg9e06SzsTsZ5fe+Ezg8CBNEOt3oG oN9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687305160; x=1689897160; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Vb4/4UZZyVYXRpnqB4ac9M1vy7WiRZuGJxfCVEQQGnA=; b=ZJnvFo+c3+02sCiwwZeWNdX+StRwK0wCB9RU1HzEnEdcwzb/4v0himR21MCKMVDTb5 grzzYhs9JWcz28Adc3cxrlwFrgqck0utnPlOKSPY/unBWmwbxrqhd5+hULwAT+C5LTjO yjl0JrEGNImn5KrRUkDCHIi+lBQ4G726LOpn2E6WfMBVQfus1uOO0kSeywpKENiJElmp Edni0t223OLal+mXwiblPvpeJtnCORKeel2/3J+QEiM/TL/x0vG+/J0WfwCVTDrhGzQ+ H/eqlJewifAahr65ORPz8cI2H9TFlToSSN6O9qyBXAX28rHo2AyXAdUbSL8ZXkdwfCu0 GUSw== X-Gm-Message-State: AC+VfDxrQ5KdM8UdBrc1qgCB5GETX/lepTnrxxymeu/7caiH2ECGxyXZ mW6wYcAIWN8+nDAG8kWfXgSRZw== X-Google-Smtp-Source: ACHHUZ541VkZsfkII91BiSyuAKxfiQdFgXBN5W+JWnHcdBPEKYx5+92MrYfFYTfliHRuIvOLaZ9Cyg== X-Received: by 2002:ad4:5bae:0:b0:62d:ddeb:3781 with SMTP id 14-20020ad45bae000000b0062dddeb3781mr16704999qvq.20.1687305159707; Tue, 20 Jun 2023 16:52:39 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-25-194.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.25.194]) by smtp.gmail.com with ESMTPSA id jo30-20020a056214501e00b006301819be40sm1816704qvb.49.2023.06.20.16.52.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Jun 2023 16:52:39 -0700 (PDT) Received: from jgg by wakko with local (Exim 4.95) (envelope-from ) id 1qBl9K-007Vog-KI; Tue, 20 Jun 2023 20:52:38 -0300 Date: Tue, 20 Jun 2023 20:52:38 -0300 From: Jason Gunthorpe To: Hugh Dickins Cc: Andrew Morton , Gerald Schaefer , Vasily Gorbik , Mike Kravetz , Mike Rapoport , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Suren Baghdasaryan , Qi Zheng , Yang Shi , Mel Gorman , Peter Xu , Peter Zijlstra , Will Deacon , Yu Zhao , Alistair Popple , Ralph Campbell , Ira Weiny , Steven Price , SeongJae Park , Lorenzo Stoakes , Huang Ying , Naoya Horiguchi , Christophe Leroy , Zack Rusin , Axel Rasmussen , Anshuman Khandual , Pasha Tatashin , Miaohe Lin , Minchan Kim , Christoph Hellwig , Song Liu , Thomas Hellstrom , Russell King , "David Sc. Miller" , Michael Ellerman , "Aneesh Kumar K.V" , Heiko Carstens , Christian Borntraeger , Claudio Imbrenda , Alexander Gordeev , Jann Horn , Vishal Moola , Vlastimil Babka , linux-arm-kernel@lists.infradead.org, sparclinux@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v2 05/12] powerpc: add pte_free_defer() for pgtables sharing page Message-ID: References: <54cb04f-3762-987f-8294-91dafd8ebfb0@google.com> <5cd9f442-61da-4c3d-eca-b7f44d22aa5f@google.com> <2ad8b6cf-692a-ff89-ecc-586c20c5e07f@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2ad8b6cf-692a-ff89-ecc-586c20c5e07f@google.com> X-Rspamd-Queue-Id: B265F180013 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 5ehngmx3kar7drtakufb5cr3nptk5jea X-HE-Tag: 1687305160-477621 X-HE-Meta: U2FsdGVkX18fVCV+nS+J/NzrYKjf7I0UTvAyHoIu+tfX/LL77OZ4DetDLUxOLlumLRQJY1pu7opRoV4Ovg+rB+IjEaz75g94hhf0ETIyPkHN31mmUa6etNnjNcPVGRXQ9PfRrvsMiAKCfQ8Sjh3ES/0dfenbm1H4xApjDCp0oN1pFi7w0gmzJpXkl7Bp8qh6knjr0VJGKzepdwGO/6SPpKsyRh3mMIlEY8aivi3VeL2JgNw3N/mF7419N4vSzavPQtgWG0tKw0pvEPOr8hDWl1uBp3Wjraf0Ms3shAWCzcu+2vpjndoww+nTMmmOi99bS+nmKOo3kw0X2k6obmD9G2wrBBaYL0iZ/pw61T5RJhJuHrBsPL73v5WNWjMMkQn3aXSwUQdi3SrWpFmp0J46dgQsrr5Vy+VijPoMUC2tNXGtOPJptOWVj70tE5YSBZ4BDoyjqygRW6XWscKlf8ya71c+J8uyG5+HKLNmOeab+STV2oI1zIJGjQLUYtTvh7k79EPPHwfP570+gPwp98mAMeks1cPrGFLaA2NNDQEdsfWaFUegUJaI7wrpPge5ODED4fE2FURoDbCHJ9TzQANlbKwYQXuQtPUODEkxn6igVBB/syXMbcwpWEbNYig5XUOCB+jr+6KT+J5Gymt2ar+slmStsvILO+J1Vf4nesXZL3IQWqXm/Dg8DcnuI9gWzx7rcXpVzLZ5YZTHjh1pQRCND2WQrnhZ6IBbZ3IFGg0osELCcSkLSSJ5Bbf8ZjHcOev+jiVLTzhWURoQbItj75JqFOse/XZvP4yCY+7E5KctWFYNDPEK10E9QAEEeEWG4hWzW+hM5yDfPmE9SnRE0zwRiaMYntT/JdgsRzXTNsca1KLWRwl0RnwkghyK4yF08Nw1sRowfX1x69W8IHMBRxMq2Mts5IpMz334tS8TbBCZawV1NqqAylErvWmXka4wItx44sBYOMLXk/lC1KQh599 bOB/404S 0lRNd0QvImr4vSfWAYgNBF5yT+P4BJRQgB4/L4NPcOBOrPJ9x7X68cCa1eUG0k7OcfbJbgh7FRi8yBSvZa4juu5nv2XFFcQzizoojOu5W1boOquSPobMt2XoNhuvrimYnKj9BLUc552K27DFjbKb+tOsuGCKSEsHYm5kLQ9HRTw+Gchv5zlICGkyvtAMlvRhSLuPiajRjBC+ED2ZB7QrV2en/XD5hBOkKV7FqNYtfjCIfqpWxQmGbmuk6wQuRtPX4MIQHhYSqexWJr6lrI34kE2Fqhd60zjhi8jecFQaMC+nluX8bZ8IAETCWU+DaKJlozHViIGc3GO7XSwUldNsDyx9ikZyTgVvO6bikEdVFMv7GV9BAmxaiuAwbo40qfPhoELt6cflzNodspKEt526XLWnQmWloXDqpMXtL4LKFMQpEesk+ClOyUx6rnf4RB9QoJV54HgG08y+8UP+c8NxZ/UzFgdtTivfU/hX3Y+9Uo+HlbEEryNY6g6xy5WimrfxZCZ4NB7z6omUUP+g= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jun 20, 2023 at 12:54:25PM -0700, Hugh Dickins wrote: > On Tue, 20 Jun 2023, Jason Gunthorpe wrote: > > On Tue, Jun 20, 2023 at 12:47:54AM -0700, Hugh Dickins wrote: > > > Add powerpc-specific pte_free_defer(), to call pte_free() via call_rcu(). > > > pte_free_defer() will be called inside khugepaged's retract_page_tables() > > > loop, where allocating extra memory cannot be relied upon. This precedes > > > the generic version to avoid build breakage from incompatible pgtable_t. > > > > > > This is awkward because the struct page contains only one rcu_head, but > > > that page may be shared between PTE_FRAG_NR pagetables, each wanting to > > > use the rcu_head at the same time: account concurrent deferrals with a > > > heightened refcount, only the first making use of the rcu_head, but > > > re-deferring if more deferrals arrived during its grace period. > > > > You didn't answer my question why we can't just move the rcu to the > > actual free page? > > I thought that I had answered it, perhaps not to your satisfaction: > > https://lore.kernel.org/linux-mm/9130acb-193-6fdd-f8df-75766e663978@google.com/ > > My conclusion then was: > Not very good reasons: good enough, or can you supply a better patch? Oh, I guess I didn't read that email as answering the question.. I was saying to make pte_fragment_free() unconditionally do the RCU. It is the only thing that uses the page->rcu_head, and it means PPC would double RCU the final free on the TLB path, but that is probably OK for now. This means pte_free_defer() won't do anything special on PPC as PPC will always RCU free these things, this address the defer concern too, I think. Overall it is easier to reason about. I looked at fixing the TLB stuff to avoid the double rcu but quickly got scared that ppc was using a kmem_cache to allocate other page table sizes so there is not a reliable struct page to get a rcu_head from. This looks like the main challenge for ppc... We'd have to teach the tlb code to not do its own RCU stuff for table levels that the arch is already RCU freeing - and that won't get us to full RCU freeing on PPC. Anyhow, this is a full version of what I was thinking: diff --git a/arch/powerpc/mm/pgtable-frag.c b/arch/powerpc/mm/pgtable-frag.c index 20652daa1d7e3a..b5dcd0f27fc115 100644 --- a/arch/powerpc/mm/pgtable-frag.c +++ b/arch/powerpc/mm/pgtable-frag.c @@ -106,6 +106,21 @@ pte_t *pte_fragment_alloc(struct mm_struct *mm, int kernel) return __alloc_for_ptecache(mm, kernel); } +static void pgtable_free_cb(struct rcu_head *head) +{ + struct page *page = container_of(head, struct page, rcu_head); + + pgtable_pte_page_dtor(page); + __free_page(page); +} + +static void pgtable_free_cb_kernel(struct rcu_head *head) +{ + struct page *page = container_of(head, struct page, rcu_head); + + __free_page(page); +} + void pte_fragment_free(unsigned long *table, int kernel) { struct page *page = virt_to_page(table); @@ -115,8 +130,13 @@ void pte_fragment_free(unsigned long *table, int kernel) BUG_ON(atomic_read(&page->pt_frag_refcount) <= 0); if (atomic_dec_and_test(&page->pt_frag_refcount)) { + /* + * Always RCU free pagetable memory. rcu_head overlaps with lru + * which is no longer in use by the time the table is freed. + */ if (!kernel) - pgtable_pte_page_dtor(page); - __free_page(page); + call_rcu(&page->rcu_head, pgtable_free_cb); + else + call_rcu(&page->rcu_head, pgtable_free_cb_kernel); } }