From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0FCFEB64D7 for ; Fri, 30 Jun 2023 13:39:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1957E8E0014; Fri, 30 Jun 2023 09:39:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 11F278E000F; Fri, 30 Jun 2023 09:39:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E8C478E0014; Fri, 30 Jun 2023 09:39:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id D394F8E000F for ; Fri, 30 Jun 2023 09:39:47 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id A4233120163 for ; Fri, 30 Jun 2023 13:39:47 +0000 (UTC) X-FDA: 80959522014.05.770FE4E Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by imf09.hostedemail.com (Postfix) with ESMTP id 0D7B014000C for ; Fri, 30 Jun 2023 13:39:44 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=WWpOuUBl; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf09.hostedemail.com: domain of imbrenda@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=imbrenda@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688132385; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MILFYyEh0bzy33IA0yZToAfVYRzb2Mhl3gp0isRO4pM=; b=cb2+ZpwbsM1XDsPYSXZ3JZmTBH0Pc22MwiqeuLNIF7+MW0My7DqqkoZV7iVG5NrNwXQHFf ond4QGmlPbNyRM/xpp8MqKWTl7pBzfNXzYBkD86AjucjqqNlBukLtiuHPbr4bv7q4uBRKl KXZwpPCPFGtpZEQ5tAt1tFO9aOYyX8U= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=WWpOuUBl; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf09.hostedemail.com: domain of imbrenda@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=imbrenda@linux.ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688132385; a=rsa-sha256; cv=none; b=mvyoSy/lIclI4Z4GPGpEn4+AwkCI1VGhLDcQGWuOwq+Fxba5PEaivSLI5RBQV1/pL9inR8 1ZuVAlOyiv/8iyOMx3J147IEQza8QT/oE4NawHbuO+fylG11kz406T9A9YCq2AunKtATO7 4qRz0iVagje5msUNLuSClGev9h2Vc3Y= Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35UDXtkF024475; Fri, 30 Jun 2023 13:39:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=date : from : to : cc : subject : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=pp1; bh=MILFYyEh0bzy33IA0yZToAfVYRzb2Mhl3gp0isRO4pM=; b=WWpOuUBll0GcMgF6UkEAPv1U/0AELyMv08YfNFH5QWoGlgD30nDBbxmr3vbfSbir1zDF v1n3VFiHTI2iKdl0ajnyVN8bMun8YmP3twJ0zeA20CJqamSbFHDi0WTEGFk62XbzPkt+ 3sQv6xziupU8flvhfEio8DJeUkwPyzL/3LA2kAIq72vI2CbDVcUIXbqzP5IohDPsH4/K nxBzmnoEp2WjhyoNKxCZ0dgzhxUGvSu0SNRrFBSAmnnreNuicuHYPiLqqUjRWB61jY4q YJ78SFBrwbAkYFxbT96VF4dSAcOaIN/gb7fOJsJGxvv6qHAnXFv+w90Oi5zo4zs12bA4 wA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3rj01vr5v8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 30 Jun 2023 13:39:02 +0000 Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35UDXvp4024557; Fri, 30 Jun 2023 13:39:01 GMT Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com [169.51.49.98]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3rj01vr5rt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 30 Jun 2023 13:39:01 +0000 Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1]) by ppma03ams.nl.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35U4eiGQ004048; Fri, 30 Jun 2023 13:38:59 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma03ams.nl.ibm.com (PPS) with ESMTPS id 3rdr45463d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 30 Jun 2023 13:38:59 +0000 Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35UDctix16253646 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 30 Jun 2023 13:38:55 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6250B2004F; Fri, 30 Jun 2023 13:38:55 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C36332004E; Fri, 30 Jun 2023 13:38:53 +0000 (GMT) Received: from p-imbrenda (unknown [9.152.224.66]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 30 Jun 2023 13:38:53 +0000 (GMT) Date: Fri, 30 Jun 2023 15:38:52 +0200 From: Claudio Imbrenda To: Hugh Dickins Cc: Andrew Morton , Gerald Schaefer , Vasily Gorbik , Mike Kravetz , Mike Rapoport , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Suren Baghdasaryan , Qi Zheng , Yang Shi , Mel Gorman , Peter Xu , Peter Zijlstra , Will Deacon , Yu Zhao , Alistair Popple , Ralph Campbell , Ira Weiny , Steven Price , SeongJae Park , Lorenzo Stoakes , Huang Ying , Naoya Horiguchi , Christophe Leroy , Zack Rusin , Jason Gunthorpe , Axel Rasmussen , Anshuman Khandual , Pasha Tatashin , Miaohe Lin , Minchan Kim , Christoph Hellwig , Song Liu , Thomas Hellstrom , Russell King , "David S. Miller" , Michael Ellerman , "Aneesh Kumar K.V" , Heiko Carstens , Christian Borntraeger , Alexander Gordeev , Jann Horn , Vishal Moola , Vlastimil Babka , linux-arm-kernel@lists.infradead.org, sparclinux@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v2 07/12] s390: add pte_free_defer() for pgtables sharing page Message-ID: <20230630153852.31163592@p-imbrenda> In-Reply-To: References: <54cb04f-3762-987f-8294-91dafd8ebfb0@google.com> Organization: IBM X-Mailer: Claws Mail 4.1.1 (GTK 3.24.38; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: EBLZMsFsdvv19REZEqy1DFVn4Lc2XsWt X-Proofpoint-GUID: t0PdQmyDLcbJVGOB3mJ29fow46RM-DV8 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-30_05,2023-06-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 clxscore=1011 phishscore=0 lowpriorityscore=0 malwarescore=0 impostorscore=0 spamscore=0 mlxscore=0 adultscore=0 mlxlogscore=999 suspectscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306300116 X-Rspamd-Queue-Id: 0D7B014000C X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: p7axhjn9cdkpnuf6zdrtek73zk5cbhcu X-HE-Tag: 1688132384-314847 X-HE-Meta: U2FsdGVkX194gIl4BlAecJRJmHqEq95a2TTLRCkgMYo7uy47BLzLQNezX/fGIKaBImCc7EzLhcv6f+2fElfQGUGii1upZQAPhlloLQuGAESdFOH50wBZnftNh+TA+tQbvdJyQntR5rrpKUpG7liyRR4Ob8WMOS6In5bIFe1rgvqtEZqupErzFkVmbrJPKV5T/SgkdCDlLehJ2WIdQ+KwR/3U+dN3L2c+ys9o4U0ixB7GKLukDKsEN2n1pA8RMx4GaU/5myOa8YZwqeB8KwKPMyppTKW5tEoCqxbCRl+Ooduk8SrITCSCZH9+m0jlYHi07SAHnjSXbYiNKjsa7Iaojucwj30y4BhHqHeCFErfEELFoMEfijBXdEaiSPJNJv/jaXWplew/iW5yM8xP9ZFYRPug5B+XWzipld+QwHTNdbUT7hkH80blYmZ9wR/Q8GALLlCG9dH9/xS+AC61DWx8TyJq/Bi7pSNL9LMoz4XSkhjRRQe5y3jWnVjNWQ34wcedGo/yxBLkJhDlHEnS9owmqMvkDr3JKVbZFdwh8IFJ4EwxP34AAVi/UjEyVS8+gi8DK+AYjViJvyS29DSqPOhANwk1qg3PKEoJhBvjKWLMndjbJg26D4KreJUSfjQBILJSlA5PZbqPDRasBuujngQfFHPBY1Fu6+uevnXQMOVpyYhjHrC/tz7dZir9uss70ofEnwlQDRj1I20ZjDpayLO8Jf4trNStMrT9jDfM3I7mbZUI1pVpOxy1GAxRE8PvUP9NiRvRq9FUj8VfQd4Qw2OYQtLRPy5EQV1LP2zji+EluCqcR850HPZpK1HArutvgU3ZQlsWbtlQ7nnUXTjgmba5UHt5X2b1SewWyXi/SglaOC/cvyHl2EJ/81KzYGNYF/uBlEEKUEEggzy8Ea5KL122IRPasv/2tRrZEmMSSMkqiRJ1Yg2UD8MAKMAWcEzaWhZDTofJyDUboOUvh0EULXx JU5IXNDI ZxVH5OwUmOV1PQu5Pc7a/gdll/zJaLMOsXwdCtuorS9DaUt3Ken/TrFzSknEGkmZ0brAplU7KauarJW9jlawZKejzJTuPveYs+F7Jn9Xqx7ivMJlYmKG2hv1u94vsvX4mDdnxwjcrKktVJLittKueHrhJ6eH23egB7apdQMT1l1QPVQjRetDQVZke2hcub3kmQw6sYxkVN3cvI6WUDZobfNZT4AZL47NsQuSdQmm1TESaPIPwciu78+sdYekHbBsw8+KsckOgNaQosNBJId3GLfKHKbJRIqMmbsjB/bX4krBZHv2lj3H6/ajH8euiOKOxcRlxJ+UN7M9FhcA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, 20 Jun 2023 00:51:19 -0700 (PDT) Hugh Dickins wrote: [...] > @@ -407,6 +429,77 @@ void __tlb_remove_table(void *_table) > __free_page(page); > } > > +#ifdef CONFIG_TRANSPARENT_HUGEPAGE > +static void pte_free_now0(struct rcu_head *head); > +static void pte_free_now1(struct rcu_head *head); > + > +static void pte_free_pgste(struct rcu_head *head) > +{ > + unsigned long *table; > + struct page *page; > + > + page = container_of(head, struct page, rcu_head); > + table = (unsigned long *)page_to_virt(page); > + table = (unsigned long *)((unsigned long)table | 0x03U); > + __tlb_remove_table(table); > +} > + > +static void pte_free_half(struct rcu_head *head, unsigned int bit) > +{ > + unsigned long *table; > + struct page *page; > + unsigned int mask; > + > + page = container_of(head, struct page, rcu_head); > + mask = atomic_xor_bits(&page->_refcount, 0x04U << (bit + 24)); > + > + table = (unsigned long *)page_to_virt(page); > + table += bit * PTRS_PER_PTE; > + table = (unsigned long *)((unsigned long)table | (0x01U << bit)); > + __tlb_remove_table(table); > + > + /* If pte_free_defer() of the other half came in, queue it now */ > + if (mask & 0x0CU) > + call_rcu(&page->rcu_head, bit ? pte_free_now0 : pte_free_now1); > +} > + > +static void pte_free_now0(struct rcu_head *head) > +{ > + pte_free_half(head, 0); > +} > + > +static void pte_free_now1(struct rcu_head *head) > +{ > + pte_free_half(head, 1); > +} > + > +void pte_free_defer(struct mm_struct *mm, pgtable_t pgtable) > +{ > + unsigned int bit, mask; > + struct page *page; > + > + page = virt_to_page(pgtable); > + if (mm_alloc_pgste(mm)) { > + call_rcu(&page->rcu_head, pte_free_pgste); so is this now going to be used to free page tables instead of page_table_free_rcu? or will it be used instead of page_table_free? this is actually quite important for KVM on s390 > + return; > + } > + bit = ((unsigned long)pgtable & ~PAGE_MASK) / > + (PTRS_PER_PTE * sizeof(pte_t)); > + > + spin_lock_bh(&mm_pgtable_list_lock); > + mask = atomic_xor_bits(&page->_refcount, 0x15U << (bit + 24)); > + mask >>= 24; > + /* Other half not allocated? Other half not already pending free? */ > + if ((mask & 0x03U) == 0x00U && (mask & 0x30U) != 0x30U) > + list_del(&page->lru); > + spin_unlock_bh(&mm_pgtable_list_lock); > + > + /* Do not relink on rcu_head if other half already linked on rcu_head */ > + if ((mask & 0x0CU) != 0x0CU) > + call_rcu(&page->rcu_head, bit ? pte_free_now1 : pte_free_now0); > +} > +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ > + > /* > * Base infrastructure required to generate basic asces, region, segment, > * and page tables that do not make use of enhanced features like EDAT1. > diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h > index 306a3d1a0fa6..1667a1bdb8a8 100644 > --- a/include/linux/mm_types.h > +++ b/include/linux/mm_types.h > @@ -146,7 +146,7 @@ struct page { > pgtable_t pmd_huge_pte; /* protected by page->ptl */ > unsigned long _pt_pad_2; /* mapping */ > union { > - struct mm_struct *pt_mm; /* x86 pgds only */ > + struct mm_struct *pt_mm; /* x86 pgd, s390 */ > atomic_t pt_frag_refcount; /* powerpc */ > }; > #if ALLOC_SPLIT_PTLOCKS