From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F433C47258 for ; Tue, 23 Jan 2024 19:43:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 828668D0001; Tue, 23 Jan 2024 14:43:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7B2236B0080; Tue, 23 Jan 2024 14:43:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 651D38D0001; Tue, 23 Jan 2024 14:43:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 529086B007E for ; Tue, 23 Jan 2024 14:43:13 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 004A71209B2 for ; Tue, 23 Jan 2024 19:43:12 +0000 (UTC) X-FDA: 81711599424.15.39A3460 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf02.hostedemail.com (Postfix) with ESMTP id 16D1A80013 for ; Tue, 23 Jan 2024 19:43:10 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; spf=pass (imf02.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706038991; a=rsa-sha256; cv=none; b=6e5+Ydhfs4Nb2O9+0X4bwqMiNR5172rWYviZ0A65OfBHSw5AWQu5iUrm+BSPP4EfHrXGKC AAnI+k5UfdbEZE/I4KL1ef2mqjWfqkgz3pdOligiL+7UNwsfaNdD/YS9fAWd6ygnhb4/2g wknAveeYbs0hQm9vlwuuAKEYEMK+qek= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; spf=pass (imf02.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706038991; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qwsU9zpTbJ8nTbPvPcmaJX9dyILbrCOm70W7gXRsL1c=; b=ma0k7iu+/F9+Z3ECaRf1zSQPy3xp/niUCAqHfn/OYjoy40Lia1CU60MJgmL0g1qqpbxZTv cV0OvSDpavlPZ0tJi4pjIcBNkv0uvu/RQI098jdnjVXnYNs+N/t/XrsrYaJi+zuz6AwCUI E+TXZdn6ykK9cxZ/BC3QvSEgQ2xWmqA= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 41E301FB; Tue, 23 Jan 2024 11:43:55 -0800 (PST) Received: from [10.57.77.165] (unknown [10.57.77.165]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id EBFA03F73F; Tue, 23 Jan 2024 11:43:04 -0800 (PST) Message-ID: <7d92d27a-44f6-47d0-8eab-3f80bd7bd75d@arm.com> Date: Tue, 23 Jan 2024 19:43:03 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v1 00/11] mm/memory: optimize fork() with PTE-mapped THP Content-Language: en-GB To: David Hildenbrand , linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, Andrew Morton , Matthew Wilcox , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org References: <20240122194200.381241-1-david@redhat.com> <56bee384-461e-4167-b7e9-4dd60666dd66@arm.com> From: Ryan Roberts In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 16D1A80013 X-Stat-Signature: 5o6kk3ptsbfnm6hc5trx38y89s9m6r8x X-HE-Tag: 1706038990-834937 X-HE-Meta: U2FsdGVkX1/zCzq5nseIVkXMdwr7CBRpVJmfhNRug0HLWolyuTmke3uuq9QvvzVLMMovgPVgcVZmyCVpd9FW2p8MjYA502vCrimyzWu8jB0wqMA1FM3bq89+l/7iDVb5e8nJNmbTuxu5dkJ3UEdwBHPVjifE6qVn0CXlirKHXBtkSZC4nAvKVKU7V3NaHr9vqlJjMmVJgZRAy0+eKp7+8rEVEY2kQU1iICmDty0C94H6Cj2pUufhRL+Hcy2Cuiwjwp19q0F0l37jGcqf09BvJK2Dz3FYpdIfrJGrwBk90On92J3slnQrxF4OU1EnxQwnccv9zz3Fi+1/CHE85hmCqYmuJK0HdW9x9NKOCgrVy67Nbo5ROgUKT8UJpz5WCp2CCAwCaWiT5lWVLCQ9UraOpt1PLnIwArjI3hNr5UOtmEOwXsejra5NZRHETTpv8v8Js2SRokiXvTQcDZRZsPuxHp73nsRQOtfST6/IE8zQug6X4iokFCzdI1y29R7WAXm69Q3yPQ0lEbis/QMEpOcbX4V7qIcWB5BHsNP1YSF8y3xLa3xRwqx4YEGmJv7ZPSMLUQv1yPenGgu9IntmclcQDgjPZf+asXj87IUR4w6zdvR3dz7edOynpZ3kyBRmTMrZM8MsQ5SpNBNmcqdmz4XoHHH5BMTYKoNQizTHTyQNMd2ZBv3GjnL5KhiF8JXOEBu6lqn5xgoe2Vi7DrVZtGqz3vFsSzj4N+tMyAo+vItCAwX404/7Z694rf30lej+VAiMNSe2/DHX9Z9uh1P4XUgVEhnRDd5LgVrPglgp25X20cfsbM3LeqvwE5rXDQXEU7T2eh+M06XNP+x27tNcHAPabeRQYfjxWvFQOjKJ3/MD7MXFGKk4kck7LA3+mnyojJM7qbCqqg/lwlL5Gvl5OPaZntsy30v7752PtX3e4QF/+RDFmQToNo/kJea/qaZ4pQIhdRdABt8MRJiNC9gkFI/ NBixT+Zp 14EqvjWgLUBdchlQn7EY5aO4nA/pkkuOVeEhzrJ4md2kcJvftt2oFHrf2AUJZzb1/1kohAWbRLUQXrIGXFHLuSZ9riKlxmjCKULBHeJGwpfumNBIAXqpSkNs+716h0Gyv3FQoUN4L3bjr5Dt6JgaUnM7E8w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 23/01/2024 19:33, David Hildenbrand wrote: > On 23.01.24 20:15, Ryan Roberts wrote: >> On 22/01/2024 19:41, David Hildenbrand wrote: >>> Now that the rmap overhaul[1] is upstream that provides a clean interface >>> for rmap batching, let's implement PTE batching during fork when processing >>> PTE-mapped THPs. >>> >>> This series is partially based on Ryan's previous work[2] to implement >>> cont-pte support on arm64, but its a complete rewrite based on [1] to >>> optimize all architectures independent of any such PTE bits, and to >>> use the new rmap batching functions that simplify the code and prepare >>> for further rmap accounting changes. >>> >>> We collect consecutive PTEs that map consecutive pages of the same large >>> folio, making sure that the other PTE bits are compatible, and (a) adjust >>> the refcount only once per batch, (b) call rmap handling functions only >>> once per batch and (c) perform batch PTE setting/updates. >>> >>> While this series should be beneficial for adding cont-pte support on >>> ARM64[2], it's one of the requirements for maintaining a total mapcount[3] >>> for large folios with minimal added overhead and further changes[4] that >>> build up on top of the total mapcount. >> >> I'm currently rebasing my contpte work onto this series, and have hit a problem. >> I need to expose the "size" of a pte (pte_size()) and skip forward to the start >> of the next (cont)pte every time through the folio_pte_batch() loop. But >> pte_next_pfn() only allows advancing by 1 pfn; I need to advance by nr pfns: >> >> >> static inline int folio_pte_batch(struct folio *folio, unsigned long addr, >>         pte_t *start_ptep, pte_t pte, int max_nr, bool *any_writable) >> { >>     unsigned long folio_end_pfn = folio_pfn(folio) + folio_nr_pages(folio); >>     const pte_t *end_ptep = start_ptep + max_nr; >>     pte_t expected_pte = __pte_batch_clear_ignored(pte_next_pfn(pte)); >> -    pte_t *ptep = start_ptep + 1; >> +    pte_t *ptep = start_ptep; >> +    int vfn, nr, i; >>     bool writable; >> >>     if (any_writable) >>         *any_writable = false; >> >>     VM_WARN_ON_FOLIO(!pte_present(pte), folio); >> >> +    vfn = addr >> PAGE_SIZE; >> +    nr = pte_size(pte); >> +    nr = ALIGN_DOWN(vfn + nr, nr) - vfn; >> +    ptep += nr; >> + >>     while (ptep != end_ptep) { >> +        pte = ptep_get(ptep); >>         nr = pte_size(pte); >>         if (any_writable) >>             writable = !!pte_write(pte); >>         pte = __pte_batch_clear_ignored(pte); >> >>         if (!pte_same(pte, expected_pte)) >>             break; >> >>         /* >>          * Stop immediately once we reached the end of the folio. In >>          * corner cases the next PFN might fall into a different >>          * folio. >>          */ >> -        if (pte_pfn(pte) == folio_end_pfn) >> +        if (pte_pfn(pte) >= folio_end_pfn) >>             break; >> >>         if (any_writable) >>             *any_writable |= writable; >> >> -        expected_pte = pte_next_pfn(expected_pte); >> -        ptep++; >> +        for (i = 0; i < nr; i++) >> +            expected_pte = pte_next_pfn(expected_pte); >> +        ptep += nr; >>     } >> >>     return ptep - start_ptep; >> } >> >> >> So I'm wondering if instead of enabling pte_next_pfn() for all the arches, >> perhaps its actually better to expose pte_pgprot() for all the arches. Then we >> can be much more flexible about generating ptes with pfn_pte(pfn, pgprot). >> >> What do you think? > > The pte_pgprot() stuff is just nasty IMHO. I dunno; we have pfn_pte() which takes a pfn and a pgprot. It seems reasonable that we should be able to do the reverse. > > Likely it's best to simply convert pte_next_pfn() to something like > pte_advance_pfns(). The we could just have > > #define pte_next_pfn(pte) pte_advance_pfns(pte, 1) > > That should be fairly easy to do on top (based on PFN_PTE_SHIFT). And only 3 > archs (x86-64, arm64, and powerpc) need slight care to replace a hardcoded "1" > by an integer we pass in. I thought we agreed powerpc was safe to just define PFN_PTE_SHIFT? But, yeah, the principle works I guess. I guess I can do this change along with my series. >