From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB00DC46CA3 for ; Mon, 4 Dec 2023 12:57:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 42C7B6B02C9; Mon, 4 Dec 2023 07:57:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3DD9B6B02CA; Mon, 4 Dec 2023 07:57:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2A5A06B02CB; Mon, 4 Dec 2023 07:57:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 1DC576B02C9 for ; Mon, 4 Dec 2023 07:57:13 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id EE0051C0485 for ; Mon, 4 Dec 2023 12:57:12 +0000 (UTC) X-FDA: 81529136304.09.94C37C3 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf22.hostedemail.com (Postfix) with ESMTP id 065F0C000C for ; Mon, 4 Dec 2023 12:57:10 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701694631; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JFhI8lD1QTu0a6aMa4k6PTk95/fWfSy6OGpp42XlRmg=; b=gqIZgfThKXqH/2qQ+0r8MQ53yRm+QM19QUIq0uI9Nl1DJikSDQIRhOV+VoNIhDoL7ESy7k YimXO905X3s0vdIIaiAcLHfgXRyeizPfPq/2pXum2AQuDFaJ7ir4l7FagHGi59XE8daOiu 5iAv3iQQPX+g7LxQgtRtwM9rF9swiU8= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701694631; a=rsa-sha256; cv=none; b=p95yh0g6onSIz6fz2SdrR9vh0ctC+Ra3mOpENBosQf1V6XSTOyQSbbOFtObcbtQljEnUjd iZsyjyvgVvl4W62+eMDH+RHM+pOJLshaUBMSwTRb9FnvID2J6GQ495jUBZBSUxSaP8dsjN Q+7W85DeC0AzMW12C6uAPUrFT42WsMk= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 421031424; Mon, 4 Dec 2023 04:57:57 -0800 (PST) Received: from [10.57.73.130] (unknown [10.57.73.130]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2E2F63F5A1; Mon, 4 Dec 2023 04:57:07 -0800 (PST) Message-ID: <83497a70-45c3-4386-8994-92207ab521bd@arm.com> Date: Mon, 4 Dec 2023 12:57:05 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v1 4/4] mm/mmu_gather: Store and process pages in contig ranges Content-Language: en-GB To: David Hildenbrand , Zi Yan , Matthew Wilcox Cc: Will Deacon , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Peter Zijlstra , Christian Borntraeger , Sven Schnelle , Arnd Bergmann , Yu Zhao , "Kirill A. Shutemov" , Yin Fengwei , Yang Shi , "Huang, Ying" , linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20230810103332.3062143-1-ryan.roberts@arm.com> <20230810103332.3062143-5-ryan.roberts@arm.com> <800937DA-BAD0-4C60-B155-AECCA21E955E@nvidia.com> <1a0f5cb8-421c-4f28-a986-f3c381406e81@arm.com> <90EC4C0D-0254-4B93-AFD5-3C09580A77DE@nvidia.com> <6dd6164a-1dd5-46e7-bcf7-b62ff5c6e8ec@arm.com> <890f4455-bf1b-437b-a355-7895e4dd3085@redhat.com> <52b042b9-ec95-4db0-b38a-f7f1cea0b90c@arm.com> <635de797-1219-40b0-b4b2-7eba758749a5@redhat.com> From: Ryan Roberts In-Reply-To: <635de797-1219-40b0-b4b2-7eba758749a5@redhat.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 065F0C000C X-Rspam-User: X-Stat-Signature: r9tyqju3xwitruqc9jbzgf64xkadcqy8 X-Rspamd-Server: rspam01 X-HE-Tag: 1701694630-581732 X-HE-Meta: U2FsdGVkX18/Z2ud8Vz5T7/bDJfuMAzZyydBrc2DZIM8r7d+aCL3jvyRi2b9qu1GsWSkR2pc8FgpSqogZEpYAjXlF8pCjsQ+dhJ7ke6NTH2hlgXB7QpUwoxGOhgiDvF0Pbo2qBYfVAI1gXoGFGGlcGPv5KPeZjDTYaeeITLEC1B0//xB1pNO6ieT24tg6KLFTxljLnq41hLkU9lE4dqelgoZ8cOLrydwMpPH3LbEBJaUqJn/U45K3NsZPRC1RCh5xx5/RLl9fPR23u14LZxCOO2rsLW82EL/FKJu/UykALQ9SO6tEch88O0zLD0fEmFQ7yTMW6XeqcXD/gEDQgiDz7EEBnKiOh5EMcvdajOikvcoGetwYSnSUooc4H+b2m6+FhYrLeumz3fQJIV6YFzsd2lx1Zxb4SWtO874lSvxKkVZS8+LkgdTkMs8mbB01z3l6EhczHg2Fe9qTcaHLbjO6fjRlNY8XIFD5wnpjlV6FEi9tc4GaqhO05KDrnFBqcgWnmMisnkqnyu/0HYxeZQTio54Pny6SDpMZJBJi3ZWFxFU0QI5fo5458A+pVuEGqwwGPBjUalvxH4Q2nwbgaOec0RDvkbQBybZBA1dkcdoRSOowARPiz1vRSHYUi8rHHdxLJJbd/5iOv00p5KsRQD4OCKUfeDhHha8J/rjD0Of2bdiB3lC5sERzSYDB7nZ8/OFIiz4jPVlnqwssNM7h3Ic6yRUkjAhYf8fukKVvdey1qU6uREjE47su2ZD8AOWy6gAFsHTQ78+YAm6WRxHrPPScbmKz7iUrrAaxNnliKOZJawQGQUff/mjeQWzdoTBNrh4Dx7JwwudJ4Vm7JLfI+N2znftmMUjeEMGtD/P9tghXlnHBi3VMK0ZoyON6JosaItBNn5aURN5J7XJYDKhXj7IaX0zmAk9VXYI2lMZ2B1cUAlIIa+pOg2BhJ49Q7W1x/wITAH/LTAHpGW7XLk1EVH CeQ7EYmd VTzLIplCUc55O2Iw2G+l4I5LQxBiHav6ZGH4aM+xiUYl32tmfnBIjkAEYizSoJoqzztgFje+41JF8JkEI5m9jpLawTRIrPBgUKGo5vw7sBMcQtT0b8QQpLrNdwCtaHJDWBUDSZl0vNwU7DfmpNJtoZqwQAkIzG2RmDxUJLVEiVqI5sXnyhCASxjmz5+6WSImbtv5/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 04/12/2023 12:43, David Hildenbrand wrote: > On 04.12.23 13:39, Ryan Roberts wrote: >> On 04/12/2023 12:28, David Hildenbrand wrote: >>> On 04.12.23 13:26, Ryan Roberts wrote: >>>>>>> >>>>>>> Also, struct page (memmap) might not be always contiguous, using struct page >>>>>>> points to represent folio range might not give the result you want. >>>>>>> See nth_page() and folio_page_idx() in include/linux/mm.h. >>>>>> >>>>>> Is that true for pages within the same folio too? Or are all pages in a folio >>>>>> guarranteed contiguous? Perhaps I'm better off using pfn? >>>>> >>>>> folio_page_idx() says not all pages in a folio is guaranteed to be contiguous. >>>>> PFN might be a better choice. >>>> >>>> Hi Zi, Matthew, >>>> >>>> Zi made this comment a couple of months back that it is incorrect to assume >>>> that >>>> `struct page`s within a folio are (virtually) contiguous. I'm not sure if >>>> that's >>>> really the case though? I see other sites in the source that do page++ when >>>> iterating over a folio. e.g. smaps_account(), splice_folio_into_pipe(), >>>> __collapse_huge_page_copy(), etc. >>>> >>>> Any chance someone could explain the rules? >>> >>> With the vmemmap, they are contiguous. Without a vmemmap, but with sparsemem, we >>> might end up allocating one memmap chunk per memory section (e.g., 128 MiB). >>> >>> So, for example, a 1 GiB hugetlb page could cross multiple 128 MiB sections, and >>> therefore, the memmap might not be virtually consecutive. >> >> OK, is a "memory section" always 128M or is it variable? If fixed, does that >> mean that it's impossible for a THP to cross section boundaries? (because a THP >> is always smaller than a section?) > > Section size is variable (see SECTION_SIZE_BITS), but IIRC, buddy allocations > will never cross them. > >> >> Trying to figure out why my original usage in this series was wrong, but >> presumably the other places that I mentioned are safe. > > If only dealing with buddy allocations, *currently* it might always fall into a > single memory section. OK that makes sense - thanks! >