From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D729EC77B7F for ; Fri, 27 Jun 2025 11:55:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 61BD36B0089; Fri, 27 Jun 2025 07:55:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5CBE56B00B0; Fri, 27 Jun 2025 07:55:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 492536B00B3; Fri, 27 Jun 2025 07:55:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 301E66B0089 for ; Fri, 27 Jun 2025 07:55:23 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id DC1838068C for ; Fri, 27 Jun 2025 11:55:22 +0000 (UTC) X-FDA: 83601025284.28.88E50C7 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf14.hostedemail.com (Postfix) with ESMTP id AE4B7100009 for ; Fri, 27 Jun 2025 11:55:20 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="SM/lBO2S"; spf=pass (imf14.hostedemail.com: domain of dhildenb@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhildenb@redhat.com; dmarc=pass (policy=quarantine) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1751025320; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wcOFieBfYhagF3JtnYzgZBrQvN7+QiFqX1LLvkigDHE=; b=fbWbHAXMNuN0eD48vrzLxsOVNHWIkz83IiW+O6EgMKLkUQb1b2hKUgtz66IRr7gPjARxL1 CD5A/CjHeyVY3NjXq2/UQxXegVlhSTaYSjZcgbpuEKnzeLF2Eu0cgLM5jQR1g0IlmxU+rL 30G4R0czakR7lX4lQI7+1juFocVzK60= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1751025320; a=rsa-sha256; cv=none; b=KTjNi8NfHqdKIoDZw/AUznlgBQEAm319FqPbtb/OZ+W95DsOqA5rpv+1mAFr7CUeQrcllD 5y44q4my+YdWr8+Q9t77zG5R2z+r9o1jJUW0BP7ofmsHwyAyY/ZtV2SDXOoEW9JQWa4z0y /ghnOwvxqvuQz7TtczRsVupzVOdfjRc= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="SM/lBO2S"; spf=pass (imf14.hostedemail.com: domain of dhildenb@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhildenb@redhat.com; dmarc=pass (policy=quarantine) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751025320; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wcOFieBfYhagF3JtnYzgZBrQvN7+QiFqX1LLvkigDHE=; b=SM/lBO2S0IBQaPL/xOsELUAWoEwLFkIaiAe1hAVwx0BO5SO1gaAkirDwor3yfxr0WpU5UX RXqzi5TDFg79Z9Hd/31QLMnln3AdZQZkZiVuBT8+ZAuTc5WtddR4zI05ozVGtdahmvfOzM RDYeYgoYyMFdbl748fUZm/Sz/oK2Q2o= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-687-ZBicA0AzOmW37S2xNbTSSg-1; Fri, 27 Jun 2025 07:55:18 -0400 X-MC-Unique: ZBicA0AzOmW37S2xNbTSSg-1 X-Mimecast-MFC-AGG-ID: ZBicA0AzOmW37S2xNbTSSg_1751025317 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-4538f375e86so5492775e9.3 for ; Fri, 27 Jun 2025 04:55:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751025317; x=1751630117; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wcOFieBfYhagF3JtnYzgZBrQvN7+QiFqX1LLvkigDHE=; b=ZNvlFQ5aS7iJa7Ov6AdvPqdTtjnb2JLmmfTIynN9BeDVDaPd8R68X/Jdp4YZFH+sbq 7zBlMqWGKfukeJurN+LsAnTMjWNd5sJJVxpZzVPWdWe5vW1izIZx/i/sjLKanEjUTYnO fbUX9Tj/AOb9JzbuOK8mvzbEzwH95Op6Aj9Ntccaie/b7qdzuYPtSfIlcZ0PTnWRokhB CsZ/hsyJHFnwoydSiq2o3xJQ8pwyLF6J3KDJZzMJVDK0715H3YhYKK2ulaz7G0Ed7+ee 7w+77mcSVwTuPIVTanVJ1qwwpli/4YrAdegOQCJaFOK1bB/fE3ze8UIjd5P/lqLEFSkq Hl4w== X-Gm-Message-State: AOJu0YzWRL5vqmpbd9opPXMzAy5v0HydCsVo7tOWW8Op0V18N+WR9uPu +lTaDl2eNpH7KbR34dWdaV0SWI5TYsIqtTCazLiYzYrDupCSxpNsIdGBav60yj61GzgieVDYySw PcwXGCJmW/cHvcUlfN9twjYAu0Y7ZRdrsdMo2x+er5YbDPpXc24kx X-Gm-Gg: ASbGncuOmb97yqJf1A7YMtnBR7xtnTzIllpypBhYN8mr87B2zxGWAAThhDZGyIKbNCg QdO8Cp79CSxf+xfS+wjBv8iB/FcX+9BSsbTtqCPuKPCXteKRBvn36h/F2VFEY4E7EX9GE3F6Q9Q ojQaQ8fT2v/J2Pg6xJJ+NG5J0Y0C7L1tpTPaB4TD59VvWG99W6ELElW/XAfabG4UvoJSS0R/Gvs cvyReI+R6QJN/AohlWmzFMPuBFtgyXED/K+OOvI7V6qeeqqJix9pwNNmRLOv/Jo4aQ+vpYfBOkY 8Y2fzzzOx7Lv0w8uUhS6fblG6+wsskcSf6HbOkMp20zKspO+xIL4s8UbXp4zbArMV5RmABVT3Yq 8KhEbmv8= X-Received: by 2002:a05:600c:a4c:b0:43d:300f:fa1d with SMTP id 5b1f17b1804b1-4538ee61b50mr37455535e9.31.1751025316843; Fri, 27 Jun 2025 04:55:16 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHxgJfDsSX5zp20qR+Gqu3ACFHA6VlqNRYklPCovpm6sCHWkL6IljTpiOqEKrLvkyyoB1vwtw== X-Received: by 2002:a05:600c:a4c:b0:43d:300f:fa1d with SMTP id 5b1f17b1804b1-4538ee61b50mr37455035e9.31.1751025316287; Fri, 27 Jun 2025 04:55:16 -0700 (PDT) Received: from localhost (p200300d82f2d5d00f1a32f3065759425.dip0.t-ipconnect.de. [2003:d8:2f2d:5d00:f1a3:2f30:6575:9425]) by smtp.gmail.com with UTF8SMTPSA id ffacd0b85a97d-3a892e6214fsm2517230f8f.98.2025.06.27.04.55.15 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 27 Jun 2025 04:55:15 -0700 (PDT) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , "Liam R. Howlett" , Lorenzo Stoakes , Vlastimil Babka , Jann Horn , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Zi Yan , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Pedro Falcato , Rik van Riel , Harry Yoo Subject: [PATCH v1 2/4] mm: smaller folio_pte_batch() improvements Date: Fri, 27 Jun 2025 13:55:08 +0200 Message-ID: <20250627115510.3273675-3-david@redhat.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250627115510.3273675-1-david@redhat.com> References: <20250627115510.3273675-1-david@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: CysmCNXHitqfqcIz8eNDrC4XdSg5FNdFfvPC0L5up2s_1751025317 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit content-type: text/plain; charset="US-ASCII"; x-default=true X-Rspam-User: X-Stat-Signature: eeqmugwwc8o4xxaqsi947mcddqgz5y4o X-Rspamd-Queue-Id: AE4B7100009 X-Rspamd-Server: rspam08 X-HE-Tag: 1751025320-119834 X-HE-Meta: U2FsdGVkX196HBPAOqBLGT2NPNtSPnQXq2UDWsq9gdGcfQH4ZLztHilCgrIW1g7SPuvaJ3QbNzdYOGxmEoy+z0bMSkbrkhSl8hWNRsrFN73CrmFJyIel98j8rNtrqqpWUbUoazt1B24b6YRUm2A2kdT9ytgmWLpmkMyOQT88m8epndZJR/fzuS4ydAZzKJbrl8SP36uVsZ8qrpR9jPgG7WIEPTtQpOsEr+vGsww0p5WqabKCfXZs72A5Pu7n/djygKxCDMC4Pdzs0B0RSIys9SY9SEjxX+CDJh4xx/n3lYos7y/b/PHviCMrsyn79G2V5B+rgco+5sZ68akP3LH9hJpgT7Bv/1s3seZ+FUw1NpHvFEUV8fuz+x8gQ1kSd6GDApiQUWSt+MNutmr8JCcU2dsbsmOm+NqAqT1eROLlteel9QAtk0VrwiO0qmgzJc6fazk96K4xv6uye1g3SHmCmfByhuu4XO4lr9Xo/I6pjGnqP9xGHHx/B3ZLK0eO0CW9MrkL9s1m7XFocywzTQzGbPlTS+Xvq6iqAOLgPpfy846m+TFnJlw7pF4C7xODYcRLik1Wn1GmrSsoqqz2FA86rGsW1XsgASJLkQiz9QGZdLZbUg4uwhj6mWsJrZDGpDkLkMzgP+5G9WR9VYZXOQijFlTpn2duPnH/V56I22nG11CKl3cG+ajGtsR42jeKmUSjKsiv/1yTNAnhLuM+IPvPig7MRvD1AJE4yJS2VxC0Z3PJYirKjBHA2+A8lDp14b0JsXuJ+lleOvEaqjYj9OeYeDFOvCM3psV40XGMhK6UVnU/jud2EUI+aC3M5l3qpum1A/i6Fi9pHsw2OzLrcxxb1sdbAZzg+YHfOAiPPFK9GEMKzHk/VOuVf9tAcSZGmpOr1dyGpHK0a4S/43ImIrxjcVfn234TWL31f78sKboCn0LZvvF1Km7LHu5mNT5/qbbAjNsSYs105Fzs5kobnbB iD+6zDyZ +/tX5a+f2aCuM9lI+BACxNe8wiEqOs0kRbrlRomuABwynA1RplL7OwLxzovJCIBugWvBXPeU2DjmRizeJgUq0oxYtMulB3JmufUAEPvMH8TsYcYox0zwsjuNIA6EAtogn+qLdO8LXMxj1GGeuQvUA4dGC2Scg2lN0XOr3aE5lafpO/bfMINIuVq2g9qnGniSKi42BvErl91PvPDkBVM556UB6RS5UkL6dKzJO5vIyio7pGb+1jOiSeytFFJjtCpiO4dP7HtFujUNEWPbJkFrD3kMDm9Ea7W6EGTrnTKn4b5gNewkBaLgWuksrve3liQTCSDa6LQE5TUheO4WurovcHl02t5z2Kiq5yA3Yev5FJI/kQ0V9BsXmnVv0fhP95JXhAvw26FdSAb32XsjR0MDtT0EDwyCzWg2++SsQ9pMHmy0xMFBsv1YHygPNfnQvUSOSdZiWdLfHF8hkzqdd9SSvV+fLfdA9cBRWGmG+5y8/ymrtIsmftHd/yx6oRZmSi9wMsEo948ApentuzNvxpMVejCBtMB4KtjAXZp/Y65F/NPiCY6FsgfmLy8VPVk3hIMi6XEWKq7KXVY3A6qLgvZJMzioKqoR8AKX1+Wdri3uPj4tYH8gMpkkZ3Q8y+7+ewit18Zh20SBYf86s4PHx++4VkbiM3eImrIfXg1Gj X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Let's clean up a bit: (1) No need for start_ptep vs. ptep anymore, we can simply use ptep (2) Let's switch to "unsigned int" for everything (3) We can simplify the code by leaving the pte unchanged after the pte_same() check. (4) Clarify that we should never exceed a single VMA; it indicates a problem in the caller. No functional change intended. Signed-off-by: David Hildenbrand --- mm/internal.h | 37 +++++++++++++++---------------------- 1 file changed, 15 insertions(+), 22 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 9690c75063881..ca6590c6d9eab 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -221,7 +221,7 @@ static inline pte_t __pte_batch_clear_ignored(pte_t pte, fpb_t flags) * folio_pte_batch - detect a PTE batch for a large folio * @folio: The large folio to detect a PTE batch for. * @addr: The user virtual address the first page is mapped at. - * @start_ptep: Page table pointer for the first entry. + * @ptep: Page table pointer for the first entry. * @pte: Page table entry for the first page. * @max_nr: The maximum number of table entries to consider. * @flags: Flags to modify the PTE batch semantics. @@ -233,24 +233,24 @@ static inline pte_t __pte_batch_clear_ignored(pte_t pte, fpb_t flags) * first one is dirty. * * Detect a PTE batch: consecutive (present) PTEs that map consecutive - * pages of the same large folio. + * pages of the same large folio in a single VMA and a single page table. * * All PTEs inside a PTE batch have the same PTE bits set, excluding the PFN, * the accessed bit, writable bit, dirty bit (unless FPB_HONOR_DIRTY is set) and * soft-dirty bit (unless FPB_HONOR_SOFT_DIRTY is set). * - * start_ptep must map any page of the folio. max_nr must be at least one and - * must be limited by the caller so scanning cannot exceed a single page table. + * @ptep must map any page of the folio. max_nr must be at least one and + * must be limited by the caller so scanning cannot exceed a single VMA and + * a single page table. * * Return: the number of table entries in the batch. */ -static inline int folio_pte_batch(struct folio *folio, unsigned long addr, - pte_t *start_ptep, pte_t pte, int max_nr, fpb_t flags, +static inline unsigned int folio_pte_batch(struct folio *folio, unsigned long addr, + pte_t *ptep, pte_t pte, unsigned int max_nr, fpb_t flags, bool *any_writable, bool *any_young, bool *any_dirty) { - pte_t expected_pte, *ptep; - bool writable, young, dirty; - int nr, cur_nr; + unsigned int nr, cur_nr; + pte_t expected_pte; if (any_writable) *any_writable = false; @@ -267,29 +267,22 @@ static inline int folio_pte_batch(struct folio *folio, unsigned long addr, max_nr = min_t(unsigned long, max_nr, folio_pfn(folio) + folio_nr_pages(folio) - pte_pfn(pte)); - nr = pte_batch_hint(start_ptep, pte); + nr = pte_batch_hint(ptep, pte); expected_pte = __pte_batch_clear_ignored(pte_advance_pfn(pte, nr), flags); - ptep = start_ptep + nr; + ptep = ptep + nr; while (nr < max_nr) { pte = ptep_get(ptep); - if (any_writable) - writable = !!pte_write(pte); - if (any_young) - young = !!pte_young(pte); - if (any_dirty) - dirty = !!pte_dirty(pte); - pte = __pte_batch_clear_ignored(pte, flags); - if (!pte_same(pte, expected_pte)) + if (!pte_same(__pte_batch_clear_ignored(pte, flags), expected_pte)) break; if (any_writable) - *any_writable |= writable; + *any_writable |= pte_write(pte); if (any_young) - *any_young |= young; + *any_young |= pte_young(pte); if (any_dirty) - *any_dirty |= dirty; + *any_dirty |= pte_dirty(pte); cur_nr = pte_batch_hint(ptep, pte); expected_pte = pte_advance_pfn(expected_pte, cur_nr); -- 2.49.0