From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A0BAEB64D9 for ; Wed, 12 Jul 2023 13:58:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 11C706B0075; Wed, 12 Jul 2023 09:58:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0CC5E6B0078; Wed, 12 Jul 2023 09:58:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EAEC56B007B; Wed, 12 Jul 2023 09:58:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id DAF816B0075 for ; Wed, 12 Jul 2023 09:58:58 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 6B96A4015A for ; Wed, 12 Jul 2023 13:58:58 +0000 (UTC) X-FDA: 81003115956.09.F8FB419 Received: from mail-ot1-f47.google.com (mail-ot1-f47.google.com [209.85.210.47]) by imf21.hostedemail.com (Postfix) with ESMTP id 5316C1C000A for ; Wed, 12 Jul 2023 13:58:55 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=dabbelt-com.20221208.gappssmtp.com header.s=20221208 header.b=D7vYLwD0; dmarc=none; spf=pass (imf21.hostedemail.com: domain of palmer@dabbelt.com designates 209.85.210.47 as permitted sender) smtp.mailfrom=palmer@dabbelt.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689170335; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:dkim-signature; bh=WtiPPmkimlf1cfbcZtKGy3NvKyESLQgOrGfUSB0R+RM=; b=BpJ9/BzDWjBf9Zr/438ivlraGmqoBCL1xfbxTgY/4O7pAI9pDvlPnENwX6XSZJ/MtIgkbT U1UWsZMrtaJwtP5kZjvf7CaZyIgvbeP/OpGOIx1AmG8hWOHwUYaD9deLlqzkKL/iaswac1 v2nixrqTpA4iMtaX7RS6l2TDkqKAkhM= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=dabbelt-com.20221208.gappssmtp.com header.s=20221208 header.b=D7vYLwD0; dmarc=none; spf=pass (imf21.hostedemail.com: domain of palmer@dabbelt.com designates 209.85.210.47 as permitted sender) smtp.mailfrom=palmer@dabbelt.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689170335; a=rsa-sha256; cv=none; b=DpKUEgNWysq3MXTsHn66Ov9IlLoDcsG02sRyY9HbuByEVLIAghh+rJ99c6GdKUhLlafO/D pXssQzRu6KCb/aUMi59pNhhSJlFObmziAGRI8tcATFcCLiRkivq9/UgyQBaFSG5QpBNigq BffhVVPVYdbBcUgGTV4Y8lELXpcOX3A= Received: by mail-ot1-f47.google.com with SMTP id 46e09a7af769-6b91e21365aso3314848a34.1 for ; Wed, 12 Jul 2023 06:58:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dabbelt-com.20221208.gappssmtp.com; s=20221208; t=1689170334; x=1691762334; h=content-transfer-encoding:mime-version:message-id:to:from:cc :in-reply-to:subject:date:from:to:cc:subject:date:message-id :reply-to; bh=WtiPPmkimlf1cfbcZtKGy3NvKyESLQgOrGfUSB0R+RM=; b=D7vYLwD0d+cm+/50I06d5LTEFId3x/YrwBjY/a4xVK74mteWzgneFjalmuxORj52ca gWxFHvKbOR5pmot+DxeOlx30OJ+VfVBw+Ar/YySRZsMUe1eviZctwGugnI4l7kp60lbQ 4NpaToE7ZtAebZrtqGNE0FRpXhOUIvaH5YPWepcTfypqK1iZ7aJOKELncvKCegDE9MUX MogrWsUyN/O3Q6STv9oJoXhzselyzbhBTRHOOepN/FHbrbYs135kip5JLwjzW0YJDdfz pfFHkxAfNaXnv6NFsXwi7SNcwWVhCW3J3hG5CJjIsaVc1zTs/cqsdKljqWy3CrC6vf7t xQmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689170334; x=1691762334; h=content-transfer-encoding:mime-version:message-id:to:from:cc :in-reply-to:subject:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=WtiPPmkimlf1cfbcZtKGy3NvKyESLQgOrGfUSB0R+RM=; b=gO4t/SnmLzBkyyEw4Yj+W6PdJzNs4nBu4DDRD843vRvG6a3KsTulNWPe+w8jaQbVqR 3CXBO0gKHD0aJK2//tiNl78U1wlrwIvvO9pY2Wwk/Hf3wRPpoX9j915a7lCsF/pkFfQA +e0pF6y2OaDcRAwl8vCZuWZz+elqnWTxy5hoqtIfL/7ybgCi2byM/CDM1m7SesWVMdcP ixJnYxELGmrB+VO3dK7jPlSxzrAyWPXoWogPNHvY/AhNyrMQGAM8u2bFSxXsfsdYalTZ 7j/Fu7GFjqh0kSokW/+/mU1+83Y9Rcx6oR00YTnw9Gj7QFWIBPqjKmPHFGGC0f61U5CV bzNw== X-Gm-Message-State: ABy/qLakMV8LKxHiyAL0y3PT+57Blicfx8JumeOzSwBJTIk3Bs17gHIb C/GTQGMA5NzNjMzeijJ07gwGjA== X-Google-Smtp-Source: APBJJlEDKV1lafoYvKOBZ1aTJjv6CVXylfJX/ok+2beQDlkAS+AoMoK8n/64bLsknN7G9FdyH8JNSw== X-Received: by 2002:a05:6358:4311:b0:133:f39:5e8f with SMTP id r17-20020a056358431100b001330f395e8fmr6770464rwc.10.1689170334098; Wed, 12 Jul 2023 06:58:54 -0700 (PDT) Received: from localhost ([50.38.6.230]) by smtp.gmail.com with ESMTPSA id f13-20020a63380d000000b0052858b41008sm3560696pga.87.2023.07.12.06.58.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Jul 2023 06:58:53 -0700 (PDT) Date: Wed, 12 Jul 2023 06:58:53 -0700 (PDT) X-Google-Original-Date: Wed, 12 Jul 2023 06:58:07 PDT (-0700) Subject: Re: [PATCH v5 22/38] riscv: Implement the new page table range API In-Reply-To: <20230710204339.3554919-23-willy@infradead.org> CC: akpm@linux-foundation.org, willy@infradead.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, alexghiti@rivosinc.com, rppt@kernel.org, Paul Walmsley , aou@eecs.berkeley.edu, linux-riscv@lists.infradead.org From: Palmer Dabbelt To: willy@infradead.org Message-ID: Mime-Version: 1.0 (MHng) Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 5316C1C000A X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: rrj76ratie5eu3je679uhey1zun8k8bq X-HE-Tag: 1689170335-858490 X-HE-Meta: U2FsdGVkX18DdqF71fkpiwnDDT19qkmMk7PFpief9oskZZqhxQEipMKZ370qgbnlywifZK7QZzATyAqs6OJaFZHUiBFkTWvLq5y3EtixDw0ZhjG85ZgeUOOx2yZSYLlMiQzq5Q5RmEz7otyA98CcIslOLrkWZ7yWz8CmB7RXaY3THZqWBb9IQWroADzWo2SYB0bFc4XmhST5Aa0Ww5rM3JFJnWMDDOSc7jZTG5HFxMHrlGec+boUuwx0j5myfz/PjItao0mSwDeIZXJ27DMkpjZcuxuVgTgoyxRsaG/nEmGQ56BabdaDgbJMu2PFuWamdbnJD1maOM+wDdmGez8jANAaJ3Gp53QXk3U61wL3go597KbuokMYErzv4FJQS6UntYrkPP2iqsH/JjNSuBhp7L2uLq6gliSh4cSRjzTt8aBeulhCEIv1KuOr/KNKDEeEIBPllq9FDJ34qSJSVZ9IEfJdLibf+hS97HNQaqf2YAx9oGTUIYAYi9Sen+BvZZ77+LDQ7b6+wBzq2FaQHiijF8S0u3LUdlGakJBfaZageJcsWef/+M9DuTiktdIYT12JED/N/WTc14g+dh9ew4yXLfWt4L+caCKnyh/BjFaldna4u92dOaOdH1sPGXHSGj4n8d4RfWxWmsEnBUYvFWemiVEzpMvHmpSnSeBLzEZCmX9xExe0aL7HDJYunrOTfEYUgwGgAtJXC6benugMz4KdqMxdFH7AzU4fW+819HBIbzad0jHie1fDOnKYFHnlkJltV78P0sAO1h3vkDMS5BXkgVy+mExKiaESBn9D/lItTMLUhpC/Ev4hZX3ZBiJmDrunpa/LZrKRxh4sIB73VhEdwkzlSRgy+6+OxqRKxclU8Xo79HN7PkAZ8kbUmkJFBKeS5p7Z5gJ+Q9If/KfEC4+NqXuTCz7ydmtwQWR+YtUCwmj70pr912Z+amLFk+SJ0ijLFY8nK5g/wfggnqEaqWF w1mvYUB/ GNq/N8AZtBbzzMQh7ALPinFiFRxNyq3V4H81yEH8hJABQqncS1iuYnXadWlgATrB4KGOGN5jRHFYrXMEm7SavjDyM58HQS1Y0/EeT0nCSyssbZgQGQ63YYkp0pFyC0N6ophu4ttXr19dODB5hF++dP+9oCcO4OOU7FsJVcRB0rmsGTn81QqXaejWuIl333m0bJw4su/y8BHD5ebqKVFKEY/Kg/mKu2ktvHsBNewKmqgXTqxVPbeKQasSmdl+YQpAv+t3iHe8aWTy/oUPWouwfFswFLvFnpvARBrUDzcoRP5tGC7OXt+O/zVUFzwi9ZX8E2wtW7y8vlqk0+JydMldJi5BpcGioSxiUo5H32nuCrbKSuo6rEzgwTmdLHNAFm60geEIys01Gbslo2z8q7HxKxHiT796V73rQqvxY0NZYuDR9j/iMhaZsKC1csYfUeU1wrSg/wdKwlzAGgNp3srK6pzYLVGxHTvSgH0EAH3xoPMT6VLGKKdnJEHb82Xz9UZnwDKlTsTM76podWSE6w268+7tosrfcTqPgn1rsIBCs3t70Jba3Mqeb/X8NHzo/PsqX4eIb81yx/fVcdF0yQ2KAMlkn2OWvxR3frubfKOlYTk/OrpezKUZ/RLkyURF8yIGZD04CRsni2P3nZPiRh/D/vWgECJAIit8nYWk++jWLG3RRwYbIKzpXelNhauggAaoqG6H5wUIT/2+zb9XAKnVRY1Irrg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, 10 Jul 2023 13:43:23 PDT (-0700), willy@infradead.org wrote: > Add set_ptes(), update_mmu_cache_range() and flush_dcache_folio(). > Change the PG_dcache_clean flag from being per-page to per-folio. > > Signed-off-by: Matthew Wilcox (Oracle) > Reviewed-by: Alexandre Ghiti > Acked-by: Mike Rapoport (IBM) > Cc: Paul Walmsley > Cc: Palmer Dabbelt > Cc: Albert Ou > Cc: linux-riscv@lists.infradead.org > --- > arch/riscv/include/asm/cacheflush.h | 19 +++++++-------- > arch/riscv/include/asm/pgtable.h | 38 +++++++++++++++++++---------- > arch/riscv/mm/cacheflush.c | 13 +++------- > 3 files changed, 37 insertions(+), 33 deletions(-) > > diff --git a/arch/riscv/include/asm/cacheflush.h b/arch/riscv/include/asm/cacheflush.h > index 8091b8bf4883..0d8c92c5dfb7 100644 > --- a/arch/riscv/include/asm/cacheflush.h > +++ b/arch/riscv/include/asm/cacheflush.h > @@ -15,20 +15,19 @@ static inline void local_flush_icache_all(void) > > #define PG_dcache_clean PG_arch_1 > > -static inline void flush_dcache_page(struct page *page) > +static inline void flush_dcache_folio(struct folio *folio) > { > - /* > - * HugeTLB pages are always fully mapped and only head page will be > - * set PG_dcache_clean (see comments in flush_icache_pte()). > - */ > - if (PageHuge(page)) > - page = compound_head(page); > - > - if (test_bit(PG_dcache_clean, &page->flags)) > - clear_bit(PG_dcache_clean, &page->flags); > + if (test_bit(PG_dcache_clean, &folio->flags)) > + clear_bit(PG_dcache_clean, &folio->flags); > } > +#define flush_dcache_folio flush_dcache_folio > #define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 1 > > +static inline void flush_dcache_page(struct page *page) > +{ > + flush_dcache_folio(page_folio(page)); > +} > + > /* > * RISC-V doesn't have an instruction to flush parts of the instruction cache, > * so instead we just flush the whole thing. > diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h > index 2137e36595b3..c8f897ed5fd0 100644 > --- a/arch/riscv/include/asm/pgtable.h > +++ b/arch/riscv/include/asm/pgtable.h > @@ -445,8 +445,9 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot) > > > /* Commit new configuration to MMU hardware */ > -static inline void update_mmu_cache(struct vm_area_struct *vma, > - unsigned long address, pte_t *ptep) > +static inline void update_mmu_cache_range(struct vm_fault *vmf, > + struct vm_area_struct *vma, unsigned long address, > + pte_t *ptep, unsigned int nr) > { > /* > * The kernel assumes that TLBs don't cache invalid entries, but > @@ -455,8 +456,11 @@ static inline void update_mmu_cache(struct vm_area_struct *vma, > * Relying on flush_tlb_fix_spurious_fault would suffice, but > * the extra traps reduce performance. So, eagerly SFENCE.VMA. > */ > - local_flush_tlb_page(address); > + while (nr--) > + local_flush_tlb_page(address + nr * PAGE_SIZE); > } > +#define update_mmu_cache(vma, addr, ptep) \ > + update_mmu_cache_range(NULL, vma, addr, ptep, 1) > > #define __HAVE_ARCH_UPDATE_MMU_TLB > #define update_mmu_tlb update_mmu_cache > @@ -487,8 +491,7 @@ static inline void set_pte(pte_t *ptep, pte_t pteval) > > void flush_icache_pte(pte_t pte); > > -static inline void __set_pte_at(struct mm_struct *mm, > - unsigned long addr, pte_t *ptep, pte_t pteval) > +static inline void __set_pte_at(pte_t *ptep, pte_t pteval) > { > if (pte_present(pteval) && pte_exec(pteval)) > flush_icache_pte(pteval); > @@ -496,17 +499,26 @@ static inline void __set_pte_at(struct mm_struct *mm, > set_pte(ptep, pteval); > } > > -static inline void set_pte_at(struct mm_struct *mm, > - unsigned long addr, pte_t *ptep, pte_t pteval) > +static inline void set_ptes(struct mm_struct *mm, unsigned long addr, > + pte_t *ptep, pte_t pteval, unsigned int nr) > { > - page_table_check_ptes_set(mm, addr, ptep, pteval, 1); > - __set_pte_at(mm, addr, ptep, pteval); > + page_table_check_ptes_set(mm, addr, ptep, pteval, nr); > + > + for (;;) { > + __set_pte_at(ptep, pteval); > + if (--nr == 0) > + break; > + ptep++; > + addr += PAGE_SIZE; > + pte_val(pteval) += 1 << _PAGE_PFN_SHIFT; > + } > } > +#define set_ptes set_ptes > > static inline void pte_clear(struct mm_struct *mm, > unsigned long addr, pte_t *ptep) > { > - __set_pte_at(mm, addr, ptep, __pte(0)); > + __set_pte_at(ptep, __pte(0)); > } > > #define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS > @@ -515,7 +527,7 @@ static inline int ptep_set_access_flags(struct vm_area_struct *vma, > pte_t entry, int dirty) > { > if (!pte_same(*ptep, entry)) > - set_pte_at(vma->vm_mm, address, ptep, entry); > + __set_pte_at(ptep, entry); > /* > * update_mmu_cache will unconditionally execute, handling both > * the case that the PTE changed and the spurious fault case. > @@ -688,14 +700,14 @@ static inline void set_pmd_at(struct mm_struct *mm, unsigned long addr, > pmd_t *pmdp, pmd_t pmd) > { > page_table_check_pmd_set(mm, addr, pmdp, pmd); > - return __set_pte_at(mm, addr, (pte_t *)pmdp, pmd_pte(pmd)); > + return __set_pte_at((pte_t *)pmdp, pmd_pte(pmd)); > } > > static inline void set_pud_at(struct mm_struct *mm, unsigned long addr, > pud_t *pudp, pud_t pud) > { > page_table_check_pud_set(mm, addr, pudp, pud); > - return __set_pte_at(mm, addr, (pte_t *)pudp, pud_pte(pud)); > + return __set_pte_at((pte_t *)pudp, pud_pte(pud)); > } > > #ifdef CONFIG_PAGE_TABLE_CHECK > diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c > index fbc59b3f69f2..f1387272a551 100644 > --- a/arch/riscv/mm/cacheflush.c > +++ b/arch/riscv/mm/cacheflush.c > @@ -82,18 +82,11 @@ void flush_icache_mm(struct mm_struct *mm, bool local) > #ifdef CONFIG_MMU > void flush_icache_pte(pte_t pte) > { > - struct page *page = pte_page(pte); > + struct folio *folio = page_folio(pte_page(pte)); > > - /* > - * HugeTLB pages are always fully mapped, so only setting head page's > - * PG_dcache_clean flag is enough. > - */ > - if (PageHuge(page)) > - page = compound_head(page); > - > - if (!test_bit(PG_dcache_clean, &page->flags)) { > + if (!test_bit(PG_dcache_clean, &folio->flags)) { > flush_icache_all(); > - set_bit(PG_dcache_clean, &page->flags); > + set_bit(PG_dcache_clean, &folio->flags); > } > } > #endif /* CONFIG_MMU */ Sorry I missed this earlier. IIRC it ended up somewhere, but Acked-by: Palmer Dabbelt anyway. Thanks!