From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44939C19F4F for ; Wed, 8 May 2024 11:41:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D8C0D8D0003; Wed, 8 May 2024 07:41:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D3C598D0001; Wed, 8 May 2024 07:41:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BDC5F8D0003; Wed, 8 May 2024 07:41:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9951B8D0001 for ; Wed, 8 May 2024 07:41:37 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 554D2A0237 for ; Wed, 8 May 2024 11:41:37 +0000 (UTC) X-FDA: 82095038634.24.0F172A7 Received: from mail-qk1-f169.google.com (mail-qk1-f169.google.com [209.85.222.169]) by imf04.hostedemail.com (Postfix) with ESMTP id 816F54000C for ; Wed, 8 May 2024 11:41:35 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=XKI1DpcW; dmarc=none; spf=pass (imf04.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.222.169 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715168495; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=p6iTAuwd3BKroPVlGkkNGENGW7uKwPOOcXR0pGzhYec=; b=hdBfElNv+nvIATyComDQ1QNyttXj4pikQxQvlP1Cr36eRjzcwyQnl7dWytJKGgzkrXGVGe gva+A2riyx8EUdlj0BuHInFCEB9NfCoNDgzSB3RPSZ25nM+QrBKoKfMrft4ddWRrz6k9iw UA0mMBVxE7DRYo/yN29QlNQtzEq0yxo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715168495; a=rsa-sha256; cv=none; b=brjkyBh6coD5JQWKS8pxMhspTLTZ6nM+yoJTHTjpQY85zLCav8mbVYuDyJcq0LQqPX1UnT lQVAaKDECPmqm0Bg1wSQAExqOGWQ8Qchaf0mzSxyC9x4oLhHz5DYUrkzbM+VSc3WRO3Rq+ gX6ofxqK624WmRseHfSO6Gi2Ijt06Nc= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=XKI1DpcW; dmarc=none; spf=pass (imf04.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.222.169 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com Received: by mail-qk1-f169.google.com with SMTP id af79cd13be357-792b8d98a56so35487985a.2 for ; Wed, 08 May 2024 04:41:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1715168494; x=1715773294; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=p6iTAuwd3BKroPVlGkkNGENGW7uKwPOOcXR0pGzhYec=; b=XKI1DpcWli9wwJubD8Nu+TMq7W5ZK/lBT7/DPgDv7cjusG8UCBUhrTcbEOJY+nhS7P csBbi0w4aC893K1UDt0yDcMxjekujtDUeUY8y5/5V1MWOkvLFJpF7dA4cEOMC9Pa0JUo 1yGCEQG29VOaxsJMKc9FYvE7fsmSa61PSlyc0AgGfUmhgzA+i4V/skhnLhKuf6eFLUS4 1MBBVq/zQeED/r2jTWokh45Et1ZgoBu9/b3HJj6R5HJthMtNJidx+6N3MtMhSY8DUlWI 91ShehUsSwb7evD5zVSgeY4T5SCsmYvluLB6qC/j0+b7VeownGYaf5BlV/dr8i2Lcds+ wZfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715168494; x=1715773294; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p6iTAuwd3BKroPVlGkkNGENGW7uKwPOOcXR0pGzhYec=; b=Iz2KEgrJuTm7DQbF4BydMs1tbMv5tAbK5IYPhW0HLzJ0hmkRC3W95zIIHpS2IOa1Ty fsLqkWlP1K8E6iCTK3mmtwWDEDAExZ9d2iWQFJpTK093YBptzQoBURL86zVefW010Ly0 lSABE4fEutstgIMoBn+XooxqZfur9p2Swfcpn0CQql/SKUMB9UiBUMQjsCYMTlBsxfQN Bbr0SOk0tJy5xZusA6EdaLyycPnPOaDA3EqaQfYbrEfWjPK0OOzMIOaVU3HWQn1mVxof wImNgl7hYorw41JChqiV4hl3HFAhs3ExLjATO1uMa1M63qyrvyJosvM6MKKRGFcF62Y1 zdUg== X-Forwarded-Encrypted: i=1; AJvYcCUzriXM8mjpT5eWM0ANOrHPkeHQyHuXCA+FZCP7+dlkLP35l6IpoEQSb1H6ryO8w7zOUNSUxiNDFzgRV/vxmwCg0Cg= X-Gm-Message-State: AOJu0Yz8YuUTSMtC0p+sBx2SF91DY++bCPIMWXfSPXjv5+iJJMsJ0vnS OW4JZDSFTVXcBx/yJ9gkuhwKSsx01GvU0Gm0NzJkrkyjuY5q0fJyMHNxV7OKZyI= X-Google-Smtp-Source: AGHT+IEf9c2lQsdOpkInfpCnqc/bzKruRrtsh40+8uzVrWPXvCoLmGvCA0UiasJn6iJt4vC5DbH2zA== X-Received: by 2002:a05:620a:394b:b0:78e:df14:19dc with SMTP id af79cd13be357-792b26d4b61mr350150385a.20.1715168494636; Wed, 08 May 2024 04:41:34 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id de42-20020a05620a372a00b0078d7213de13sm5753377qkb.136.2024.05.08.04.41.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 May 2024 04:41:34 -0700 (PDT) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Ryan Roberts , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH RESEND v2 7/9] mm: Use common huge_ptep_set_access_flags() function for riscv/arm64 Date: Wed, 8 May 2024 13:34:17 +0200 Message-Id: <20240508113419.18620-8-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240508113419.18620-1-alexghiti@rivosinc.com> References: <20240508113419.18620-1-alexghiti@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 816F54000C X-Stat-Signature: d33ta136wyij7cr14mde9js6mzmq888p X-Rspam-User: X-HE-Tag: 1715168495-51737 X-HE-Meta: U2FsdGVkX1+/vrGDzPyONMGuzTztE6BO4AMd9rJQPU082BrLLVtMOzn+sweE0iYE7lRjYDdDqmbRyfeLK7XXcwC3wF4ud3Tru1pMlOEMBrG2pRG4mBQQLBnZQ99j8kkxq5LHNezA0aBvKsGZG0JWkDczPTiosL3qhuT4YyPcqgCETRSNR+obSfySayXeFjejvXjM5w64K+CcJdEPXnfQMvw7f9o8Vd8aOK0CFiTrgVuQGJQHKhW+5BDs2kguPEPktBsLPfxI6Ym55GHZ/p4KG96YSn5Edo5WINsG4wG4O1XuUHjqewNuhPIrbqX61c9MKzXuCJtkg/KckxdOOAN5fX5u/zJwF+1S+MgvNN9ejERWdeUJFNRFoMTGjdpJja5Ow0+hKNm4zs2Tb5x9/g2jxxaHy+2pZ9/5zdduhAS5Bbjc+LdFmBFGflOWgp6QwUptr57RWKhp2M2ikiRiUbm2GUnvDzw6LnOud55Q78tZXBhXGHzC5t3PJIJue2qTKqoYDoyxRKPluQuA5QhDD+3fV/z353U3PPfFeF3JOqR+VvdC4iRLi6BhjL5Nli1mIALqTVPThFdf5e+55zr8mNWuQiRm1krCMtvunKHPmaQ73icxcYk2XXZfTkDDAynb1XS6wMuFX3euJG3nc66KNyLfSxyhZvN12aguvBD+lTO7EtWhbsu/1cvuY2L0H8+jIvVs2ceeLX0Ugz2Qqrcq0/14fjmEc4Vc8UNpvx+OJ9t5XiHH0yfW+ZTxdID52flSdmGd8unqOSVfukeIqyZMk/wZquyZltXGYFhNdEoVyttGI7ZljgRHXTqHLZeiAMtnnHoQk4wqnC3c5+zCVPXspea+3XZdRdjxhQpWtW9Xgm+rCm9eKwgPdNUEnu24rEd+6qHvCKvoPolPu51IdA+gsL3rvllrs4+WBNpFDHJSXaeVeyFQTKqbxRUsBx5e8+/GM/pv6h1CEOmokFotpUUMSiB Nhvy1S0b M0hl4RskQfMvsgCpprxlGim5/A79VFSM1adCAu2y5zfN3nTNqVZc4ez23hdK1276sS9/PJXtvhOG/FlgoLaqkGVI8O9+NDsru2SNr52UVkqRtyflGbEliBGcW8TUKUqM1uROAIs3tmaULIIWuJA0UBP23jX56ciiwrj9hs4cxgD3s7lHjTcyrGiQ7wxdTEmn1AUrWpNKJE4zNpXy7U2ZiopTRw4iK1Cvr9BNP0S8UFfBTPj4CMBkdjamRwiN9B0tuijSsAvj8JkdwKSNrDC8sbx+jGMKSySMLLK5cxqV7Ncg5OfdDwQem0BQZQqneU/w99MnJ8IFG6NE+uSKFhhs9vKKki+ItARQH7YWzjPndMnnCFwXXQckqV2RlnNxueEATtF5qrmaZAbfUefizw6tSeeL0xTp1ZLP1VHbdVWaOOG0mbcxtWFLjw4Q5hG2hZZ633oGyNh25MyCTHi8IEYR9SUkVHjSimuQ9kHBWVWYmalr1SVHB4p09AeIS2A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Both architectures have almost the same implementation: __cont_access_flags_changed() is also correct on riscv and brings the same benefits (ie don't do anything if the flags are unchanged). As in the previous commit, get_clear_contig_flush() is duplicated in both the arch and the generic codes, it will be removed from the arch code when the last reference there gets moved to the generic code. Signed-off-by: Alexandre Ghiti --- arch/arm64/mm/hugetlbpage.c | 65 --------------------------- arch/riscv/include/asm/pgtable.h | 7 +-- arch/riscv/mm/hugetlbpage.c | 29 ------------ arch/riscv/mm/pgtable.c | 6 +-- mm/contpte.c | 75 ++++++++++++++++++++++++++++++++ 5 files changed, 82 insertions(+), 100 deletions(-) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 5ace4bf7ce35..052a5bf2926c 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -277,71 +277,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -/* - * huge_ptep_set_access_flags will update access flags (dirty, accesssed) - * and write permission. - * - * For a contiguous huge pte range we need to check whether or not write - * permission has to change only on the first pte in the set. Then for - * all the contiguous ptes we need to check whether or not there is a - * discrepancy between dirty or young. - */ -static int __cont_access_flags_changed(pte_t *ptep, pte_t pte, int ncontig) -{ - int i; - - if (pte_write(pte) != pte_write(__ptep_get(ptep))) - return 1; - - for (i = 0; i < ncontig; i++) { - pte_t orig_pte = __ptep_get(ptep + i); - - if (pte_dirty(pte) != pte_dirty(orig_pte)) - return 1; - - if (pte_young(pte) != pte_young(orig_pte)) - return 1; - } - - return 0; -} - -int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, - pte_t pte, int dirty) -{ - int ncontig, i; - size_t pgsize = 0; - unsigned long pfn = pte_pfn(pte), dpfn; - struct mm_struct *mm = vma->vm_mm; - pgprot_t hugeprot; - pte_t orig_pte; - - if (!pte_cont(pte)) - return __ptep_set_access_flags(vma, addr, ptep, pte, dirty); - - ncontig = find_num_contig(mm, addr, ptep, &pgsize); - dpfn = pgsize >> PAGE_SHIFT; - - if (!__cont_access_flags_changed(ptep, pte, ncontig)) - return 0; - - orig_pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); - - /* Make sure we don't lose the dirty or young state */ - if (pte_dirty(orig_pte)) - pte = pte_mkdirty(pte); - - if (pte_young(orig_pte)) - pte = pte_mkyoung(pte); - - hugeprot = pte_pgprot(pte); - for (i = 0; i < ncontig; i++, ptep++, addr += pgsize, pfn += dpfn) - __set_ptes(mm, addr, ptep, pfn_pte(pfn, hugeprot), 1); - - return 1; -} - void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index d976113a370d..20f62505d0bc 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -652,9 +652,8 @@ static inline void __pte_clear(struct mm_struct *mm, __set_pte_at(mm, ptep, __pte(0)); } -#define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS /* defined in mm/pgtable.c */ -extern int ptep_set_access_flags(struct vm_area_struct *vma, unsigned long address, - pte_t *ptep, pte_t entry, int dirty); +extern int __ptep_set_access_flags(struct vm_area_struct *vma, unsigned long address, + pte_t *ptep, pte_t entry, int dirty); #define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG /* defined in mm/pgtable.c */ extern int ptep_test_and_clear_young(struct vm_area_struct *vma, unsigned long address, pte_t *ptep); @@ -703,6 +702,8 @@ static inline int ptep_clear_flush_young(struct vm_area_struct *vma, #define __HAVE_ARCH_PTEP_GET_AND_CLEAR #define ptep_get_and_clear __ptep_get_and_clear #define pte_clear __pte_clear +#define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS +#define ptep_set_access_flags __ptep_set_access_flags #define pgprot_nx pgprot_nx static inline pgprot_t pgprot_nx(pgprot_t _prot) diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index a757e0b2f090..a59b776e9c8b 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -173,35 +173,6 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) return entry; } -int huge_ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long addr, - pte_t *ptep, - pte_t pte, - int dirty) -{ - struct mm_struct *mm = vma->vm_mm; - size_t pgsize; - pte_t orig_pte; - int pte_num; - - if (!pte_napot(pte)) - return ptep_set_access_flags(vma, addr, ptep, pte, dirty); - - pte_num = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, &pgsize); - - orig_pte = get_clear_contig_flush(mm, addr, ptep, pte_num); - - if (pte_dirty(orig_pte)) - pte = pte_mkdirty(pte); - - if (pte_young(orig_pte)) - pte = pte_mkyoung(pte); - - set_ptes(mm, addr, ptep, pte, pte_num); - - return true; -} - void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) diff --git a/arch/riscv/mm/pgtable.c b/arch/riscv/mm/pgtable.c index 533ec9055fa0..e86df7ef193c 100644 --- a/arch/riscv/mm/pgtable.c +++ b/arch/riscv/mm/pgtable.c @@ -5,9 +5,9 @@ #include #include -int ptep_set_access_flags(struct vm_area_struct *vma, - unsigned long address, pte_t *ptep, - pte_t entry, int dirty) +int __ptep_set_access_flags(struct vm_area_struct *vma, + unsigned long address, pte_t *ptep, + pte_t entry, int dirty) { if (!pte_same(ptep_get(ptep), entry)) __set_pte_at(vma->vm_mm, ptep, entry); diff --git a/mm/contpte.c b/mm/contpte.c index 68eb1634b922..9c7a9f250bca 100644 --- a/mm/contpte.c +++ b/mm/contpte.c @@ -13,6 +13,7 @@ * - __set_ptes() * - __ptep_get_and_clear() * - __pte_clear() + * - __ptep_set_access_flags() * - pte_cont() * - arch_contpte_get_num_contig() */ @@ -23,6 +24,7 @@ * - set_huge_pte_at() * - huge_pte_clear() * - huge_ptep_get_and_clear() + * - huge_ptep_set_access_flags() */ pte_t huge_ptep_get(pte_t *ptep) @@ -158,3 +160,76 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm, return get_clear_contig(mm, addr, ptep, pgsize, ncontig); } + +/* + * huge_ptep_set_access_flags will update access flags (dirty, accesssed) + * and write permission. + * + * For a contiguous huge pte range we need to check whether or not write + * permission has to change only on the first pte in the set. Then for + * all the contiguous ptes we need to check whether or not there is a + * discrepancy between dirty or young. + */ +static int __cont_access_flags_changed(pte_t *ptep, pte_t pte, int ncontig) +{ + int i; + + if (pte_write(pte) != pte_write(__ptep_get(ptep))) + return 1; + + for (i = 0; i < ncontig; i++) { + pte_t orig_pte = __ptep_get(ptep + i); + + if (pte_dirty(pte) != pte_dirty(orig_pte)) + return 1; + + if (pte_young(pte) != pte_young(orig_pte)) + return 1; + } + + return 0; +} + +static pte_t get_clear_contig_flush(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long pgsize, + unsigned long ncontig) +{ + pte_t orig_pte = get_clear_contig(mm, addr, ptep, pgsize, ncontig); + struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); + + flush_tlb_range(&vma, addr, addr + (pgsize * ncontig)); + return orig_pte; +} + +int huge_ptep_set_access_flags(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep, + pte_t pte, int dirty) +{ + int ncontig; + size_t pgsize = 0; + struct mm_struct *mm = vma->vm_mm; + pte_t orig_pte; + + if (!pte_cont(pte)) + return __ptep_set_access_flags(vma, addr, ptep, pte, dirty); + + ncontig = arch_contpte_get_num_contig(vma->vm_mm, addr, ptep, 0, &pgsize); + + if (!__cont_access_flags_changed(ptep, pte, ncontig)) + return 0; + + orig_pte = get_clear_contig_flush(mm, addr, ptep, pgsize, ncontig); + + /* Make sure we don't lose the dirty or young state */ + if (pte_dirty(orig_pte)) + pte = pte_mkdirty(pte); + + if (pte_young(orig_pte)) + pte = pte_mkyoung(pte); + + set_contptes(mm, addr, ptep, pte, ncontig, pgsize); + + return 1; +} -- 2.39.2