From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5E47C04FFE for ; Wed, 8 May 2024 19:24:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 491276B009C; Wed, 8 May 2024 15:24:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 440E26B009D; Wed, 8 May 2024 15:24:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2E1BB6B009E; Wed, 8 May 2024 15:24:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 10A896B009C for ; Wed, 8 May 2024 15:24:45 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 9B6E01C159A for ; Wed, 8 May 2024 19:24:44 +0000 (UTC) X-FDA: 82096205688.01.439C509 Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) by imf01.hostedemail.com (Postfix) with ESMTP id AB63840012 for ; Wed, 8 May 2024 19:24:42 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b="uLEtc8/o"; dmarc=none; spf=pass (imf01.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.41 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715196282; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bID6WfA+atBgi3UdotMTiBHvNsJiSSNtZP3FlzKWl8Q=; b=CEAOCz/np4HBGk45TSv65Mm4o04FpS3ZIgA5zGOTRyfWFLaG3KF4Xu5NQYXkmQ+tzI5clZ wp3we1z8rbzAHHPDQBqRmBhN+fzvlX7B8LM2KkpefRhuWpfuol/Xx3nZQHff5Z0RHmbklP wtgZK8tJeoODwv9k3RscksTwRg22j2s= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b="uLEtc8/o"; dmarc=none; spf=pass (imf01.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.41 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715196282; a=rsa-sha256; cv=none; b=nh19IDBcT38KBWMRrb0KPnehpXn+g86OAzcciW/8U5781n/M8wjpADYbTZ0dUFPK/Dg9bJ r3lTMLYZIjZDZoreeeoqv9wA3Xc/6xILmzObdv452Ihw2O3xYVDrI4mEUOyc6DOaYaovSK CcWxJxa3286QZf3BkWIlonB7Q7HqrPE= Received: by mail-wm1-f41.google.com with SMTP id 5b1f17b1804b1-41f9ce16ed8so1067025e9.0 for ; Wed, 08 May 2024 12:24:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1715196281; x=1715801081; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bID6WfA+atBgi3UdotMTiBHvNsJiSSNtZP3FlzKWl8Q=; b=uLEtc8/oMEzMe2XEqGj5TT6PBM+MtHk13ehyAmfmsRryCwBwFtnhqKEU801hfOhKVR /9LBiXhXYAqTc9J0ZZZuXYg7zxUQayHhHG6a+z1GIvnSzAYV7MHemeqomDcHaOTtSiDq NaA072OoT5/heaDf7k0P1nT71PeeTnf9GzlyMMoclFkct7VLBJ2cj+cgKigp0UwxPX8n rBXGy+bTIlhJJCV3CDPDmNRbGaKa19LE57Vm0i14gzg6cYYKkn2DhWVGN9gRWGjTDOnl EVq/6XMZPhPotBmKkVgEZhhStFLy/Bgze1rXyiCOanc43DDhZVG9LVpYxQKuKIRZox71 gdiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715196281; x=1715801081; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bID6WfA+atBgi3UdotMTiBHvNsJiSSNtZP3FlzKWl8Q=; b=ePNvGb2bKp5ypS8lCH6N5Rh0m02oXy9hh8538HzA9rSNeu+zIj+q01bCLtkg1cRhDp mZNFmEDeZc1gCVcjoSaAzM+hY0Dc3YXJ2+T+GnpiT1/qeuqyZouJFPwGTZWD3VivLtHM dIegN5HZDZXMj1PJxLA2gupOhJbldykbfb9E3z3kqZqRLDPyKHCl2A2Emg43inqj76oq xls04P9zdW6svcmr2JhYmylaKb94c2KMKDdw4EaX0SNDAsuhmbR16Fri8TI1ajOYbJWH 2Xsha1H4BbC6q17L+Wp7TC7BXk9BDksJqjmh0/IjD2FQcgzT+VY8mblSJvEWFs1q8KXI Vj2Q== X-Forwarded-Encrypted: i=1; AJvYcCX6vI92icnxs4++6xyrewMGHV844AVMmCqINhffjq4VWAJnMXS0sPvkqU50gtrrVh+R4y0k8NI4EJf2DdB+mNpPnCc= X-Gm-Message-State: AOJu0YwQ6aLB1WWn0mPOEJP9oPef5Rlu+MZ8jJchOPMc8LLNkaGmk31Z k4sXXQ/yBla5RntFewoZFVv1+ZtiUad1qwiCJJrwCMyrrQUQNhSSe1IyBj8Y+lg= X-Google-Smtp-Source: AGHT+IFH7sC2GowLam3Jd+gaWz8gHH4sQsrwabW1J5xMuNqZ6dTlKzWFO1Hh7H4Mfqebgr3mIcqVow== X-Received: by 2002:a05:600c:1991:b0:41d:803c:b945 with SMTP id 5b1f17b1804b1-41f71309fafmr39869075e9.10.1715196281225; Wed, 08 May 2024 12:24:41 -0700 (PDT) Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-41f882089cbsm32567815e9.48.2024.05.08.12.24.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 May 2024 12:24:40 -0700 (PDT) From: Alexandre Ghiti To: Ryan Roberts , Catalin Marinas , Will Deacon , Alexander Potapenko , Marco Elver , Dmitry Vyukov , Paul Walmsley , Palmer Dabbelt , Albert Ou , Ard Biesheuvel , Anup Patel , Atish Patra , Andrey Ryabinin , Andrey Konovalov , Vincenzo Frascino , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-riscv@lists.infradead.org, linux-efi@vger.kernel.org, kvm@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH 05/12] mm, riscv, arm64: Use common set_pte() function Date: Wed, 8 May 2024 21:19:24 +0200 Message-Id: <20240508191931.46060-6-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240508191931.46060-1-alexghiti@rivosinc.com> References: <20240508191931.46060-1-alexghiti@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: AB63840012 X-Stat-Signature: jc7d57rwqny4gn5xmyxq7pddte8jbfgt X-Rspam-User: X-HE-Tag: 1715196282-794489 X-HE-Meta: U2FsdGVkX187U+J8Aq03HCUNF/zvvX0JQ7t4eznsBK4oOLc60yReH05ba/gxn+9pwi4ZijE3Zw31p5uxfnZ6a92g1L8n54YVzYYaCr7AoTYepLFg6ZlcYr9P6AIAd8tYa/TbfXGPac1kw5dLNz5JPdtRfD/XQIsXBCUA5im1wN3coWlvMi8lfBcyLNfGXQfHMmfL7LCBz3atmM7r8hz/r+aM0P1QCyE+qF9cvmc0/UwNoHPRvFmZaIiCG2pdjt4g9eYxWGE6o4jxv0FU/PpKhoW3iBQJz0uBQZ41mFu9sIZ40ABsHeP0PRRotEj0ALB6hGEN4qsiSfLWZQKxvER9JIgPZ/YhffqhQNGEgqKK+yYA9QJAwunVa6gIOEj1lDxsfcHrKzGpCGNXc0grm0SEmVkooVGB3n6opMOsE5ZnQJ5q2jqdYFUCENufMGg4SyuxzxwvzAmTKsY3BQL1Al09cZBUaqI0mYRHSIijl4PVXSFeFknilrBAsORrbF4neV5enVaTNjllZVjYfbOX+/WZjy1lQnqj63S5uE5p2StebtdWLOHsgEAvhEvjtd+eYMGN8pKOv9LN636qXMe74f9I8Wl84hhWTxNqV+21LaG2fCNBrlkKbqoOLJuFmZ9A2FCaugxlnYRN8PtUnSTn23Qe9QXawDHgwchnYAzRTDWBwxwNwkngRwL/9Cy6/CaXX2Y2laUtF6tw5O/8Jzp1YZxa2tHjbrk1IjEF/ZI/LkVlcCsQV44fMvL1UXywTx8goY0PHkmRWv3DV5MJwDMKodPuGbxt3xrrMlZJ86UWI15st8iV1duMFiZ2YBN7MOny9KzSMMw0h/e3BMCj1k6b17nPbnrFID540LGMdgNP4G7SU36U4EuFURTCcOIrh0j8G+QIggPh0hZZIQS9uxozFdX1eW4b7B5G8v5FMrBqDo3MloID+JUSVGCYcyBxAyFh4JXTX2SAzL7SnzNUs1gcFtz WijD5LNU XJGtV5dUEx4VjE9Y7JueviGgHl5khtaVtfX+QycVfeO0lz5o4eD2uqtuxvPl7asm9DWs64mfp/mcgG80mrUw6o5bGPN1ys+rpqc8rO3ERYHomODYAi0Au4rmqnp5tKopruBuQ95eOlIIbMRFXalJt98MjbNVEAlAmxOuaTHS1T7Jl5xxy6ZAcYvLyzZk2d0L6aXTCYyM0Co3lmiNsHY8XFrSEycymi9jX5KjsjnYZspvo5okXLlOOriifpcaAu2bwJmtqdvaynWkqM34qPAUlusH5KvfTGbmQViyiBCFXXSuetRaCn201EjHZFFDvaHLICQUl28XVS5zIqcTkHZjsir6aDBvIsMqentAK5lTkmzrPwpWem7xa0WYBwobhcoYTunJgqDI0yfXTNi7sZrr9xgsM0l6h2+DM1DlbQizYxxmOnRTgz6lCmSATZyvAH2LxYGSSgGdUk0Bd04cmyQ3/JT4oylj1gbAzLVlBy22McUJJIozYUPuPQIt0PO3+C5T5zYs3/7/O4uapJQbhKVx877fUMkg7ekBwpd5QpritzvFDnZ5Nf8fiaDuJFXuEvRkbTJz9 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Make riscv use the contpte aware set_pte() function from arm64. Signed-off-by: Alexandre Ghiti --- arch/arm64/include/asm/pgtable.h | 16 ++-------------- arch/riscv/include/asm/kfence.h | 4 ++-- arch/riscv/include/asm/pgtable.h | 7 +++++-- arch/riscv/kernel/efi.c | 2 +- arch/riscv/kernel/hibernate.c | 2 +- arch/riscv/kvm/mmu.c | 10 +++++----- arch/riscv/mm/init.c | 2 +- arch/riscv/mm/kasan_init.c | 14 +++++++------- arch/riscv/mm/pageattr.c | 4 ++-- mm/contpte.c | 18 ++++++++++++++++++ 10 files changed, 44 insertions(+), 35 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 8a0603257436..bb6210fb72c8 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -1432,20 +1432,8 @@ extern pte_t ptep_get(pte_t *ptep); extern pte_t ptep_get_lockless(pte_t *ptep); #define ptep_get_lockless ptep_get_lockless -static inline void set_pte(pte_t *ptep, pte_t pte) -{ - /* - * We don't have the mm or vaddr so cannot unfold contig entries (since - * it requires tlb maintenance). set_pte() is not used in core code, so - * this should never even be called. Regardless do our best to service - * any call and emit a warning if there is any attempt to set a pte on - * top of an existing contig range. - */ - pte_t orig_pte = __ptep_get(ptep); - - WARN_ON_ONCE(pte_valid_cont(orig_pte)); - __set_pte(ptep, pte_mknoncont(pte)); -} +extern void set_pte(pte_t *ptep, pte_t pte); +#define set_pte set_pte extern void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte, unsigned int nr); diff --git a/arch/riscv/include/asm/kfence.h b/arch/riscv/include/asm/kfence.h index f303fef8591c..36e9f638abf6 100644 --- a/arch/riscv/include/asm/kfence.h +++ b/arch/riscv/include/asm/kfence.h @@ -18,9 +18,9 @@ static inline bool kfence_protect_page(unsigned long addr, bool protect) pte_t *pte = virt_to_kpte(addr); if (protect) - set_pte(pte, __pte(pte_val(__ptep_get(pte)) & ~_PAGE_PRESENT)); + __set_pte(pte, __pte(pte_val(__ptep_get(pte)) & ~_PAGE_PRESENT)); else - set_pte(pte, __pte(pte_val(__ptep_get(pte)) | _PAGE_PRESENT)); + __set_pte(pte, __pte(pte_val(__ptep_get(pte)) | _PAGE_PRESENT)); flush_tlb_kernel_range(addr, addr + PAGE_SIZE); diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 62cad1b974f1..4f8f673787e7 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -539,7 +539,7 @@ static inline int pte_same(pte_t pte_a, pte_t pte_b) * a page table are directly modified. Thus, the following hook is * made available. */ -static inline void set_pte(pte_t *ptep, pte_t pteval) +static inline void __set_pte(pte_t *ptep, pte_t pteval) { WRITE_ONCE(*ptep, pteval); } @@ -551,7 +551,7 @@ static inline void __set_pte_at(struct mm_struct *mm, pte_t *ptep, pte_t pteval) if (pte_present(pteval) && pte_exec(pteval)) flush_icache_pte(mm, pteval); - set_pte(ptep, pteval); + __set_pte(ptep, pteval); } #define PFN_PTE_SHIFT _PAGE_PFN_SHIFT @@ -790,11 +790,14 @@ extern pte_t ptep_get_lockless(pte_t *ptep); extern void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pteval, unsigned int nr); #define set_ptes set_ptes +extern void set_pte(pte_t *ptep, pte_t pte); +#define set_pte set_pte #else /* CONFIG_THP_CONTPTE */ #define ptep_get __ptep_get #define set_ptes __set_ptes +#define set_pte __set_pte #endif /* CONFIG_THP_CONTPTE */ diff --git a/arch/riscv/kernel/efi.c b/arch/riscv/kernel/efi.c index 3d2a635c69ac..673eca7705ba 100644 --- a/arch/riscv/kernel/efi.c +++ b/arch/riscv/kernel/efi.c @@ -72,7 +72,7 @@ static int __init set_permissions(pte_t *ptep, unsigned long addr, void *data) val = pte_val(pte) & ~_PAGE_EXEC; pte = __pte(val); } - set_pte(ptep, pte); + __set_pte(ptep, pte); return 0; } diff --git a/arch/riscv/kernel/hibernate.c b/arch/riscv/kernel/hibernate.c index 671b686c0158..97ed3df7a308 100644 --- a/arch/riscv/kernel/hibernate.c +++ b/arch/riscv/kernel/hibernate.c @@ -186,7 +186,7 @@ static int temp_pgtable_map_pte(pmd_t *dst_pmdp, pmd_t *src_pmdp, unsigned long pte_t pte = READ_ONCE(*src_ptep); if (pte_present(pte)) - set_pte(dst_ptep, __pte(pte_val(pte) | pgprot_val(prot))); + __set_pte(dst_ptep, __pte(pte_val(pte) | pgprot_val(prot))); } while (dst_ptep++, src_ptep++, start += PAGE_SIZE, start < end); return 0; diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c index 70c6cb3864d6..1ee6139d495f 100644 --- a/arch/riscv/kvm/mmu.c +++ b/arch/riscv/kvm/mmu.c @@ -155,7 +155,7 @@ static int gstage_set_pte(struct kvm *kvm, u32 level, next_ptep = kvm_mmu_memory_cache_alloc(pcache); if (!next_ptep) return -ENOMEM; - set_pte(ptep, pfn_pte(PFN_DOWN(__pa(next_ptep)), + __set_pte(ptep, pfn_pte(PFN_DOWN(__pa(next_ptep)), __pgprot(_PAGE_TABLE))); } else { if (gstage_pte_leaf(ptep)) @@ -167,7 +167,7 @@ static int gstage_set_pte(struct kvm *kvm, u32 level, ptep = &next_ptep[gstage_pte_index(addr, current_level)]; } - set_pte(ptep, *new_pte); + __set_pte(ptep, *new_pte); if (gstage_pte_leaf(ptep)) gstage_remote_tlb_flush(kvm, current_level, addr); @@ -251,7 +251,7 @@ static void gstage_op_pte(struct kvm *kvm, gpa_t addr, return; if (op == GSTAGE_OP_CLEAR) - set_pte(ptep, __pte(0)); + __set_pte(ptep, __pte(0)); for (i = 0; i < PTRS_PER_PTE; i++) gstage_op_pte(kvm, addr + i * next_page_size, &next_ptep[i], next_ptep_level, op); @@ -259,9 +259,9 @@ static void gstage_op_pte(struct kvm *kvm, gpa_t addr, put_page(virt_to_page(next_ptep)); } else { if (op == GSTAGE_OP_CLEAR) - set_pte(ptep, __pte(0)); + __set_pte(ptep, __pte(0)); else if (op == GSTAGE_OP_WP) - set_pte(ptep, __pte(pte_val(__ptep_get(ptep)) & ~_PAGE_WRITE)); + __set_pte(ptep, __pte(pte_val(__ptep_get(ptep)) & ~_PAGE_WRITE)); gstage_remote_tlb_flush(kvm, ptep_level, addr); } } diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index fe8e159394d8..bb5c6578204c 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -325,7 +325,7 @@ void __set_fixmap(enum fixed_addresses idx, phys_addr_t phys, pgprot_t prot) ptep = &fixmap_pte[pte_index(addr)]; if (pgprot_val(prot)) - set_pte(ptep, pfn_pte(phys >> PAGE_SHIFT, prot)); + __set_pte(ptep, pfn_pte(phys >> PAGE_SHIFT, prot)); else pte_clear(&init_mm, addr, ptep); local_flush_tlb_page(addr); diff --git a/arch/riscv/mm/kasan_init.c b/arch/riscv/mm/kasan_init.c index 381d61f42ab8..b5061cb3ce4d 100644 --- a/arch/riscv/mm/kasan_init.c +++ b/arch/riscv/mm/kasan_init.c @@ -41,7 +41,7 @@ static void __init kasan_populate_pte(pmd_t *pmd, unsigned long vaddr, unsigned do { if (pte_none(__ptep_get(ptep))) { phys_addr = memblock_phys_alloc(PAGE_SIZE, PAGE_SIZE); - set_pte(ptep, pfn_pte(PFN_DOWN(phys_addr), PAGE_KERNEL)); + __set_pte(ptep, pfn_pte(PFN_DOWN(phys_addr), PAGE_KERNEL)); memset(__va(phys_addr), KASAN_SHADOW_INIT, PAGE_SIZE); } } while (ptep++, vaddr += PAGE_SIZE, vaddr != end); @@ -327,8 +327,8 @@ asmlinkage void __init kasan_early_init(void) KASAN_SHADOW_END - (1UL << (64 - KASAN_SHADOW_SCALE_SHIFT))); for (i = 0; i < PTRS_PER_PTE; ++i) - set_pte(kasan_early_shadow_pte + i, - pfn_pte(virt_to_pfn(kasan_early_shadow_page), PAGE_KERNEL)); + __set_pte(kasan_early_shadow_pte + i, + pfn_pte(virt_to_pfn(kasan_early_shadow_page), PAGE_KERNEL)); for (i = 0; i < PTRS_PER_PMD; ++i) set_pmd(kasan_early_shadow_pmd + i, @@ -523,10 +523,10 @@ void __init kasan_init(void) kasan_mem_to_shadow((const void *)MODULES_VADDR + SZ_2G)); for (i = 0; i < PTRS_PER_PTE; i++) - set_pte(&kasan_early_shadow_pte[i], - mk_pte(virt_to_page(kasan_early_shadow_page), - __pgprot(_PAGE_PRESENT | _PAGE_READ | - _PAGE_ACCESSED))); + __set_pte(&kasan_early_shadow_pte[i], + mk_pte(virt_to_page(kasan_early_shadow_page), + __pgprot(_PAGE_PRESENT | _PAGE_READ | + _PAGE_ACCESSED))); memset(kasan_early_shadow_page, KASAN_SHADOW_INIT, PAGE_SIZE); init_task.kasan_depth = 0; diff --git a/arch/riscv/mm/pageattr.c b/arch/riscv/mm/pageattr.c index 98c9dc4b983c..d623e4fc11fc 100644 --- a/arch/riscv/mm/pageattr.c +++ b/arch/riscv/mm/pageattr.c @@ -71,7 +71,7 @@ static int pageattr_pte_entry(pte_t *pte, unsigned long addr, pte_t val = __ptep_get(pte); val = __pte(set_pageattr_masks(pte_val(val), walk)); - set_pte(pte, val); + __set_pte(pte, val); return 0; } @@ -121,7 +121,7 @@ static int __split_linear_mapping_pmd(pud_t *pudp, ptep_new = (pte_t *)page_address(pte_page); for (i = 0; i < PTRS_PER_PTE; ++i, ++ptep_new) - set_pte(ptep_new, pfn_pte(pfn + i, prot)); + __set_pte(ptep_new, pfn_pte(pfn + i, prot)); smp_wmb(); diff --git a/mm/contpte.c b/mm/contpte.c index 060e0bc1a2a3..543ae5b5a863 100644 --- a/mm/contpte.c +++ b/mm/contpte.c @@ -17,6 +17,7 @@ * - __pte_clear() * - __ptep_set_access_flags() * - __ptep_set_wrprotect() + * - __set_pte() * - pte_cont() * - arch_contpte_get_num_contig() * - pte_valid_cont() @@ -43,6 +44,7 @@ * - ptep_get() * - set_ptes() * - ptep_get_lockless() + * - set_pte() */ pte_t huge_ptep_get(pte_t *ptep) @@ -658,4 +660,20 @@ __always_inline pte_t ptep_get_lockless(pte_t *ptep) return contpte_ptep_get_lockless(ptep); } + +void set_pte(pte_t *ptep, pte_t pte) +{ + /* + * We don't have the mm or vaddr so cannot unfold contig entries (since + * it requires tlb maintenance). set_pte() is not used in core code, so + * this should never even be called. Regardless do our best to service + * any call and emit a warning if there is any attempt to set a pte on + * top of an existing contig range. + */ + pte_t orig_pte = __ptep_get(ptep); + + WARN_ON_ONCE(pte_valid_cont(orig_pte)); + __set_pte(ptep, pte_mknoncont(pte)); +} + #endif /* CONFIG_THP_CONTPTE */ -- 2.39.2