From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBFAED0E6DD for ; Mon, 21 Oct 2024 10:13:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 72D3D6B0082; Mon, 21 Oct 2024 06:13:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6DD656B0088; Mon, 21 Oct 2024 06:13:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5A4ED6B0089; Mon, 21 Oct 2024 06:13:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 333446B0082 for ; Mon, 21 Oct 2024 06:13:41 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 916461A1899 for ; Mon, 21 Oct 2024 10:13:13 +0000 (UTC) X-FDA: 82697197170.08.ADBEF23 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf24.hostedemail.com (Postfix) with ESMTP id 538A0180015 for ; Mon, 21 Oct 2024 10:13:36 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=O8CVaWgK; spf=pass (imf24.hostedemail.com: domain of chenhuacai@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=chenhuacai@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729505542; a=rsa-sha256; cv=none; b=S08zgkOpACQmrxKY8Z6Y937CRbW/jv160d3UBYukGJAPHHQnHuzFCkCKkP23qvgqDghsGW Dr+KgTq0KIdbzjMB07x+A1t2V/NJw+Ww2A8x2t1HKzn+ypaBospWsLzYBS3eSfs0gTOs/h SDg1yavKX51huC4E0flVQSQBckO8PzA= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=O8CVaWgK; spf=pass (imf24.hostedemail.com: domain of chenhuacai@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=chenhuacai@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729505542; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mSq8UGZQ3NJRlapaTTvgYjTUniLDeiulD6xo6BZpX8U=; b=UXth9UDJ76FHuOR8D8y4+/NV7AaRFFzAN823NveyTMOgeNKnbQFaUz5Wbi+Eer6+uWXZNf ltYW9u8o8atuN2Thtyd/xwI2/cf5q5MQ7jduSjrdrtGSRnuZHpGf9ORx1PNTGgE2aACRRl Gzh2qweSKPhV+cTW4o0ctmL1nMoIfXU= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id CCA42A42159 for ; Mon, 21 Oct 2024 10:13:28 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8139BC4CEE7 for ; Mon, 21 Oct 2024 10:13:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729505617; bh=jgibUuUlmzffYqsQkg5W+yVcI04OIHEGmRwn6NuO/wU=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=O8CVaWgKGMhB0cBPkApEVVEUDRMYrD7xqVCOt0yoZtyt6QGgwVQ3ptHt0jB7djrkB /mlbBfaR2RXCgCKzS25NVvzir/j4HKgSlEoKlHOzDW+xePTLFusb6wI0Cv80FyMkFI 8OTUo0f4qHR3syPBXiYCDPCToRNIQP0iwdsz/TFvZCpnfLEfrsnKbFzrMnCIHlUr8j 8gqpGl3AoPfp3a2+ejLi4UKG1bh0GpFt1mCu9/TfA7f2E4CLXNK4q5UfREhRAE08Ur LQXANATocN1JCmkfbwpA3a5SgUBtqbtChK+nOY9tIF6e+GsoD6rmvYXgP4H9Lq33fq X3UGYlLa4/2ZQ== Received: by mail-ed1-f51.google.com with SMTP id 4fb4d7f45d1cf-5c9404c0d50so4359261a12.3 for ; Mon, 21 Oct 2024 03:13:37 -0700 (PDT) X-Forwarded-Encrypted: i=1; AJvYcCWgsrPgD+mAAMoCciWpZ3KqRHHqO8ktNDA63cM9iJm/NIMFG4cMnm5CAU+4jwegjiD6FmeyIoY+RA==@kvack.org X-Gm-Message-State: AOJu0Yz8t/zdeAiEiKXhUQf0R0SisQXguZsq5U2yPfY1m0eGx9rPG3p1 0SwmMuF+R8X0MG+hJpM2jHftg/c9iINsiKQjgu0bC3bvnShpCwjo3eDuuXCMmMb9mk8yK4hjlY+ xQHq0gVufLnKXC6XmM0x747x5yZo= X-Google-Smtp-Source: AGHT+IGkScV1UZ4+JsX1t5mnmGxuvMdzS/NJr4OXeJMsMFge9N+OTDfUV2GMn59vK/I6+b7B/D8r6K8TkoU1q96HgQc= X-Received: by 2002:a17:906:478b:b0:a99:4025:82e1 with SMTP id a640c23a62f3a-a9a69c55c81mr1013062566b.41.1729505615843; Mon, 21 Oct 2024 03:13:35 -0700 (PDT) MIME-Version: 1.0 References: <20241014035855.1119220-1-maobibo@loongson.cn> <20241014035855.1119220-2-maobibo@loongson.cn> <5f76ede6-e8be-c7a9-f957-479afa2fb828@loongson.cn> In-Reply-To: From: Huacai Chen Date: Mon, 21 Oct 2024 18:13:23 +0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v2 1/3] LoongArch: Set initial pte entry with PAGE_GLOBAL for kernel space To: maobibo Cc: wuruiyang@loongson.cn, Andrey Ryabinin , Andrew Morton , David Hildenbrand , Barry Song , loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: 6rnf6f1r8a3zc7tg6sqjqk5xsci6kpho X-Rspamd-Queue-Id: 538A0180015 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1729505616-394781 X-HE-Meta: U2FsdGVkX1+kyK+5ScC7M5GStrVHB04+wwt5Jpw9b6E0N2SpV7k+Pu7gCozvo+qr+Mb0m2eInYI7IxVafiB6GmFmox2PUiTn30Gc+BNraEU6Q32W7hMSciURmCRAbCjA+QpQmOY3xlRG0NGssSxowa5w6nR2VpbN0djKDBktWJjVrwAqlSZvfWBMO+ZPI8GLQwVsadBen6jSkJO7Tsf/Eoh/wmsCgbkimBYbC+Si2bT/3FxqzzNI6D+O3z7qgJJyiEMi26empcnNN1b7RuJZOcSyp/ey4dmrDm6dTYEChOvQMBWMT4m7Xivtou5V3W98tLz+cGfh+6xvhLdCY+OYsD9dPLL40/QrpUOWm7FwyBhwzi6SKF7lt8DKvgxCASQ9x21TJdfkxfM82x+PPYgWG75EqwKAXNSa1H8PnX+6AEafBjd6PsdujvB09nTuS54BcWGKPoTb1s7uvkeJc9gj52GHPsJvhHK8KnZfKstHAp+jVxbkym4WIQhhjtqEt53dPJyViwmIb16WovCoNkp/8PMyQxeEIdHwFv5pmsXqrSlr/GTOKlwDS2rOfQPSRpz9hmPaBSmBsNJyk7W7/Fkq1Ts5Y4E7LOXxyv3+LATcjaxpsUoTYRP6xPPN2mZhXXuUGUT11xqZWTnasWYcxV0OmldNgKXyjEwEVncwtBipyGQeJiDsI209ZxePrV6mU2tHWu/M7b0PfmeNSF+89rqJFiP6x4qVd71xzJDxX1cGynekicpzc41xzseLXAhAzwHrh/o5s5pXaBY1sq2W/l+CEf1jG2kVY6+FUE8Zb+Y3Ean75OD3Yc9s+H7qnpDBaf0lzpey48Jr+w3L1+K+5c37jcsCkrXBiUjMtChIaRzC3dapFWzwExmwB9uQMtFbKmSak7tMcoyE9r+ek9+AROmvLt1yl11HOsIThxmdT9ew+W4ZshuCCIGLWgciQI8zirmJpSYS/mmO74Xm7yzAM4D 3qw4jsQH tGhMK5EExp26RYJo1DMbGPtF44vetZAfoIGFSIo/mVYE0cuWuQphYMKZ6DkdRw9pCaJXtZFyxO84gz634NGv7Cu9rr1Deajo8y4ipge7JierVdmbqpxnx2xNIjxKSfZX/s1W9ezh2rTPTJAnxWrpKwkIcxwfUlVn5Hjx1u9TuOc89zCzkUvEXNNFdYCrbq4WRgI+IHTRDXD+AHMtJ6FQJrHbfpyBMtb1Wx84YbahkQnu4WF4dNiDOtAcM+a4f1tBZi6IRrpIuq6AogkqMRlECTG7PGraz84TemjdmqBHc64BIY1L5jJeqw5znezFRgUOv3/06uq+tyopv37g1cDz/ltwSo9qxPKORSM96/aiO1BmgDYrXM6niNsMoxcP9Y9cR3lt0PleVdtjBUhJxMYgbea2t3QaTUpST1D+ra5Mbj3I5fno= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Oct 21, 2024 at 9:23=E2=80=AFAM maobibo wrote= : > > > > On 2024/10/18 =E4=B8=8B=E5=8D=882:32, Huacai Chen wrote: > > On Fri, Oct 18, 2024 at 2:23=E2=80=AFPM maobibo w= rote: > >> > >> > >> > >> On 2024/10/18 =E4=B8=8B=E5=8D=8812:23, Huacai Chen wrote: > >>> On Fri, Oct 18, 2024 at 12:16=E2=80=AFPM maobibo wrote: > >>>> > >>>> > >>>> > >>>> On 2024/10/18 =E4=B8=8B=E5=8D=8812:11, Huacai Chen wrote: > >>>>> On Fri, Oct 18, 2024 at 11:44=E2=80=AFAM maobibo wrote: > >>>>>> > >>>>>> > >>>>>> > >>>>>> On 2024/10/18 =E4=B8=8A=E5=8D=8811:14, Huacai Chen wrote: > >>>>>>> Hi, Bibo, > >>>>>>> > >>>>>>> I applied this patch but drop the part of arch/loongarch/mm/kasan= _init.c: > >>>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-= loongson.git/commit/?h=3Dloongarch-next&id=3D15832255e84494853f543b4c70ced5= 0afc403067 > >>>>>>> > >>>>>>> Because kernel_pte_init() should operate on page-table pages, not= on > >>>>>>> data pages. You have already handle page-table page in > >>>>>>> mm/kasan/init.c, and if we don't drop the modification on data pa= ges > >>>>>>> in arch/loongarch/mm/kasan_init.c, the kernel fail to boot if KAS= AN is > >>>>>>> enabled. > >>>>>>> > >>>>>> static inline void set_pte(pte_t *ptep, pte_t pteval) > >>>>>> { > >>>>>> WRITE_ONCE(*ptep, pteval); > >>>>>> - > >>>>>> - if (pte_val(pteval) & _PAGE_GLOBAL) { > >>>>>> - pte_t *buddy =3D ptep_buddy(ptep); > >>>>>> - /* > >>>>>> - * Make sure the buddy is global too (if it's !non= e, > >>>>>> - * it better already be global) > >>>>>> - */ > >>>>>> - if (pte_none(ptep_get(buddy))) { > >>>>>> -#ifdef CONFIG_SMP > >>>>>> - /* > >>>>>> - * For SMP, multiple CPUs can race, so we = need > >>>>>> - * to do this atomically. > >>>>>> - */ > >>>>>> - __asm__ __volatile__( > >>>>>> - __AMOR "$zero, %[global], %[buddy] \n" > >>>>>> - : [buddy] "+ZB" (buddy->pte) > >>>>>> - : [global] "r" (_PAGE_GLOBAL) > >>>>>> - : "memory"); > >>>>>> - > >>>>>> - DBAR(0b11000); /* o_wrw =3D 0b11000 */ > >>>>>> -#else /* !CONFIG_SMP */ > >>>>>> - WRITE_ONCE(*buddy, __pte(pte_val(ptep_get(= buddy)) | _PAGE_GLOBAL)); > >>>>>> -#endif /* CONFIG_SMP */ > >>>>>> - } > >>>>>> - } > >>>>>> + DBAR(0b11000); /* o_wrw =3D 0b11000 */ > >>>>>> } > >>>>>> > >>>>>> No, please hold on. This issue exists about twenty years, Do we ne= ed be > >>>>>> in such a hurry now? > >>>>>> > >>>>>> why is DBAR(0b11000) added in set_pte()? > >>>>> It exists before, not added by this patch. The reason is explained = in > >>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/= commit/?h=3Dv6.12-rc3&id=3Df93f67d06b1023313ef1662eac490e29c025c030 > >>>> why speculative accesses may cause spurious page fault in kernel spa= ce > >>>> with PTE enabled? speculative accesses exists anywhere, it does not > >>>> cause spurious page fault. > >>> Confirmed by Ruiyang Wu, and even if DBAR(0b11000) is wrong, that > >>> means another patch's mistake, not this one. This one just keeps the > >>> old behavior. > >>> +CC Ruiyang Wu here. > >> Also from Ruiyang Wu, the information is that speculative accesses may > >> insert stale TLB, however no page fault exception. > >> > >> So adding barrier in set_pte() does not prevent speculative accesses. > >> And you write patch here, however do not know the actual reason? > >> > >> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/com= mit/?h=3Dv6.12-rc3&id=3Df93f67d06b1023313ef1662eac490e29c025c030 > > I have CCed Ruiyang, whether the description is correct can be judged b= y him. > > There are some problems to add barrier() in set_pte(): > > 1. There is such issue only for HW ptw enabled and kernel address space, > is that? Also it may be two heavy to add barrier in set_pte(), comparing > to do this in flush_cache_vmap(). So adding a barrier in set_pte() may not be the best solution for performance, but you cannot say it is a wrong solution. And yes, we can only care the kernel space, which is also the old behavior before this patch, so set_pte() should be: static inline void set_pte(pte_t *ptep, pte_t pteval) { WRITE_ONCE(*ptep, pteval); #ifdef CONFIG_SMP if (pte_val(pteval) & _PAGE_GLOBAL) DBAR(0b11000); /* o_wrw =3D 0b11000 */ #endif } Putting a dbar unconditionally in set_pte() is my mistake, I'm sorry for t= hat. > > 2. LoongArch is different with other other architectures, two pages are > included in one TLB entry. If there is two consecutive page mapped and > memory access, there will page fault for the second memory access. Such > as: > addr1 =3Dpercpu_alloc(pagesize); > val1 =3D *(int *)addr1; > // With page table walk, addr1 is present and addr2 is pte_none > // TLB entry includes valid pte for addr1, invalid pte for addr2 > addr2 =3Dpercpu_alloc(pagesize); // will not flush tlb in first time > val2 =3D *(int *)addr2; > // With page table walk, addr1 is present and addr2 is present also > // TLB entry includes valid pte for addr1, invalid pte for addr2 > So there will be page fault when accessing address addr2 > > There there is the same problem with user address space. By the way, > there is HW prefetching technology, negative effective of HW prefetching > technology will be tlb added. So there is potential page fault if memory > is allocated and accessed in the first time. As discussed internally, there may be three problems related to speculative access in detail: 1) a load/store after set_pte() is prioritized before, which can be prevented by dbar, 2) a instruction fetch after set_pte() is prioritized before, which can be prevented by ibar, 3) the buddy tlb problem you described here, if I understand Ruiyang's explanation correctly this can only be prevented by the filter in do_page_fault(). >From experiments, without the patch "LoongArch: Improve hardware page table walker", there are about 80 times of spurious page faults during boot, and increases continually during stress tests. And after that patch which adds a dbar to set_pte(), we cannot observe spurious page faults anymore. Of course this doesn't mean 2) and 3) don't exist, but we can at least say 1) is the main case. On this basis, in "LoongArch: Improve hardware page table walker" we use a relatively cheap dbar (compared to ibar) to prevent the main case, and add a filter to handle 2) and 3). Such a solution is reasonable. > > 3. For speculative execution, if it is user address, there is eret from > syscall. eret will rollback all speculative execution instruction. So it > is only problem for speculative execution. And how to verify whether it > is the problem of speculative execution or it is the problem of clause 2? As described above, if spurious page faults still exist after adding dbar to set_pte(), it may be a problem of clause 2 (case 3 in my description), otherwise it is not a problem of clause 2. At last, this patch itself is attempting to solve the concurrent problem about _PAGE_GLOBAL, so adding pte_alloc_one_kernel() and removing the buddy stuff in set_pte() are what it needs. However it shouldn't touch the logic of dbar in set_pte(), whether "LoongArch: Improve hardware page table walker" is right or wrong. Huacai > > Regards > Bibo Mao > > > > > > Huacai > > > >> > >> Bibo Mao > >>> > >>> Huacai > >>> > >>>> > >>>> Obvious you do not it and you write wrong patch. > >>>> > >>>>> > >>>>> Huacai > >>>>> > >>>>>> > >>>>>> Regards > >>>>>> Bibo Mao > >>>>>>> Huacai > >>>>>>> > >>>>>>> On Mon, Oct 14, 2024 at 11:59=E2=80=AFAM Bibo Mao wrote: > >>>>>>>> > >>>>>>>> Unlike general architectures, there are two pages in one TLB ent= ry > >>>>>>>> on LoongArch system. For kernel space, it requires both two pte > >>>>>>>> entries with PAGE_GLOBAL bit set, else HW treats it as non-globa= l > >>>>>>>> tlb, there will be potential problems if tlb entry for kernel sp= ace > >>>>>>>> is not global. Such as fail to flush kernel tlb with function > >>>>>>>> local_flush_tlb_kernel_range() which only flush tlb with global = bit. > >>>>>>>> > >>>>>>>> With function kernel_pte_init() added, it can be used to init pt= e > >>>>>>>> table when it is created for kernel address space, and the defau= lt > >>>>>>>> initial pte value is PAGE_GLOBAL rather than zero at beginning. > >>>>>>>> > >>>>>>>> Kernel address space areas includes fixmap, percpu, vmalloc, kas= an > >>>>>>>> and vmemmap areas set default pte entry with PAGE_GLOBAL set. > >>>>>>>> > >>>>>>>> Signed-off-by: Bibo Mao > >>>>>>>> --- > >>>>>>>> arch/loongarch/include/asm/pgalloc.h | 13 +++++++++++++ > >>>>>>>> arch/loongarch/include/asm/pgtable.h | 1 + > >>>>>>>> arch/loongarch/mm/init.c | 4 +++- > >>>>>>>> arch/loongarch/mm/kasan_init.c | 4 +++- > >>>>>>>> arch/loongarch/mm/pgtable.c | 22 +++++++++++++++++= +++++ > >>>>>>>> include/linux/mm.h | 1 + > >>>>>>>> mm/kasan/init.c | 8 +++++++- > >>>>>>>> mm/sparse-vmemmap.c | 5 +++++ > >>>>>>>> 8 files changed, 55 insertions(+), 3 deletions(-) > >>>>>>>> > >>>>>>>> diff --git a/arch/loongarch/include/asm/pgalloc.h b/arch/loongar= ch/include/asm/pgalloc.h > >>>>>>>> index 4e2d6b7ca2ee..b2698c03dc2c 100644 > >>>>>>>> --- a/arch/loongarch/include/asm/pgalloc.h > >>>>>>>> +++ b/arch/loongarch/include/asm/pgalloc.h > >>>>>>>> @@ -10,8 +10,21 @@ > >>>>>>>> > >>>>>>>> #define __HAVE_ARCH_PMD_ALLOC_ONE > >>>>>>>> #define __HAVE_ARCH_PUD_ALLOC_ONE > >>>>>>>> +#define __HAVE_ARCH_PTE_ALLOC_ONE_KERNEL > >>>>>>>> #include > >>>>>>>> > >>>>>>>> +static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) > >>>>>>>> +{ > >>>>>>>> + pte_t *pte; > >>>>>>>> + > >>>>>>>> + pte =3D (pte_t *) __get_free_page(GFP_KERNEL); > >>>>>>>> + if (!pte) > >>>>>>>> + return NULL; > >>>>>>>> + > >>>>>>>> + kernel_pte_init(pte); > >>>>>>>> + return pte; > >>>>>>>> +} > >>>>>>>> + > >>>>>>>> static inline void pmd_populate_kernel(struct mm_struct *mm= , > >>>>>>>> pmd_t *pmd, pte_t *pt= e) > >>>>>>>> { > >>>>>>>> diff --git a/arch/loongarch/include/asm/pgtable.h b/arch/loongar= ch/include/asm/pgtable.h > >>>>>>>> index 9965f52ef65b..22e3a8f96213 100644 > >>>>>>>> --- a/arch/loongarch/include/asm/pgtable.h > >>>>>>>> +++ b/arch/loongarch/include/asm/pgtable.h > >>>>>>>> @@ -269,6 +269,7 @@ extern void set_pmd_at(struct mm_struct *mm,= unsigned long addr, pmd_t *pmdp, pm > >>>>>>>> extern void pgd_init(void *addr); > >>>>>>>> extern void pud_init(void *addr); > >>>>>>>> extern void pmd_init(void *addr); > >>>>>>>> +extern void kernel_pte_init(void *addr); > >>>>>>>> > >>>>>>>> /* > >>>>>>>> * Encode/decode swap entries and swap PTEs. Swap PTEs are = all PTEs that > >>>>>>>> diff --git a/arch/loongarch/mm/init.c b/arch/loongarch/mm/init.c > >>>>>>>> index 8a87a482c8f4..9f26e933a8a3 100644 > >>>>>>>> --- a/arch/loongarch/mm/init.c > >>>>>>>> +++ b/arch/loongarch/mm/init.c > >>>>>>>> @@ -198,9 +198,11 @@ pte_t * __init populate_kernel_pte(unsigned= long addr) > >>>>>>>> if (!pmd_present(pmdp_get(pmd))) { > >>>>>>>> pte_t *pte; > >>>>>>>> > >>>>>>>> - pte =3D memblock_alloc(PAGE_SIZE, PAGE_SIZE); > >>>>>>>> + pte =3D memblock_alloc_raw(PAGE_SIZE, PAGE_SIZE)= ; > >>>>>>>> if (!pte) > >>>>>>>> panic("%s: Failed to allocate memory= \n", __func__); > >>>>>>>> + > >>>>>>>> + kernel_pte_init(pte); > >>>>>>>> pmd_populate_kernel(&init_mm, pmd, pte); > >>>>>>>> } > >>>>>>>> > >>>>>>>> diff --git a/arch/loongarch/mm/kasan_init.c b/arch/loongarch/mm/= kasan_init.c > >>>>>>>> index 427d6b1aec09..34988573b0d5 100644 > >>>>>>>> --- a/arch/loongarch/mm/kasan_init.c > >>>>>>>> +++ b/arch/loongarch/mm/kasan_init.c > >>>>>>>> @@ -152,6 +152,8 @@ static void __init kasan_pte_populate(pmd_t = *pmdp, unsigned long addr, > >>>>>>>> phys_addr_t page_phys =3D early ? > >>>>>>>> __pa_symbol(kasan_ea= rly_shadow_page) > >>>>>>>> : kasan_alloc_= zeroed_page(node); > >>>>>>>> + if (!early) > >>>>>>>> + kernel_pte_init(__va(page_phys)); > >>>>>>>> next =3D addr + PAGE_SIZE; > >>>>>>>> set_pte(ptep, pfn_pte(__phys_to_pfn(page_phy= s), PAGE_KERNEL)); > >>>>>>>> } while (ptep++, addr =3D next, addr !=3D end && __p= te_none(early, ptep_get(ptep))); > >>>>>>>> @@ -287,7 +289,7 @@ void __init kasan_init(void) > >>>>>>>> set_pte(&kasan_early_shadow_pte[i], > >>>>>>>> pfn_pte(__phys_to_pfn(__pa_symbol(ka= san_early_shadow_page)), PAGE_KERNEL_RO)); > >>>>>>>> > >>>>>>>> - memset(kasan_early_shadow_page, 0, PAGE_SIZE); > >>>>>>>> + kernel_pte_init(kasan_early_shadow_page); > >>>>>>>> csr_write64(__pa_symbol(swapper_pg_dir), LOONGARCH_C= SR_PGDH); > >>>>>>>> local_flush_tlb_all(); > >>>>>>>> > >>>>>>>> diff --git a/arch/loongarch/mm/pgtable.c b/arch/loongarch/mm/pgt= able.c > >>>>>>>> index eb6a29b491a7..228ffc1db0a3 100644 > >>>>>>>> --- a/arch/loongarch/mm/pgtable.c > >>>>>>>> +++ b/arch/loongarch/mm/pgtable.c > >>>>>>>> @@ -38,6 +38,28 @@ pgd_t *pgd_alloc(struct mm_struct *mm) > >>>>>>>> } > >>>>>>>> EXPORT_SYMBOL_GPL(pgd_alloc); > >>>>>>>> > >>>>>>>> +void kernel_pte_init(void *addr) > >>>>>>>> +{ > >>>>>>>> + unsigned long *p, *end; > >>>>>>>> + unsigned long entry; > >>>>>>>> + > >>>>>>>> + entry =3D (unsigned long)_PAGE_GLOBAL; > >>>>>>>> + p =3D (unsigned long *)addr; > >>>>>>>> + end =3D p + PTRS_PER_PTE; > >>>>>>>> + > >>>>>>>> + do { > >>>>>>>> + p[0] =3D entry; > >>>>>>>> + p[1] =3D entry; > >>>>>>>> + p[2] =3D entry; > >>>>>>>> + p[3] =3D entry; > >>>>>>>> + p[4] =3D entry; > >>>>>>>> + p +=3D 8; > >>>>>>>> + p[-3] =3D entry; > >>>>>>>> + p[-2] =3D entry; > >>>>>>>> + p[-1] =3D entry; > >>>>>>>> + } while (p !=3D end); > >>>>>>>> +} > >>>>>>>> + > >>>>>>>> void pgd_init(void *addr) > >>>>>>>> { > >>>>>>>> unsigned long *p, *end; > >>>>>>>> diff --git a/include/linux/mm.h b/include/linux/mm.h > >>>>>>>> index ecf63d2b0582..6909fe059a2c 100644 > >>>>>>>> --- a/include/linux/mm.h > >>>>>>>> +++ b/include/linux/mm.h > >>>>>>>> @@ -3818,6 +3818,7 @@ void *sparse_buffer_alloc(unsigned long si= ze); > >>>>>>>> struct page * __populate_section_memmap(unsigned long pfn, > >>>>>>>> unsigned long nr_pages, int nid, struct vmem= _altmap *altmap, > >>>>>>>> struct dev_pagemap *pgmap); > >>>>>>>> +void kernel_pte_init(void *addr); > >>>>>>>> void pmd_init(void *addr); > >>>>>>>> void pud_init(void *addr); > >>>>>>>> pgd_t *vmemmap_pgd_populate(unsigned long addr, int node); > >>>>>>>> diff --git a/mm/kasan/init.c b/mm/kasan/init.c > >>>>>>>> index 89895f38f722..ac607c306292 100644 > >>>>>>>> --- a/mm/kasan/init.c > >>>>>>>> +++ b/mm/kasan/init.c > >>>>>>>> @@ -106,6 +106,10 @@ static void __ref zero_pte_populate(pmd_t *= pmd, unsigned long addr, > >>>>>>>> } > >>>>>>>> } > >>>>>>>> > >>>>>>>> +void __weak __meminit kernel_pte_init(void *addr) > >>>>>>>> +{ > >>>>>>>> +} > >>>>>>>> + > >>>>>>>> static int __ref zero_pmd_populate(pud_t *pud, unsigned lon= g addr, > >>>>>>>> unsigned long end) > >>>>>>>> { > >>>>>>>> @@ -126,8 +130,10 @@ static int __ref zero_pmd_populate(pud_t *p= ud, unsigned long addr, > >>>>>>>> > >>>>>>>> if (slab_is_available()) > >>>>>>>> p =3D pte_alloc_one_kernel(&= init_mm); > >>>>>>>> - else > >>>>>>>> + else { > >>>>>>>> p =3D early_alloc(PAGE_SIZE,= NUMA_NO_NODE); > >>>>>>>> + kernel_pte_init(p); > >>>>>>>> + } > >>>>>>>> if (!p) > >>>>>>>> return -ENOMEM; > >>>>>>>> > >>>>>>>> diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c > >>>>>>>> index edcc7a6b0f6f..c0388b2e959d 100644 > >>>>>>>> --- a/mm/sparse-vmemmap.c > >>>>>>>> +++ b/mm/sparse-vmemmap.c > >>>>>>>> @@ -184,6 +184,10 @@ static void * __meminit vmemmap_alloc_block= _zero(unsigned long size, int node) > >>>>>>>> return p; > >>>>>>>> } > >>>>>>>> > >>>>>>>> +void __weak __meminit kernel_pte_init(void *addr) > >>>>>>>> +{ > >>>>>>>> +} > >>>>>>>> + > >>>>>>>> pmd_t * __meminit vmemmap_pmd_populate(pud_t *pud, unsigned= long addr, int node) > >>>>>>>> { > >>>>>>>> pmd_t *pmd =3D pmd_offset(pud, addr); > >>>>>>>> @@ -191,6 +195,7 @@ pmd_t * __meminit vmemmap_pmd_populate(pud_t= *pud, unsigned long addr, int node) > >>>>>>>> void *p =3D vmemmap_alloc_block_zero(PAGE_SI= ZE, node); > >>>>>>>> if (!p) > >>>>>>>> return NULL; > >>>>>>>> + kernel_pte_init(p); > >>>>>>>> pmd_populate_kernel(&init_mm, pmd, p); > >>>>>>>> } > >>>>>>>> return pmd; > >>>>>>>> -- > >>>>>>>> 2.39.3 > >>>>>>>> > >>>>>> > >>>>>> > >>>> > >>>> > >> > >> >