From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72337D2069A for ; Wed, 16 Oct 2024 06:09:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F21916B0083; Wed, 16 Oct 2024 02:09:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EAA046B0088; Wed, 16 Oct 2024 02:09:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D4A196B0089; Wed, 16 Oct 2024 02:09:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B2BE86B0083 for ; Wed, 16 Oct 2024 02:09:30 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 98F1C1A1A1A for ; Wed, 16 Oct 2024 06:09:12 +0000 (UTC) X-FDA: 82678438206.26.F541F9B Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by imf20.hostedemail.com (Postfix) with ESMTP id 3AFA11C0007 for ; Wed, 16 Oct 2024 06:09:15 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of maobibo@loongson.cn designates 114.242.206.163 as permitted sender) smtp.mailfrom=maobibo@loongson.cn; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729058893; a=rsa-sha256; cv=none; b=7qIJi0r63EM22gMNg/Oks7XO6UPEpJdK9fTFu8DCJ0yuOJVaP7I8n7SAeapSLdD0Nt8jQN E6vf26++4nSh4WjvRP6/HAgEnGBt8730SPxOsH/5rs2Ir+X63YyPNVaWsoOS3dOLjXdvrW yTMaz2pazYRVgI0ZIJo1Hg0YTi/rRdQ= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of maobibo@loongson.cn designates 114.242.206.163 as permitted sender) smtp.mailfrom=maobibo@loongson.cn; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729058893; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s5GauQ51CQYdbsZdOQ6plFGnwMuUiYdffpMo6dLdCS8=; b=VRQZvm+Mu8fSl4H9//CSYpiE2p+yAgXsYRnub0d5RkLDNl3Vv1kS+fXKvgWoDef+scFVD6 zOuJ8rodtFlFIxL6yPFQDO3Nob1gwrstwNn2Cdlrwix+L5VsK57lPp9JSJClSE6/OrsWE+ aPmijhug7Zt1sQSxx6wJRfEQ6cgNvCM= Received: from loongson.cn (unknown [10.20.42.62]) by gateway (Coremail) with SMTP id _____8BxHLOSWA9nUIYfAA--.47020S3; Wed, 16 Oct 2024 14:09:22 +0800 (CST) Received: from [10.20.42.62] (unknown [10.20.42.62]) by front1 (Coremail) with SMTP id qMiowMDx_9ePWA9nMj8sAA--.15772S3; Wed, 16 Oct 2024 14:09:21 +0800 (CST) Subject: Re: [PATCH v2 2/3] LoongArch: Add barrier between set_pte and memory access To: Huacai Chen Cc: Andrey Ryabinin , Andrew Morton , David Hildenbrand , Barry Song , loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org References: <20241014035855.1119220-1-maobibo@loongson.cn> <20241014035855.1119220-3-maobibo@loongson.cn> From: maobibo Message-ID: <1b4070c9-921e-65e3-c2a7-dab486d4f17f@loongson.cn> Date: Wed, 16 Oct 2024 14:09:01 +0800 User-Agent: Mozilla/5.0 (X11; Linux loongarch64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-CM-TRANSID:qMiowMDx_9ePWA9nMj8sAA--.15772S3 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBj93XoWxtr18CF4xtw18JF45Ww48AFc_yoW7AFW5pr W2k3Z8Kr4kXF1Fvw12vw1fWr1ft39rWFy8Xw1FqryDCw1qqFy29ry8WrW8uryxXa4rJa1x uw4Utr13WFWUJagCm3ZEXasCq-sJn29KB7ZKAUJUUUU8529EdanIXcx71UUUUU7KY7ZEXa sCq-sGcSsGvfJ3Ic02F40EFcxC0VAKzVAqx4xG6I80ebIjqfuFe4nvWSU5nxnvy29KBjDU 0xBIdaVrnRJUUUv2b4IE77IF4wAFF20E14v26r1j6r4UM7CY07I20VC2zVCF04k26cxKx2 IYs7xG6rWj6s0DM7CIcVAFz4kK6r1Y6r17M28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48v e4kI8wA2z4x0Y4vE2Ix0cI8IcVAFwI0_Gr0_Xr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI 0_Gr0_Cr1l84ACjcxK6I8E87Iv67AKxVWxJVW8Jr1l84ACjcxK6I8E87Iv6xkF7I0E14v2 6r4UJVWxJr1le2I262IYc4CY6c8Ij28IcVAaY2xG8wAqjxCEc2xF0cIa020Ex4CE44I27w Aqx4xG64xvF2IEw4CE5I8CrVC2j2WlYx0E2Ix0cI8IcVAFwI0_Jw0_WrylYx0Ex4A2jsIE 14v26r4j6F4UMcvjeVCFs4IE7xkEbVWUJVW8JwACjcxG0xvEwIxGrwCYjI0SjxkI62AI1c AE67vIY487MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8C rVAFwI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVWUtVW8Zw CIc40Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r4j6ryUMIIF0xvE2Ix0cI8IcVCY1x02 67AKxVW8JVWxJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Gr 0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8JVW8JrUvcSsGvfC2KfnxnUUI43ZEXa7IU8l3 8UUUUUU== X-Stat-Signature: fhbaabn16qi57r13gj1ohqurp5zo18fh X-Rspamd-Queue-Id: 3AFA11C0007 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1729058955-888974 X-HE-Meta: U2FsdGVkX1/78+Oc+5Jyc8/KjgTF5rcYiDLuCPkvshA66vswKduoO0MrdXd/eUBPkeSsVQCw8bh8N7MroSa7dqQhqk07EjL2zaNDhr/DL2tImkK/WUTPAKt0wS8pDaqE4bDTX4uo9jJbyHNNwo4gK2S2FiLuBlKAe4EGZyrAR6DglDWNT0NWruMZ+ZjIyes/dw0mxCAWdY4b0HMQaz+HVdP9iZR3uSM1Btn21LUMFK8s8MrebzY6yQFeNuHuK63xfL9GFVoVpKvedVJKNbAOqmEs24ZkJJ/2Whd+TjNq6Bj1+S43gJK4h41wFwpZTlivVgfujaOj2M0HyzC8TcOdsJvl2pBnpaQzgvZ/Aa7TM/e9hclYoKWPlQkyR9Nl2w2IVVfbHaA/Y59gD8SBfYQArTrSygb4l/Vlj9xRKKydTYhtYWbwDapDX5kjW28N83o7Jjph9z1kTDB4w4jTqMZcu7YTAk0VEr1KTvplxP2QD72yUUv/IoGS1Sf/2a3xF2AYLF8LyJ85HmGKEOdIB2AEvdg3EnsZkh+/urarOIW9vGCQ/22tKR5CRhI05kc9t9d74qDA+iXXCQ6oEKpoyooYtsIPnk4WOPHTJ2qfUWq9Xaq21df37GosUxwEojA+XrhUsz0Xg7ibDpTTNIGHll4OsCN6H4vt4/bvbFxPVKaFL4C+2DaQoqSoCzFS/IZciuiysikmiXYadOqB5z/g2L1kbGpBa2iOPu0fMYV47YFvuyFCtxzQ29UKYqCBe8D/CnXsxDPubbrP5/WIYnhtk/G3fEcDlwLXTK/2NbNLJ7IkziePScRwwN0znpnFN1Gi4gr4I0/DDvqwtcz1SoN8tIbZJaXGuTA8bedJBw20afitvpMXRN3sOsXxs4HdSXS9ZryHsoBb5D5qJ47xnlwcvqnzlB4VrXN6z5+10zqOCy/9dtU6fzQWn0ImWgjnPXQFzW+10M+tcx4xPvWeUmUvJu4 bBIYQwk8 eFml9uSPDtBYiM/X0SE1w6qN5XswQ31eQiggMruI8FYy3y2uGHHqvdxxm8oA3o+yFpmpwmQKD6azQrgn647tOZM3qOqVO5CoNKpPmRkFEmCp/UnI8hT1xg/IePQO6kMencgvDDNa0RFD+UloTfHMWMASpOl7txkoCtbXhjoTdyVXhT6DlKp8h+ZHw6Komie1sGfViCcgwYQBcli5/s/0yLhvy2w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/10/15 下午8:27, Huacai Chen wrote: > On Tue, Oct 15, 2024 at 10:54 AM maobibo wrote: >> >> >> >> On 2024/10/14 下午2:31, Huacai Chen wrote: >>> Hi, Bibo, >>> >>> On Mon, Oct 14, 2024 at 11:59 AM Bibo Mao wrote: >>>> >>>> It is possible to return a spurious fault if memory is accessed >>>> right after the pte is set. For user address space, pte is set >>>> in kernel space and memory is accessed in user space, there is >>>> long time for synchronization, no barrier needed. However for >>>> kernel address space, it is possible that memory is accessed >>>> right after the pte is set. >>>> >>>> Here flush_cache_vmap/flush_cache_vmap_early is used for >>>> synchronization. >>>> >>>> Signed-off-by: Bibo Mao >>>> --- >>>> arch/loongarch/include/asm/cacheflush.h | 14 +++++++++++++- >>>> 1 file changed, 13 insertions(+), 1 deletion(-) >>>> >>>> diff --git a/arch/loongarch/include/asm/cacheflush.h b/arch/loongarch/include/asm/cacheflush.h >>>> index f8754d08a31a..53be231319ef 100644 >>>> --- a/arch/loongarch/include/asm/cacheflush.h >>>> +++ b/arch/loongarch/include/asm/cacheflush.h >>>> @@ -42,12 +42,24 @@ void local_flush_icache_range(unsigned long start, unsigned long end); >>>> #define flush_cache_dup_mm(mm) do { } while (0) >>>> #define flush_cache_range(vma, start, end) do { } while (0) >>>> #define flush_cache_page(vma, vmaddr, pfn) do { } while (0) >>>> -#define flush_cache_vmap(start, end) do { } while (0) >>>> #define flush_cache_vunmap(start, end) do { } while (0) >>>> #define flush_icache_user_page(vma, page, addr, len) do { } while (0) >>>> #define flush_dcache_mmap_lock(mapping) do { } while (0) >>>> #define flush_dcache_mmap_unlock(mapping) do { } while (0) >>>> >>>> +/* >>>> + * It is possible for a kernel virtual mapping access to return a spurious >>>> + * fault if it's accessed right after the pte is set. The page fault handler >>>> + * does not expect this type of fault. flush_cache_vmap is not exactly the >>>> + * right place to put this, but it seems to work well enough. >>>> + */ >>>> +static inline void flush_cache_vmap(unsigned long start, unsigned long end) >>>> +{ >>>> + smp_mb(); >>>> +} >>>> +#define flush_cache_vmap flush_cache_vmap >>>> +#define flush_cache_vmap_early flush_cache_vmap >>> From the history of flush_cache_vmap_early(), It seems only archs with >>> "virtual cache" (VIVT or VIPT) need this API, so LoongArch can be a >>> no-op here. > OK, flush_cache_vmap_early() also needs smp_mb(). > >> >> Here is usage about flush_cache_vmap_early in file linux/mm/percpu.c, >> map the page and access it immediately. Do you think it should be noop >> on LoongArch. >> >> rc = __pcpu_map_pages(unit_addr, &pages[unit * unit_pages], >> unit_pages); >> if (rc < 0) >> panic("failed to map percpu area, err=%d\n", rc); >> flush_cache_vmap_early(unit_addr, unit_addr + ai->unit_size); >> /* copy static data */ >> memcpy((void *)unit_addr, __per_cpu_load, ai->static_size); >> } >> >> >>> >>> And I still think flush_cache_vunmap() should be a smp_mb(). A >>> smp_mb() in flush_cache_vmap() prevents subsequent accesses be >>> reordered before pte_set(), and a smp_mb() in flush_cache_vunmap() >> smp_mb() in flush_cache_vmap() does not prevent reorder. It is to flush >> pipeline and let page table walker HW sync with data cache. >> >> For the following example. >> rb = vmap(pages, nr_meta_pages + 2 * nr_data_pages, >> VM_MAP | VM_USERMAP, PAGE_KERNEL); >> if (rb) { >> <<<<<<<<<<< * the sentence if (rb) can prevent reorder. Otherwise with >> any API kmalloc/vmap/vmalloc and subsequent memory access, there will be >> reorder issu. * >> kmemleak_not_leak(pages); >> rb->pages = pages; >> rb->nr_pages = nr_pages; >> return rb; >> } >> >>> prevents preceding accesses be reordered after pte_clear(). This >> Can you give an example about such usage about flush_cache_vunmap()? and >> we can continue to talk about it, else it is just guessing. > Since we cannot reach a consensus, and the flush_cache_* API look very > strange for this purpose (Yes, I know PowerPC does it like this, but > ARM64 doesn't). I prefer to still use the ARM64 method which means add > a dbar in set_pte(). Of course the performance will be a little worse, > but still better than the old version, and it is more robust. > > I know you are very busy, so if you have no time you don't need to > send V3, I can just do a small modification on the 3rd patch. No, I will send V3 by myself. And I will drop the this patch in this patchset since by actual test vmalloc_test works well even without this patch on 3C5000 Dual-way, also weak function kernel_pte_init will be replaced with inline function rebased on https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/mm-define-general-function-pxd_init.patch I dislike the copy-paste method without further understanding :(, although I also copy and paste code, but as least I try best to understand it. Regards Bibo Mao > > > Huacai > >> >> Regards >> Bibo Mao >>> potential problem may not be seen from experiment, but it is needed in >>> theory. >>> >>> Huacai >>> >>>> + >>>> #define cache_op(op, addr) \ >>>> __asm__ __volatile__( \ >>>> " cacop %0, %1 \n" \ >>>> -- >>>> 2.39.3 >>>> >>>> >> >>