From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1972BD21696 for ; Tue, 15 Oct 2024 12:28:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9BC596B0083; Tue, 15 Oct 2024 08:28:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 945096B0085; Tue, 15 Oct 2024 08:28:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7E5486B0088; Tue, 15 Oct 2024 08:28:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5CE086B0083 for ; Tue, 15 Oct 2024 08:28:04 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 56DDD4158B for ; Tue, 15 Oct 2024 12:27:58 +0000 (UTC) X-FDA: 82675763352.13.818EE45 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf23.hostedemail.com (Postfix) with ESMTP id 34F5C140014 for ; Tue, 15 Oct 2024 12:27:56 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=DKRcuqPu; spf=pass (imf23.hostedemail.com: domain of chenhuacai@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=chenhuacai@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728995208; a=rsa-sha256; cv=none; b=n67qnLOHZ7XT/X+rWYk3L29wsmrtIOuhY5DvUWQGyq4ZdvYMaZMn/UYgrD9WM1+6Io+N96 FF0AVyBW8BKbgO8YfsugMJwzR0+JG+RjbkOe9nHoG0RRfrVBRBRxek1hxn4G+ENtV+I2Xh Q3YSc4z6Wu+ivAYuZhBGAuJCgCe7Y+Q= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=DKRcuqPu; spf=pass (imf23.hostedemail.com: domain of chenhuacai@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=chenhuacai@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728995208; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=m72B8Ax7Cs86MwObcwcARk1nTXi8mBNE2H4xmkM4IxM=; b=S0az8YDkM1sGiVEHxxCKwLE9j7VGQS2H5rffhrxPHOKLy5MrneCbEY6c0+5hAVMAAJExut 6htsLPttnUIdo52scihMO7V3kt/D3wMB28djElsMZamCae/e1OgTuZJITh8BESV6aGk2dt 0wKb+zIGX8TYCRFKM4dh6ILMj2L+svw= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id EE9F8A42577 for ; Tue, 15 Oct 2024 12:27:51 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 41C6CC4CED4 for ; Tue, 15 Oct 2024 12:28:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728995280; bh=PNLn1/pCtXUyStU0U2BVMLfpIt+9X20DAEgGORY0iH4=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=DKRcuqPufglS2KDcdgst9CtDv9nI0ZLxzJ6tvHWGLDLJ0fQDbK31YVylSYaLqcq0i Uxkg+ll5WZV+HMnHSv1bqOEcntWVPdennBVpf/oiBeaMyR+ysDH+0ceixuwOJrgl3G veEGqMZEuNR3/dTgJqknrlBGhyLsp4om/nl9FPotEgoDsp0VCuai7u4F/RU+K8OYlj rJL6/ZwRQ5ucGp5CCwd+9gfcAPKwH49ZGqZE/j6gwrCko1vhQkkXGfXEA7Gs/Xkw9a XEqI1cNbTm9KN78ga0Y86dynyneFcuGeDqb22sjVSAsk6uphgCXGDOBpqbQhQw6fD9 3tLgDjQ999qsg== Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-a994ecf79e7so808150866b.0 for ; Tue, 15 Oct 2024 05:28:00 -0700 (PDT) X-Forwarded-Encrypted: i=1; AJvYcCVCF/qJgajRpo+AqHrsdTcgu7FPEklByPRkeQcUVe6W5Yi8yAwzh1GR+HwyzJGobrSWoc+9Pgaq6g==@kvack.org X-Gm-Message-State: AOJu0Yz8JacC17YO+4yFMYOBAtd05ZGVBEq8AjAs3iZv1iUWJ9826zVo 8SfknLcoSSOUOqRmBp7ACgdtFn/4iQsXNqqbfJsbDYJQbDf15n7LfxnG+Twly3f3yARJKOMVdha w/pdQgXW65InCZxyfwtgz0rrAXgY= X-Google-Smtp-Source: AGHT+IF7wVxa4z12T17CxaCFQcgsO/SFzkpZWZ6Gldwe9M2owZr6r0gxkCYAZsQRGltQQSnQfoF0tccs3lOOlaBOnJY= X-Received: by 2002:a17:907:934e:b0:a9a:26dd:16bc with SMTP id a640c23a62f3a-a9a34d3b5dcmr396366b.5.1728995278707; Tue, 15 Oct 2024 05:27:58 -0700 (PDT) MIME-Version: 1.0 References: <20241014035855.1119220-1-maobibo@loongson.cn> <20241014035855.1119220-3-maobibo@loongson.cn> In-Reply-To: From: Huacai Chen Date: Tue, 15 Oct 2024 20:27:46 +0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v2 2/3] LoongArch: Add barrier between set_pte and memory access To: maobibo Cc: Andrey Ryabinin , Andrew Morton , David Hildenbrand , Barry Song , loongarch@lists.linux.dev, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: i9ukcjuuji6561qm1g5d3xbstejpjg56 X-Rspamd-Queue-Id: 34F5C140014 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1728995276-674727 X-HE-Meta: U2FsdGVkX19d4REgBgeePNmBebd+h7RIMVPCUTISnLItbGuVyAEguUiBH3NWlZ7aoODgS+qv+bosAMWSnCEr+lU2AVn29fjkoNEw75sc3wm0eYUoGUbvFLOQLe+gMGsIYkaP7dO4+ZEfCGcFRZo9RUgS9/t6Kjy5DG0F1GmxI7BF6pifJjH+gOgPDfMojIAxgwXLQXOl7z6kUwz3TZrQuSSFDMkGRpzjLWbMB6AqqYYFtc1X9SfsBZn9K/DGCsLNS2nQ7Od0Wuq9uYWAOlojEnCL9x3lMSEPrskJmjI9IwYYQI4uJHRpCSfNQKaCoh6oKfvF9QE1gXnojZFKyO7CpXUPMFfSkR1xBkW/qOQPIR0Phw9eo6Y+zXL1HOM2h6HxHQvp8cOArj8xWtHv5CNMB6nzh9xwvrGf72HnrITtoO0WbjI4yTj5IkMS9+0BcG/dMbhlKJBL+bFAiTORBLCuLrwPaIC28/TLuc8DqBXz2S6Br0MLl8hV8mSx63emIdDCR3zlFdMjiIfdkAwm2pqDra8zpvyyfqi0XQjlvjq6EoOQAjtwE/je84D4IUH7/pYhJRYXLTGrkSazcCjkAj6RNcSrCrbldKMFhbNUtdfDTN7JCdr2Ls0QLIhiDCQRkuhq1oKEequGL1bl2sRhfglI5ObOmjko5oXo0o4Zf9nQPQmyUDzCkALWSzahv9ZMtU7t8Y0yzDNtlV5YD5ROWhHOCxY1aznEvxiae/Jwssh38yXlmFY+QrpBMBs8TTPjFsGCsF6E6sZlwIaD90c/Dx9SpAQJOgcDm6vUZ0LmlDYOQsleTmVn+OmJFA2UsF+59oEgekjzTDeJ2vRsBXEYfgm6FbucAhyRrfpuFCXpTU8paTlhJmH19El8Lb5ryMH/3LAwkbjZNtXCPyrqWr19+w0KGKfWBf7ZYNMBq00offkTapNkdCIxS4s6Wj4ZYXq73c8EYlcmRB/upZmbAVIumTt sngzFN3k OULpyqMxFvJFXQv1dB2mMZCC6lkdnGuYPRCvpxn9/brHsqLmLVDNgC0e4KTEmC/AfR8T/XjcnNIc3IkBLyMfFKz5WSuD2juxg0n5gEsJdkhANjcVErxAwh3CHhHgygn8GySKy7yO05OAdRcDlybjYcfaFCFsNteMkR3tXJAl7oQGO3IxRzpzW8LF6CvhvOMRProcf1jF+aKsQkQ1+Vo6YJpOOOS+OwfgDsY0eMIPyY/f2WwmEYgTZ6z40JhTF9+9m3SYPNHOlh6Xjr+cYFNxwvNuDyijk/wQA69g7yEkEUziCi6zXQ4n8YksWerjQfNLj7LI2ZFKE/K5BcudSoOZZKExuCGFF4T+cMV4jUdj5U2WKj+AYO/M3nw57o4XSblglrCtChLIYD1r/ne/61xYhM9JQ9w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Oct 15, 2024 at 10:54=E2=80=AFAM maobibo wrot= e: > > > > On 2024/10/14 =E4=B8=8B=E5=8D=882:31, Huacai Chen wrote: > > Hi, Bibo, > > > > On Mon, Oct 14, 2024 at 11:59=E2=80=AFAM Bibo Mao = wrote: > >> > >> It is possible to return a spurious fault if memory is accessed > >> right after the pte is set. For user address space, pte is set > >> in kernel space and memory is accessed in user space, there is > >> long time for synchronization, no barrier needed. However for > >> kernel address space, it is possible that memory is accessed > >> right after the pte is set. > >> > >> Here flush_cache_vmap/flush_cache_vmap_early is used for > >> synchronization. > >> > >> Signed-off-by: Bibo Mao > >> --- > >> arch/loongarch/include/asm/cacheflush.h | 14 +++++++++++++- > >> 1 file changed, 13 insertions(+), 1 deletion(-) > >> > >> diff --git a/arch/loongarch/include/asm/cacheflush.h b/arch/loongarch/= include/asm/cacheflush.h > >> index f8754d08a31a..53be231319ef 100644 > >> --- a/arch/loongarch/include/asm/cacheflush.h > >> +++ b/arch/loongarch/include/asm/cacheflush.h > >> @@ -42,12 +42,24 @@ void local_flush_icache_range(unsigned long start,= unsigned long end); > >> #define flush_cache_dup_mm(mm) do { } while = (0) > >> #define flush_cache_range(vma, start, end) do { } while = (0) > >> #define flush_cache_page(vma, vmaddr, pfn) do { } while = (0) > >> -#define flush_cache_vmap(start, end) do { } while (= 0) > >> #define flush_cache_vunmap(start, end) do { } while = (0) > >> #define flush_icache_user_page(vma, page, addr, len) do { } while = (0) > >> #define flush_dcache_mmap_lock(mapping) do { = } while (0) > >> #define flush_dcache_mmap_unlock(mapping) do { } while = (0) > >> > >> +/* > >> + * It is possible for a kernel virtual mapping access to return a spu= rious > >> + * fault if it's accessed right after the pte is set. The page fault = handler > >> + * does not expect this type of fault. flush_cache_vmap is not exactl= y the > >> + * right place to put this, but it seems to work well enough. > >> + */ > >> +static inline void flush_cache_vmap(unsigned long start, unsigned lon= g end) > >> +{ > >> + smp_mb(); > >> +} > >> +#define flush_cache_vmap flush_cache_vmap > >> +#define flush_cache_vmap_early flush_cache_vmap > > From the history of flush_cache_vmap_early(), It seems only archs with > > "virtual cache" (VIVT or VIPT) need this API, so LoongArch can be a > > no-op here. OK, flush_cache_vmap_early() also needs smp_mb(). > > Here is usage about flush_cache_vmap_early in file linux/mm/percpu.c, > map the page and access it immediately. Do you think it should be noop > on LoongArch. > > rc =3D __pcpu_map_pages(unit_addr, &pages[unit * unit_pages], > unit_pages); > if (rc < 0) > panic("failed to map percpu area, err=3D%d\n", rc); > flush_cache_vmap_early(unit_addr, unit_addr + ai->unit_size); > /* copy static data */ > memcpy((void *)unit_addr, __per_cpu_load, ai->static_size); > } > > > > > > And I still think flush_cache_vunmap() should be a smp_mb(). A > > smp_mb() in flush_cache_vmap() prevents subsequent accesses be > > reordered before pte_set(), and a smp_mb() in flush_cache_vunmap() > smp_mb() in flush_cache_vmap() does not prevent reorder. It is to flush > pipeline and let page table walker HW sync with data cache. > > For the following example. > rb =3D vmap(pages, nr_meta_pages + 2 * nr_data_pages, > VM_MAP | VM_USERMAP, PAGE_KERNEL); > if (rb) { > <<<<<<<<<<< * the sentence if (rb) can prevent reorder. Otherwise with > any API kmalloc/vmap/vmalloc and subsequent memory access, there will be > reorder issu. * > kmemleak_not_leak(pages); > rb->pages =3D pages; > rb->nr_pages =3D nr_pages; > return rb; > } > > > prevents preceding accesses be reordered after pte_clear(). This > Can you give an example about such usage about flush_cache_vunmap()? and > we can continue to talk about it, else it is just guessing. Since we cannot reach a consensus, and the flush_cache_* API look very strange for this purpose (Yes, I know PowerPC does it like this, but ARM64 doesn't). I prefer to still use the ARM64 method which means add a dbar in set_pte(). Of course the performance will be a little worse, but still better than the old version, and it is more robust. I know you are very busy, so if you have no time you don't need to send V3, I can just do a small modification on the 3rd patch. Huacai > > Regards > Bibo Mao > > potential problem may not be seen from experiment, but it is needed in > > theory. > > > > Huacai > > > >> + > >> #define cache_op(op, addr) = \ > >> __asm__ __volatile__( = \ > >> " cacop %0, %1 \n" = \ > >> -- > >> 2.39.3 > >> > >> > >