From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57CD1EE14D3 for ; Thu, 7 Sep 2023 09:06:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 900AB280016; Thu, 7 Sep 2023 05:06:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8B0778E000F; Thu, 7 Sep 2023 05:06:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 77858280016; Thu, 7 Sep 2023 05:06:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 67E3D8E000F for ; Thu, 7 Sep 2023 05:06:15 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 36C261A10CB for ; Thu, 7 Sep 2023 09:06:15 +0000 (UTC) X-FDA: 81209219910.19.4D5B6D3 Received: from mail-lj1-f177.google.com (mail-lj1-f177.google.com [209.85.208.177]) by imf10.hostedemail.com (Postfix) with ESMTP id 40E42C000F for ; Thu, 7 Sep 2023 09:06:12 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=Mh4hwoR9; dmarc=none; spf=pass (imf10.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.177 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694077573; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QwR4d+5UoVgi3hAZhhlc3KZ970LG1A0zS/bDs+8zFKk=; b=LoALUSCJyTLt1YuQiQCBQFHGhaoOEb51+jCWNB0D5K0THrl7fd3r+wc1+6cgkC/xwGsvGt Bh4f0jt2Pf8nZggfYZgVvIl7E+PQwpaa0nFh3U9/UqxBRtjFVVzE19Lpc1NvFL1zzU/zf1 +M6Djr2iIOMAMygKwqWZAmjg1pwEUc4= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=Mh4hwoR9; dmarc=none; spf=pass (imf10.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.177 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694077573; a=rsa-sha256; cv=none; b=PzBVtYE1wXqcjPQwXnaFcYJ8uO/LLAT7dm+B2g92L2ZoM5t7IkNgwF5+GkOudq8fbkA24W Hu+s1eTzcHu3G38JQMYXG/aO4ag6S89xvcgrp9nnIpFEhKQH8s0iXqEtETBSrtfqdgMaN/ 4ad5zh2ochs+r5Wk8PIN4Nw0mVjWDVc= Received: by mail-lj1-f177.google.com with SMTP id 38308e7fff4ca-2b95d5ee18dso13334761fa.1 for ; Thu, 07 Sep 2023 02:06:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1694077571; x=1694682371; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=QwR4d+5UoVgi3hAZhhlc3KZ970LG1A0zS/bDs+8zFKk=; b=Mh4hwoR9p3mfUCHH95vrXCyWGZ2H/T48tYuI7wbf8HfYjj6mHp/DtJBgHyBt/aoDqc NTJJtQVy+UUtIVH8YJUPCzgyOBm7xf8xUmgnfhNIOy+5zKSM+Bksscnp1r801YiCYjWx uFRj14xW5H0ppn4A2WzcT/8Uyc/SJ6iJmcu/FoK98QgMl19SSIksnDHQtsDw/PcoajXs Z2iZNzYdytndE5Nj+j43cxGRDjU5+yzmxB796vtH+4eJ16mIx+pZt143bod0KNifw/ks NjTeCFxWUnFH3rDQTs+b60jhJHiM50oHFKJe+4BaUin5dQyr/ugTaM05nJkEsuPtZkxv k8tA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1694077571; x=1694682371; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QwR4d+5UoVgi3hAZhhlc3KZ970LG1A0zS/bDs+8zFKk=; b=XA3/hvfy07gKdHZsM7hl/QqqmFWFqgyO5z396+omeRLvXgl+5wCwdDW+h8ieliULZW bj6dF5JNOdI3EHTUdpk1Dbz5XTUmrtDHqYCjiUlWE+lLP0/ahVL1NzHCm+R1d1WBl3Zv RM90aHZP9WK2jyhqKTPtb/OS/BIEBBow5tefeyicnJz1X3z2R/OD5112elGQ5C8XwfCW b0On3tvLIB2hwVjesjGwnj4jfDwoCIrKEqua8KRjx+uA7qOLJ5yy3RNnXMLXrbxvwARp K5uAO36f3jpmTqp8sUpEp9SjrJFe2NwkideBv5lP3Jx2uTh52UBon/UrWpiFoWdSxh8U t/uA== X-Gm-Message-State: AOJu0YyoYPK1oskmonEwG4+gDRQIsY52r8RUL56tOc4c9suye6Q7qYgu vP97WPcnS87J6IgtX7nH5me2+1ZCDox5TAsiCX59ug== X-Google-Smtp-Source: AGHT+IFJTavyxjhxdaZZ701qmmj1qQoYaTn/kfRbKFSawpT28Zk2UYqvQGaxKIpZhSb6/hBu/iAUJVkTBPu3r6ZOQ5U= X-Received: by 2002:a2e:b013:0:b0:2bc:b29e:8ff7 with SMTP id y19-20020a2eb013000000b002bcb29e8ff7mr4344328ljk.20.1694077571114; Thu, 07 Sep 2023 02:06:11 -0700 (PDT) MIME-Version: 1.0 References: <20230801085402.1168351-1-alexghiti@rivosinc.com> <20230801085402.1168351-5-alexghiti@rivosinc.com> In-Reply-To: From: Alexandre Ghiti Date: Thu, 7 Sep 2023 11:05:59 +0200 Message-ID: Subject: Re: [PATCH v3 4/4] riscv: Improve flush_tlb_kernel_range() To: "Lad, Prabhakar" Cc: Geert Uytterhoeven , Will Deacon , "Aneesh Kumar K . V" , Andrew Morton , Nick Piggin , Peter Zijlstra , Mayuresh Chitale , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Andrew Jones Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 40E42C000F X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 3by5entxx8ufamphg3ewjcbwcxpaz931 X-HE-Tag: 1694077572-776104 X-HE-Meta: U2FsdGVkX1/yfgXb+0b+nxpR1jisFB8CFFxZbCy1QKCUrt+3CdwAkEpE79TTF/fXvYfd/JPuF5gl5D6wK+aBxQcPe+1OjvHP7iO6GCWn3zNDD3iT640X5A5j2kCoNzTi6ZWFQ9Jabs4DVMHz2XofeYe5Cxqvx1GGaeKbWC0vwA3YfLGbjE0MNA3MGPgPYGOc3yS1fjYFzb8v0PYYwoonN5oOJ3Yqq/P5dyk0OXl4A6ABrtan4IjohvbYnIY0KMv5raDN5WKn9PPAyJjadbdJUcaopBpJtpcPiRIAIGdcq6K0W6NT6fBr0fb4FDBeXlpwqOBT/ar68ZRdRXvCPk/PQPdzqnXCwrFUMdQk0CFH2hKgTDypyQUMorgRI2qT1HKmGSPkeaUT/PqJltFjNeMtxytMLlQVkfUWa0w9aFz8inxNpIHJWcVL48h/oyN2TrpN1jV+dfQ4zsEaf4To9ezXVg9v997zm4/ZlnAryrNF8ZPO1WdOnBDijRJOrJ8WNm8IrAU5QOz9pGy91qleTrzvpz8vcvTpUKBBl+3L/vObfTrHfOufWZHiuvyrw+WUuVgPr+NGcjXjo+qlPj2LAEzctPDE8J+8r9P6J7SlavqRlx0QUsCp+qLgYje3+J8KAAv/et8/coZlnjkAs85LvXrVLDr1iTerouIKzeW65N6kr9ozh+WRFb2I33zJxCv6dhJrRY1CrapfCBF/4DYw2c88z/Lyk8EIZ2X76lKgs3+k0rjnA9OYX8OTeF+ASYsoGAPoZx86TCPLnCEsf0qqd98HWW45sSTyugl+TE8VUni8QsGH6WvQZN8oUJNjry841XpJC6Ud7BP5xRmvQMs8nDcGqctpFCK5DXjvQSVksa6fwHQngivC9MVrpik6VdKJV7JvdR7dYCwjJ1F5O1ypKJfhyTuAlw2C4ZhzmAet8e6PDAVLQUxOqt9NXaksnCUxnpbRCrr9nhWztUpqTb/RN3k AMaWS8Z/ fM+031YyIxnQOI+iJ6j9EX8Kv8H7nUmPeHVmwtiw1bgCttIHBuQ1KR8XS3Tf3ovlt19T18ErREkChIHA6npf1yhx4hM5idcuh2bGLZbDwicJuy84saJLparvLLcTsPz6H9yPJHZUmydgLgsPUqqfi/+4SW3vsfpUijXLYp1uDF/sbGtF8L6cbO6C+FQaCw6Wv53KBd6ouVRSV06Scos0r+H+IJPurCITGj1MQ20IKvqjjIraTdq8cykaRrGgxN5e0o+3bUe+v1cYFM+h4kOWu9J546jL/xkynou+BuDtmViXYAQ3XA0eAS46xlTeL1nTQovDo6giCmYTIgytsyPV8c6ZaTt5X94szrk4nelcEPwTensewJJZ7tAwaZqDw4ayPCQj5cYNzCf01S6NaVepkJbDZqRqYnh6N9qwvS54UDX6UkgO+ywL9SB5B9GeZMBEe8t1NJtSUXd5DU65e/cqf63tRUZYse3aRqGuFYhIpGX5kLkZvpLVDTJuL3oH7vRu3p1QCPsNxIskURQSBuon0XjDcI4JRv1nQG86BOMdMh8BtIS3iNMWtomCAkcdggFL1p/b3 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Prabhakar, On Wed, Sep 6, 2023 at 3:55=E2=80=AFPM Lad, Prabhakar wrote: > > Hi Alexandre, > > On Wed, Sep 6, 2023 at 1:43=E2=80=AFPM Alexandre Ghiti wrote: > > > > On Wed, Sep 6, 2023 at 2:24=E2=80=AFPM Lad, Prabhakar > > wrote: > > > > > > Hi Alexandre, > > > > > > On Wed, Sep 6, 2023 at 1:18=E2=80=AFPM Alexandre Ghiti wrote: > > > > > > > > On Wed, Sep 6, 2023 at 2:09=E2=80=AFPM Lad, Prabhakar > > > > wrote: > > > > > > > > > > Hi Alexandre, > > > > > > > > > > On Wed, Sep 6, 2023 at 1:01=E2=80=AFPM Alexandre Ghiti wrote: > > > > > > > > > > > > Hi Prabhakar, > > > > > > > > > > > > On Wed, Sep 6, 2023 at 1:49=E2=80=AFPM Lad, Prabhakar > > > > > > wrote: > > > > > > > > > > > > > > Hi Alexandre, > > > > > > > > > > > > > > On Tue, Aug 1, 2023 at 9:58=E2=80=AFAM Alexandre Ghiti wrote: > > > > > > > > > > > > > > > > This function used to simply flush the whole tlb of all har= ts, be more > > > > > > > > subtile and try to only flush the range. > > > > > > > > > > > > > > > > The problem is that we can only use PAGE_SIZE as stride sin= ce we don't know > > > > > > > > the size of the underlying mapping and then this function w= ill be improved > > > > > > > > only if the size of the region to flush is < threshold * PA= GE_SIZE. > > > > > > > > > > > > > > > > Signed-off-by: Alexandre Ghiti > > > > > > > > Reviewed-by: Andrew Jones > > > > > > > > --- > > > > > > > > arch/riscv/include/asm/tlbflush.h | 11 +++++----- > > > > > > > > arch/riscv/mm/tlbflush.c | 34 +++++++++++++++++++= ++++-------- > > > > > > > > 2 files changed, 31 insertions(+), 14 deletions(-) > > > > > > > > > > > > > > > After applying this patch, I am seeing module load issues on = RZ/Five > > > > > > > (complete log [0]). I am testing defconfig + [1] (rz/five rel= ated > > > > > > > configs). > > > > > > > > > > > > > > Any pointers on what could be an issue here? > > > > > > > > > > > > Can you give me the exact version of the kernel you use? The tr= ap > > > > > > addresses are vmalloc addresses, and a fix for those landed ver= y late > > > > > > in the release cycle. > > > > > > > > > > > I am using next-20230906, Ive pushed a branch [1] for you to have= a look. > > > > > > > > > > [0] https://github.com/prabhakarlad/linux/tree/rzfive-debug > > > > > > > > Great, thanks, I had to get rid of this possibility :) > > > > > > > > As-is, I have no idea, can you try to "bisect" the problem? I mean > > > > which patch in the series leads to those traps? > > > > > > > Oops sorry for not mentioning earlier, this is the offending patch > > > which leads to the issues seen on rz/five. > > > > Ok, so at least I found the following problem, but I don't see how > > that could fix your issue: can you give a try anyway? I keep looking > > into this, thanks > > > > diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c > > index df2a0838c3a1..b5692bc6c76a 100644 > > --- a/arch/riscv/mm/tlbflush.c > > +++ b/arch/riscv/mm/tlbflush.c > > @@ -239,7 +239,7 @@ void flush_tlb_range(struct vm_area_struct *vma, > > unsigned long start, > > > > void flush_tlb_kernel_range(unsigned long start, unsigned long end) > > { > > - __flush_tlb_range(NULL, start, end, PAGE_SIZE); > > + __flush_tlb_range(NULL, start, end - start, PAGE_SIZE); > > } > > > I am able to reproduce the issue with the above change too. I can't reproduce the problem on my Unmatched or Qemu, so it is not easy to debug. But I took another look at your traces and something is weird to me. In the following trace (and there is another one), the trap is taken at 0xffffffff015ca034, which is the beginning of rz_ssi_probe(): that's a page fault, so no translation was found (or an invalid one is cached). [ 16.586527] Unable to handle kernel paging request at virtual address ffffffff015ca034 [ 16.594750] Oops [#3] ... [ 16.622000] epc : rz_ssi_probe+0x0/0x52a [snd_soc_rz_ssi] ... [ 16.708697] status: 0000000200000120 badaddr: ffffffff015ca034 cause: 000000000000000c [ 16.716580] [] rz_ssi_probe+0x0/0x52a [snd_soc_rz_ssi] ... But then here we are able to read the code at this same address: [ 16.821620] Code: 0109 6597 0000 8593 5f65 7097 7f34 80e7 7aa0 b7a9 (7139) f822 So that looks like a "transient" error. Do you know if you uarch caches invalid TLB entries? If you don't know, I have just written some piece of code to determine if it does, let me know. Do those errors always happen? > > Cheers, > Prabhakar