From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81B91CD128D for ; Wed, 3 Apr 2024 08:38:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0C7676B008C; Wed, 3 Apr 2024 04:38:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ECD6E6B0092; Wed, 3 Apr 2024 04:38:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CFAE26B0093; Wed, 3 Apr 2024 04:38:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A19486B008C for ; Wed, 3 Apr 2024 04:38:32 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 6C0FC80CC6 for ; Wed, 3 Apr 2024 08:38:32 +0000 (UTC) X-FDA: 81967569264.13.A300A41 Received: from szxga04-in.huawei.com (szxga04-in.huawei.com [45.249.212.190]) by imf26.hostedemail.com (Postfix) with ESMTP id 0755F14000A for ; Wed, 3 Apr 2024 08:38:29 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.190 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712133510; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vhZLrHVELrtbu5DW0tOonOUKFnnhSmrXd1Sgos/yGkA=; b=qrmAUalkvfWui39hqHzv1UvQtcFq0GSNMZxzFx+HQmG65EgsYirx8Nhl52aY2P8kEktlyJ eQQPqv55ImPw5oSImmsL+pOjNwh3L69hXT8Z62vQYsHvk2WPJXps1HcSGBpWs9PeI86LLc vAA34tOdGr5FOY9Y0TfcJu24Q8/bNmw= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.190 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712133510; a=rsa-sha256; cv=none; b=xPCEQulRBcXEqy6EvBcQYkCIg5nhHq6LcoPwGIikNK+quNT5kkwYmBHTLUUh5RIMdKJwCR D/CbzbtDGJ0kxXqZpXUZzZbQBhxQ35zpLlPcOQFtthDlYREMXV/H/fMY9M9AllD1yuNKd7 7ULKMUy3wX/1x+ZJkVjaKPPz3ad98kI= Received: from mail.maildlp.com (unknown [172.19.163.44]) by szxga04-in.huawei.com (SkyGuard) with ESMTP id 4V8dP90NXPz29lXd; Wed, 3 Apr 2024 16:35:41 +0800 (CST) Received: from dggpemm100001.china.huawei.com (unknown [7.185.36.93]) by mail.maildlp.com (Postfix) with ESMTPS id B1225140410; Wed, 3 Apr 2024 16:38:26 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Wed, 3 Apr 2024 16:38:25 +0800 From: Kefeng Wang To: CC: Russell King , Catalin Marinas , Will Deacon , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Dave Hansen , Andy Lutomirski , Peter Zijlstra , , , , , , , , Kefeng Wang Subject: [PATCH v2 4/7] powerpc: mm: accelerate pagefault when badaccess Date: Wed, 3 Apr 2024 16:38:02 +0800 Message-ID: <20240403083805.1818160-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240403083805.1818160-1-wangkefeng.wang@huawei.com> References: <20240403083805.1818160-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To dggpemm100001.china.huawei.com (7.185.36.93) X-Rspamd-Queue-Id: 0755F14000A X-Rspam-User: X-Stat-Signature: k6j6c14wk3efx3iozatm81ewq5ehzqpt X-Rspamd-Server: rspam01 X-HE-Tag: 1712133509-986337 X-HE-Meta: U2FsdGVkX1/7S76NOUpjVPItyc3SJiPHjhZvnPB1/6ktNFWJQf8cIRV4Zcw2mTvSU/Qot8gw7vuot6nteMrDHnsdxvFRob/qFDmMCJuX4gYku6wn22IzbTjE/lKntwX5OdVARGGdyY1LdXDCyiJ6Vbyuo/ubxxTCX88pQXGHt3+YiCBusbzA+OXX+MZrYN21sjuI68UgUIBM+t1e8RvPUM++s6PDcPz2N/rAEYt6HbyjCTpEAmK+Z8jfg60h5m1vowYS9Pfd6rIbN/2Yw6Z4jJGAKB6CW7Q7K0J52alZwKTvsVhnsS13xp2vYRuFvMu7UDMgWScs4BDyI8B61jmw9wm4O8Z/j2t34tBP7TVsx/c3xDBcaR0TjBh6Y9yHBKOGwUfi/3wiop/PPh58ICyFarzaI6QQfvyzy/lEFhwQSusiF+LxL5ovC7FlKCE6prjum53pVdsy/Myfm2hd/K+fVGz4hldFLloVNeF5FpTrjkqtNVGOWuDFpg/wi+rUuYRFVO0n9GcptoBBw3wJ74h8SIjFfRlo/+mx7ukwTxq1N7vXaBsNqGYcgIq/QCuvD0QGDnJG62GCtUGcNWhe31PrJHER3Cma7Lgb/v5F9OCMo2+bnz0rV83/tu0hvNugXQ22HMxzn7Gy3dhEKOfqWQfCtAFIZdIbBEnqhVxC5uOmbLcwp+hEUHMxaK3G4a65n4SnHsuH9G3TwhOlPwe1uYK0kYAnT5hv/jjIotK/AUuKgKTEnBobKxfFd1t4lhjfYBqAV6MYt5HxJYckVtQ+bQtseTLNZgbkYWMeJcgNX5HlliNO7Ad7yPWXAxCmPjd6zlyhVjE07FEj1HKml2W2Jjy12ngDW2R8AX1xritXbwk+44EsgJ7Agizjdf+in42745Q/A5GupfhjvqLA2qoo1ygoa8RJBiTX6mNqwypDzbvF2t1xSUM5pURDU01rXd6dhtx2P+0ReBjuN1GFrqNNT2H 93Ntg+B3 ra46ymwTJjE8acO86POT5yH3TjzCBP2XTaWioOlv7R2SToGo/Mzigua5lEXW8tJerRNnSbSQxUaAlwFUo49CofFSNYtdOqszNov1m8GncTIOuSDl4yYK0NsxwkXZS2QurM/BC/iSCI4tUgPGfZRPPUoXThA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The access_[pkey]_error() of vma already checked under per-VMA lock, if it is a bad access, directly handle error, no need to retry with mmap_lock again. In order to release the correct lock, pass the mm_struct into bad_access_pkey()/bad_access(), if mm is NULL, release vma lock, or release mmap_lock. Since the page faut is handled under per-VMA lock, count it as a vma lock event with VMA_LOCK_SUCCESS. Signed-off-by: Kefeng Wang --- arch/powerpc/mm/fault.c | 33 ++++++++++++++++++++------------- 1 file changed, 20 insertions(+), 13 deletions(-) diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c index 53335ae21a40..215690452495 100644 --- a/arch/powerpc/mm/fault.c +++ b/arch/powerpc/mm/fault.c @@ -71,23 +71,26 @@ static noinline int bad_area_nosemaphore(struct pt_regs *regs, unsigned long add return __bad_area_nosemaphore(regs, address, SEGV_MAPERR); } -static int __bad_area(struct pt_regs *regs, unsigned long address, int si_code) +static int __bad_area(struct pt_regs *regs, unsigned long address, int si_code, + struct mm_struct *mm, struct vm_area_struct *vma) { - struct mm_struct *mm = current->mm; /* * Something tried to access memory that isn't in our memory map.. * Fix it, but check if it's kernel or user first.. */ - mmap_read_unlock(mm); + if (mm) + mmap_read_unlock(mm); + else + vma_end_read(vma); return __bad_area_nosemaphore(regs, address, si_code); } static noinline int bad_access_pkey(struct pt_regs *regs, unsigned long address, + struct mm_struct *mm, struct vm_area_struct *vma) { - struct mm_struct *mm = current->mm; int pkey; /* @@ -109,7 +112,10 @@ static noinline int bad_access_pkey(struct pt_regs *regs, unsigned long address, */ pkey = vma_pkey(vma); - mmap_read_unlock(mm); + if (mm) + mmap_read_unlock(mm); + else + vma_end_read(vma); /* * If we are in kernel mode, bail out with a SEGV, this will @@ -124,9 +130,10 @@ static noinline int bad_access_pkey(struct pt_regs *regs, unsigned long address, return 0; } -static noinline int bad_access(struct pt_regs *regs, unsigned long address) +static noinline int bad_access(struct pt_regs *regs, unsigned long address, + struct mm_struct *mm, struct vm_area_struct *vma) { - return __bad_area(regs, address, SEGV_ACCERR); + return __bad_area(regs, address, SEGV_ACCERR, mm, vma); } static int do_sigbus(struct pt_regs *regs, unsigned long address, @@ -479,13 +486,13 @@ static int ___do_page_fault(struct pt_regs *regs, unsigned long address, if (unlikely(access_pkey_error(is_write, is_exec, (error_code & DSISR_KEYFAULT), vma))) { - vma_end_read(vma); - goto lock_mmap; + count_vm_vma_lock_event(VMA_LOCK_SUCCESS); + return bad_access_pkey(regs, address, NULL, vma); } if (unlikely(access_error(is_write, is_exec, vma))) { - vma_end_read(vma); - goto lock_mmap; + count_vm_vma_lock_event(VMA_LOCK_SUCCESS); + return bad_access(regs, address, NULL, vma); } fault = handle_mm_fault(vma, address, flags | FAULT_FLAG_VMA_LOCK, regs); @@ -521,10 +528,10 @@ static int ___do_page_fault(struct pt_regs *regs, unsigned long address, if (unlikely(access_pkey_error(is_write, is_exec, (error_code & DSISR_KEYFAULT), vma))) - return bad_access_pkey(regs, address, vma); + return bad_access_pkey(regs, address, mm, vma); if (unlikely(access_error(is_write, is_exec, vma))) - return bad_access(regs, address); + return bad_access(regs, address, mm, vma); /* * If for any reason at all we couldn't handle the fault, -- 2.27.0