From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85FC6C4167B for ; Thu, 9 Nov 2023 14:09:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 046E28D00D7; Thu, 9 Nov 2023 09:09:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F379F8D0073; Thu, 9 Nov 2023 09:09:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DFF038D00D7; Thu, 9 Nov 2023 09:09:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id CD0D88D0073 for ; Thu, 9 Nov 2023 09:09:02 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 97156140ED4 for ; Thu, 9 Nov 2023 14:09:02 +0000 (UTC) X-FDA: 81438597324.30.D4004E4 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf21.hostedemail.com (Postfix) with ESMTP id 9C91B1C0026 for ; Thu, 9 Nov 2023 14:08:58 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=none; spf=pass (imf21.hostedemail.com: domain of zhangpeng362@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=zhangpeng362@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699538939; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references; bh=KuUf0XYf058sGGUYE6o8VIgIZZ1R0vThyQm96wl8m6g=; b=yyiy9b7PMTjzd6uitn1vHxlKejZOtYEPnAzmiFWvt0VZLdoNvZ4seqEUePIcKaa7O0rawh sXlleLKixob45q1b/RoO3uDV+4LUnm9h5rkf3u1ny4iVH8iWzZff9djSM6fJ4a21LO2nfD C24ZGaalOiRSh7XFFi8wvgeJZIJ84H4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699538939; a=rsa-sha256; cv=none; b=pFLfFMsmP/c1AXiDj26Qgq67Dvc9I1ZEzsMAUAEjBEBOcDQxNW/lznq6kyH76WXPIE1zbW iY86cQGIJw0LPhbhT3aYc8wNzo36qs/aygywHUBAioeXHMMEB0SLfr0PS0P+9oITHYh0Ia DYl0YBLJq/nbJVgWhNRCLc8+e2EO+KE= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=none; spf=pass (imf21.hostedemail.com: domain of zhangpeng362@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=zhangpeng362@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from kwepemm000020.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4SR39D27d5zmXCt; Thu, 9 Nov 2023 21:44:48 +0800 (CST) Received: from [10.174.179.160] (10.174.179.160) by kwepemm000020.china.huawei.com (7.193.23.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Thu, 9 Nov 2023 21:47:24 +0800 Message-ID: <9e62fd9a-bee0-52bf-50a7-498fa17434ee@huawei.com> Date: Thu, 9 Nov 2023 21:47:24 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.9.0 Content-Language: en-US To: , CC: , Matthew Wilcox , , , , , , , , , , , , Nanyong Sun , Kefeng Wang From: "zhangpeng (AS)" Subject: [Question]: major faults are still triggered after mlockall when numa balancing Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.174.179.160] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To kwepemm000020.china.huawei.com (7.193.23.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 9C91B1C0026 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: pmkoab5s8wkmeucwio6ct3kmd138fmms X-HE-Tag: 1699538938-587582 X-HE-Meta: U2FsdGVkX19PoTHJ/vySEpGKDND6ExsNgqOD57wtNjA99V2SXf6w6sxlCpGZgmsBnOMe9UmpnmnBkGn4SHy3jwEqm3fcX+DBp9ftzz06r1oKyKW2Xy/pNdQWDp22KmLBQd0Pa/oTBeF+crFzp5tuLLBx7ipAs3UPDhN0akyBtq16kMUB+5q/DrO0YIJlcfQWImkBvjMbMzLebPfCKdiLGeJb0tSpfGfWR9Cgqog1YjCkvZljUjYF3j3rW44LLHuCu+FvJm8YxJhFgrJVud35fQ5V2JEUZuFPPRh2BDhy7erkhurTdAYOEkpA8i/4TlgTZ4R9waGZ6/4HxlI5Ju4WHw8JhZZFG6k93tR2AZsywvEJjGyi6bz18VSpfPQKaX06n59fxCphcouErFKSJeUuXf8LiwgtVHggsTMcX9nD0GouPEmKFY09EjexSWsu3xHnzwWCyoaC1ZmQoHmwRf6zncF+ZunSb44UNbpK1HSO1NoQztKc4rkZO4uzNJY1JIa9rOz7XQWV6b1NFkxPIsC4xq7LKFyaN7LuNkEJ5dVJe+TJgARUyOY8FzOix35dUNRM9ZKevnbYuQ0Nr4KtP+F3FWFd6cY8PO5wCoRcqruwxeyEW+jk4yv4MRfAUSkcDrJDG0/13BieyqOVznj1UKJqNtLJbBEusrbHdMXYCgHRKI1TXaADsy5LeaTsKdp3QUa65o5RXUqx0xdOTijj7GjVDEV5labR5n2yxT3HbKSuA/81QjRwp2NHsmBApzJ5KbyfUNTNudLm8WYjdHC2Yzt98b7xHskiM24E2RGDHKNIssE//ssTLeEtkcllM0emG8YjV/qCTC49Q8D7KxWaDaOwXKUZ0OS8oXOiiVBAiDJGkgfmc0xiW3nECa3M0ldeCuKIzM7GoHrGAzy6OBCOMrDc8rmZPag3lfZF5iC2MNVwRIn/I+X51daaVCNSHc4Gxp0pa3+P2jEt1b8Cbubpi8g AAOhf3FZ J1hn/9SHrI+F4M3m+y1jILhJkNMO4c4+jG2BN02d3Sccx0R7WHOwz0EQQXP0Mnx/PkNEaY0gT9GxYIqVc5KINjS8NdcuKw0msuov0khM/UALtcmbuZ4cnVBjZsAUvV2pQNrqbHj1v4kBaSL8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi everyone, There is a performance issue that has been bothering us recently. This problem can reproduce in the latest mainline version (Linux 6.6). We use mlockall(MCL_CURRENT | MCL_FUTURE) in the user mode process to avoid performance problems caused by major fault. There is a stage in numa fault which will set pte as 0 in do_numa_page() : ptep_modify_prot_start() will clear the vmf->pte, until ptep_modify_prot_commit() assign a value to the vmf->pte. For the data segment of the user-mode program, the global variable area is a private mapping. After the pagecache is loaded, the private anonymous page is generated after the COW is triggered. Mlockall can lock COW pages (anonymous pages), but the original file pages cannot be locked and may be reclaimed. If the global variable (private anon page) is accessed when vmf->pte is zero which is concurrently set by numa fault, a file page fault will be triggered. At this time, the original private file page may have been reclaimed. If the page cache is not available at this time, a major fault will be triggered and the file will be read, causing additional overhead. Our problem scenario is as follows: task 1 task 2 ------ ------ /* scan global variables */ do_numa_page() spin_lock(vmf->ptl) ptep_modify_prot_start() /* set vmf->pte as null */ /* Access global variables */ handle_pte_fault() /* no pte lock */ do_pte_missing() do_fault() do_read_fault() ptep_modify_prot_commit() /* ptep update done */ pte_unmap_unlock(vmf->pte, vmf->ptl) do_fault_around() __do_fault() filemap_fault() /* page cache is not available and a major fault is triggered */ do_sync_mmap_readahead() /* page_not_uptodate and goto out_retry. */ Is there any way to avoid such a major fault? -- Best Regards, Peng