From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7756FC4332F for ; Fri, 10 Nov 2023 09:50:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 86FF64401C7; Fri, 10 Nov 2023 04:50:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 81FCE280009; Fri, 10 Nov 2023 04:50:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6E8074401C7; Fri, 10 Nov 2023 04:50:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 5D0B0280009 for ; Fri, 10 Nov 2023 04:50:57 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 296304030B for ; Fri, 10 Nov 2023 09:50:57 +0000 (UTC) X-FDA: 81441575754.23.B6F19D1 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf13.hostedemail.com (Postfix) with ESMTP id 2B8C72000F for ; Fri, 10 Nov 2023 09:50:53 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; spf=pass (imf13.hostedemail.com: domain of zhangpeng362@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=zhangpeng362@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699609855; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hqx4naJGEpHYm3Ps8GGYdbYFPF8NGTbPwA6+k94n7YE=; b=Jiha19hdaCNCpiwUe2x6htQ16ruU/MV20rCYNOc9DCAzOzdtuIwchq4E4ZUkdX4DwzEGk+ XKzOKkBXh35gZOJuveC3DHK5w6Jfrx0MQhC5YuN4BVT2m0L0WYpY880Qmu2tDj7lORDaE6 NmkiClbeCLtEHnPOnCAf33+/T3F+uCs= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; spf=pass (imf13.hostedemail.com: domain of zhangpeng362@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=zhangpeng362@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699609855; a=rsa-sha256; cv=none; b=BX9BVbkzuRCIf+Fie5qY+CAm7vpzB98/p2tgF8jU1RT/xwqrMrYA+v/u03CM3x/T+rlxIb bgIgh1CpV6LiPl+CvZJyCwzBqBQo+A7VGDadBze1AsUuS9oY4vu2Koxznhii9aDNlbuVBP gc7eVLII2CIP+reTq5gnm1uPGa7hK20= Received: from kwepemm000020.china.huawei.com (unknown [172.30.72.56]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4SRYs20s9Xz1P86T; Fri, 10 Nov 2023 17:47:34 +0800 (CST) Received: from [10.174.179.160] (10.174.179.160) by kwepemm000020.china.huawei.com (7.193.23.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Fri, 10 Nov 2023 17:50:45 +0800 Message-ID: Date: Fri, 10 Nov 2023 17:50:45 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.9.0 Subject: Re: [Question]: major faults are still triggered after mlockall when numa balancing Content-Language: en-US To: Aneesh Kumar K.V , , CC: , Matthew Wilcox , , , , , , , , , , , , Nanyong Sun , Kefeng Wang References: <9e62fd9a-bee0-52bf-50a7-498fa17434ee@huawei.com> <87edgynht3.fsf@linux.ibm.com> From: "zhangpeng (AS)" In-Reply-To: <87edgynht3.fsf@linux.ibm.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.174.179.160] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemm000020.china.huawei.com (7.193.23.93) X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 2B8C72000F X-Rspam-User: X-Stat-Signature: 77wj1h7b6zarjc64grqr9r3geaiy1zq4 X-Rspamd-Server: rspam01 X-HE-Tag: 1699609853-553559 X-HE-Meta: U2FsdGVkX1/SnQ1NpRckoeEbiPOO9XRy+SXf+s6LiIZpimFo0yPcumt/MdEyK1k2hhql/GSXdnd10VkAzZ6AHys6fWy4b9rXyQhLfw0w9yuQGUQYakpvP1ZN3d8PP1zrzPbCfauINPS9x3w3aJodsyy79MgSHyVwf8mqhQJRv/xlxx0maZEt/BGmosow2riRIRd1tgkiRbt7teKGuF9zTRjsFphgwAdLij0chjyWu2tbPeUpaBsDq95EwA8YJznsJnzcV6iUDwi6YqFvFmrBj1GYFy8gpafuKo7kRUQHbqYFs2R/ct6fz3U9p2IMwPECH27CY+fY5NA4NlDKNPCCKCRiYVUTp6lqVp3FF0sVWEny8xyDthxDi1sh3hd9Aft/RP95Wx9oM82uW552YAvjPJ0F1JsJlsCpofaQ2RSu8B7jycHmMq1Zh1t5+QmFP6fulowgDK4bWS9DKoXNfO4QpcGkttc3UrLF0mcMKz1ygbVRxOgL1ng/5Ad5546U5smN1kBPNFJS8hB14E5SyW+CRVsqjnYpyBf+ZC54sX82xejza2sCCY986C5HdkCZuik2HBCoHSxQwxJyy+rvHnjf+NqRwY0CvaF+/logZAd/wb8s+5LsFo4WdZTo/wRSwq5VpDr2amjFXGqmJicDu4//1ChP6W5YnGFtTVo9Wi9LuK5GBBj8eNx73kMWQyyI2QQph93kV+7cx0Gz/QZt/VQnXKqI43HVHmVkTXJ8IjsJ1LrE/Axh+bsv/jxL2y94lfxRFBuaUtFNA6sPwv9mKpc4QBqJ/kifglRPVyogHpp8kjOyKC4rgttSUiwN3jYmuHtidPcfUPQtBKsDlrVdZUUUZG9mD7d9PSI58pgIyXe99WqFFHadg4t4nHcl2MnKKelZz4jqS7xfox9sh5q5gbGO4xsutRF6G2/U4n/L05mEn29QNtcwYbQYAmN6Mgc6zlUOdTAo5PPYwkhJsLvnNoN leyUPo0h vsnxpdolWd2CWi1byh1uhL7vgdOvRqNOukMkDA5sJKbXRohPmC9iLN9zmam822L541vuw8QIzM5TduDzR5T3Okq/wd16cGAUGftU7+pxUVo/Jjxd+/wU9QUp7yf+hA55GtkJcBg/xF91ijPqlHNeZDHOkBg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2023/11/10 16:17, Aneesh Kumar K.V wrote: > "zhangpeng (AS)" writes: > >> Hi everyone, >> >> There is a performance issue that has been bothering us recently. >> This problem can reproduce in the latest mainline version (Linux 6.6). >> >> We use mlockall(MCL_CURRENT | MCL_FUTURE) in the user mode process >> to avoid performance problems caused by major fault. >> >> There is a stage in numa fault which will set pte as 0 in do_numa_page() : >> ptep_modify_prot_start() will clear the vmf->pte, until >> ptep_modify_prot_commit() assign a value to the vmf->pte. >> > pte lookup don't expect the pte to be 0 after it got initialized (We do > check pte value without holding ptl and if we find the pte val 0 we > return). So the read-modify-write updates to the pte should make sure we > don't clear the pte right? powerpc did that by marking the pte present > but invalid. Can we do similar for other architecture? The default > implementation of ptep_modify_prot_start() to ptep_get_and_clear() can > result in pte lookup returning wrong pte as explained in the report > because we don't hold ptl and recheck if we find pte == 0 > Thank you for helping to clarify the problem. >> For the data segment of the user-mode program, the global variable area >> is a private mapping. After the pagecache is loaded, the private >> anonymous page is generated after the COW is triggered. Mlockall can >> lock COW pages (anonymous pages), but the original file pages cannot >> be locked and may be reclaimed. If the global variable (private anon page) >> is accessed when vmf->pte is zero which is concurrently set by numa fault, >> a file page fault will be triggered. >> >> At this time, the original private file page may have been reclaimed. >> If the page cache is not available at this time, a major fault will be >> triggered and the file will be read, causing additional overhead. >> >> Our problem scenario is as follows: >> >> task 1 task 2 >> ------ ------ >> /* scan global variables */ >> do_numa_page() >> spin_lock(vmf->ptl) >> ptep_modify_prot_start() >> /* set vmf->pte as null */ >> /* Access global variables */ >> handle_pte_fault() >> /* no pte lock */ >> do_pte_missing() >> do_fault() >> do_read_fault() >> ptep_modify_prot_commit() >> /* ptep update done */ >> pte_unmap_unlock(vmf->pte, vmf->ptl) >> do_fault_around() >> __do_fault() >> filemap_fault() >> /* page cache is not available >> and a major fault is triggered */ >> do_sync_mmap_readahead() >> /* page_not_uptodate and goto >> out_retry. */ >> >> Is there any way to avoid such a major fault? >> > -aneesh -- Best Regards, Peng