From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA4BEEB64D7 for ; Mon, 26 Jun 2023 10:52:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3D9638D0002; Mon, 26 Jun 2023 06:52:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 38A768D0001; Mon, 26 Jun 2023 06:52:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2511C8D0002; Mon, 26 Jun 2023 06:52:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 16CDA8D0001 for ; Mon, 26 Jun 2023 06:52:48 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id E3B511C75C8 for ; Mon, 26 Jun 2023 10:52:47 +0000 (UTC) X-FDA: 80944585974.27.497A984 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by imf16.hostedemail.com (Postfix) with ESMTP id 6F6D2180016 for ; Mon, 26 Jun 2023 10:52:45 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b="A5m/YXm9"; spf=pass (imf16.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687776765; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=G7UXEJVfOH8njSn3vIZel71lMNZ+KAhjbUKwumTdS0Y=; b=leUU5IIWNsi34jy4o+wTZOhd7W2CjpLv68Fxzy8sUAhmd0gnxZuAfy3FthFKAESL3T+kQq 1+DsApS6eB7pKLQ5FVQIwgazgWUk+p5SBwjfuJabE5S4N9lLgI/Xez7LcS3QgvP3S0JE7k xh2l+Xwe1TX9J0frxF+ptCToQt05Qog= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687776765; a=rsa-sha256; cv=none; b=W9TU3SME+Zslsj7QJMQcC+hLTZMpJFDubIdCGy9Id0ZCairsd8VDeBKpt+Hu7BzQO5BHnL LK/bLtlzCSMrCPKTp4SswnLWHittT8cGTAlHvtRFlxdpn6R9t1UcGV8a+gSV6fHoxkuQdR IGM1GHfjJt4VFIXSSEI5v2UJW6ksIhg= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b="A5m/YXm9"; spf=pass (imf16.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com Received: from pps.filterd (m0353723.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35QAlFLb024534; Mon, 26 Jun 2023 10:52:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : date : mime-version : subject : to : cc : references : from : in-reply-to : content-type : content-transfer-encoding; s=pp1; bh=G7UXEJVfOH8njSn3vIZel71lMNZ+KAhjbUKwumTdS0Y=; b=A5m/YXm9cZrp5e5NJBUinTFolc37y+hGVtDL5LhaLq4OziMVmsH9KXzeTb0JLTfkoicJ TASlR9kl94QeKeB1a7V9OhjKP9N3xhvkNK/pHTQuZFiP28BzMg8VXyc5J86/kiaX6qIi digJE5pK2Op7YL9CoAL6kxFIcHpjHRKhwBwtDe1zKFQaGbZyh/vrDD/P3qZYJNu7xWnX E3InEcWk0R4vKRE1B2nk3B4ItX86xOyPduqgAj7263S5NtHlP+6jz7dR/MlcYOF8SIU2 XNOXPK6GLxdux3ksUH2Ug15gTBKzViVdT+PBJc8PH7cfyEKFmhro3nidMvdFyN0Ldppu Wg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3rf97r039a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 26 Jun 2023 10:52:44 +0000 Received: from m0353723.ppops.net (m0353723.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35QApdE0003926; Mon, 26 Jun 2023 10:52:43 GMT Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3rf97r038q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 26 Jun 2023 10:52:43 +0000 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35Q1wglL006616; Mon, 26 Jun 2023 10:52:41 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma06ams.nl.ibm.com (PPS) with ESMTPS id 3rdqre13rk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 26 Jun 2023 10:52:41 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35QAqd2p14353010 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 26 Jun 2023 10:52:39 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2C0C02004E; Mon, 26 Jun 2023 10:52:39 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6942F20040; Mon, 26 Jun 2023 10:52:38 +0000 (GMT) Received: from [9.43.116.175] (unknown [9.43.116.175]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 26 Jun 2023 10:52:38 +0000 (GMT) Message-ID: <959537fc-cee4-f5e8-d7be-5e4402feda9f@linux.ibm.com> Date: Mon, 26 Jun 2023 16:22:37 +0530 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: [PATCH 3/3] mm/lru_gen: Don't build multi-gen LRU page table walk code on architecture not supported To: Yu Zhao Cc: linux-mm@kvack.org, akpm@linux-foundation.org References: <20230613120047.149573-1-aneesh.kumar@linux.ibm.com> <20230613120047.149573-3-aneesh.kumar@linux.ibm.com> <87bkh4x661.fsf@linux.ibm.com> Content-Language: en-US From: Aneesh Kumar K V In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 7ik-mh8GnTwvFmNWX4Aa8ybApNmvluan X-Proofpoint-ORIG-GUID: PLdtxKePpU_FsnYViqmstmAK2nwY_K0P X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-26_06,2023-06-22_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 clxscore=1015 impostorscore=0 mlxscore=0 phishscore=0 bulkscore=0 malwarescore=0 spamscore=0 adultscore=0 suspectscore=0 lowpriorityscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306260096 X-Stat-Signature: crx5irq8e8txtpwg6j5a643nfdzdsdso X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 6F6D2180016 X-Rspam-User: X-HE-Tag: 1687776765-971832 X-HE-Meta: U2FsdGVkX1/cqyYP91CitwdjzDoiUHy4avfpdXOYJ7+C7a4viRTupQTzVpvOyxCp02+LdK/wTK8AM+afmNizicAXbNCzi0vGACc685APw8gfKM6zjwwdQgUy9PqU3GhFErj1I9UWHBfViu517ecfc0bA2BJaR678huoBrW5U3Ut2L4LEPYy8DE6jjJAraLtE6ZOazIwnZitrPOFY9aTIIq5Ei8W17c9Vcv2G+XeE8JgtKimIApC3Q9QfDUYPi1s5OGEREoJ885xaeZzxgVvC4udMlaFms351LrV/FMqGgkdSABV7keveCYY9Xz2o9xZb2411/aetuWejwUpwLMzjUIWtJjU2qnHPZUAF2RHST7WAg/i0eUCZ2jg3RfQbias6iBtxhd5mtUGi+z0vEEtOeD2V0X7GU4C4LCbE6edyciTv5gk6O1VTroeV0Omkzjn8s4k/GjdoNkg0FxSdx/4qSt8cudlYJHQrmM/qK4kXFTUzXpayUKYxhl5nNU54ZpgtFWYUNDriJIYsthhqkeacdBRy5NOBTZLFb4tMr7c6qn5kW3bAgjPmv3xc2gv1MRxSjyhyQl3+g7QV4cewh5ioXe+PN6icKTlAHuGgSkiIIzGxxhuJajOWtbVom1wuCi/1rcmrWHlcCpzF9ah7GymF+syiSU6Y+21ElQJE1w5IkBZZGJpYguns7hEx9N+GPM/NUnRd/AagHr7U2xNqIFg++NdgmCfm7yR44S3axgsB0XW2W8YXTEgO2DdjF0Fwy3snsHOW31m2vTNG8bjyyumybKC6d+EUcmk1/S2FQ9B78rd9oQMExkhVhRSflrhoqiJhbH4lVeNSAJjtUbEhoV8/isnz/GZXH1oinlbbhYZH3sGx+5Ce3WHGh/RSdYHxRldK3wQgKF2iYu6dMokXR27b+UtlUQdhheKXHkhK2rBh8sfR5z2x+STQ3zG7xn4cU6baKwcIhB8/7CYAGMTVCN4 pX3OZ7jD H/6dJufJngq1e5w/C7+ZxFJNaW63le0rCGOas/y8bXNpDiJp05kUYDCK9FDYtnlDcFotYxk5JLyNcSkAWZVh9VBzOf5nr9Rj1K+rudlHv2EfSwpq47IaAXR65rk1sVjL3AcYdZQEzeoldxJKp0xY6CvIsZ4qSkRlFTLvpVuSWqvvHt5BIO0bxDFFv9Qzw/brqq7UH4KdG8U8zHe0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 6/26/23 1:04 AM, Yu Zhao wrote: > On Sat, Jun 24, 2023 at 8:54 AM Aneesh Kumar K.V > wrote: >> >> Hi Yu Zhao, >> >> "Aneesh Kumar K.V" writes: >> >>> Not all architecture supports hardware atomic updates of access bits. On >>> such an arch, we don't use page table walk to classify pages into >>> generations. Add a kernel config option and remove adding all the page >>> table walk code on such architecture. >>> >>> No preformance change observed with mongodb ycsb test: >>> >>> Patch details Throughput(Ops/sec) >>> without patch 93278 >>> With patch 93400 >>> >>> Without patch: >>> $ size mm/vmscan.o >>> text data bss dec hex filename >>> 112102 42721 40 154863 25cef mm/vmscan.o >>> >>> With patch >>> >>> $ size mm/vmscan.o >>> text data bss dec hex filename >>> 105430 41333 24 146787 23d63 mm/vmscan.o >>> >> >> Any feedback on this patch? Can we look at merging this change? > > Just want to make sure I fully understand the motivation: are there > any other end goals besides reducing the footprint mentioned above? > E.g., preparing for HCA, etc. (My current understanding is that HCA > shouldn't care about it, since it's already runtime disabled if HCA > doesn't want to use it.) > My goal with this change was to remove all those dead code from getting complied in for ppc64. > Also as explained offline, solely relying on folio_activate() in > lru_gen_look_around() can cause a measure regression on powerpc, > because > 1. PAGEVEC_SIZE is 15 whereas pglist_data->mm_walk.batched is > virtually unlimited. > 2. Once folio_activate() reaches that limit, it takes the LRU lock on > top of the PTL, which can be shared by multiple page tables on > powerpc. > > In fact, I think we try the opposite direction first, before arriving > at any conclusions, i.e., > #define arch_has_hw_pte_young() radix_enabled() The reason it is disabled on powerpc was that a reference bit update takes a pagefault on powerpc irrespective of the translation mode. > on powerpc. This might benefit platforms with the A-bit but not HCA, > e.g., POWER9. I just ran a quick test (memcached/memtier I previously > shared with you) and it showed far less PTL contention in kswapd. I'm > attaching the flamegraphs for you to analyze. Could you try some > benchmarks with the above change on your end as well? > The ptl lock is a valid concern even though i didn't observe the contention increasing with the change. I will rerun the test to verify. We have possibly two options here 1) Delay the lruvec->nr_pages update until the sort phase. But as you explained earlier, that can impact should_run_aging(). 2) Add another batching mechanism similar to pglist_data->mm_walk which can be used on architecture that don't support hardware update of access/reference bit to be used only by lru_gen_look_around() -aneesh