From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C25D9C433F5 for ; Fri, 3 Sep 2021 22:16:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 17F7B6101A for ; Fri, 3 Sep 2021 22:16:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 17F7B6101A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=hisilicon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 856436B0071; Fri, 3 Sep 2021 18:16:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7DDA46B0072; Fri, 3 Sep 2021 18:16:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6A5336B0073; Fri, 3 Sep 2021 18:16:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0207.hostedemail.com [216.40.44.207]) by kanga.kvack.org (Postfix) with ESMTP id 564286B0071 for ; Fri, 3 Sep 2021 18:16:25 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id F04E22043C for ; Fri, 3 Sep 2021 22:16:24 +0000 (UTC) X-FDA: 78547671888.12.A886E30 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf30.hostedemail.com (Postfix) with ESMTP id E4BB6E0016BE for ; Fri, 3 Sep 2021 22:16:23 +0000 (UTC) Received: from dggemv711-chm.china.huawei.com (unknown [172.30.72.53]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4H1XDK3Sdmz1DGXt; Sat, 4 Sep 2021 06:15:29 +0800 (CST) Received: from dggemi709-chm.china.huawei.com (10.3.20.108) by dggemv711-chm.china.huawei.com (10.1.198.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2176.2; Sat, 4 Sep 2021 06:16:15 +0800 Received: from dggemi761-chm.china.huawei.com (10.1.198.147) by dggemi709-chm.china.huawei.com (10.3.20.108) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.8; Sat, 4 Sep 2021 06:16:14 +0800 Received: from dggemi761-chm.china.huawei.com ([10.9.49.202]) by dggemi761-chm.china.huawei.com ([10.9.49.202]) with mapi id 15.01.2308.008; Sat, 4 Sep 2021 06:16:15 +0800 From: "Song Bao Hua (Barry Song)" To: Huang Shijie , Florian Weimer CC: Huang Shijie via Libc-alpha , "carlos@systemhalted.org" , "zwang@amperecomputing.com" , "patches@amperecomputing.com" , "akpm@linux-foundation.org" , "torvalds@linux-foundation.org" , "viro@zeniv.linux.org.uk" , "willy@infradead.org" , "linux-mm@kvack.org" Subject: RE: [PATCH] Add LD_NUMA_REPLICATION for glibc Thread-Topic: [PATCH] Add LD_NUMA_REPLICATION for glibc Thread-Index: AQHXoNaLIDPd6kWWt0CtjlSjT9ni+auS3vng Date: Fri, 3 Sep 2021 22:16:14 +0000 Message-ID: References: <20210903121434.12162-1-shijie@os.amperecomputing.com> <87bl5a6ul2.fsf@oldenburg.str.redhat.com> In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.126.200.215] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-CFilter-Loop: Reflected Authentication-Results: imf30.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=hisilicon.com; spf=pass (imf30.hostedemail.com: domain of song.bao.hua@hisilicon.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=song.bao.hua@hisilicon.com X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: E4BB6E0016BE X-Stat-Signature: ziqpq61ykzccwzds1uz7wb11pcyj4qft X-HE-Tag: 1630707383-587907 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > -----Original Message----- > From: Huang Shijie [mailto:shijie@os.amperecomputing.com] > Sent: Saturday, September 4, 2021 3:16 AM > To: Florian Weimer > Cc: Huang Shijie via Libc-alpha ; > carlos@systemhalted.org; zwang@amperecomputing.com; > patches@amperecomputing.com; akpm@linux-foundation.org; > torvalds@linux-foundation.org; viro@zeniv.linux.org.uk; willy@infradead.o= rg; > Song Bao Hua (Barry Song) ; linux-mm@kvack.or= g > Subject: Re: [PATCH] Add LD_NUMA_REPLICATION for glibc >=20 > On Fri, Sep 03, 2021 at 08:28:57AM +0200, Florian Weimer wrote: > > * Huang Shijie via Libc-alpha: > > > > > This patch adds LD_NUMA_REPLICATION which influences the linkage of s= hared > libraries at run time. > > > > > > If LD_NUMA_REPLICATION is set for program foo like this: > > > #LD_NUMA_REPLICATION=3D1 ./foo > > > > > > At the time ld.so mmaps the shared libraries, it will uses > > > mmap(, c->prot | PROT_WRITE, MAP_COPY | MAP_FILE | MAP_POPULATE,) > > > for them, and the mmap will trigger COW(copy on write) for the shared > > > libraries at the NUMA node which the program `foo` runs. After the > > > COW, the foo will have a copy of the shared library segment(mmap > > > covered) which belong to the same NUMA node. > > > > > > So when enable LD_NUMA_REPLICATION, it will consume more memory, > > > but it will reduce the remote-access in NUMA. > > > > I think the kernel could do this in a much better way, avoiding > > duplicating the pages within the same NUMA node. > I think maybe only the per-NUMA-node page cache can avoid it.. >=20 > > > > The other issue is the temporary RWX mapping, which does not > > interoperate well with some security hardening features. > > > Thanks for the feedback. > I CC more people who may have interesting about this. >=20 Hi Shijie, Thanks! Could we have some benchmark data for this patch? I assume it will benefit frontend-bound large workset which might face relatively more icache miss? >=20 > Thanks > Huang Shijie Thanks Barry