From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DATE_IN_FUTURE_06_12, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D33BEC433F5 for ; Fri, 3 Sep 2021 07:17:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 76E9161056 for ; Fri, 3 Sep 2021 07:17:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 76E9161056 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=os.amperecomputing.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id EBA9F900002; Fri, 3 Sep 2021 03:17:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E6ADF8D0001; Fri, 3 Sep 2021 03:17:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D0B1A900002; Fri, 3 Sep 2021 03:17:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0072.hostedemail.com [216.40.44.72]) by kanga.kvack.org (Postfix) with ESMTP id BE5D58D0001 for ; Fri, 3 Sep 2021 03:17:15 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 807442B36D for ; Fri, 3 Sep 2021 07:17:15 +0000 (UTC) X-FDA: 78545406030.05.5B2AEB3 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1anam02on2107.outbound.protection.outlook.com [40.107.96.107]) by imf22.hostedemail.com (Postfix) with ESMTP id EE9921907 for ; Fri, 3 Sep 2021 07:17:14 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dYYOTBdtc28dWSO0N8nvojgFomxFVj0Vpb9MBiXRhG5k8XKrfMwr8YcN//OnNysrNah8eLfiuqU2OyIMhqOS5otyfIsOZKAi8wPRfqmKM8DloMLZWXV9D3uWro1gYap4C8xoR5JrrM4VZTU5zZ/Msv+hohT1WxKfzViJfnPpQ9f/E+voMpIqzlZr4+8maVG9s5dkXYgmPXGKF/9Aa2ODR1DouMYA5LpoZvyoKTc6//G5Lie878/O382x2dOal6Q5VIAxWuHjDXM3pcIoR+NESJH/s7emfPgux+0Oh4turYmxCpcuxlX5FQt/d0JVtktBhlbJPu9eujUTTMAC0SL/3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=8LxkugM4gauNDtZRyfYPUEi/OCLLf5jw823zfLf2soo=; b=gtXOkMyPTNUII/PkNY9EUYkbv+tDzbtzZG2IT8GL3halY6VS7VTHbInAT9PuY8NsbMCRJPj3/ojKGpB7bzWYP5pd0Cme8j55bHHB/E99iGz3povePhe96GNKneJk0JoXO9sgTJ3NA38rYtzvgUQFzGgJkMisvds2tnWERipXxqxV/vQJWA76SY7iwllJmIyuSyAoUbFakCc4O3eF5z+ZibVn/tjnnZvyLo7OMVB7zmpqSlX5eTLKDdsMfwZ91NifEuqLHg8Ze9CXyOBJ8am6vbsfUIKk9SdsGg5DrftMH4lQHcyl6AkQblRHF1VsvE0XRKfW5knnY76wXXIM6giUdA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8LxkugM4gauNDtZRyfYPUEi/OCLLf5jw823zfLf2soo=; b=VuCK2hMIo/+p9RJIlqK4+ZzQQOIM8v4F+VF+LpjjGBcinElShj4rCukidkt+L57HoZBYeyhKbmGnwjZdoXs8WNOFjBS7S02uZHtakEm+SG1g4rbfDdo6mQ/m5I3Mjwuwg2L3buyE1XsbBEsOK5jEt1NKRDTTB0+9olp66tyAS+k= Received: from MWHPR0101MB3165.prod.exchangelabs.com (2603:10b6:301:2f::19) by CO1PR01MB6776.prod.exchangelabs.com (2603:10b6:303:f4::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.21; Fri, 3 Sep 2021 07:17:08 +0000 Received: from MWHPR0101MB3165.prod.exchangelabs.com ([fe80::ed89:1b21:10f4:ed56]) by MWHPR0101MB3165.prod.exchangelabs.com ([fe80::ed89:1b21:10f4:ed56%3]) with mapi id 15.20.4478.022; Fri, 3 Sep 2021 07:17:08 +0000 Date: Fri, 3 Sep 2021 15:15:35 +0000 From: Huang Shijie To: Florian Weimer Cc: Huang Shijie via Libc-alpha , carlos@systemhalted.org, zwang@amperecomputing.com, patches@amperecomputing.com, akpm@linux-foundation.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, willy@infradead.org, song.bao.hua@hisilicon.com, linux-mm@kvack.org Subject: Re: [PATCH] Add LD_NUMA_REPLICATION for glibc Message-ID: References: <20210903121434.12162-1-shijie@os.amperecomputing.com> <87bl5a6ul2.fsf@oldenburg.str.redhat.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87bl5a6ul2.fsf@oldenburg.str.redhat.com> X-ClientProxiedBy: CH2PR04CA0015.namprd04.prod.outlook.com (2603:10b6:610:52::25) To MWHPR0101MB3165.prod.exchangelabs.com (2603:10b6:301:2f::19) MIME-Version: 1.0 Received: from hsj (180.167.209.74) by CH2PR04CA0015.namprd04.prod.outlook.com (2603:10b6:610:52::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.19 via Frontend Transport; Fri, 3 Sep 2021 07:17:04 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0d622bdb-822e-4f0e-b52e-08d96eaad71b X-MS-TrafficTypeDiagnostic: CO1PR01MB6776: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yxmL0ByRCvZQOSg27lTSMNHdSLO+VlbX2ZGD3f/vyUJopDUoJ2W6O1QpzxNSEgZQWV/z7uGdd8v2NojlCtVwXl5vvv6DuME/Pj8y/waeKxNu387Xeg+aha0w8M1jjAj9U83Dw8XIopOJ2bHxEQDFkXwHvc1Qg3cXmEQN7B0zgvpv3mEB64CoqXtjMnOsjP0pvscbEGfY1FoaJMheAHi63H2svyFoaLOgecn76964CbTsAtCcStGJm/Xnop2xXbPfcTWoQpU3G7ewF+605Ab13ZUA2YuJTfpz75bHZE1NeM4VZ/gDtjrXU1ZciHhlcqjVx0ka1w/IGtlK9IBwrKQYpLEKeszCA0HHkJ5m9FQRnt74EEYCjBHQQu48C7tF5yq55iya5m2f/WrHj8tvHti3D7droscIBVpAJ+x+TM3NiS3wNtUXN6JI8JXqGHLQ02JAd4AQM85Ik8tIL3Co4X1QEARHt+eS8T1nsKKivj00xukvB9QFAm86ka+IWctQGR2fLcRu1mveQVAqpkPWukztG4erBlfZFDR9UaFjHBVnzu7E3f/UJ5FGIPE2NNgUzld+k814AfyDBbK2PTpvqjXYE2gLSOuTAG9iHzgmajJTry6Q3CSDZ6RMILf9xbkdMFJfviUP68yN2maeS64AId5RvAkZUEDn+EmTdySoMiUcTBVatG5LbmVnhW7p8ic7oezng6mOlgSAlj4k0T1vwPfmYUynnpGcF+eYd30x3zHsYLE= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR0101MB3165.prod.exchangelabs.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(2906002)(33716001)(6916009)(66946007)(86362001)(66476007)(66556008)(6666004)(9686003)(316002)(8936002)(956004)(55016002)(8676002)(38350700002)(5660300002)(6496006)(508600001)(9576002)(52116002)(38100700002)(26005)(186003)(4326008)(83380400001)(67856001);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?2Wv3MdhC3x7gvf4+tPla1MU60yZhMUOM/RipcV0hG7mQptABei/RCgxevRm5?= =?us-ascii?Q?midSPEh+ZDK9YOKQoLi1fU0gYXasrHkeO3Z6/UroM8UbAPUz3VE4/56WuYaq?= =?us-ascii?Q?bfFxK6KscbjRDKpQSNJNvqvWvUku8bnDDTXCmYK66nibe6Sh6UBwuZAkzm1O?= =?us-ascii?Q?5APdnZtZ7RBcAoxPLZ58edJDsscst1pyQvzVXxLLCep5ZSTS6TKWGNCqJXFK?= =?us-ascii?Q?MOvv0+gVeogz5z88w+3N/2dOEs4Q+kbsjWtScnOwBZ070xa611yIVKl2799W?= =?us-ascii?Q?FtBboS6o6khs8eieSViO7rSuYOgqKu7BfxivhlkWOMoJ3ilVPsg7DiyDriMd?= =?us-ascii?Q?CyuHwq5cMcMO48EGy8mSbj8V0mpTZUer1mCEWSdUzA2TXQ1ClWDR8Hln7Hc7?= =?us-ascii?Q?NwxXNBQA3RLpjYi6i16FvY05/wpVqQz7cBnK/dPQhXAjhWZMgO4kSU13Gl9R?= =?us-ascii?Q?G8YwXLQyHpIaJCETGIhSUg/rQBMNbDvFoTRcTHbnP6sR6exkEvelu7cDnfJA?= =?us-ascii?Q?yq9L0ZemerQTHm0Ix1COWs69N8+tFyNQ9hGpR0sl68iD2VaaG7iSPCUDeblY?= =?us-ascii?Q?7WTNggbcThHLOjTOA8OR4AMg5EN4Oq224U7AQ7qgbpxixAn9RlPHV4Rz/dQL?= =?us-ascii?Q?o0eQrRQ/oj9deJNXeULBFhcuhFvJ/r0jWJbTwfZBcl+hb/fYLJUMiMdCUeK9?= =?us-ascii?Q?0TwuG1aHasdd8mT3Cs8taHxEKgYcceFGONsd8lY7vZQWqSQtMz2NSQq3GnSt?= =?us-ascii?Q?jVrGH0XvV7kdfj9BLBPeBiHCYQwsocOaSQkZtBK+YI+ZRC1UZ4pQ44Atv3ah?= =?us-ascii?Q?ttUHdAgt2Bui/3F/yxUyxckBng2DlBKeFWkav4CWiYEVKLuG5BWiDmFK+hFH?= =?us-ascii?Q?0qmXd/IW4VI638zcXqE+/7xc/FDL9D5M+Hh4Cs7vcOTT6GkESuPHTqH6BWrC?= =?us-ascii?Q?4Miq9ripxKCPVSNHFioP4uH50NnF9jTiiXWoFG/8p9fBewILV2DXYrWLoWyK?= =?us-ascii?Q?nLoiPJCut3VVY28pvYy9ZGdQEnXgiUyofnuqtooy0bZ9I5J2r4mz2izv0nhN?= =?us-ascii?Q?9GqJInB2tyDDvyobdG7cl6ULOc6li97pAL2HUqbVJdtFmqmpMhXNZD8cZTzw?= =?us-ascii?Q?EHv/Ln+cXeRVLCqGlhisGepgup2xnuTRWIrqSlUdsaEvlMzDAuwJGTEMcqJ4?= =?us-ascii?Q?Wh4WBjY0MMKLOF6ahpsveT0wqWD+34d8n+6c71CxsiID1DOIGJof+51h4jqL?= =?us-ascii?Q?NXTiEI02uWeE63m/eywPlycrc4CtN6e3Nw03kEVwjrleK1sXkXotx8UnlmcD?= =?us-ascii?Q?bcSIK/VXxJ07PsIkFM3BCXFD?= X-OriginatorOrg: os.amperecomputing.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0d622bdb-822e-4f0e-b52e-08d96eaad71b X-MS-Exchange-CrossTenant-AuthSource: MWHPR0101MB3165.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Sep 2021 07:17:08.6486 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: LeEM6sfdDbOCx+frhMNtkvJtEj73Dgr134QeCGEiiQAel95eLAHIpZ3EMw76dem1QF8SBJRQPwM9oyvwSHRDM9d5dg1GtqRTymLRm/rxCiBlHv0wqPf5KhokEFtkZ4vC X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR01MB6776 Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=os.amperecomputing.com header.s=selector2 header.b=VuCK2hMI; spf=pass (imf22.hostedemail.com: domain of Shijie@os.amperecomputing.com designates 40.107.96.107 as permitted sender) smtp.mailfrom=Shijie@os.amperecomputing.com; dmarc=pass (policy=quarantine) header.from=amperecomputing.com X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: EE9921907 X-Stat-Signature: bgtz1rdh3ecnc6qrk7zag63w9cmkif89 X-HE-Tag: 1630653434-287247 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Sep 03, 2021 at 08:28:57AM +0200, Florian Weimer wrote: > * Huang Shijie via Libc-alpha: > > > This patch adds LD_NUMA_REPLICATION which influences the linkage of shared libraries at run time. > > > > If LD_NUMA_REPLICATION is set for program foo like this: > > #LD_NUMA_REPLICATION=1 ./foo > > > > At the time ld.so mmaps the shared libraries, it will uses > > mmap(, c->prot | PROT_WRITE, MAP_COPY | MAP_FILE | MAP_POPULATE,) > > for them, and the mmap will trigger COW(copy on write) for the shared > > libraries at the NUMA node which the program `foo` runs. After the > > COW, the foo will have a copy of the shared library segment(mmap > > covered) which belong to the same NUMA node. > > > > So when enable LD_NUMA_REPLICATION, it will consume more memory, > > but it will reduce the remote-access in NUMA. > > I think the kernel could do this in a much better way, avoiding > duplicating the pages within the same NUMA node. I think maybe only the per-NUMA-node page cache can avoid it.. > > The other issue is the temporary RWX mapping, which does not > interoperate well with some security hardening features. > Thanks for the feedback. I CC more people who may have interesting about this. Thanks Huang Shijie