From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8436C4332F for ; Fri, 10 Nov 2023 22:29:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 06BEC280009; Fri, 10 Nov 2023 17:29:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 01BEA8D0005; Fri, 10 Nov 2023 17:29:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD77E280009; Fri, 10 Nov 2023 17:29:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id C91838D0005 for ; Fri, 10 Nov 2023 17:29:42 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 998491402B9 for ; Fri, 10 Nov 2023 22:29:42 +0000 (UTC) X-FDA: 81443487804.13.EC2110A Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by imf24.hostedemail.com (Postfix) with ESMTP id C9229180004 for ; Fri, 10 Nov 2023 22:29:39 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=memverge.com header.s=selector2 header.b=Dm+U0w3u; dmarc=pass (policy=none) header.from=memverge.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf24.hostedemail.com: domain of gregory.price@memverge.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=gregory.price@memverge.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699655380; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CJ6HzzRCDC4mF5y3X9giyKkmGrV+CkPT9s51r/kBVE4=; b=uTXhITvJ5uA5wO1gkivpyEJjJ4RjMFB+t2KBucEs4f5nCPnqvfyYKz2GqmlGeb+VUcwywV MHeCroWt2KPsO/wooyNjuA+O6evRT5nPD3CJ54CHCjOEQ5TUGErV/0uPOMZT9R81DRAcJN rj6Q0m8XRodRYAdKhRpIPmcbzegmG18= ARC-Authentication-Results: i=2; imf24.hostedemail.com; dkim=pass header.d=memverge.com header.s=selector2 header.b=Dm+U0w3u; dmarc=pass (policy=none) header.from=memverge.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf24.hostedemail.com: domain of gregory.price@memverge.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=gregory.price@memverge.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1699655380; a=rsa-sha256; cv=pass; b=I3dbBQRsX7c5uB9bYAZpcl0DvFALHBL/Tq89PAtPHJcTCWfGotIDpntccQNZbSqt5es64M U1L/G4aiMBfMCXK6q6swFhw9YvSEDHg+0krihEpjNXn2FeLO5JTpm5ZEV1POCilOKmz2zM emv0ryfW+34tTUt8s5dyIwWbhY2kRjo= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GQjQM8qwyEroilZhZwnspwXRlZQDnf/kpJ+GjV/1DJykTV8dD5WpBdoM3MCUgZsX9WVwtIqipauR9AOXmOK0G9Mj0KgjyyVdO1cf0aXPIcGLsb0KSVXKfC8jNyaVB7gXjldiUrLCpLnwg2WUT4ioe2fwpgbpmUNogDrrJ+pkg22bEL+qZxWIzEws9S+BmxBVWM36AxR2q+41Ha50C1+Fky4+maUitoJp1MEgY/3J8mBcN68rkg/l6VAt3f5YTCdv9qQIss0OaRQQFV3UJGxS/g90c0X+6Aw9HN0ogD9oTDfVhhPx8p65eY6H7iNfM24SWvIdPUsAS/RrarCKcarv2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CJ6HzzRCDC4mF5y3X9giyKkmGrV+CkPT9s51r/kBVE4=; b=HA6wgp+R2kQqrEe15MOaOXrtUfNZrJnjqItwST5BDS6Igt63lfevo9B32KTf/4csb8k78KbSImg66KogMTA6FLe82vuhV+6j4NzeguANDjUlnC9c5hix8bhh98H8D6MzztcPLor+DWPz1v1Nel5IKzf6SpT24/AByYowWsk3tcbo3IesRT4sOm2THjDuELZ+Fu1k3wnRYAhn1FVaU4VAJBPMmdUYHSGd0bRhtleAeZNsF+mVfEpQQmP9q4RuQZHJ62prQRarzb8bNx7p/LU+6DMDNxivPsODW4y7VD+mD+UgwpBVIyyeq6Fe57HlABK6T3jJmEnrHvOs8ETNThYxMg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=memverge.com; dmarc=pass action=none header.from=memverge.com; dkim=pass header.d=memverge.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=memverge.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CJ6HzzRCDC4mF5y3X9giyKkmGrV+CkPT9s51r/kBVE4=; b=Dm+U0w3ubVPlhfMP9gULdzMvCMigi0Wj7V9dMsfXMqJBkxeBRNGg8/kHSq0H7de8yjiwQqSAbxcpW24NIOeyGpPQRmHD+p3ESLZFKUrW+7klJDInyz72o5nDrDakdvD4v4rzxzfMMS2svqZwMO/XnX2kvp3R5SxOQGgarOjpyd8= Received: from SJ0PR17MB5512.namprd17.prod.outlook.com (2603:10b6:a03:394::19) by CH3PR17MB6714.namprd17.prod.outlook.com (2603:10b6:610:132::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7002.9; Fri, 10 Nov 2023 22:29:34 +0000 Received: from SJ0PR17MB5512.namprd17.prod.outlook.com ([fe80::381c:7f11:1028:15f4]) by SJ0PR17MB5512.namprd17.prod.outlook.com ([fe80::381c:7f11:1028:15f4%5]) with mapi id 15.20.7002.010; Fri, 10 Nov 2023 22:29:33 +0000 Date: Fri, 10 Nov 2023 17:29:25 -0500 From: Gregory Price To: "tj@kernel.org" Cc: John Groves , Gregory Price , "linux-kernel@vger.kernel.org" , "linux-cxl@vger.kernel.org" , "linux-mm@kvack.org" , "cgroups@vger.kernel.org" , "linux-doc@vger.kernel.org" , "ying.huang@intel.com" , "akpm@linux-foundation.org" , "mhocko@kernel.org" , "lizefan.x@bytedance.com" , "hannes@cmpxchg.org" , "corbet@lwn.net" , "roman.gushchin@linux.dev" , "shakeelb@google.com" , "muchun.song@linux.dev" , "jgroves@micron.com" Subject: Re: [RFC PATCH v4 0/3] memcg weighted interleave mempolicy control Message-ID: References: <20231109002517.106829-1-gregory.price@memverge.com> <0100018bb64636ef-9daaf0c0-813c-4209-94e4-96ba6854f554-000000@email.amazonses.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: SJ0PR03CA0070.namprd03.prod.outlook.com (2603:10b6:a03:331::15) To SJ0PR17MB5512.namprd17.prod.outlook.com (2603:10b6:a03:394::19) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR17MB5512:EE_|CH3PR17MB6714:EE_ X-MS-Office365-Filtering-Correlation-Id: 13b215eb-5256-48b2-1a97-08dbe23c8382 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Jy4zek7vDboqlotz2wtfAa2akNGWkku3IEDq1zI1DpMN8c5Uf7iGZ78tfVCmpRlfvwghZCHtqHZWmJs3KTazOV7SX56EKJFlvPyF0lyG1Z4GJxy89lQWZidYcUnHuWJmHkl9VV/2ZaqYpGQTDDo8aiPL59EEnBK69nYKPQxfZlbOATUaxJKhKs46963UlD6aifwNZOeBjVvJBTGcx6SIA+FLthx0BI2shFLWTwb2vmh9OiA6HKO1O49LNMEv9Ke9z9O5pNt2MqeU6N59BjU+o3PWDfQHwzImx9O2OwOZ8S+juzzNuA0OoMB/Ot6bJn+ONQWH255MHPiZV7Y85QAuzm2BUVFbR55FR2FEEyCXLsTRmb9X5lTs1RzNHkh8rmt6EVkZpQeXymuRK860B668LzE5khBYQjiw/MvQ1l9PG34XRv+uER4x0T03+VM7hxFFniProFBv0HvtAbEmyRAZCOBUtHa5Q/hH6Z6IU5DBvBM3JauHyhMNlu241+1h6ZgZhicHw63J8VWGXCOVnc/aqhlf88edKqtoSZXFmATEnJJQZrf/R3Ed0kAL8BfhFUrWQ8kUcxnfLjiE8N8WAPynjPAyCmMWoYiGPVG9jQbnlzU= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR17MB5512.namprd17.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(346002)(39840400004)(366004)(376002)(136003)(396003)(230922051799003)(64100799003)(1800799009)(186009)(451199024)(2616005)(26005)(5660300002)(83380400001)(7416002)(6512007)(86362001)(36756003)(2906002)(38100700002)(8676002)(316002)(66946007)(54906003)(66476007)(6666004)(66556008)(6916009)(41300700001)(4326008)(6506007)(44832011)(8936002)(478600001)(6486002)(966005)(16393002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?KhD1W7PcrTxKpYP67+AcLFC7Qoy0b+Djsapn3eOvvRO4SKSCPC0NC8O08NEo?= =?us-ascii?Q?O79j1a3pevwx0Hvkv//bY/lZ6zZ0sWdSl8n/DMO+owIuJyncnzWC58tb+cvD?= =?us-ascii?Q?Nhbu4NtO2bX0FaScgLnFgu874Y5qSV5QVwRcT+hJKecdq7hUiEHmKf/eRAT5?= =?us-ascii?Q?b5RPxHF4Xwb30asp2bqbIfJIotAhSIdVXyMtdvZ5E8+dPZI8H9XJNaLST+3j?= =?us-ascii?Q?EGggoHLaIEM2U5duLCVZUlg9sNEPvx0PtPif2ZeSqXYCWcVH9ZtSfY/Dk0ox?= =?us-ascii?Q?bftgFNerq/sbRpIxU8Ecd44ZMN2VBJGGj3rHSu69Zo0tNKgn6zQ7SYtIcCdW?= =?us-ascii?Q?dFBjBLZ2q4oxl2ROxU0p/nednahig9JxP/3wzROuzMVsNRshe11XWigRMN31?= =?us-ascii?Q?yAH4UP+UrGq+9BO0epUztw7fOCECB+OyJbvznbli48WyYl/+bEgoVdWMAm+U?= =?us-ascii?Q?bCvJgjczPz8nKsTU8iEpP1iYA9K1bWzCcgb8vbeUzDJFptR4KV1p+HJgHbHf?= =?us-ascii?Q?dy9BBPqOZQNc7wQGO7XnUhM1ehcRfK82zqitZrz/l1G4qF/NXVvomykm13dA?= =?us-ascii?Q?7IH1jhFYljPU3sQtCU1W0bdOSTVMOLnW55StOcwORtLaaJYm9oOa0rg87Pzp?= =?us-ascii?Q?4cyT32MTI/B18r7dM+kDaQx9jIZ36r/57Wek0RWJgwaJF9SUHxS4v37U+bTU?= =?us-ascii?Q?qs+i7H4zHiL6h6KGSjd3wx90AHxiHQP0SI34Y17uHbikDbgY3nu6iDtOG7zT?= =?us-ascii?Q?BamUeQg/Ap3YdQeFa+TN3ThP/LGVY+c4k4Fvsu5vystEVQKUfigHnHN/iVI+?= =?us-ascii?Q?K0TFPceC6Ov1vHSqwnzg2AO+LaV/p9KAfiljI7TdOdb4KKW26hAmeA3GcRyA?= =?us-ascii?Q?Fg9cPjiOt1zgxow8iXpJ1I8LrPCIaQZUwigrmjKIPg1aypRXq5a1YWDGBZd+?= =?us-ascii?Q?qdEkwvaAIzTnhOA+8wjLPVRc9zly1SW+0Vqy+veREM78N/nxg89i5lHQqj2B?= =?us-ascii?Q?o7KyHMKGXkICY/td0tQzXPFmANOoZFSXuzZSX6Id3IM2wEDywRWKIkB67A+Y?= =?us-ascii?Q?xnMJ3sPb57+mtZ0turVZ4wtYjwWu8lDvwQ6xRQ7wVLJvo0LVMsJ9fEBpy5ls?= =?us-ascii?Q?UNKfdmXs2Wn+Ks9UFR4ORAQjDph0qhYdBG7R1omMcE5odnV9df3ND5CH/rgh?= =?us-ascii?Q?kpDRllsz0UgG4b7NWArBVSm1Wrj8N5TKOsbE5X93dCSQAEfcjjXceXDy7zOX?= =?us-ascii?Q?Sb0Un6mOxqTLKCf6oNpQIfnRB6YQXYrtzchrWNHfOdcg7fMwEQmT+OktCZry?= =?us-ascii?Q?4ofY3ai7Z1KyJvleX96O1g0h/gyFvyIISPg6p8Ree833qfAVpeF4N5+cmC2w?= =?us-ascii?Q?ovFdsu0ieNhBXgS8nC8JreLw0q65htXMdnjEYGN3absxK0KUYdjtzJSuwIi9?= =?us-ascii?Q?JYSWilW8FzRo1hZlR9lHkTZm5WfzYyuEdIQV56QKflqNKBzbp2UJox0Xypwt?= =?us-ascii?Q?CVOanyXQVBL1B8uK9pHvMhbVHCSfJ5xeKbwKrjvyb03CyUJmKHBy8ZBKZf+b?= =?us-ascii?Q?/nuqf6gIwMPmB66AvzZlgqBGmaBWJwXdDa27k8Fxhptotc+sv8VtR9ycYiv3?= =?us-ascii?Q?Uw=3D=3D?= X-OriginatorOrg: memverge.com X-MS-Exchange-CrossTenant-Network-Message-Id: 13b215eb-5256-48b2-1a97-08dbe23c8382 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR17MB5512.namprd17.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2023 22:29:33.7238 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5c90cb59-37e7-4c81-9c07-00473d5fb682 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1LFtGjgZNYW39WiMDTibchSGHW9I/guWfXM2FyIIVz3nXZNGLPXdxXktnN328WMKQ2FMLINfvKIc3JIY1WVljPsjDBr1wL1AvUSO7+KkvI0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR17MB6714 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: C9229180004 X-Stat-Signature: r7rzx7tbhe6zbfhpgfawbnojts7kdqtp X-HE-Tag: 1699655379-339256 X-HE-Meta: U2FsdGVkX1+KC+wIq++pRveMcx1hfYIdG2ftCEkQP9JnkpcDy4lxjf+/3WjnUlKlrwLAn0qqifH8xppwHNEU7OvXNW2Obas90KdfRqknyh2V3MMYVe8kWjJFf5/Yw+9UFrGP2LYFQ7VeH/gTP67EEEbWtVgyfFdccEAzOxsScXR67WMrMc4dPAylEkgbLpYOfzvabm6YjfW/BX/1y9X0fxTWyq1rRuOuc79i75rockKJiojr3oRQT/VhK71YXN1QYa74EXfyEmO0eoYBtH3cFMj4DIs4ChepHgjR6aRga8QtyW9Hy+wXKdmpBqMEaNuvbSpLcMfebuLgKteTY9890KDdQv5CBA7HyOmlQOnVspIAJOMOF8bLuizi5LDtxQLDbAS85kEIwfFg9zkfBAgKmeNeoHCJi4ZzkJ4QBKSh0lTaGBEM0gBFOEJ8aeOJpEFD45xT2mlYD2urjY+WNcx7Rd1FJUXuLlRUb84PCvIU8BR25MlHAyEbVLIYE4BkPUUSOJ7Db0VVgzfveRT/57uBIp1ekjatU3XWWC0S6dQxjur0PNGpSv8zDxYS6cBhT8+ry2q1gmq/0/UFQ0d+dXzJ1T2+wKUzslFMNVvYV9+HC+//oIZ+tJ1wrkGou+O3LMZkFAzn3Hqn/V86bVf0nMdKuTCVrBjqiq3GgSY0sUXOZVrZOfUPyACfKTiO0l3+tvdCO50rnOKIQgszm2gUCl598Ye0SBcmOkoMz4wDasES8AEoXuek7lHyj87BAn6pB0AB1aUsDVFvCD6FmxZxmJrQOeToOYB+uFNFGMLC1UWE3WJm/9kWbfd024AUOXSKJZQ03sTDbyhPdy6jP5Hx66Yd46FATQfEMitY5tD8Rqj3R2XIKJ4r++UqdbHPCGShBdOXrzujiShELzYnaCtz3fNzZ/C1UsPvTVJF2cZVT0kGWkAeqJPTkl962XoL4XiAoUM0p9q9X0VYhkOMPql2431 rT8c5XRQ qPxEKPnHhOetV4hXcob/MJkIsF/InzuTXL1bPEuBCCz6miir7vHVAxyBqyucPvB9nGyDVERXwx1J57IJ44AR2sd7lhWWOLXq1iN+c8k+DSq6Nfjr7upXI9DbeTtlPILmjf7lTYLD1JEACRAYO3XpWz7Fu4DJ13na5WW8oSBLb5vnNlTAKwWs4BA4bOlrjuNbKfHmpQtcm9VrDcn68eTVP6saCtJEqGXoyRe5GLBVuojqxluOHyOLA9IZ09P7hSfSXxq8LsszLkeoL5mkEw2WbWcS0GqdKQKcDiQQ94/79gSkonSDXPcCfg3HnMAJ66zoqULSDc7FUp7B8QVbp8CaoVQKpnH6qNZqCs3Oew7BdPurgrs9WVoEd2hKyRt/n2/wmT8EtN44QNDEUJKIunZTIC2gbCTyemGpxbUT/m/d5vOiHUaHMWQ0khA5jC0SWz2oTq8ao X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Nov 10, 2023 at 12:05:59PM -1000, tj@kernel.org wrote: > Hello, > > On Thu, Nov 09, 2023 at 10:48:56PM +0000, John Groves wrote: > > This approach checks all the important boxes: it only applies to apps where > > it's enabled, the weighting can vary from one app to another, the > > kernel is not affected, and the numa topology is not buried. > > Can't it be a mempol property which is inherited by child processes? Then > all you'll need is e.g. adding systemd support to configure this at service > unit level. I'm having a bit of hard time seeing why this needs to be a > cgroup feature when it doesn't involve dynamic resource accounting / > enforcement at all. > > Thanks. > > -- > tejun I did originally implement it this way, but note that it will either require some creative extension of set_mempolicy or even set_mempolicy2 as proposed here: https://lore.kernel.org/all/20231003002156.740595-1-gregory.price@memverge.com/ One of the problems to consider is task migration. If a task is migrated from one socket to another, for example by being moved to a new cgroup with a different cpuset - the weights might be completely nonsensical for the new allowed topology. Unfortunately mpol has no way of being changed from outside the task itself once it's applied, other than changing its nodemasks via cpusets. So one concrete use case: kubernetes might like change cpusets or move tasks from one cgroup to another, or a vm might be migrated from one set of nodes to enother (technically not mutually exclusive here). Some memory policy settings (like weights) may no longer apply when this happens, so it would be preferable to have a way to change them. ~Gregory