From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0D68C43334 for ; Fri, 15 Jul 2022 05:57:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BC6EB9401C9; Fri, 15 Jul 2022 01:57:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B77B19401A5; Fri, 15 Jul 2022 01:57:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A17C79401C9; Fri, 15 Jul 2022 01:57:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 926289401A5 for ; Fri, 15 Jul 2022 01:57:08 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 5B6942049C for ; Fri, 15 Jul 2022 05:57:08 +0000 (UTC) X-FDA: 79688276136.21.FD10CF2 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2052.outbound.protection.outlook.com [40.107.92.52]) by imf30.hostedemail.com (Postfix) with ESMTP id CAFDB80085 for ; Fri, 15 Jul 2022 05:57:07 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DW78V/HQv0TL55rprt08eOyg8vcxp74YngSRMd8CxtMhIfHmsJBBA2cblcTeubcfF7OiczPnP62hPmqza95UzXpSYnedzs5iJKMuUAKbxhX6M9irnJTXSywX8VesFU47m4kZkufMxItw8KPGai1DmMgTYAEFXI2jQbsKxNvVUJ6PEnceN7KRCwKV6+zsnYa3vp/NC9zZKqxGaijo6CXZQBMbXeS39kZO3w3i/b9EM06rtEEOJw8bU1xbKb655AjX0dAGHqROBdujDb8n3ZFPwt155hvO4bd7u0Bc3pwqp8nFSbX4TAOwVJzt8f2MP5GDcS9kX2+EI8oiBIptoVdLjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=a696pMQYvRSUk4fO5hzX2hjnPUvLB1n+mRt8BPdfZZg=; b=A+daJ/2tQEVjNVJDvCDivYtFXHimWgcKxiG7i0JdU6KZ5TXOlwHMa/vZPm31QgLDNSXIQ4OxFRTawvbpFCP9bsCsZWl52jySvvI8xIMT9mjDCP6BER++gnnADXHelp7DoCJwkI8mZgvGxk2rURKOMshVe8kvFy24PYki81Ygw97dhWwmImBgxIvgPQl7MI2NUoptUy/edKe+4Ry2f1lKyQkMEVL51ipXASEKTwgmZcc70iWwKF3RhFYMtStM2L1ppJGJN0sxjA084Z2SgZsRvksAhZGN8ALxtMsSQ5YUruvmnf42Fcx9NeCDeexD/7Gw9OyR9XXuEHTktICiwxuMXw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=a696pMQYvRSUk4fO5hzX2hjnPUvLB1n+mRt8BPdfZZg=; b=oz3lbvVMFwB1ceDXqH9ZSi1yOV99LL8d02doQRxjjEqCzpCnt7LmZNCVMnK02Tk2k+bqnTFEHObWhVTYLs4lqZVwf/Dbmt6QS2PjGL8WghiyYdDzQ9eMPf/gB8fQ0Sd9uAE3Yc6i7yMLUXJlpB1aImb4bkOAbluQTCJ6djstSV/1UvfnfVuE0Lu2UHM3KkAYTcXQ7Ctt84GIBNaVaCyEJe/5lKNbJWRh+XrPDZMN2y/5r/oWOw8EiH/CwukrSwvi/iJfZfO9YJYd+Yj7KOW2va/vXHJ7L1GEjuN5mzpYiFcMT5K37oaXy4v4cjmaJXTzWOPQKUiQ9f7wWwDRaHixag== Received: from BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) by DM4PR12MB5167.namprd12.prod.outlook.com (2603:10b6:5:396::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5438.13; Fri, 15 Jul 2022 05:57:05 +0000 Received: from BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::eca6:a4a7:e2b2:27e7]) by BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::eca6:a4a7:e2b2:27e7%5]) with mapi id 15.20.5417.020; Fri, 15 Jul 2022 05:57:05 +0000 References: <20220714045351.434957-1-aneesh.kumar@linux.ibm.com> <20220714045351.434957-5-aneesh.kumar@linux.ibm.com> User-agent: mu4e 1.6.9; emacs 27.1 From: Alistair Popple To: "Aneesh Kumar K.V" Cc: linux-mm@kvack.org, akpm@linux-foundation.org, Wei Xu , Huang Ying , Yang Shi , Davidlohr Bueso , Tim C Chen , Michal Hocko , Linux Kernel Mailing List , Hesham Almatary , Dave Hansen , Jonathan Cameron , Dan Williams , Johannes Weiner , jvgediya.oss@gmail.com Subject: Re: [PATCH v9 4/8] mm/demotion: Add hotplug callbacks to handle new numa node onlined Date: Fri, 15 Jul 2022 14:38:09 +1000 In-reply-to: <20220714045351.434957-5-aneesh.kumar@linux.ibm.com> Message-ID: <87h73iapg1.fsf@nvdebian.thelocal> Content-Type: text/plain X-ClientProxiedBy: SJ0P220CA0013.NAMP220.PROD.OUTLOOK.COM (2603:10b6:a03:41b::20) To BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 64ef9eac-e6bc-4eec-bc41-08da6626d8a5 X-MS-TrafficTypeDiagnostic: DM4PR12MB5167:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: jY8n3eWGEQuObsWEzYNWudHbP1PDe83zVYX/Dm8tqZkwKZk8H8NfuRdz6hnPpaYwyikpaQ7yHmAEnEBakZpN7j7lVwhc9Yt8QYW2Y1JEfqhvK/VkCQXpkBRaEh7+jSXhAzYPucQVQtqMs0Z2sgbHya12H2dJailM9E7PZc7EUjuID+jkGJdGnyNG73Y8NOs7lhrlqnqphiId/Ibw0ZqoWNl769QO0g5KpEzYUqn1OD99jAvrZdPyK7RVAeNtJkDvqShlzDHWcBUjKHlwiDcU12I542jP/E2nE8PGQHY8okKgkiPWsMRNZav9hABvMziqSTFNRhSnQKZ5/NMZHGSesga6tQ4UKa9g7CvI4mItPf1bYZG4BVs2Ckcl2QcvbetbfmDjSjP+DksEHN3zLyJJlcwsSD0y4RtreQiitYolaLWj/dn83au8qpCfr6MK2TZbhr5qgkxVHIu4V/DRFykmLjJDE+MW8NgXUznF9ohuVIVyiqcBdbH1MyPe7/hidP5UeNdHKUqFwnENvtGr4E2P9znhbtNRY/HdTgr6OiBEjNvhlNzbMZ0H1A9y2wZ3cZkzPOVs3xZI8Lis6tWaGwgFDz8a1HQfk+u9CdbBbRKNF/l91y8GOAOgoIg8T99vO9gH5ox990CVMox19H69ko+dqpiUvLbrr0qHzBFR5W8zna+COBN6JD+h4Uyqbp0/FJg5NhWdC5MB2hREj78DnrTvu0u/eEaKTBwWV7H6vgewfVIPFtqfypD1pdW3LU2fjfBb X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR12MB3176.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(136003)(39860400002)(346002)(366004)(376002)(396003)(66476007)(4326008)(6512007)(9686003)(6486002)(83380400001)(26005)(5660300002)(38100700002)(2906002)(186003)(86362001)(66946007)(8936002)(8676002)(66556008)(41300700001)(6506007)(478600001)(7416002)(54906003)(6916009)(316002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?nPbou2kX3IxRq2tVOOY3dOVd5QNzL7B+XnrOdcsxLd9f9TIDdvXvwJYp3MV8?= =?us-ascii?Q?65mufVElSE7IJcwlCy9lvS6Si+OuD3SleS7bOofoWgkcFtVQUJ9zpLHu3XlP?= =?us-ascii?Q?1gHIry8Z/O61l+kyHRwRnGx15BzfNPa/Npm8rJtYBcwEjNVi/pzGB/F4nsn5?= =?us-ascii?Q?t0EM43y45IuLHjHfDEP/LWLodb/442Xo+6KG0yV5Y8HTOG/YzVPbUozkJHfm?= =?us-ascii?Q?ENc+9qOSKoj5hydpdjqRrKCVTIYTE2q6ykejLv/rAf5CSf5eSjGpdiyJtTyq?= =?us-ascii?Q?Ul5y/sFXCjWQOxRV6BFkGJbzAckihHT1cz9DPLQ1NGDEIeOsB+UQLSDXL0+b?= =?us-ascii?Q?y4kV09Hr5vY2xb3cf3M6GYDP4GEWwsc0Rva7ymAa8XoSiedv+Nw32kyk/764?= =?us-ascii?Q?E/6ljn1tC929CRZE+KZfhfzFfde5fhIg+Ikfpp6JfoDQu2go+S0HoCnn9sYR?= =?us-ascii?Q?1XVjs3DGPvLYCyhBCE17rP/tt73mawY6kzSVu0n3nqGiSLJNlVBM8udkrxZf?= =?us-ascii?Q?Sim/rdlJxD6ZAkRjaSYxScw/EUSFjR9b9nU4UC3SWwV8p+yC1efFPvuULW0T?= =?us-ascii?Q?HgET0mFl08x4AV+8MgcFCbDEGPJH+tO69OXecDTpG0R9wi8bRzvYH2R1zh6j?= =?us-ascii?Q?1D470SA5jIMVCzI0NhX1vW8qEgZtarCe7EqbmSydy3LhbIpk041eylVKBYwn?= =?us-ascii?Q?JPVr2mZcDLsoIJ6bLuViwIJWfvl65QO1TS3FlcpSTHeIzm/w/TkwZx0+23/y?= =?us-ascii?Q?12yBvtZEKv3LAvlK1dnng+cWer0bUT0O6sp4gj7vrsZywBjV11/wPQo2k5Tj?= =?us-ascii?Q?wjjM1Wl3O+1HiX6PFxlhLrkgeZ7SUK6m67Fj4+OctJjiUXm7CygbBwO1MU9H?= =?us-ascii?Q?CMdGeqSbrmlQGX2FBOGiSgqHvQNTxBTWtU+gpUv7gaDnvbbDffvlUOAl646O?= =?us-ascii?Q?X20mA4Xzvhbn9YG085P2tqFbkyBDfvP/WaskwDNqEYvwagnlFliiW62VVLpZ?= =?us-ascii?Q?9v+ITn4WtIoXssrIyYcZYUJUhKRnyQJFNx7I2nY9Oq/Yds3CX4U2UrKpy43Y?= =?us-ascii?Q?pQzbQa9TETeJQ/zw5+QLgkFzsxZMBKAx321RFWOh7etGMiu56s9efZICfEc7?= =?us-ascii?Q?PuBXXNodCtNxk1RYYlyb5NjrwR6xCCkv3Po9KIjA7Cpooxsv4En8VMdZBOXA?= =?us-ascii?Q?knYTwj0aLYsmGdr+bam9Njbjycxn8wLgV2H0iqSgdMMirJgC7DHgcDk5Q6/q?= =?us-ascii?Q?xDlZqsg7FUXcH+XC5IszyeyRZB/jlJum0xrtsNJcFosooiUUX0LdEVJUlR2t?= =?us-ascii?Q?0y5zz4f1LESbI4TimypIo7/OUlib9dcJ6fRwioD3qvvvBCbVay4qENjg31+g?= =?us-ascii?Q?BUu5luDNZ6gLQia8tZcHAj+p7rTXLve7zspExr/4pXwwXP3nonpolE9+KLiq?= =?us-ascii?Q?uLpXhTvb5c8k9zbe87zxbRGsSo0wUt1NpesReubby3/H8lb1MbW0HdQN3UuH?= =?us-ascii?Q?la6qvv+2nqjH8ksYeEA+P7SGbNUzlbaMHKmrr7h/vXtsOHIQHqxSnjBgTYPE?= =?us-ascii?Q?gC/Sd39A4VNywT5+LMsaMPl+Fwap26SiuNFHMJnc?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 64ef9eac-e6bc-4eec-bc41-08da6626d8a5 X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3176.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2022 05:57:05.7439 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: EKysqL6InQrhys5MlLuOCrmrdBmzKqWr5FvwGaIWe5dLFZoMD0w+WX55yFddKWPsZlI7ul8l8Y+Blw8pV1CuzA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5167 ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657864628; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=a696pMQYvRSUk4fO5hzX2hjnPUvLB1n+mRt8BPdfZZg=; b=lyFz2O1DepDARBZO6h2DN3kubk/LYDTVZ1G2qJ27yiIvtupw1npsgmnoWn3ZaChgUDLJw9 jgPvF5OhXR94/Cn28IdBr6sZ49wAB6508zIBYYVKMZ+QFN9asH/K41TIG1e7yEgN3D18s8 m6meDIsraoV4J8qzlStzahWMkhvEy8w= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1657864628; a=rsa-sha256; cv=pass; b=fjFHwXWAF2zIhoUNP+h7wQVe3Xg2KkrzpWmdXjxIKNzVFipVQnRhm5FsG26SDez+8uQPg4 5tJVTXzvORu+JgYo950ADGgZ11VyAG6U8q1jIlzTITBVrTLeHN9x8lqqqJnUw6xMiAUUob 8Rr1qyUKOLSJy2p91MOTI6r3c80qbOw= ARC-Authentication-Results: i=2; imf30.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=oz3lbvVM; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=none (imf30.hostedemail.com: domain of apopple@nvidia.com has no SPF policy when checking 40.107.92.52) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=oz3lbvVM; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=none (imf30.hostedemail.com: domain of apopple@nvidia.com has no SPF policy when checking 40.107.92.52) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com X-Rspamd-Queue-Id: CAFDB80085 X-Stat-Signature: e6px3f3i8z8rye1d6qw31au89kbqsik8 X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1657864627-425490 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: "Aneesh Kumar K.V" writes: > If the new NUMA node onlined doesn't have a memory tier assigned, > the kernel adds the NUMA node to default memory tier. > > Signed-off-by: Aneesh Kumar K.V > --- > mm/memory-tiers.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 68 insertions(+) > > diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c > index 79347d4ab05e..5706ad647136 100644 > --- a/mm/memory-tiers.c > +++ b/mm/memory-tiers.c > @@ -5,6 +5,7 @@ > #include > #include > #include > +#include > #include > > struct memory_tier { > @@ -130,8 +131,73 @@ int node_create_and_set_memory_tier(int node, int tier) > } > EXPORT_SYMBOL_GPL(node_create_and_set_memory_tier); > > +static int __node_set_memory_tier(int node, int tier) > +{ > + int ret = 0; > + struct memory_tier *memtier; > + > + memtier = __get_memory_tier_from_id(tier); > + if (!memtier) { > + ret = -EINVAL; > + goto out; > + } > + node_set(node, memtier->nodelist); > +out: > + return ret; > +} > + > +static int node_set_memory_tier(int node, int tier) Minor comment, but I don't like the name of this function as it doesn't always set the node to the given tier. Something like this would make it clearer the tier value is only used if the node isn't already assigned to a tier: static int init_node_memory_tier(int node, int default_tier) > +{ > + struct memory_tier *memtier; > + int ret = 0; > + > + mutex_lock(&memory_tier_lock); > + memtier = __node_get_memory_tier(node); > + if (!memtier) > + ret = __node_set_memory_tier(node, tier); > + > + mutex_unlock(&memory_tier_lock); > + > + return ret; > +} > + > static unsigned int default_memtier = DEFAULT_MEMORY_TIER; > core_param(default_memory_tier, default_memtier, uint, 0644); > +/* > + * This runs whether reclaim-based migration is enabled or not, > + * which ensures that the user can turn reclaim-based migration > + * at any time without needing to recalculate migration targets. > + */ > +static int __meminit migrate_on_reclaim_callback(struct notifier_block *self, > + unsigned long action, void *_arg) > +{ > + struct memory_notify *arg = _arg; > + > + /* > + * Only update the node migration order when a node is > + * changing status, like online->offline. > + */ > + if (arg->status_change_nid < 0) > + return notifier_from_errno(0); > + > + switch (action) { > + case MEM_ONLINE: > + /* > + * We ignore the error here, if the node already have the tier > + * registered, we will continue to use that for the new memory > + * we are adding here. > + */ > + node_set_memory_tier(arg->status_change_nid, default_memtier); > + break; > + } > + > + return notifier_from_errno(0); > +} > + > +static void __init migrate_on_reclaim_init(void) > +{ > + hotplug_memory_notifier(migrate_on_reclaim_callback, 100); > +} > > static int __init memory_tier_init(void) > { > @@ -153,6 +219,8 @@ static int __init memory_tier_init(void) > /* CPU only nodes are not part of memory tiers. */ > memtier->nodelist = node_states[N_MEMORY]; > mutex_unlock(&memory_tier_lock); > + > + migrate_on_reclaim_init(); > return 0; > } > subsys_initcall(memory_tier_init);