From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 146ADE77188 for ; Fri, 20 Dec 2024 15:17:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7ADDF6B0082; Fri, 20 Dec 2024 10:17:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 75D3C6B0088; Fri, 20 Dec 2024 10:17:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 64B2D6B0089; Fri, 20 Dec 2024 10:17:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 456496B0082 for ; Fri, 20 Dec 2024 10:17:08 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 9B306AFE66 for ; Fri, 20 Dec 2024 15:17:07 +0000 (UTC) X-FDA: 82915689654.15.9692A0D Received: from mail-qt1-f172.google.com (mail-qt1-f172.google.com [209.85.160.172]) by imf19.hostedemail.com (Postfix) with ESMTP id A888E1A0023 for ; Fri, 20 Dec 2024 15:16:29 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b="H1KJ/DUv"; spf=pass (imf19.hostedemail.com: domain of gourry@gourry.net designates 209.85.160.172 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734707799; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pR8tfRixeKD43F1oU2tExUTza9QeFOTPElUlwwuE5Tg=; b=IRwEs+nAXNhbund128drb6y880Bi/31uCSN5bIPqNHLGQoteGXZ1kvI53wiaOIwzPAZRMW E6qxwsCj3h0NqXphCPegXd9KWqqWAka9g998J1cUuqRWnTjyEgNVXgJii7lxktGIPUNBgH 8V5U9ijstfUAAOh2eunNMBCAVOWQw+g= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b="H1KJ/DUv"; spf=pass (imf19.hostedemail.com: domain of gourry@gourry.net designates 209.85.160.172 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734707799; a=rsa-sha256; cv=none; b=33HcsJhtCqM2R5zlKzwZgb/H4XSOZtMegb0GBcLDKO55BwjIN0i9a4gZprOT4+qPoM4sEm VjSwzJMrlC9CM/AmrC/qSGdFjjDYAJzqeNYanfiWRDmQbgUzYorRkWnqrE4AbUin1BFgPZ y/p+kHG3R/ukIV5rQGXcHYC1Lyb5d+E= Received: by mail-qt1-f172.google.com with SMTP id d75a77b69052e-467a1ee7ff2so15767431cf.0 for ; Fri, 20 Dec 2024 07:17:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1734707825; x=1735312625; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:from:to:cc:subject:date:message-id:reply-to; bh=pR8tfRixeKD43F1oU2tExUTza9QeFOTPElUlwwuE5Tg=; b=H1KJ/DUvcrb1nJq5VaPj0PViKl/qyLqpYie/4ER/10hb4zrIvK1/lFfbgFnNLNt+GE bHqoNtVgHF3yuax1M7/YVVwRiIKcesAzUu4tB4WzIIf6cP9bJIcmLOhdR3D4RL/gYT2n 5Hma1MeROKBIcNHcQJOCHQTEsyHgQU9JlGys/M/OoYdyhsDccQ6dPLFAQV3ZyDRpMz0R kP8NzDGqC0iGrf8IEl0M5LEVrGtRzXlQhUtctjWbrgboQ2hV572fXlU2i71GgpJn+UxB JfgeGKbh+3C3c7bzZBkmt7NspUkthTHZX6SJOb4LbVC0irKlOo0DNZkuml8U0oMBhRS8 548g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734707825; x=1735312625; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=pR8tfRixeKD43F1oU2tExUTza9QeFOTPElUlwwuE5Tg=; b=kPhZysdBoRPrKGypYIzUAxW9YCgqODqF1YC9Gjid1JJAqxcmD47BfZ8BYOZCHnqqO9 FdTAek6KG5v5rh873QH9qyUx9nYcH6kpY6hDkfxHWs/Qz8LWiywBWLU1XpzDgnedsSrU nRIIPvS9GT9loKM5raa15en/pmupcSjcokiCSNWFSJwPjHH9kOT7ON0vBEYjenF9ltMK EyrEndIVUnUPRF0X2lGT8QhXh51rKA+Gt/rxL/B2cXr8LCZMsq/8rwgb0zOeQigduz2v 03NFbHIFlXtDttsWhvydD773eID/c4ImWIBKBnyqH+paeyy8Kykgtig3km05QZqduRXR vebg== X-Forwarded-Encrypted: i=1; AJvYcCUH1BbcCW7mRV5fdt3GG1EXUApoTBkf7YGrkNokoJPIpParnAokYg4T1xejRuSq/dkWkH8VJNXQsQ==@kvack.org X-Gm-Message-State: AOJu0YwXRZ8204etg1JzmT1fjDa9Y94mrGnKDmnP0NFgzjTK7DzejH7F /dQKFjVHN4ror+YWonIP601ibF89l1xqSjRdt2VGM/M42edo+uLG0tq83sBE71E= X-Gm-Gg: ASbGnctyoX7SsdScM7asOR2IPvtTKFxhaSApZp76D6GzQQSP2tnFMgPnWBu1n/av/xA JlEN9zjfPSgNPES+4atEBAfmgBy9nEHupCVxGOH3+Zud2xQ2h7f+iUSkukRDEFXocDl+Sf0g3Gf s+8EdVV5P+YJQR/b+11TxnveJ4AvnmN1L+Vy5SNTZyveOAnlvhHEIVc2IKmm9W2N43JY7LZl1kZ WKQTKTZ7gidG9txVljbKu9nnaTIWsNyzsx5cZT156GGq0ifhch5GgqAuP44al85GigIivUlr9g/ 5PGDvo5y7UpU8I9q99Dc7QpGDUDDOEKt6UDYYM1TnVkneui1Ee+jJOw= X-Google-Smtp-Source: AGHT+IFYzQ12D81op7AvMTHyJ6Un6tsFLrLquKGsSUAjVvBadxuah2bmouvnm+CN2kkF5zd1ZnxwdA== X-Received: by 2002:ac8:7d46:0:b0:466:9bc4:578 with SMTP id d75a77b69052e-46a4a8e2899mr53472111cf.22.1734707824747; Fri, 20 Dec 2024 07:17:04 -0800 (PST) Received: from gourry-fedora-PF4VCD3F (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-46a3eb16ca0sm17361201cf.65.2024.12.20.07.17.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Dec 2024 07:17:04 -0800 (PST) From: Gregory Price X-Google-Original-From: Gregory Price Date: Fri, 20 Dec 2024 10:17:02 -0500 To: David Hildenbrand Cc: Gregory Price , linux-mm@kvack.org, linux-kernel@kvack.org, osalvador@suse.de, gregkh@linuxfoundation.org, rafael@kernel.org, akpm@linux-foundation.org Subject: Re: [PATCH] mm: add build-time option to set hotplug default type Message-ID: References: <20241220144518.206208-1-gourry@gourry.net> <11911ad7-527b-411b-9896-fea06bf38a26@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <11911ad7-527b-411b-9896-fea06bf38a26@redhat.com> X-Rspamd-Queue-Id: A888E1A0023 X-Rspamd-Server: rspam12 X-Stat-Signature: roxjx6xrhqpysfa1qges6xxwgubbsuzc X-Rspam-User: X-HE-Tag: 1734707789-914119 X-HE-Meta: U2FsdGVkX19lT99DuEOxhA1Wk1dnLNGPR7deeDZT5wB1knicpwWnzojYylZEEoDTNhmk1SydnGy6AnATEIhZ1Ml7sfHufkkk9b6Vlao1U/1jLYguM+Gk9KSDLNI4PY0gwO0JANpcp07Q3OOwh1yenJdu6JK5piddfwk426bXG4lxRh1zolIvOYVtOJKiv2TmlzXLomw+GC9YXLSmjHdJjGKth+2lOpIl89EJ8CIvfJID28tBCK2bDKT7xlHRzbz/dG/d6KJ6BbSj7Me/DznoxV03W3WM2jyLTWBfV/HzlOG5STG/GxQ4Aj09OQI2HPUOuyp3cUD0CFQrYBBWDGTfy0UOrhaCgwxNBvB0RVFxsAXuuZcsiz5PPmDGtcxGNx0MEMDEgM4+ZlBagIqI2JP8g9tnxk4XhoyFz7mFYQ+CUTM7NiexUoOiylDSUDT2kW4nNbIMm1H6qChdQLWy+1Mx85ukBUQb/P1O+7ZE8j6/M1GsVjLOuMuOjaUYkPE6VG2GZaQTIKP+WT+XdbNRA6Jax5uQo+mDai9wgfX/wAbZ8sBjAyvlCv40GBo+25hhvMubk7Q3eRYF7k2evhwIm2UpNHZyb7vvrDuHlfsF3f2ib1e+X8awwAWIgjH0VTsq9srXfSGLQYYxTBVu3CnKk7fibdgG7gN2C7iKG2e1PNI0l4SYTIxmEi4Lu/m/B2w5SfLzFwzSoTFql4DKryjjsE+ibboMQHo5rX4iHK+ie6OdzwyUCmMKWkAvMaeolXVdJryakFJKPlQabkSIK9siLwJ6Lc7YeEp1Lqr7a3r8H2iKiqnbpfyT0Ydb0ACQ92RxkCT/oJ5As84qvu7rE+KaIObEVNIV2znSu9j8kFTw4cMziCpiDQV9+vN36hdE/4aki081p5nKlwc0uk4iD8DU5IuEEKpaptk8Or62cyRJspu6hrKHfrHR2Y+dpFI+hNhPZNtmnIhLBRbpZgiZ2wyAHqD /4d4QbWn gK4r3qtxaTdyunXK/+qPiaRUArpcga2utcnLVi1OsyRb7nFtLBCturMhjPYhyeRSQyY6R/DIcAfwPz1Jw0zoku2FeV+bP59KuZcWXeKsw27UFFKHYCG/I3Zg86szeEjGQ8E7HxGs/5IsYYqsfrVqZkk1VyRTGEU7EOnv/OTRGo1HMyoFhLIgfvxPzCC3kiMTQVvFqcOD1G1Pj6CiMl0JHXL4OXEUkDuac7e+s6HFteoLKtIt4cGZVxU4MIy4Od1fnU5sa2ICU/PGqMzq7PQ/enYP7SgW3bSU2EQMlrst2yn8kgxkg1J7qlfR4y4OKUXZ5tVPU/22jfvYW9yQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.006372, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Dec 20, 2024 at 03:59:58PM +0100, David Hildenbrand wrote: > On 20.12.24 15:45, Gregory Price wrote: > > When memory hotplug auto-online is enabled, hotplug memory blocks are > > onlined into ZONE_NORMAL by default. The `memhp_default_state` boot > > param allows runtime configuration, but no build-time config exists. > > + you can configure it at runtime. > > > > > Add a build-time configuration option to change default hotplug zone. > > > > build config: > > MEMHP_DEFAULT_TYPE > > > > Selections: > > MEMHP_DEFAULT_TYPE_NORMAL => mhp_default_online_type = "online" > > MEMHP_DEFAULT_TYPE_MOVABLE => mhp_default_online_type = "online_movable" > > > > When MEMORY_HOTPLUG_DEFAULT_ONLINE is disabled, MEMHP_DEFAULT_TYPE is > > set to "offline" to match the current system behavior. > > > > ZONE_NORMAL still remains the default, because for systems with a large > > amount of hotplug memory, defaulting it to ZONE_MOVABLE may result in > > portions failing to online if sufficient ZONE_NORMAL memory does not > > exist to describe it. > > > > What's the use case? > > I'm hoping that we can move away from the compile-time option and let user > space, who better knows what to do (especially with different kinds of > memory having different requirements) configure auto-onlining or online > manually (e.g., devdax). > At Meta we have a fairly complex boot process that goes through multiple kernels before we get to a target kernel to run workloads. Each of those kernels may have health-checks that want to see the memory is online. The build switch makes this particular feature consistent for us across all those kernels without having to carry the boot parameter. Eventually we'd like to move to udev, but it's not feasible for us right now due to the state of CXL BIOS/Platform/Drivers - driver-management does not work reliable for all platforms and all devices. This gets us where we're going while the rest catch up. > For example, in RHEL we traditionally use udev rules, because we want a > different behavior on bare-metal vs. VMs, but they are not particularly easy > to extend to implement wilder policies. > > For a while I worked on a systemd unit [1] to configure+handle memory > onlining so we can get rid of the udev rules we use in RHEL. But it only > configured+handled having "one type of hotplugged memort". > > I'm planning on picking that up again at some point, to also make it > possible to handle different policies for different memory types. > > For example, maybe someone wants to auto-online virtio-mem memory to > ZONE_NORMAL, but let onlining of devdax memory be handled by the devdax > utility (e.g. ZONE_MOVABLE). We can identify in some cases "what" memory was > added using /proc/iomem. > Generally I agree this is the way to go. But in those cases - you're probably not turning MEMORY_HOTPLUG_DEFAULT_ONLINE on anyway. So this doesn't really affect that usage pattern. ~Gregory