From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C350C433FE for ; Wed, 2 Nov 2022 18:59:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ED6328E0002; Wed, 2 Nov 2022 14:59:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E86D48E0001; Wed, 2 Nov 2022 14:59:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D4DFB8E0002; Wed, 2 Nov 2022 14:59:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id C5E178E0001 for ; Wed, 2 Nov 2022 14:59:06 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 8C379C030A for ; Wed, 2 Nov 2022 18:59:06 +0000 (UTC) X-FDA: 80089414692.23.A6AAF20 Received: from mail-lj1-f169.google.com (mail-lj1-f169.google.com [209.85.208.169]) by imf27.hostedemail.com (Postfix) with ESMTP id 2681540005 for ; Wed, 2 Nov 2022 18:59:04 +0000 (UTC) Received: by mail-lj1-f169.google.com with SMTP id d3so26288665ljl.1 for ; Wed, 02 Nov 2022 11:59:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=t75DvuVjvU8ErKkB8sfDwtGNgCI+Z6sqJYaVzvmVK/A=; b=MUP/AG+NNiZBTgmNQ6OfMmSid3DDYp7HtQco7foq508VnUZGteDOI+eFJ14oaHh0kV zy5pxl4sKYUklIfLktqU4bfR3vTxjchIkJZCU+8SnbaLYpmjAAfnl+D7T507mSmJGUDP W4sLJeyFJbgStOpJ0ZewD6MNe7V036GCuC0ctZbubE9g3url7E6yH3rpCGF2fwcOu/v9 uQrqpV2jjd+sYeXxVUcrnLfoV4/QdnuPNaVf9/QZgNAIRJPGygLTuwin+mm0CrDJ9yMl TEzTwBrKzWwqeXds+YBIeHyfOFyvjP9ZgggNMUJtiNgSuj4A8y61GD1gqp+lP6Ks8ljI 7P1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=t75DvuVjvU8ErKkB8sfDwtGNgCI+Z6sqJYaVzvmVK/A=; b=8Hirue74p0bTqr6eOnWI0m6yD9ByGLj00GqD+NFGuZA7ePL8LfAvG4HEMam4zIbi+1 WqEVaAf8tC9f+nk2ktSlg+dgBv2zXBuj+6GsOsGaQ3KaIJ5/UqiGpbCScn6ICUYCgSGa LX0BDMwewXOu43vehwBm+Ba0lmxT2wKpgYwPx/p8/wJgDZg4xSNXwDa3Q8zksNfS/58r NFX4YWf8il82F7xt1YHfoEMTUJFHD5H0A/vN80CwvYQ7+6Vg/WvsULPFyAoXeCW4R0TP hgeh3ZjSZZGZtxOtcVGM751hHjkDGmBdPvGA0zVhXWTG+OOBTZxKPEL5VYQcCdrGuqP/ WeQA== X-Gm-Message-State: ACrzQf2WfQR/HKC9a26Bn6rexohdzJvzkboXmHKxgMjmNSh10bAnsRmH paFFDAFrAKeW52sn91mTTHUdRejvR3fASJdiDl1B6w== X-Google-Smtp-Source: AMsMyM7BJbGl07H3PmzfwOoksLa52lkbMBKu2vdfiDt6LeVz6JfHt+wc3jYY53SxUNF0LsDH0MlDeXt7k/2BrK6oRhI= X-Received: by 2002:a2e:98ce:0:b0:277:4887:944d with SMTP id s14-20020a2e98ce000000b002774887944dmr7585040ljj.426.1667415543120; Wed, 02 Nov 2022 11:59:03 -0700 (PDT) MIME-Version: 1.0 References: <20221031183122.470962-1-shy828301@gmail.com> In-Reply-To: From: "Zach O'Keefe" Date: Wed, 2 Nov 2022 11:58:26 -0700 Message-ID: Subject: Re: [PATCH] mm: don't warn if the node is offlined To: Yang Shi Cc: Michal Hocko , akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrew Davidoff , Bob Liu Content-Type: text/plain; charset="UTF-8" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667415545; a=rsa-sha256; cv=none; b=2lyMEeZWH4R+8xi50liItwKh/+cQRXEOceVksnN2cjCKEEcveNFc99Akl1KOtwRgNediqR T9H0drDYrnSw5OTVc4n/HcVZtCYMG+alBh9yyAzBAv7sbi/kSNDJtntNIa9fTc48Tk/Vkd sryOcncPYvljglq+aUbcSyoCbk6AwHU= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="MUP/AG+N"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of zokeefe@google.com designates 209.85.208.169 as permitted sender) smtp.mailfrom=zokeefe@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667415545; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=t75DvuVjvU8ErKkB8sfDwtGNgCI+Z6sqJYaVzvmVK/A=; b=0CnD/ZHqIlZ5f4syvqJDkeJg7x05qf12h6BlYBkDeed+6LZwcV1RfA2aSrRN8n7OEkerrx g6466t+gPWCLWY6b/JOXKAyWLE3gPC2PTAFqwH3FUXExSJOIUy8IevEBXr2telgploTW3z MpbFLyQbTSANwTht17s53/d5Mbz+qas= X-Stat-Signature: 45u6x9dzotj7etk9makwo5pq1ct187bz X-Rspamd-Queue-Id: 2681540005 Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="MUP/AG+N"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of zokeefe@google.com designates 209.85.208.169 as permitted sender) smtp.mailfrom=zokeefe@google.com X-Rspam-User: X-Rspamd-Server: rspam07 X-HE-Tag: 1667415544-841163 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Nov 2, 2022 at 11:18 AM Yang Shi wrote: > > On Wed, Nov 2, 2022 at 10:47 AM Michal Hocko wrote: > > > > On Wed 02-11-22 10:36:07, Yang Shi wrote: > > > On Wed, Nov 2, 2022 at 9:15 AM Michal Hocko wrote: > > > > > > > > On Wed 02-11-22 09:03:57, Yang Shi wrote: > > > > > On Wed, Nov 2, 2022 at 12:39 AM Michal Hocko wrote: > > > > > > > > > > > > On Tue 01-11-22 12:13:35, Zach O'Keefe wrote: > > > > > > [...] > > > > > > > This is slightly tangential - but I don't want to send a new mail > > > > > > > about it -- but I wonder if we should be doing __GFP_THISNODE + > > > > > > > explicit node vs having hpage_collapse_find_target_node() set a > > > > > > > nodemask. We could then provide fallback nodes for ties, or if some > > > > > > > node contained > some threshold number of pages. > > > > > > > > > > > > I would simply go with something like this (not even compile tested): > > > > > > > > > > Thanks, Michal. It is definitely an option. As I talked with Zach, I'm > > > > > not sure whether it is worth making the code more complicated for such > > > > > micro optimization or not. Removing __GFP_THISNODE or even removing > > > > > the node balance code should be fine too IMHO. TBH I doubt there would > > > > > be any noticeable difference. > > > > > > > > I do agree that an explicit nodes (quasi)round robin sounds over > > > > engineered. It makes some sense to try to target the prevalent node > > > > though because this code can be executed from khugepaged and therefore > > > > allocating with a completely different affinity than the original fault. > > > > > > Yeah, the corner case comes from the node balance code, it just tries > > > to balance between multiple prevalent nodes, so you agree to remove it > > > IIRC? > > > > Yeah, let's just collect all good nodes into a nodemask and keep > > __GFP_THISNODE in place. You can consider having the nodemask per collapse_control > > so that you allocate it only once in the struct lifetime. > > Actually my intention is more aggressive, just remove that node balance code. > The balancing code dates back to 2013 commit 9f1b868a13ac ("mm: thp: khugepaged: add policy for finding target node") where it was made to satisfy "numactl --interleave=all". I don't know why any real workloads would want this -- but there very well could be a valid use case. If not, I think it could be removed independent of what we do with __GFP_THISNODE and nodemask. Balancing aside -- I haven't fully thought through what an ideal (and further overengineered) solution would be for numa, but one (perceived - not measured) issue that khugepaged might have (MADV_COLLAPSE doesn't have the choice) is on systems with many, many nodes with source pages sprinkled across all of them. Should we collapse these pages into a single THP from the node with the most (but could still be a small %) pages? Probably there are better candidates. So, maybe a khugepaged-only check for max_value > (HPAGE_PMD_NR >> 1) or something makes sense. > > > > And as mentioned in other reply it would be really nice to hide this > > under CONFIG_NUMA (in a standalong follow up of course). > > The hpage_collapse_find_target_node() function itself is defined under > CONFIG_NUMA. > > > > > -- > > Michal Hocko > > SUSE Labs