From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CDF3C433FE for ; Thu, 3 Nov 2022 07:51:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D7CF58E0002; Thu, 3 Nov 2022 03:51:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D2C2C8E0001; Thu, 3 Nov 2022 03:51:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF3798E0002; Thu, 3 Nov 2022 03:51:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id ABD358E0001 for ; Thu, 3 Nov 2022 03:51:06 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 59533120C19 for ; Thu, 3 Nov 2022 07:51:06 +0000 (UTC) X-FDA: 80091360132.11.8FE6EEB Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf20.hostedemail.com (Postfix) with ESMTP id A85701C0007 for ; Thu, 3 Nov 2022 07:51:05 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 14BE4221E1; Thu, 3 Nov 2022 07:51:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1667461864; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=xMZ1/xJ+ivqqXiqEy69V2CepWd5ZrOeZBYR6ed7f3e8=; b=aOiZS7XFJ3YklcHmkjQTN1uqrMjkvOInWO2MC2j4n5zWROVryAl7PshMGM+tnxeKXjoD00 Wh4N+RX6R4FMmGBRCQzD1fKJ0Mx2RBjZ5uEOov2HZyxx9G5m4uPZKmgfUUI0miVLzm+WJi 4WkY3IZsSyKxTFKozZjDQAWKIq11S8o= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id E910E13AAF; Thu, 3 Nov 2022 07:51:03 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id hF0hNudyY2PgJwAAMHmgww (envelope-from ); Thu, 03 Nov 2022 07:51:03 +0000 Date: Thu, 3 Nov 2022 08:51:03 +0100 From: Michal Hocko To: Zach O'Keefe Cc: Yang Shi , akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrew Davidoff , Bob Liu Subject: Re: [PATCH] mm: don't warn if the node is offlined Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=aOiZS7XF; spf=pass (imf20.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667461866; a=rsa-sha256; cv=none; b=BMZWuyVUYHN3cN5OBt8pqEF6ZKPQOQi7wFXYadLC0mKvFI4Oj8K1RnLIXk4tIuNFBodU1d 6yr0DH8hsAFKhcp5bGY5AskH6xCq9TfYSgsOresFWZP7Z1hM1fZVgZ9RRoFQWhIzw61OBk YJlg4ePRQOh0O/1goXriPmAdFeo1Xns= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667461866; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xMZ1/xJ+ivqqXiqEy69V2CepWd5ZrOeZBYR6ed7f3e8=; b=IFcyoiiCqHI0bBP3HEPZmPrw4xUJcwdTLkv//qZtRLVQsMzWxKDY+7VaaiMghYkNPC+R2U wMbiC4H6StUgt122dagfYpZfvFUxoT3RlBbjTVeIzFS3T7aAVn/2McYlR9RaBC/uZodXFK 2l1/UHm9qaNpCqlXMhYLZoKBMYMucZk= X-Stat-Signature: kb6utm5rczsc67ttmwk3t8wat3zat3cb X-Rspamd-Queue-Id: A85701C0007 Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=aOiZS7XF; spf=pass (imf20.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1667461865-318084 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed 02-11-22 11:58:26, Zach O'Keefe wrote: > On Wed, Nov 2, 2022 at 11:18 AM Yang Shi wrote: > > > > On Wed, Nov 2, 2022 at 10:47 AM Michal Hocko wrote: > > > > > > On Wed 02-11-22 10:36:07, Yang Shi wrote: > > > > On Wed, Nov 2, 2022 at 9:15 AM Michal Hocko wrote: > > > > > > > > > > On Wed 02-11-22 09:03:57, Yang Shi wrote: > > > > > > On Wed, Nov 2, 2022 at 12:39 AM Michal Hocko wrote: > > > > > > > > > > > > > > On Tue 01-11-22 12:13:35, Zach O'Keefe wrote: > > > > > > > [...] > > > > > > > > This is slightly tangential - but I don't want to send a new mail > > > > > > > > about it -- but I wonder if we should be doing __GFP_THISNODE + > > > > > > > > explicit node vs having hpage_collapse_find_target_node() set a > > > > > > > > nodemask. We could then provide fallback nodes for ties, or if some > > > > > > > > node contained > some threshold number of pages. > > > > > > > > > > > > > > I would simply go with something like this (not even compile tested): > > > > > > > > > > > > Thanks, Michal. It is definitely an option. As I talked with Zach, I'm > > > > > > not sure whether it is worth making the code more complicated for such > > > > > > micro optimization or not. Removing __GFP_THISNODE or even removing > > > > > > the node balance code should be fine too IMHO. TBH I doubt there would > > > > > > be any noticeable difference. > > > > > > > > > > I do agree that an explicit nodes (quasi)round robin sounds over > > > > > engineered. It makes some sense to try to target the prevalent node > > > > > though because this code can be executed from khugepaged and therefore > > > > > allocating with a completely different affinity than the original fault. > > > > > > > > Yeah, the corner case comes from the node balance code, it just tries > > > > to balance between multiple prevalent nodes, so you agree to remove it > > > > IIRC? > > > > > > Yeah, let's just collect all good nodes into a nodemask and keep > > > __GFP_THISNODE in place. You can consider having the nodemask per collapse_control > > > so that you allocate it only once in the struct lifetime. > > > > Actually my intention is more aggressive, just remove that node balance code. > > > > The balancing code dates back to 2013 commit 9f1b868a13ac ("mm: thp: > khugepaged: add policy for finding target node") where it was made to > satisfy "numactl --interleave=all". I don't know why any real > workloads would want this -- but there very well could be a valid use > case. If not, I think it could be removed independent of what we do > with __GFP_THISNODE and nodemask. Thanks for the reference. The patch is really dubious. If the primary usecase is a memory policy then one should be used. We have the vma handy. Sure per task policy would be a bigger problem but interleaving is a mere hint rather than something that has hard requirements. > Balancing aside -- I haven't fully thought through what an ideal (and > further overengineered) solution would be for numa, but one (perceived > - not measured) issue that khugepaged might have (MADV_COLLAPSE > doesn't have the choice) is on systems with many, many nodes with > source pages sprinkled across all of them. Should we collapse these > pages into a single THP from the node with the most (but could still > be a small %) pages? Probably there are better candidates. So, maybe a > khugepaged-only check for max_value > (HPAGE_PMD_NR >> 1) or something > makes sense. Honestly I do not see any problem to be solved here. -- Michal Hocko SUSE Labs