From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C95BC46CD2 for ; Wed, 3 Jan 2024 01:32:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 302796B02AA; Tue, 2 Jan 2024 20:32:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2B1F46B02FB; Tue, 2 Jan 2024 20:32:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 152756B02CA; Tue, 2 Jan 2024 20:32:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 03EF86B02FB for ; Tue, 2 Jan 2024 20:32:20 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C3E73A06B5 for ; Wed, 3 Jan 2024 01:32:19 +0000 (UTC) X-FDA: 81636274398.03.1424C81 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by imf14.hostedemail.com (Postfix) with ESMTP id 08912100008 for ; Wed, 3 Jan 2024 01:32:17 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=QaOd6MmJ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf14.hostedemail.com: domain of rientjes@google.com designates 209.85.214.174 as permitted sender) smtp.mailfrom=rientjes@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704245538; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IqPVmDTLcp0lm6JkLsZ7c/JcNa8c/Le1eOgSpFlwcuI=; b=qyu+2qofrn8kT/G9XyOVlwexKsrzyvUtjxm+r4WnHYE1QdflOrNzFan/9eN5V2D+NWU+OL nUT2ecsLYQGTIx0dVckLWCZZDD77pPM3K0xnxfZ74aCEXPVY7Fm1uR1WIba5/HybS2G8d9 r3L+Rnv+KkCZbG3ESusMvFwBU02SE98= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=QaOd6MmJ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf14.hostedemail.com: domain of rientjes@google.com designates 209.85.214.174 as permitted sender) smtp.mailfrom=rientjes@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704245538; a=rsa-sha256; cv=none; b=ha6EEUTKVOsGnQkTnDdIX0yoPg46ay6fhTegbg5wV7wopnVAehoDebr/vlrz8xpHgfHdDu AvQKnmTiIUhQuOs9RBtTWr+PqDwxliUTc39brIzdPt9T0JrMAXXt3miwkuCla80vrTCWuE N7p4iDn2Kjz2VVRHshlIS2BWYHZ0oQs= Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-1d42ed4cdc7so73735ad.0 for ; Tue, 02 Jan 2024 17:32:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1704245537; x=1704850337; darn=kvack.org; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=IqPVmDTLcp0lm6JkLsZ7c/JcNa8c/Le1eOgSpFlwcuI=; b=QaOd6MmJPaZNBsOZg0SgomFb47OUJxc2Nb5wx7W+I1VCuNnaq2KSJ1In8d9JCQhuWz oaFEhBkchfHD94UuoBgPLg4ml2SLp0x3kS8WMFLgtUe5bJM0ebjLBEXdpooAGcNBRRqr iMVJiVuql0nS3WRGJYi6L6sviQzKKUzhV6MrQKuuJ6SpVaBERxAPPq0IZVE5XStAWYTH YHTREkHYG2rHbfEgDIMPSrV42EnQfJ1pMeQfRySHB3GxLhwlhVC4UNm5egKrLQGqi0bR 1yddyR14LBUEALIdkczYhVWHaDG28W8xwISNr2pX14eWBfN35JcM14qhnVGtErQsKib/ WO/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704245537; x=1704850337; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=IqPVmDTLcp0lm6JkLsZ7c/JcNa8c/Le1eOgSpFlwcuI=; b=iT7dd3f1XW+ni/Pd4cqxSL4N3FaN0TWwIzGKzsf3a9k9dQbcqUbbyqy5lnpoLq+oNl eTMyO46EJhFiR8hEhKtYsq4vgUZF9STL8r1ArY/syQPBnStnSRI+GGX5uUSm6heimYgj EMW3lwRVV++wrc+d5CZtarKKewIPhlx3/Hqjb8QtnUPKg+3hNbyY7gkfV24tyx3I7VnO RGygg6EY/wC2x1cUMinx4jAATQbcWZseytX88DNUyIP4Iglupk6Ir0LbjKlCJfcu5twY UnzxGPbXdA5Q9HDaUmYp9QyrEl6J5JWft9tldjn+yTQlISV1SIIog2mQslUnfudm43DA AOog== X-Gm-Message-State: AOJu0YxuETrcV208DiUf6lP23KZ/8U57u6JbpizJb1ZaTfhExBG+MVYE KnipV1Cv7ZqqXBhDfsL054ikHBGH41x1 X-Google-Smtp-Source: AGHT+IG7XtbtkOp8HKw3smQ60PS3HuaDsC6br2LR43feUh9XWs4NZY7BQuSI3RZwcDhAAmIxspxKVQ== X-Received: by 2002:a17:902:d508:b0:1d3:9d94:3309 with SMTP id b8-20020a170902d50800b001d39d943309mr76340plg.22.1704245536522; Tue, 02 Jan 2024 17:32:16 -0800 (PST) Received: from [2620:0:1008:15:c73b:7876:89ec:9102] ([2620:0:1008:15:c73b:7876:89ec:9102]) by smtp.gmail.com with ESMTPSA id e15-20020aa7824f000000b006d9879ba6besm20155240pfn.170.2024.01.02.17.32.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 17:32:15 -0800 (PST) Date: Tue, 2 Jan 2024 17:32:15 -0800 (PST) From: David Rientjes To: Gang Li cc: David Hildenbrand , Mike Kravetz , Muchun Song , Andrew Morton , Tim Chen , linux-mm@kvack.org, linux-kernel@vger.kernel.org, ligang.bdlg@bytedance.com Subject: Re: [PATCH v3 4/7] hugetlb: pass *next_nid_to_alloc directly to for_each_node_mask_to_alloc In-Reply-To: <20240102131249.76622-5-gang.li@linux.dev> Message-ID: <19b08bb8-1fbd-557b-2684-5c29c5c783e6@google.com> References: <20240102131249.76622-1-gang.li@linux.dev> <20240102131249.76622-5-gang.li@linux.dev> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 08912100008 X-Stat-Signature: gy1oonfd76ekenzuj7ghg4cbu9bxfw1x X-HE-Tag: 1704245537-909787 X-HE-Meta: U2FsdGVkX19+/Ft6TVcYOH5OUpkCEN70gwpl9dOioUzQX6QFrYWZGjZZoy0MBpxY+yKnpODY8LAedeLljFN6O255iwzEvlJ8PPusflycltnT4BEJidSvO3oO4FWilYhvwylNKwj9vC4a4BjQGTigTjtrA/R4IlXpGWIq2IJfENvTbyzeqjPKdTGwO+uKpcwdTbDW6hZPjeEG5s5lP64i0N36KBmGjZLHS3JgV8hyK+mbmi7A1pAT1tJuaCohRWOgGmtUqDs3U1nYZE6MBHh8MDBp1n2Dj7FOhdOyPbpi6sx8aUyEm5pOa5DaGFHy2MvFgK6v/vXrfhIULVljFl+motBfHeo97QPkPay3MT/2SFzE/E+k+2ZIV2EQi0O31LzZl1j8sw3TGV+h9xnAAXPE7ObfK4df4q72QRipxCKNIJwAS6sE0Qg8TSwI8X6V+d1vzMqxnytPhdI5o1/iAZ78uj1NcG/wyH3iQQNTx3Mj77JigsRm2N8sCFdjiys3OW7KmrU962UBmfJJQIzr3+bgXMiR1frf1sEgSJtdl6UNaFVCsjy1hgLIMOWQy10FrCRfmCbpzMa1rlhqes6kDV1S+MSI3JTij2MQQBKpllXjneBF/SD1BNGaFjoz5eiHq9f22Hswtd0aLC6NHo0jhReuCQc2A8SGoOQ+d3gBHdYcpJm6YSKUUqn+cOlAZi4gWYq+XKPUaux6OCne6NzfGHQeGJFSRzuy6ZTT/Cl/4KWZ6anyGqIdXIfc4hCbFU5yI9ktrZWZH+8t+8bM8/1FiUi+ljmg0BWOqgRVKHcUcobE0Dq50F5yDx7lJDrIOY3rUPKshRPq8sGm1qAvt0e99Gq4AOwniRXnnx/lGj4MjxWDCYk9gDrd8XSgCeekA5/tooS0YNepTemfmjeALRvBcie1WL4J58Z7IdhzADZenLQYV/pGiTOuVsWNtX9pl6w0VEV9cSTs5mIZr1T1OEI+SPj BNaDRZYV VdEju3VcRxvJA5F0QGZX4kdGOrTt/ph/ETL/LKlWLLHGuekI9t1wLXXIxpl7lovCyWHNJ957mI53P0MbtIzrjCooB7r38ENeeIY+pmkyucefCsB1FKte1p9ZFegPkNO7JX7IOWGDiH6rsyQdFlHiP6lfYdlYkIIpiH5QByPFRmB44Vjf+ChI5enABwaxEd6IoRiiloS9EHditlvqkVeAD+oJbggUGRwuJMhGi6yNqF1tXa1Bitl8p4ta3uXc7OR4uiSzVvPb7f0mNc2lNj0Bo3xpKW0Tfl2B1N977k5PvYB8qrzp0F5YxeGw8aIb0e97k/PwihVvbtJ7a0ZvyQKbJ9SFoBiPaOZkmX97nla0pbZ+n07waiHVY/zKrEOaPVOi8emMv2NJ37fsV6v7TvIjd3SQ4vg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, 2 Jan 2024, Gang Li wrote: > The parallelization of hugetlb allocation leads to errors when sharing > h->next_nid_to_alloc across different threads. To address this, it's > necessary to assign a separate next_nid_to_alloc for each thread. > > Consequently, the hstate_next_node_to_alloc and for_each_node_mask_to_alloc > have been modified to directly accept a *next_nid_to_alloc parameter, > ensuring thread-specific allocation and avoiding concurrent access issues. > > Signed-off-by: Gang Li > --- > This patch seems not elegant, but I can't come up with anything better. > Any suggestions will be highly appreciated! Same error as v2: mm/hugetlb.c:3315:53: warning: variable 'node' is used uninitialized whenever '&&' condition is false [-Wsometimes-uninitialized] for_each_node_mask_to_alloc(&h->next_nid_to_alloc, nr_nodes, node, &node_states[N_MEMORY]) { ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ mm/hugetlb.c:1501:3: note: expanded from macro 'for_each_node_mask_to_alloc' nr_nodes > 0 && \ ^~~~~~~~~~~~ mm/hugetlb.c:3342:38: note: uninitialized use occurs here list_add(&m->list, &huge_boot_pages[node]); ^~~~ mm/hugetlb.c:3315:53: note: remove the '&&' if its condition is always true for_each_node_mask_to_alloc(&h->next_nid_to_alloc, nr_nodes, node, &node_states[N_MEMORY]) { ^ mm/hugetlb.c:3310:7: warning: variable 'node' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized] if (!m) ^~ mm/hugetlb.c:3342:38: note: uninitialized use occurs here list_add(&m->list, &huge_boot_pages[node]); ^~~~ mm/hugetlb.c:3310:3: note: remove the 'if' if its condition is always true if (!m) ^~~~~~~ mm/hugetlb.c:3304:20: note: initialize the variable 'node' to silence this warning int nr_nodes, node; ^ = 0 2 warnings generated. > --- > mm/hugetlb.c | 22 ++++++++++++---------- > 1 file changed, 12 insertions(+), 10 deletions(-) > > diff --git a/mm/hugetlb.c b/mm/hugetlb.c > index 92448e747991d..a71bc1622b53b 100644 > --- a/mm/hugetlb.c > +++ b/mm/hugetlb.c > @@ -1464,15 +1464,15 @@ static int get_valid_node_allowed(int nid, nodemask_t *nodes_allowed) > * next node from which to allocate, handling wrap at end of node > * mask. > */ > -static int hstate_next_node_to_alloc(struct hstate *h, > +static int hstate_next_node_to_alloc(int *next_nid_to_alloc, > nodemask_t *nodes_allowed) > { > int nid; > > VM_BUG_ON(!nodes_allowed); > > - nid = get_valid_node_allowed(h->next_nid_to_alloc, nodes_allowed); > - h->next_nid_to_alloc = next_node_allowed(nid, nodes_allowed); > + nid = get_valid_node_allowed(*next_nid_to_alloc, nodes_allowed); > + *next_nid_to_alloc = next_node_allowed(nid, nodes_allowed); > > return nid; > } > @@ -1495,10 +1495,10 @@ static int hstate_next_node_to_free(struct hstate *h, nodemask_t *nodes_allowed) > return nid; > } > > -#define for_each_node_mask_to_alloc(hs, nr_nodes, node, mask) \ > +#define for_each_node_mask_to_alloc(next_nid_to_alloc, nr_nodes, node, mask) \ > for (nr_nodes = nodes_weight(*mask); \ > nr_nodes > 0 && \ > - ((node = hstate_next_node_to_alloc(hs, mask)) || 1); \ > + ((node = hstate_next_node_to_alloc(next_nid_to_alloc, mask)) || 1); \ > nr_nodes--) > > #define for_each_node_mask_to_free(hs, nr_nodes, node, mask) \ > @@ -2350,12 +2350,13 @@ static void prep_and_add_allocated_folios(struct hstate *h, > */ > static struct folio *alloc_pool_huge_folio(struct hstate *h, > nodemask_t *nodes_allowed, > - nodemask_t *node_alloc_noretry) > + nodemask_t *node_alloc_noretry, > + int *next_nid_to_alloc) > { > gfp_t gfp_mask = htlb_alloc_mask(h) | __GFP_THISNODE; > int nr_nodes, node; > > - for_each_node_mask_to_alloc(h, nr_nodes, node, nodes_allowed) { > + for_each_node_mask_to_alloc(next_nid_to_alloc, nr_nodes, node, nodes_allowed) { > struct folio *folio; > > folio = only_alloc_fresh_hugetlb_folio(h, gfp_mask, node, > @@ -3310,7 +3311,7 @@ int __alloc_bootmem_huge_page(struct hstate *h, int nid) > goto found; > } > /* allocate from next node when distributing huge pages */ > - for_each_node_mask_to_alloc(h, nr_nodes, node, &node_states[N_MEMORY]) { > + for_each_node_mask_to_alloc(&h->next_nid_to_alloc, nr_nodes, node, &node_states[N_MEMORY]) { > m = memblock_alloc_try_nid_raw( > huge_page_size(h), huge_page_size(h), > 0, MEMBLOCK_ALLOC_ACCESSIBLE, node); > @@ -3684,7 +3685,7 @@ static int adjust_pool_surplus(struct hstate *h, nodemask_t *nodes_allowed, > VM_BUG_ON(delta != -1 && delta != 1); > > if (delta < 0) { > - for_each_node_mask_to_alloc(h, nr_nodes, node, nodes_allowed) { > + for_each_node_mask_to_alloc(&h->next_nid_to_alloc, nr_nodes, node, nodes_allowed) { > if (h->surplus_huge_pages_node[node]) > goto found; > } > @@ -3799,7 +3800,8 @@ static int set_max_huge_pages(struct hstate *h, unsigned long count, int nid, > cond_resched(); > > folio = alloc_pool_huge_folio(h, nodes_allowed, > - node_alloc_noretry); > + node_alloc_noretry, > + &h->next_nid_to_alloc); > if (!folio) { > prep_and_add_allocated_folios(h, &page_list); > spin_lock_irq(&hugetlb_lock); > -- > 2.20.1 > >