From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BEADC2BBCA for ; Tue, 25 Jun 2024 20:05:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CF7A96B00B1; Tue, 25 Jun 2024 16:05:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CA75E6B00B4; Tue, 25 Jun 2024 16:05:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B47B96B00B6; Tue, 25 Jun 2024 16:05:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 97D206B00B1 for ; Tue, 25 Jun 2024 16:05:58 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 5351D1203E3 for ; Tue, 25 Jun 2024 20:05:58 +0000 (UTC) X-FDA: 82270491996.09.49E22B6 Received: from mail-lf1-f47.google.com (mail-lf1-f47.google.com [209.85.167.47]) by imf24.hostedemail.com (Postfix) with ESMTP id 5564B180016 for ; Tue, 25 Jun 2024 20:05:56 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=jr9CUImt; spf=pass (imf24.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.47 as permitted sender) smtp.mailfrom=urezki@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719345936; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2J4hGqbcUzUN0UCwy+GvYQrnL7nzLlsNWsttcQw1Uqw=; b=re91H+sYPU7TRLKUTEtWDeCmiESMvIM+AnqpB3IlMlEz8Xkqx3NUQNsfaHI1aln5Q86ndp lTpcHvN1liYHbn0I+y3A1xr0m1EcOn1b8OyxcYm81YX8oi4L4LvATJsWL8emBZ8N4PXyi9 znzu8FhG20CV5hHmBNc5qTRT6wqMA68= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=jr9CUImt; spf=pass (imf24.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.47 as permitted sender) smtp.mailfrom=urezki@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719345936; a=rsa-sha256; cv=none; b=qoS8ZkRZyqzymKBqp+oLsm4afpx+slVDg5Pjt9eHbc94c0VDfTAiZ6sK77yI4AOQ6QVXSX 1hzixTtIeXkOWjkFSqVBRJIoghXRLc0SjjjWheY+rgoB+Kw07fiR2v8VAs1XgM/ZCm+6FK 6uCR8RSWpeYj8F+PAvqBS9pUj1rxQjI= Received: by mail-lf1-f47.google.com with SMTP id 2adb3069b0e04-52cdc4d221eso4326720e87.3 for ; Tue, 25 Jun 2024 13:05:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719345954; x=1719950754; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:from:to:cc:subject:date:message-id:reply-to; bh=2J4hGqbcUzUN0UCwy+GvYQrnL7nzLlsNWsttcQw1Uqw=; b=jr9CUImt4x83DpzB819+DuC+W7Sb3ebO2X/T1Z/oZzVemAMiFXwO6f/TKJMY91rx2T 86D2ovK4KpRZg2VFrqpwI9Bh7KCSoYz85LncOb+5MEBSESNe7VuNYOp3WDf9HtJNykhn Qdm2qEFfaLwPWNmhvO7OpjG6ASkAqggh/+wEIOn9WP+BZubBTKKINbLN30CQc6EQm+Xe yLnm3lk4vufKyPZ0X3BFy5unxf2003aU/dt+gquP63S2zfPnKMsl4KjytgvOBDJZUaZz r62mlA83lcn81EnyTiIynTxYwMetLTQxOKRIS1RHRARmYUIHM/B8WsT0/M3Dux/4YC+E H66w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719345954; x=1719950754; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=2J4hGqbcUzUN0UCwy+GvYQrnL7nzLlsNWsttcQw1Uqw=; b=vbM1HApN5wX7BeR3KhrbOmcx2oN1QiKbxQiVzaEKB6+z1VlJpmzwHhJq4zbpBj1zv1 Edx+zyGLeLJeE383laN+xT8FyoC/8fBQCvTc9JXVZRxfq6A/ORx7mAHR61IaRfm8vWtC iptmB9Zts0VBY36cSSOkJYvn9If+PQmYFXDfd2bkwWX6qgEbq/w1PzN/uiU27gHyTQsX F+J2mKTahZZmjI3qdSHd7at4yH9JJMkNjHXkVueOu4ezIKboxtm77CrW7OKCVAKtRg/s fl6zT4hcjo5SJGWG55O91m43/IM/TFl0CPjuzXrQs1PEKZbyY6uPBqdXHKx2OfXnOEF1 U/oQ== X-Forwarded-Encrypted: i=1; AJvYcCV6xJhCJoCeROOxqMqauEGMdy2sqWT/DdRlS91KzodjUbon/N/wLQV2qRNYL0fBX/63L2xRSDBa6IX8jj0IjdrRGFQ= X-Gm-Message-State: AOJu0YynmxLbFJVX3SETMDHfAQhg2v6WysPqR8cqYYrrSFnn8L2n5olR 4/Elgg094upykNzLypgkj40eDw2sOKSucKFCnPtbYFI2rU4WfzZ+ X-Google-Smtp-Source: AGHT+IGRklP8XrpFFDL63kjNNjIm7656v8ToUlPmmfs60nq9WNExmpGsQL6esjys/LPNfYORPh2NXA== X-Received: by 2002:ac2:528c:0:b0:52c:df51:20bc with SMTP id 2adb3069b0e04-52ce18350e0mr4502213e87.16.1719345954075; Tue, 25 Jun 2024 13:05:54 -0700 (PDT) Received: from pc636 (host-185-121-47-193.sydskane.nu. [185.121.47.193]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a6fe5e27995sm416499666b.0.2024.06.25.13.05.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Jun 2024 13:05:53 -0700 (PDT) From: Uladzislau Rezki X-Google-Original-From: Uladzislau Rezki Date: Tue, 25 Jun 2024 22:05:52 +0200 To: Baoquan He Cc: Nick Bowler , Hailong Liu , linux-kernel@vger.kernel.org, Linux regressions mailing list , linux-mm@kvack.org, sparclinux@vger.kernel.org, Andrew Morton Subject: Re: PROBLEM: kernel crashes when running xfsdump since ~6.4 Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 5564B180016 X-Stat-Signature: cgqrgymjgh7rgtdupqrjgqb8uftit79e X-Rspam-User: X-HE-Tag: 1719345956-66098 X-HE-Meta: U2FsdGVkX1/4RaqD/ensDUXu/B83+D9Rd5170ncJ1krIp1pKEIIyRb3/ELDp2i94O6GQRCYFvcF5Frfsg9Wv2xDt+I4qVER/JvIpD9dp3A4HirlLw0G2AtSrHDwj4Juf+/pGkAz11s+xmfD6MH+nfv479jbaQap138R2b9qIZygZDGXr0RcoCYsSlMfAGIVF52xtZ7FAf+UA5mKbUoJVWXWWVJ6jLss6sTdw226mel3Cqo3If2hJ52n6zT7TZlVxQG6Kool7FkBN1qRyQwKR0O2hyyiPLyrTVxnofKDSg/n+KSIcUagE5O9Ewnt5mMvOusGZTKerwWMzqPCK7pIEI8dDD2CnhNTJ8EjobiD7DFc1NLwZncqClh9NowJj06y1ej0MQks5Worjjmm9/kVI2FDfuUvDlFZJUfttDBkWrbyXU04kH//hLf4iLWHULYrcM4y7NFDAdDM8HeIknc0gXFREYxHoTq3npaz+AgcbwKaJl1gKw7ThVgV4DuaYdLGZaQZ+fIMU6O34OIAa+358FDBO9oMhAt7ngIMIa8N2SUB3d2uTgBuaT3e+mMlqfRwZrz5T9p6+xzqhCvemibVDj69oFKtklHmVjZA8Y0WBvtZxrmgz+5z8DIEw9JBGL6rdks9SAbRs4V+NIqltMBEhUaLKzHHvJxUxZCPvoh3Tx3HcZn8VihbQEpogwGHcIKyJg47i9zdG/9uwflpsJoASSWu5pyuOiba+YvylXkpxmZJ0Dlu7nw1VgwCH4L9k61X40mgjPLIw60fzUxSVzF046a5uP3+HliobAVKtvS1MIfRw+wTFTO1DHU98lYLAs791+M47uzki5cxWdpcQO3lqLCrC+ftM08Ztc6FHSgctO2ssAL8x4O0wIy97x/myjajugYLqLKtn3JEKb+xHg/Y7sGCGYxCFjBiGRp0JuYptb6cH2aQzZf1ua9ymmqogh4gTa/GD5454z5/bbGB1ZEi 6O5TOPgs lTIZQDlVR0dF1bT/chebqL7byfrjeACwl1e4gC+F2fJISfLcaurfaukTgINp35TSchNc9knV3U0kkeM+RiaI8qXR6WGopGaiC39cZNZIGB2SeKce1w3jbMMF/2Jeg7EjCCbLYGz5TvnVrUAEvEdfkh/qTbpB+EGAtg6GKDTIe5Dud9eTEUXO+3/hZSJUNV6K/wuXgJ+0D+ydc434VwVTKA/W2G/qIapf7S3z7pYwocFOSPSQIlWS5SL9QGqJSIpw9yJVnpyTOoFe6Yd8Xs4q9st9T/vG0hraxGm4yb6J78WkTXPxkw+rGeXDFFRj7Zpnn8Z8BLH0IJaSSMM8on4zWKsxJMIrEaj+hFSqiZQRXGgNgrGSbS2cdPptoew== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: > > > > > /** > > > > > * cpumask_next - get the next cpu in a cpumask > > > > > * @n: the cpu prior to the place to search (i.e. return will be > @n) > > > > > * @srcp: the cpumask pointer > > > > > * > > > > > * Return: >= nr_cpu_ids if no further cpus set. > > > > > > > > Ah, I got what you mean. In the vbq case, it may not have chance to get > > > > a return number as nr_cpu_ids. Becuase the hashed index limits the > > > > range to [0, nr_cpu_ids-1], and cpu_possible(index) will guarantee it > > > > won't be the highest cpu number [nr_cpu_ids-1] since CPU[nr_cpu_ids-1] must > > > > be possible CPU. > > > > > > > > Do I miss some corner cases? > > > > > > > Right. We guarantee that a highest CPU is available by doing: % nr_cpu_ids. > > > So we do not need to use *next_wrap() variant. You do not miss anything :) > > > > > > Hailong Liu has proposed more simpler version: > > > > > > > > > diff --git a/mm/vmalloc.c b/mm/vmalloc.c > > > index 11fe5ea208aa..e1e63ffb9c57 100644 > > > --- a/mm/vmalloc.c > > > +++ b/mm/vmalloc.c > > > @@ -1994,8 +1994,9 @@ static struct xarray * > > > addr_to_vb_xa(unsigned long addr) > > > { > > > int index = (addr / VMAP_BLOCK_SIZE) % num_possible_cpus(); > > > + int cpu = cpumask_nth(index, cpu_possible_mask); > > > > > > - return &per_cpu(vmap_block_queue, index).vmap_blocks; > > > + return &per_cpu(vmap_block_queue, cpu).vmap_blocks; > > > > > > > > > which just takes a next CPU if an index is not set in the cpu_possible_mask. > > > > > > The only thing that can be updated in the patch is to replace num_possible_cpu() > > > by the nr_cpu_ids. > > > > > > Any thoughts? I think we need to fix it by a minor change so it is > > > easier to back-port on stable kernels. > > > > Yeah, sounds good since the regresson commit is merged in v6.3. > > Please feel free to post this and the hash array patch separately for > > formal reviewing. > > > Agreed! The patch about hash array i will post later. > > > By the way, when I am replying this mail, I check the cpumask_nth() > > again. I doubt it may take more checking then cpu_possible(), given most > > of systems don't have gaps in cpu_possible_mask. I could be dizzy at > > this moment. > > > > static inline unsigned int cpumask_nth(unsigned int cpu, const struct cpumask *srcp) > > { > > return find_nth_bit(cpumask_bits(srcp), small_cpumask_bits, cpumask_check(cpu)); > > } > > > Yep, i do not think it is a big problem based on your noted fact. > Checked. There is a difference: 1. Default ... + 15.95% 6.05% [kernel] [k] __vmap_pages_range_noflush + 15.91% 1.74% [kernel] [k] addr_to_vb_xa <--------------- + 15.13% 12.05% [kernel] [k] vunmap_p4d_range + 14.17% 13.38% [kernel] [k] __find_nth_bit <-------------- + 10.62% 0.00% [kernel] [k] ret_from_fork_asm + 10.62% 0.00% [kernel] [k] ret_from_fork + 10.62% 0.00% [kernel] [k] kthread ... 2. Check if cpu_possible() and then fallback to cpumask_nth() if not ... + 6.84% 0.29% [kernel] [k] alloc_vmap_area + 6.80% 6.70% [kernel] [k] native_queued_spin_lock_slowpath + 4.24% 0.09% [kernel] [k] free_vmap_block + 2.41% 2.38% [kernel] [k] addr_to_vb_xa <----------- + 1.94% 1.91% [kernel] [k] xas_start ... It is _worth_ to check if an index is in possible mask: diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 45e1506d58c3..af20f78c2cbf 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2542,7 +2542,10 @@ static DEFINE_PER_CPU(struct vmap_block_queue, vmap_block_queue); static struct xarray * addr_to_vb_xa(unsigned long addr) { - int index = (addr / VMAP_BLOCK_SIZE) % num_possible_cpus(); + int index = (addr / VMAP_BLOCK_SIZE) % nr_cpu_ids; + + if (!cpu_possible(index)) + index = cpumask_nth(index, cpu_possible_mask); return &per_cpu(vmap_block_queue, index).vmap_blocks; } cpumask_nth() is not cheap. My measurements are based on a synthetic tight test and it detects a difference. In a real workloads it should not be visible. Having gaps is not a common case plus a "slow path" will be mitigated by the hit against possible mask. -- Uladzislau Rezki