From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8B78C76196 for ; Wed, 22 Mar 2023 17:47:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D1C7E6B0071; Wed, 22 Mar 2023 13:47:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CCCBC6B0072; Wed, 22 Mar 2023 13:47:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BBB036B0075; Wed, 22 Mar 2023 13:47:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id ACDB16B0071 for ; Wed, 22 Mar 2023 13:47:40 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 51F3B4045F for ; Wed, 22 Mar 2023 17:47:40 +0000 (UTC) X-FDA: 80597266680.12.E65F6A6 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf22.hostedemail.com (Postfix) with ESMTP id 312E0C0021 for ; Wed, 22 Mar 2023 17:47:37 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=kRa6Gmr6; dmarc=none; spf=none (imf22.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679507258; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5U7snE0+aYLeGUT8P2wlcKvFPCtGDHM/aotSxqaNJaY=; b=s+KhRuRjbcRVGGYR+++sopCjS+q0loX6q+jqyeAY8NuS2ess2vWOIXmWglHDt8RnuHRJOp 7ounX/RahBOruHgCUWkzlUSDw4ksG64aTSoFiHpPu+a7PclTQ3hDkPFwccF5/89SfPUEgZ gI6aoXpK1LzgNwpHDIAVzBKgOfzfMd0= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=kRa6Gmr6; dmarc=none; spf=none (imf22.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679507258; a=rsa-sha256; cv=none; b=GNGAM3IAOe+wtMJVloUTbVFj/yh2iwubZNj6QA5UlITQmSc74/jF9MagPuRRjlCX0ph/LF W+ZFZPQtw6FFw6omTXYXrZ98TwRYMKKrKkvh2OioYtJY6wPzUZIko5FELOatzBtgdYJPii VA1HuN2nCwOFOZGWTEmPRlcorjgDAXY= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=5U7snE0+aYLeGUT8P2wlcKvFPCtGDHM/aotSxqaNJaY=; b=kRa6Gmr61/CqeuJfus2d86d5LH 8I7TRNzJAuYyRJK3566ZY/aV8ud0L7Ni7asXjiIcUh5tzJpC3b6SSQIz7ePhJkK1+P0qqt2Zt8CT7 JMZDTsQaGvD9+rfE12U0lzjc6njS/SFjWftGekcZJVZV6CSNmXJTveemmLLB8VX5QptyEJOZ/Z3EV xSb7MtWcVltTHKnReTp5Fxcm31HwcO3vd07oB4lN455EauTZ8xm6cEdSTz3jDXf894+/8TCCXkR51 vyTO3PFdAmHkPwon04jRo8QFkrQJ+WqbPAL4O5vt2LG1crSZBeiksKLWFm8rYua1AKscPYo/RIhPj /yRJCxaQ==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1pf2Ya-003DFb-CA; Wed, 22 Mar 2023 17:47:28 +0000 Date: Wed, 22 Mar 2023 17:47:28 +0000 From: Matthew Wilcox To: Uladzislau Rezki Cc: Dave Chinner , Lorenzo Stoakes , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Andrew Morton , Baoquan He , David Hildenbrand , Liu Shixin , Jiri Olsa Subject: Re: [PATCH v2 2/4] mm: vmalloc: use rwsem, mutex for vmap_area_lock and vmap_block->lock Message-ID: References: <6c7f1ac0aeb55faaa46a09108d3999e4595870d9.1679209395.git.lstoakes@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 312E0C0021 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: 18wkb44rbcimup4jm76ucrp1fnt5jy7z X-HE-Tag: 1679507257-174150 X-HE-Meta: U2FsdGVkX1+tSmsSiqgwJBQS2Cz8qkiEmI2ocI6TjOzsCkPKnGBjN60WiXsUG4jJlcW4AjhmOFo3pK3fOv0tKo9GcsbbitgY8ECtipg0N1lAtrboV8MBnwU8oAWhfcrfGagxQgBJeklqWNnFsjVNBaj32UbzpzZv09keBErlf/sC316fjlz9FhqZHEPcfNJeK19EUx5dlpxzfy+fNHm54INEnFyiHSh5kYabb0w/nmlyJqrhcyqODsZa0APRj9F7zmxEeo04eReHfsxhxk8ES2T8spkYzh5KgMNXu7BzDtwMLOwzyZm/jptBPjdY/jiWEZQ3lZvlmEBm2ysotSLncDNookNeuJW5UHIHzxbXzns5am5T4t8aET/ivGkqyalxs5XnGg5aawTvF4EuMxUyHkhU5xE2nYlLj3uD3GL+xKTuaL2SGV5AO967cLWo0BhzTMq3GsddnUhnBlb+6DihZwgzsHVIyDndufoQCCvHEX/9M5DceoPRDkQpuqSlQGTKwDtbWirvozt8g/N1mCJO2r/ieOhwj0M+ZTQs6qXHXyqyhOoHiseXbqZ++dk4wgOS97W17zhEblw5zoTa/sqEut13gOYY1qTa8mdZGblYvqCCBDFomt6PITNtrXMz2YFpwuD8PzlYOw8JehQVMxqmybRSrzAMWcWum4XYxHjiHlaE6NgWkieeXkb5oOHKN122xe2KbAIf8Wj21G6Mxt41DA1lCxhL2Ss1UqpDZSdo4Wp6DJAujqd8XOrID0G9YJXlPW8yWnMeISTRk7/kA8lR5X7NaYRKy23Q+jLtPH6+cII/5EQU53+a0wPCSSFl7BsddLK9lDIsHAgWiJaUYbgtI1rb8FfTpjyZw+LIMxSwaz+BW3gXcKbbXHp06QBwHjdqokHWtvKMfdqlNX3MdSHHV54WFm8NFlORWpJ87/CSiq5aEBAsvK8nYjVFhOsWjnTYpDXQf5JyuDkJMFWmBsB HKjP9X8G m3M2cYb9k7ZEI4FkgDIUIKZPu2eGS/KjQtm+AKLm0oznUHQmELvAk4XVwrFDf1nFlAxfMw9d4F+PGMAppexF5HKFiJS0MrUnq3vycBmY+IwzzyZbSxllkQVTVK+H/DLBuNT61WHD3Q8ChjjTo4t85VsWdahTOWYEvfe567vO2JufO/JNf5mwq8Vl+yXMDugCxgqQ7HWEn1aXup6g= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Mar 22, 2023 at 02:18:19PM +0100, Uladzislau Rezki wrote: > Hello, Dave. > > > > > I'm travelling right now, but give me a few days and I'll test this > > against the XFS workloads that hammer the global vmalloc spin lock > > really, really badly. XFS can use vm_map_ram and vmalloc really > > heavily for metadata buffers and hit the global spin lock from every > > CPU in the system at the same time (i.e. highly concurrent > > workloads). vmalloc is also heavily used in the hottest path > > throught the journal where we process and calculate delta changes to > > several million items every second, again spread across every CPU in > > the system at the same time. > > > > We really need the global spinlock to go away completely, but in the > > mean time a shared read lock should help a little bit.... > > > Could you please share some steps how to run your workloads in order to > touch vmalloc() code. I would like to have a look at it in more detail > just for understanding the workloads. > > Meanwhile my grep agains xfs shows: > > > urezki@pc638:~/data/raid0/coding/linux-rcu.git/fs/xfs$ grep -rn vmalloc ./ You're missing: fs/xfs/xfs_buf.c: bp->b_addr = vm_map_ram(bp->b_pages, bp->b_page_count, which i suspect is the majority of Dave's workload. That will almost certainly take the vb_alloc() path.