From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9AE21C7EE29 for ; Thu, 25 May 2023 10:20:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 04F3B6B007D; Thu, 25 May 2023 06:20:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F19F3900003; Thu, 25 May 2023 06:20:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DBAA8900002; Thu, 25 May 2023 06:20:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C848C6B007D for ; Thu, 25 May 2023 06:20:41 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 929A4160B1D for ; Thu, 25 May 2023 10:20:41 +0000 (UTC) X-FDA: 80828383482.16.795F92F Received: from mail-lj1-f180.google.com (mail-lj1-f180.google.com [209.85.208.180]) by imf06.hostedemail.com (Postfix) with ESMTP id 96468180011 for ; Thu, 25 May 2023 10:20:39 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=EP4wH3Tc; spf=pass (imf06.hostedemail.com: domain of urezki@gmail.com designates 209.85.208.180 as permitted sender) smtp.mailfrom=urezki@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685010039; a=rsa-sha256; cv=none; b=S7gdxtCtvb8p3kyd3VbIvJOYMyY3JmWF/BiFX5CMvFOPV5ErGvXeZrpWW0aZIMle8QmF8g RPqiuzPI+vPIb9quIQ1eVpqR3pwtwvAu/WLJ1cARq+1OdUQ8rZPdue+vCzbq1PziBiQCgg 34Du1Xh/Z8MvqBnu24Muz7XLvn+GgJs= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=EP4wH3Tc; spf=pass (imf06.hostedemail.com: domain of urezki@gmail.com designates 209.85.208.180 as permitted sender) smtp.mailfrom=urezki@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685010039; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sKTxoI+93bAIesA0outxo0/LkqsnMtrIeM8EPXBLaGM=; b=GYGZecNIwqXrVVJ88g5BB4WdZuqy5wiIX+s/M+Sr5Wd1eJ6V7Qy4IAFfVySucE7xgef9PV dP6zhJDkR+iH5t+G7imiXSnvm/edINMR4uG3Bgd+kYTQnFCeDYLcdBuEaTGfZRYnHLlotn rP/Z03pdQuTaSnD4dzdtAPXmDADTZTM= Received: by mail-lj1-f180.google.com with SMTP id 38308e7fff4ca-2afb2875491so4057911fa.1 for ; Thu, 25 May 2023 03:20:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685010038; x=1687602038; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:from:to:cc:subject:date:message-id:reply-to; bh=sKTxoI+93bAIesA0outxo0/LkqsnMtrIeM8EPXBLaGM=; b=EP4wH3TcMzGQvrjNpxSmcpliomfcgcSpXFDtVbADG7nXItV0KO7c5H3ErQUKRcsLiv X8aGCGDERCQys08Mdu8fGaCufpXUP0kqQm1fJR9lKVq2yGash1b2UMUoHfup8iPmvBE4 Cg3+MLQbgP4GfVuxADVSHFnN/GoM/HRc9oGdX5WtoT0nd2zsn2L9uVFBatDvyXb/ObNi ZtQniUVQf4dlMYjdYLnXBp97gpHTZSnLIGzhbCmBqGsIv5OldhYNbNh5D/HR1JUHWI0B +CQCZ2kvE0aI6sbDdf8bFnEbmhNbBeVFpVzuCoyswkGY+hpdVcFwxdNDLROYEeA5UvYz PlUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685010038; x=1687602038; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=sKTxoI+93bAIesA0outxo0/LkqsnMtrIeM8EPXBLaGM=; b=aRrGUGkLoMM1YQrX7p5O609AlVPcVXUpTogkfbGCQrU+8wJSA/cl4nXhWvTM91l/XJ n4YVHI7rzc++rJJHl6EgzrYSnHhmYZFPLIqKd8/mMCky39hlagWLPYlwU3RLiXH6TJ8g VK5IJE7ad02DisYaFeUibSsJ1sqrRqbO+YUWSeykCRq+tSycPrefji4aSESzHqVF2OKl aOJ3HXpxzQ65Mgocq0outWjnvmS8s9iB0diPjNY+myuKNiOKmt7mcdy1aAcRblPrEqyT ewqXOfEzBhfjVsujoB9Z98B/Vjt0ir1zS9zas4Rt3lFGFdAI4p2ShUJhX1bNR/mXUHHC fsVg== X-Gm-Message-State: AC+VfDzaBQSWlcpE1a7G7uh/u5b38qK0+m7bDhoY+i4HX/MkHnfI9HEF vCHtCa4GCrUiqMe/rGM1Zfg= X-Google-Smtp-Source: ACHHUZ6cvCJ7fl4YemnOhhNmxtMKSJrQrkm0PvdEtVa52SjGhqXx97JNXm3GE91koWePebmkiftUCw== X-Received: by 2002:a05:651c:102f:b0:2af:b260:fd4d with SMTP id w15-20020a05651c102f00b002afb260fd4dmr1165908ljm.44.1685010037360; Thu, 25 May 2023 03:20:37 -0700 (PDT) Received: from pc636 (host-90-235-19-70.mobileonline.telia.com. [90.235.19.70]) by smtp.gmail.com with ESMTPSA id y28-20020a05651c021c00b002adf8d948dasm186282ljn.35.2023.05.25.03.20.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 May 2023 03:20:36 -0700 (PDT) From: Uladzislau Rezki X-Google-Original-From: Uladzislau Rezki Date: Thu, 25 May 2023 12:20:34 +0200 To: Dave Chinner Cc: Uladzislau Rezki , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Andrew Morton , LKML , Baoquan He , Lorenzo Stoakes , Christoph Hellwig , Matthew Wilcox , "Liam R . Howlett" , "Paul E . McKenney" , Joel Fernandes , Oleksiy Avramchenko , linux-xfs@vger.kernel.org Subject: Re: [PATCH 0/9] Mitigate a vmap lock contention Message-ID: References: <20230522110849.2921-1-urezki@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 96468180011 X-Stat-Signature: gkqsm7ak81u1saymhezg8qt9fawifcn3 X-Rspam-User: X-HE-Tag: 1685010039-858118 X-HE-Meta: U2FsdGVkX1+k6j4rYeTo+AZrvF8SqITZVorvHlZpL0TIY9ufCdSRJ4Hqzfe1/EvMbX9gwtAl3OcfWqbiir43SFCaA1RQrDTHjZ2on4M91A+c0290WKCDHNHAQVHTeiOuP4ZXM01u+e3GT/xv8mQDlqqtUBwzSBw75Devm8I3UhIx5Y2ovZkqMYWd2mlV+03ibuPxnpZKgCofXQmtfM4F4FW8tRONutfrJNRDa0+sd0u1emu7jf3VG3bbD5J+HT3ndTDRuadziXSRVMZc0yE+HVqO5dS62kWkRgYL37nHpiJwYvYtax7Qca7oMTvP7LBv9MSb/BrtlF6Ky/He43ndo8iWZN9K/Q1d72QkELEawn8YLE9JfgczEmw5xYGqYU2cVqz7eSLiJCL8N5pyJjbSix1zfDze36Sby9v4d39AfepuPBSxwme9m9KtAYx1HZjIl5bUHge6ehQAqTh1Q8CHbWj5YH2Aopm9h1Xc0aj31uyif6NxevQLoAyfefg0K5GGUxV4hgq+CRPqQFwsDp3RJO1N9d7vUTojEzboyM2CT8aQ9INO1VQCEDLnYIc/llEAjuKZZaHVwBc0R4O+CNYLbawfw66xkv/iig7LDcmCXSdW5b/Ie1D6dnxw2aAXUK106I6VS2+aKRI727n4uuyc3WQCS5nuYCtbFuUqRIZqb5sutaTMx9GDUHMQEcLfL4WXS0DbJJYBMEXJYN+wIggprJAP7oV3ucGYlYAC3YDY4VLxWalRAtoKarJ/seCnliPkRsDc97IChtsUAKzpjYcpIQX++/h9phd78hOh++LmmhzPoQBN/DBzgJof8fuyi+qbgzGRIOVWDCcjJQ5w9XKrgBcYDnyKy+kIjC+LrJ3OjuX1URYSuKLUKSq6oh5Xhpcv816bhcGm4fYVnBVgBxB9MfIhhesnCQxAicKCQY5zZrxz+B4whtH2NwKN1YBaj7OENcUqcPy8xcc+yRPEuCS h0QBB/bC 3VILoiORxvzdY4QsD1f7kDmY1SoDjUQDHr/wxOEV5bbKJn8k0f7eAZ4ou1YQfTYcJT3eRNmiSJDQW4dN0I6t+SaK9y8UC+K2Q4JzPzwOTX2OidkMscaLT+uIpywNFTrCLDKctgQgnt4KtvuiR6xcOjVovU0A767JKbWymrPBSUUc5kni2Cl4DYoiFd0T9Xose6RaJtzwKORfufOkndmHvTU25y+s8mIc/FN/FL+kbpIpDShOcCZTWtzg75eefjrHD/VOA99l8P92YyqEF4vZ6y9DAKWoxfP7iUq02hR+uTHrpjCtb2qbppf+x+QSE0i9NAcjcYpqlSzzS7yyzTLLJlRmZa/7EuHdKLe7znP14Wqxrun9JJcVN7lIdQAKQzsq0OZEIc2IrsAfuUeAf6g8mli4bpIuYRrHpkoAjL0iXj9EQa1hI74ihshkmpKZusiT46o9PKV+hKcCsDJPCuT9cJezBNi/4FiUE605c4e41KjKAeWWpue/Vjp6IJNYUoKygIyqockzgJqwp/epC3NsDIdud0+sFD2+rDU7tIA4Tpka2u+agWt09jF1qp8Zfb+815DV8ouktQTYqeVO0Qzulj31/KOcsUk+Ml25DNkHFkKw3XAqepJxNRncU0Z/RZby0LZJUrDp1xp0FlH0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, May 25, 2023 at 07:56:56AM +1000, Dave Chinner wrote: > On Wed, May 24, 2023 at 11:50:12AM +0200, Uladzislau Rezki wrote: > > On Wed, May 24, 2023 at 03:04:28AM +0900, Hyeonggon Yoo wrote: > > > On Tue, May 23, 2023 at 05:12:30PM +0200, Uladzislau Rezki wrote: > > > And I would like to ask some side questions: > > > > > > 1. Is vm_[un]map_ram() API still worth with this patchset? > > > > > It is up to community to decide. As i see XFS needs it also. Maybe in > > the future it can be removed(who knows). If the vmalloc code itself can > > deliver such performance as vm_map* APIs. > > vm_map* APIs cannot be replaced with vmalloc, they cover a very > different use case. i.e. vmalloc allocates mapped memory, > vm_map_ram() maps allocated memory.... > > > vm_map_ram() and friends interface was added because of vmalloc drawbacks. > > No. vm_map*() were scalability improvements added in 2009 to replace > on vmap() and vunmap() to avoid global lock contention in the vmap > allocator that XFS had been working around for years with it's own > internal vmap cache.... > > commit 95f8e302c04c0b0c6de35ab399a5551605eeb006 > Author: Nicholas Piggin > Date: Tue Jan 6 14:43:09 2009 +1100 > > [XFS] use scalable vmap API > > Implement XFS's large buffer support with the new vmap APIs. See the vmap > rewrite (db64fe02) for some numbers. The biggest improvement that comes from > using the new APIs is avoiding the global KVA allocation lock on every call. > > Signed-off-by: Nick Piggin > Reviewed-by: Christoph Hellwig > Signed-off-by: Lachlan McIlroy > > vmap/vunmap() themselves were introduce in 2.5.32 (2002) and before > that XFS was using remap_page_array() and vfree() in exactly the > same way it uses vm_map_ram() and vm_unmap_ram() today.... > > XFS has a long, long history of causing virtual memory allocator > scalability and contention problems. As you can see, this isn't our > first rodeo... > Let me be more specific, sorry it looks like there is misunderstanding. I am talking about removing of vb_alloc()/vb_free() per-cpu stuff. If alloc_vmap_area() gives same performance: diff --git a/mm/vmalloc.c b/mm/vmalloc.c index d50c551592fc..a1687bbdad30 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2503,12 +2503,6 @@ void vm_unmap_ram(const void *mem, unsigned int count) kasan_poison_vmalloc(mem, size); - if (likely(count <= VMAP_MAX_ALLOC)) { - debug_check_no_locks_freed(mem, size); - vb_free(addr, size); - return; - } - va = find_unlink_vmap_area(addr); if (WARN_ON_ONCE(!va)) return; @@ -2539,12 +2533,6 @@ void *vm_map_ram(struct page **pages, unsigned int count, int node) unsigned long addr; void *mem; - if (likely(count <= VMAP_MAX_ALLOC)) { - mem = vb_alloc(size, GFP_KERNEL); - if (IS_ERR(mem)) - return NULL; - addr = (unsigned long)mem; - } else { struct vmap_area *va; va = alloc_vmap_area(size, PAGE_SIZE, VMALLOC_START, VMALLOC_END, @@ -2554,7 +2542,6 @@ void *vm_map_ram(struct page **pages, unsigned int count, int node) addr = va->va_start; mem = (void *)addr; - } if (vmap_pages_range(addr, addr + size, PAGE_KERNEL, pages, PAGE_SHIFT) < 0) { + other related parts. -- Uladzislau Rezki