From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07433C74A5B for ; Tue, 21 Mar 2023 07:46:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A3D3A6B0075; Tue, 21 Mar 2023 03:46:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9ECBE6B0078; Tue, 21 Mar 2023 03:46:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8B44E6B007B; Tue, 21 Mar 2023 03:46:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 7DC846B0075 for ; Tue, 21 Mar 2023 03:46:02 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 51EC4A1122 for ; Tue, 21 Mar 2023 07:46:02 +0000 (UTC) X-FDA: 80592121764.09.60F9D3D Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) by imf19.hostedemail.com (Postfix) with ESMTP id 742361A0010 for ; Tue, 21 Mar 2023 07:46:00 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=U5P6dkc4; spf=pass (imf19.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.42 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679384760; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WGNPqTjSMUqDfHrgW6kgUvbw3qknVLkyvvVwqcSq3M8=; b=haQboaEak4P2nrVpOcq1SrlXpztUc6xw41g7ry6pv8FyumglBNFimQ6MV01s6nKZpTvgiS 2okKH27OZgwHpNQflwgw48DaL94ccL/TwW2ishsm4ZsPEQ22nIPg9pISiQslnDR1Gp2dPn vLJf1PnZt3mxY3qCsdfRmiGe+606Rwg= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=U5P6dkc4; spf=pass (imf19.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.42 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679384760; a=rsa-sha256; cv=none; b=oMSv7iH8hCGkywhqVscDe9Ctl0oMHLXsZuHuL84ARIGcNF765sOkFaRhiiD4UiG4dN5rKk 6VAVyYmas5M8TbtD8s5Q76RikZJVOWICdtSX6oeHhGVvAIMarpDui2W3mMEpUwW4ZUOQ72 BWje1IU+HCfUP73b5Oy2OvBTVCGQYhc= Received: by mail-wm1-f42.google.com with SMTP id o11-20020a05600c4fcb00b003eb33ea29a8so8884021wmq.1 for ; Tue, 21 Mar 2023 00:46:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679384759; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=WGNPqTjSMUqDfHrgW6kgUvbw3qknVLkyvvVwqcSq3M8=; b=U5P6dkc4hbEDC/eQltNr2iI40AnzZGA5AIz637Rqk5+jaPXD9z2dIAd+F2YMI1opDp 8m0wLRKI280ryNuNwnqZxB3JEN6fjayqp4Xc+tzeUMUYjREhDTOYb+1BbmRlDF5jUbHj QlRURQ0Ghzr1+Q8BGWNAyWS/PZ1lBuDsuMdlFzRUFVF0uK6NsxM6vS23THO4+DU2bO4U 8Yw6UdqwgwSYIQuROvjkDYl1nmiHE43bXKuw0+o8LpgKXcLPXvx3pI9oU8SmxKK9BZ18 wws7FzZx4GkcwtOrC/ZfgzYJ24ndDuqpLa44vt+z61bjHB7xIrdFdazU6a/tAS17I1A6 FGMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679384759; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=WGNPqTjSMUqDfHrgW6kgUvbw3qknVLkyvvVwqcSq3M8=; b=pRD+lH92hXnyFzUo2edvvfYM6E2zhwdqkQX4JWax+6MksVOuN080UyqIFLWp2DkWnq 8xjqrNlYPijGR+S568HofrABARjNEQV3APs0ernY3wP7s2pcJat1B2xVfRNvJS9pkqWb PjHnNWU7b+vPxicEwQ188wP+IEAHWDbvFyJLk+SU8RHVBG+4NpaffZ14X2zBJevtXeNm 6qjFFplg+XMXxOlzrtrjhADOC8CnjVP55MdZgqZAq1sfWw+el/0bUWzWznioqVR6e0Uu 2CT/XPNnq4Vz+bsmeSqe05udknrGgrsgSB0zI2SEk3JSccsycQwGXTQaNBx6cFOaX54h DsTg== X-Gm-Message-State: AO0yUKX8pJJ00q4uOj5AI43ryAEkWgEUwYsmXk2gEU3sZtin+4DB5tf+ Ol29iyK6wZNY2OX1VPF9UW0= X-Google-Smtp-Source: AK7set995a5DnVD7LqGUABlMiy0yRpKpPaU+FTuy57OjMDKiF9VDwShiiFTEvhhWTpHc/8vvKPPzhg== X-Received: by 2002:a7b:c409:0:b0:3e2:19b0:887d with SMTP id k9-20020a7bc409000000b003e219b0887dmr1539717wmi.25.1679384758498; Tue, 21 Mar 2023 00:45:58 -0700 (PDT) Received: from localhost (host86-146-209-214.range86-146.btcentralplus.com. [86.146.209.214]) by smtp.gmail.com with ESMTPSA id p5-20020a05600c05c500b003edf2ae2432sm5496183wmd.7.2023.03.21.00.45.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Mar 2023 00:45:57 -0700 (PDT) Date: Tue, 21 Mar 2023 07:45:56 +0000 From: Lorenzo Stoakes To: Uladzislau Rezki Cc: Dave Chinner , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Andrew Morton , Baoquan He , Matthew Wilcox , David Hildenbrand , Liu Shixin , Jiri Olsa Subject: Re: [PATCH v2 2/4] mm: vmalloc: use rwsem, mutex for vmap_area_lock and vmap_block->lock Message-ID: <8cd31bcd-dad4-44e3-920f-299a656aea98@lucifer.local> References: <6c7f1ac0aeb55faaa46a09108d3999e4595870d9.1679209395.git.lstoakes@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 742361A0010 X-Stat-Signature: qkfbsh5tczopg8cjkknqoewo8x4bhtis X-HE-Tag: 1679384760-42549 X-HE-Meta: U2FsdGVkX1/xnnCWoe46JPNNoFB1vRjV14v+YB4XTs+yuu8Zd9M/E29d76kyRw/tCBwQh5brZbAenaqnKqvLoMvMwUJLaQ1M9m/3SNLQCEN2L5WTSlNiRiJWxm17fr8j/jz94MdcargTa/djTk/uL8G24mMwixoKKjO2pjXbyAL+6j5ol0h5qWVmHqn0ICTiyCCoHzmwjzyivPkgM/6ZitZfAqFF0nrec/2W3J3el7etv+neT1gV97217ZLlRAlLdc+kHvcfxm9IJf0I2iVh5GGdDpUHMMQFhFn/L04rxg561hSKr9shs8l1s/JZVW+wKsdutr2S1XxZ2YIfnYj4+F5Vo/2lHEZzdUbBfUR2EB9vWkRJ5ADyzSN6o7d3aij3wXkKBfSaAM9kZhRW//W0W6RdfaSXhlXcteJAWly5/rj77UAiTvmuGLLNTYP3WE3hfpOdrDwjmn/4FSYwUweuQJRbK3bQtzbFg95VGmVPXrbHK669hb2wE7jnDWBCvx7YxsWjb8p2N9S/4PLsgpqdLaPo11yrpl8fjNjhLws66q6CY2rAfkzMS/aDKSQ2IZEIxWAXTwFp7V6t70LyDgm69FMJdE1tm9NObcbySywLA99Me+XnfDombYj8Ge0f4E8ptfLvabdQztYkwCKnsxOwKmfXeBfmQJSomLT9JrOkQMgOrhUOn00A6VBtnHdvSQuLZXYSxvCzE4CZHDD6eVw5Wgb4JisgEXIfs4TnQn8YTztQBzV+CqIboriC2jIbxwjSSu1/LKmtSDwGfJNwMbSvhXrcSyguruBoPIE8kAxrcRm2h8s4texahsOgvmWZgbwPgaH5B9njiJUsZIOGCs5HHTaoF2U+u/sjO0PGGcEqjre6wMjeS8+d0jJK5lvRIncdRpOR92kcE0+AQKhc/FtCicQ+p/uYaM369L7FzRa86/7XpseG8HZCpgpg/83qusbxQ4BvVbiW8EMVt1Bo3l8 rBaUga/H mZ8Ek1Wc4E0EimU9YAxmzoYaN3ZuixCxUpHgsf4LhGQsVyllOVWYoMUMxs8qhzOygAMTcJa4T5zgkVW93di4uMvEMl284F+DjO8QguyHibGCDanHW1d7rZAesjYV6sON1KSMQjzMyfSwxNPePDzM4bdhMPg3QEaKFzHO/b0Ns8MR4/o8jsPvfe5Y3KnAoCw+kV2uEJcXqpXsd3WcDnbEju8gR7CDdrrQ01R9nTKlvZRb0seBLn1FbtYhFjs9nh8W4MvONyk9AfQftmrUV9TkEcKXmLjJybDsmECm22JriSVF7Zp236QyBf+/b3aKPH0p2cRZ1yNOK9D4q/G51M11aoPgPiatQO/W5dnKQazBrEn4X2m24eT7PY8xsGhUT2EQtb7z1366c8DvszjqgLXmW4+BWaDi2N7r+Pix8WMxa8oYKPT0AjH4X1WUJD7JIX29XhvGHNBXLxJlGMLaSJxoEreoBdsqJZKUI6LIH6Oe6QN9s5KoG5Lg5k6FrIxwkmiFvHPql0bkRAoguHc3g83b6j0V7fCJTtZRxKNsm X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Mar 21, 2023 at 06:23:39AM +0100, Uladzislau Rezki wrote: > On Tue, Mar 21, 2023 at 12:09:12PM +1100, Dave Chinner wrote: > > On Sun, Mar 19, 2023 at 07:09:31AM +0000, Lorenzo Stoakes wrote: > > > vmalloc() is, by design, not permitted to be used in atomic context and > > > already contains components which may sleep, so avoiding spin locks is not > > > a problem from the perspective of atomic context. > > > > > > The global vmap_area_lock is held when the red/black tree rooted in > > > vmap_are_root is accessed and thus is rather long-held and under > > > potentially high contention. It is likely to be under contention for reads > > > rather than write, so replace it with a rwsem. > > > > > > Each individual vmap_block->lock is likely to be held for less time but > > > under low contention, so a mutex is not an outrageous choice here. > > > > > > A subset of test_vmalloc.sh performance results:- > > > > > > fix_size_alloc_test 0.40% > > > full_fit_alloc_test 2.08% > > > long_busy_list_alloc_test 0.34% > > > random_size_alloc_test -0.25% > > > random_size_align_alloc_test 0.06% > > > ... > > > all tests cycles 0.2% > > > > > > This represents a tiny reduction in performance that sits barely above > > > noise. > > > > I'm travelling right now, but give me a few days and I'll test this > > against the XFS workloads that hammer the global vmalloc spin lock > > really, really badly. XFS can use vm_map_ram and vmalloc really > > heavily for metadata buffers and hit the global spin lock from every > > CPU in the system at the same time (i.e. highly concurrent > > workloads). vmalloc is also heavily used in the hottest path > > throught the journal where we process and calculate delta changes to > > several million items every second, again spread across every CPU in > > the system at the same time. > > > > We really need the global spinlock to go away completely, but in the > > mean time a shared read lock should help a little bit.... > > Hugely appreciated Dave, however I must disappoint on the rwsem as I have now reworked my patch set to use the original locks in order to satisfy Willy's desire to make vmalloc atomic in future, and Uladzislau's desire to not have a ~6% performance hit - https://lore.kernel.org/all/cover.1679354384.git.lstoakes@gmail.com/ > I am working on it. I submitted a proposal how to eliminate it: > > > > Hello, LSF. > > Title: Introduce a per-cpu-vmap-cache to eliminate a vmap lock contention > > Description: > Currently the vmap code is not scaled to number of CPU cores in a system > because a global vmap space is protected by a single spinlock. Such approach > has a clear bottleneck if many CPUs simultaneously access to one resource. > > In this talk i would like to describe a drawback, show some data related > to contentions and places where those occur in a code. Apart of that i > would like to share ideas how to eliminate it providing a few approaches > and compare them. > > Requirements: > * It should be a per-cpu approach; > * Search of freed ptrs should not interfere with other freeing(as much as we can); > * - offload allocated areas(buzy ones) per-cpu; > * Cache ready sized objects or merge them into one big per-cpu-space(split on demand); > * Lazily-freed areas either drained per-cpu individually or by one CPU for all; > * Prefetch a fixed size in front and allocate per-cpu > > Goals: > * Implement a per-cpu way of allocation to eliminate a contention. > > Thanks! > > > -- > Uladzislau Rezki > That's really awesome! I will come to that talk at LSF/MM :) being able to sustain the lock in atomic context seems to be an aspect that is important going forward also.