From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15DACC021B2 for ; Mon, 24 Feb 2025 01:43:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8393F6B007B; Sun, 23 Feb 2025 20:43:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7E93C6B0083; Sun, 23 Feb 2025 20:43:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6B00C6B0085; Sun, 23 Feb 2025 20:43:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 4D6A46B007B for ; Sun, 23 Feb 2025 20:43:15 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id F30FAC0620 for ; Mon, 24 Feb 2025 01:43:14 +0000 (UTC) X-FDA: 83153140308.09.932CD40 Received: from mail-qt1-f174.google.com (mail-qt1-f174.google.com [209.85.160.174]) by imf05.hostedemail.com (Postfix) with ESMTP id 27BCF100005 for ; Mon, 24 Feb 2025 01:43:12 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=j5CYVofg; spf=pass (imf05.hostedemail.com: domain of surenb@google.com designates 209.85.160.174 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740361393; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hs4/qefU1QtpU6+89lIxVt3csQSR47g1ulJgvu+2+tI=; b=rrKw+aufEO+T8jhbGr+MqFd8hNfDkJWofYG4cxH5rAHip3j7mp8unDUsNCatqexlPPClcx qxH9NfTmP3Z5Ky9E52lKraeveDmYSz2C+bAdLZ7ibpYjyGgoPto/4PTJYKmnMtwxzXTXuO Q4Vg58rpvVkmpsXoIlsyX9OEg+fglck= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=j5CYVofg; spf=pass (imf05.hostedemail.com: domain of surenb@google.com designates 209.85.160.174 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740361393; a=rsa-sha256; cv=none; b=vNeBfS64BoicCKmkeYcjkKlqOUb07NEEe0r69LiXF9WemxO6yFP8M+vM+151EXD7gIjioM CDgLEiYGm+07naJ+99nkeCLxkbo//0Gq2WIVufZoUtr9icORBWnRFYnzEBIn1ki2dmWofq VDxn6qkd1nMgb7DU0hs++OFSchkYCzw= Received: by mail-qt1-f174.google.com with SMTP id d75a77b69052e-472098e6e75so382311cf.1 for ; Sun, 23 Feb 2025 17:43:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740361392; x=1740966192; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=hs4/qefU1QtpU6+89lIxVt3csQSR47g1ulJgvu+2+tI=; b=j5CYVofg5UaZ1lkvbGU9hI/upNSb37YiA8vp2ZEztYR94xSb61fPvIltySOBwQ+hEL ipjlTA03dCujtGg+cIlML/y3EQScQoQLbcTvbQ+FJSRFUwtrwVsslKMcXJ5u/fLm6S2Y AQCdIhJuJzhldCfz5iwnbixqf+fUvblTsnvmvEx97B6T3oozwF52dnt05/4WhSSSyAm2 oMs3mtAkFEs12IC4qnKmQW1vOVsnXVUWrOlQ/CyVZjU8I+28q0fzSBgjDoE/B6pxxbxT 58Fg/OtEPIixxWPkPc37TF2QnbSfxtaXILj2DGtL7ZmDhXqOMVyUL4wGayLfxqoS4ROk 3kaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740361392; x=1740966192; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hs4/qefU1QtpU6+89lIxVt3csQSR47g1ulJgvu+2+tI=; b=XOjHzHcqwgl6tCAQucdjHviVbWEyFUcwBttERPpJk9yuutYlsXUGRP89pmMMDAUlQD GVjriiUIcEG43OJLGZaBCH2K3MYWHX1TGlZKedCR5Bj/x0GRgk96/rwjdhu1Ggy8r16w Xp2OGegT3Pzq1DZf+ilAnipcXyXtmF9xU9scOnRUHPJNJigN1lwAPdNQpByRS/+4usah in4D0niQz/m6UEvLyRSTb63y67tT7SYkixRRhy2gn7ZQJQZIqKUG8/pr4rFK+U00ZVvj IoIpVTWl7yVTKhW9Id+xMyNKBTq1GpR4hppoBaSrproTqq2wLu/TKlkpMHdAGyx82EtO 2CkA== X-Forwarded-Encrypted: i=1; AJvYcCX9EX9EUt2BwnsMwRd2NRZ5PTlwuOfIERv8QKfMoRnUyv/dvJnyAPHtkjfNDSRBidGIopo+NzUH6g==@kvack.org X-Gm-Message-State: AOJu0YzVbXG7fbqxwSEnAKVhtDJe+t3psgVciwU5ZJxH7b90pufxX5wr Y4UrhtRr//FYK5OwI25QtxesPTK1q83uckVGlwEXBUOeTgOZXErGUgmuBrLBLZTu2a7X7Qc77MQ DH6BEnaQdyklpet9+ZGcxxtAvmmXU47NRn41h X-Gm-Gg: ASbGncu+q5GqlcT6wrkFr1uSEeQKDaMoQpl2pEW/KH1yG9TJemsnvsZFlqY3wyew73u w2rYZTW09MAfaotRdGOQtl9Id0+r9VdVdXCObIUY3qGwaFwq48hL8/9bmbEMedMOMR/uYyNkitR 5a87h6c/Y= X-Google-Smtp-Source: AGHT+IF7PPzw9XcMzPgLGpS7L2yRjtVaj+8b437HR+N/kpecVcMii6hLGNH3rtRgLFLnrd2gUYIVZgktNFtVKAFAVsU= X-Received: by 2002:a05:622a:120f:b0:472:744:e271 with SMTP id d75a77b69052e-47233587224mr4504221cf.24.1740361391878; Sun, 23 Feb 2025 17:43:11 -0800 (PST) MIME-Version: 1.0 References: <20250214-slub-percpu-caches-v2-0-88592ee0966a@suse.cz> In-Reply-To: From: Suren Baghdasaryan Date: Sun, 23 Feb 2025 17:43:00 -0800 X-Gm-Features: AWEUYZmBheVTFZ7LsU1XyyiDk7lQ-8DjR4Kpy3qUz4phiAg_D9I2iBy4_sFiFc8 Message-ID: Subject: Re: [PATCH RFC v2 00/10] SLUB percpu sheaves To: Kent Overstreet Cc: Vlastimil Babka , "Liam R. Howlett" , Christoph Lameter , David Rientjes , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Uladzislau Rezki , linux-mm@kvack.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, maple-tree@lists.infradead.org, Sebastian Andrzej Siewior , Alexei Starovoitov Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Stat-Signature: ggdffesj1568s4ruggjfu8ys86d3ch58 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 27BCF100005 X-HE-Tag: 1740361392-633483 X-HE-Meta: U2FsdGVkX1+0EJaOPS8OxJJrHrz7nF8NTb9e+bjUZsqNlAyy7ow1hH+oJ+DR5d8D0BfSzUfdNqARqo+kw3yfb0Py3rMCOd2Y/+sQoveQWrnyoPMf6D17ZXEUJLqBzny5IrEFQdlQXlWcEy/SQQdBTx2tKmJm1b3ivhsVU8961oIdKi5KfiILBxr/qYPmEPqlS+BqVC2S234ZBGxLRdtGXZ4h9+x5laWBD+cIIoitTD8DsABo3E79ylNgzkcnT3qTj3VocB1IvLtNzAX9zrYTtM9AX+BeG+Xuj1se2uxIwf8bn54Wa/CPJ2q9JKSkY9tpOED+VzbD5aDjYHgqWVRw4vOvff7NNf3MpCsgH3LccFmG2nh4MMvyax0acGkTlzM+UNZ6P7rl4kO/CnOIB2D/Ak+OqMhf9skUGrfMyjXOtNf6lvXB2s8rXg54gf/Kd18sRd0NSVV9o2SmEIrxuXjxdKpaB+ieqa6JEtYAhNFiVmZFbtrRXaTLmQUXhBm//18V7cXpU1z+SYxKl9QU/v56UBpeOTBqq1qWoGxEIpwj6MgOrW3gz2u9sZGTRqBarJFO7OLoOgeVTS7TyNO4Gy9B5AWBBmx1m2kpJnPjD3WBt+ywREDyjsX89DjhCnoGtQvT5lsaWeWz7yHaX4YImtFMwGJG8dBPZbiduFc5u5m4JOs/mhnnOS2HeSaZL/9qVn8gVv7QjJnwZ8QAjIuiRgbAw5/h4nwgnr4WIyU42mBnXsccKLmuJAgEQkO5LFosu7hWEm+iKKmKddbPJlDU9joOgg+hSe8UXWjH2NZpm0iTfB9VTBWki1trdlqkfQwdzBCWbIpEHstNJhLKXKTPuwgGBK+w2cvQbPeP0tHQtZ9DdQ0POGSkfacBGNTiasvMcROhaTIXq8wX7OvXMEXcmcswhmtFvOEXYZuYYyjhR0eN6ffLGwdMxPjZCscj9sjZvt+iIbTkQ2a/228YNX1Q1iy luXCS7le VvkSjfbOs7QTy0clCUN+apORkS8TekxyP+Q9vOIQN5PUXZCUHw0cWMIVd++rj8rcYnal3Soqb1ynuO6ohcv8HW6ZcXgoaljsr1TqISZF9jqSGDRyuT1jCO5CsGv0GPn2kjdouCDU3DFca94XOz+uwrSw/Fd+Z6soR5UBEqG7Ro6rmMVkyWPnjMrG26wcmNfEfxaqXnwwHbkfj4HhiiWTMlTikr7yLFJjWSs9HGBljCoApGLQOkN2i6j6DPZweajW3c5PQl/wXFI8NWluct3BNU6wa4Q1a4RgWkvm+lSJfaNnJx43q+XMwiuhf4w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sun, Feb 23, 2025 at 5:36=E2=80=AFPM Suren Baghdasaryan wrote: > > On Sat, Feb 22, 2025 at 8:44=E2=80=AFPM Suren Baghdasaryan wrote: > > > > On Sat, Feb 22, 2025 at 4:19=E2=80=AFPM Kent Overstreet > > wrote: > > > > > > On Fri, Feb 14, 2025 at 05:27:36PM +0100, Vlastimil Babka wrote: > > > > - Cheaper fast paths. For allocations, instead of local double cmpx= chg, > > > > after Patch 5 it's preempt_disable() and no atomic operations. Sa= me for > > > > freeing, which is normally a local double cmpxchg only for a shor= t > > > > term allocations (so the same slab is still active on the same cp= u when > > > > freeing the object) and a more costly locked double cmpxchg other= wise. > > > > The downside is the lack of NUMA locality guarantees for the allo= cated > > > > objects. > > > > > > Is that really cheaper than a local non locked double cmpxchg? > > > > Don't know about this particular part but testing sheaves with maple > > node cache and stress testing mmap/munmap syscalls shows performance > > benefits as long as there is some delay to let kfree_rcu() do its job. > > I'm still gathering results and will most likely post them tomorrow. > > Here are the promised test results: > > First I ran an Android app cycle test comparing the baseline against shea= ves > used for maple tree nodes (as this patchset implements). I registered abo= ut > 3% improvement in app launch times, indicating improvement in mmap syscal= l > performance. > Next I ran an mmap stress test which maps 5 1-page readable file-backed > areas, faults them in and finally unmaps them, timing mmap syscalls. I forgot to mention that I also added a 500us delay after each cycle described above to give kfree_rcu() a chance to run. > Repeats that 200000 cycles and reports the total time. Average of 10 such > runs is used as the final result. > 3 configurations were tested: > > 1. Sheaves used for maple tree nodes only (this patchset). > > 2. Sheaves used for maple tree nodes with vm_lock to vm_refcnt conversion= [1]. > This patchset avoids allocating additional vm_lock structure on each mmap > syscall and uses TYPESAFE_BY_RCU for vm_area_struct cache. > > 3. Sheaves used for maple tree nodes and for vm_area_struct cache with vm= _lock > to vm_refcnt conversion [1]. For the vm_area_struct cache I had to replac= e > TYPESAFE_BY_RCU with sheaves, as we can't use both for the same cache. > > The values represent the total time it took to perform mmap syscalls, les= s is > better. > > (1) baseline control > Little core 7.58327 6.614939 (-12.77%) > Medium core 2.125315 1.428702 (-32.78%) > Big core 0.514673 0.422948 (-17.82%) > > (2) baseline control > Little core 7.58327 5.141478 (-32.20%) > Medium core 2.125315 0.427692 (-79.88%) > Big core 0.514673 0.046642 (-90.94%) > > (3) baseline control > Little core 7.58327 4.779624 (-36.97%) > Medium core 2.125315 0.450368 (-78.81%) > Big core 0.514673 0.037776 (-92.66%) > > Results in (3) vs (2) indicate that using sheaves for vm_area_struct > yields slightly better averages and I noticed that this was mostly due > to sheaves results missing occasional spikes that worsened > TYPESAFE_BY_RCU averages (the results seemed more stable with > sheaves). > > [1] https://lore.kernel.org/all/20250213224655.1680278-1-surenb@google.co= m/ > > > > > > > > > Especially if you now have to use pushf/popf... > > > > > > > - kfree_rcu() batching and recycling. kfree_rcu() will put objects = to a > > > > separate percpu sheaf and only submit the whole sheaf to call_rcu= () > > > > when full. After the grace period, the sheaf can be used for > > > > allocations, which is more efficient than freeing and reallocatin= g > > > > individual slab objects (even with the batching done by kfree_rcu= () > > > > implementation itself). In case only some cpus are allowed to han= dle rcu > > > > callbacks, the sheaf can still be made available to other cpus on= the > > > > same node via the shared barn. The maple_node cache uses kfree_rc= u() and > > > > thus can benefit from this. > > > > > > Have you looked at fs/bcachefs/rcu_pending.c?