From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62764D743F9 for ; Thu, 21 Nov 2024 00:05:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DCB676B007B; Wed, 20 Nov 2024 19:05:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D7A5A6B0083; Wed, 20 Nov 2024 19:05:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C425B6B0085; Wed, 20 Nov 2024 19:05:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A3A036B007B for ; Wed, 20 Nov 2024 19:05:12 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 46815121095 for ; Thu, 21 Nov 2024 00:05:12 +0000 (UTC) X-FDA: 82808155710.24.53E05EB Received: from out-175.mta0.migadu.com (out-175.mta0.migadu.com [91.218.175.175]) by imf04.hostedemail.com (Postfix) with ESMTP id 840DE40006 for ; Thu, 21 Nov 2024 00:04:04 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=V+LqOzat; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf04.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.175 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732147325; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/3aB6iWKBZ4UsqZogpLqRAn1MMLH9eAxB//u6YU0jng=; b=27TUf9VT/EeUfw3LCY3Zz2osjXbXDqxd5WaH4qQzdt1LIVnPC/VosxEHku8yL5tWI6qNS8 Y7hWlBLHaG7QWpHuAmEb4bk50qIZIhBuoS1udKn3xC8k4l9bOarfessRsr78hEjvcm2aH+ tq/u21pLtlgUjjWAtuz9WgWhcx8wnk0= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=V+LqOzat; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf04.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.175 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732147325; a=rsa-sha256; cv=none; b=lJZYTNb8fCdDwW44KUl97wK1Rf4BnehUC39VpLnjv2GXR+JzdW4K7XJpTB46uoH9R62qJH YWOArbq2nVLnbnuKTGC9rzOr56vQHwfz38WatXgdisO00Yfecsef36gfKqSddNL0O2IT+H dWCQHYSiHSS9uhvW1Jp04bxRK9QJ2X4= Date: Wed, 20 Nov 2024 16:04:59 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1732147508; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/3aB6iWKBZ4UsqZogpLqRAn1MMLH9eAxB//u6YU0jng=; b=V+LqOzatU5Ivl6Cf2KD1FyV2ON79Lx4pFkr8issZZA6XRQhqAKVDx0eHlOzrhUPuHAh3IJ s0GL20YUSyYEXADVKco/jZ/mwE7i7BHXe9+t5ZByGMSysWD6DwuSkuWk7mCcDN01WZGgse s17wSYswscYvDA5lhSjB0FSoE+WpoEg= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Shakeel Butt To: Suren Baghdasaryan Cc: akpm@linux-foundation.org, willy@infradead.org, liam.howlett@oracle.com, lorenzo.stoakes@oracle.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mjguzik@gmail.com, oliver.sang@intel.com, mgorman@techsingularity.net, david@redhat.com, peterx@redhat.com, oleg@redhat.com, dave@stgolabs.net, paulmck@kernel.org, brauner@kernel.org, dhowells@redhat.com, hdanton@sina.com, hughd@google.com, minchan@google.com, jannh@google.com, souravpanda@google.com, pasha.tatashin@soleen.com, corbet@lwn.net, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@android.com Subject: Re: [PATCH v4 2/5] mm: move per-vma lock into vm_area_struct Message-ID: References: <20241120000826.335387-1-surenb@google.com> <20241120000826.335387-3-surenb@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Migadu-Flow: FLOW_OUT X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 840DE40006 X-Stat-Signature: t87ii98em4ah7tfx8cjequ7ktjn583gt X-Rspam-User: X-HE-Tag: 1732147444-794297 X-HE-Meta: U2FsdGVkX18JikODL5oLzbFvpoVW7TbJqXDnI8CH4QQ3TxXC7JKfnURYd+BVVex7zW7NaanKhRL6HPz/KGaBwqUuXTWRvPljR8ecQh2l+qiXiTA/0b9EXmlrvU6mIoaFTLop/AZnZ/FIevty9Z4estgcPJElq0BcWJmLH4a74qaDcQrSO6Pgclx8YrYs3uB42sntroP+1Nu+YQwn5+iCUm0WpATXscB3zYnnisrz23Vjnezk8q3dIAMFx55HpaIUAEorGZRosN/1/FN10MpMo1nWRMI6zPngeHed5KedY+fYTJee3NDkQQJVSMgnfR4247WgxUtIrDQ9tE245TuvaDe3YPAPPmrgIA4bPaDNmSH5Ed4b2BL2ahC7a2hr5Ea26YvTWFr6OqqG61JJk7x4vLWr85KDphG6q4PX0QVEWrPBEVWMxZscoUoudiKR8yHoBd+jsySDN4vIAUBAjlf74IpSdCGU/Cpm2F2UiwrzYED3xLrTV9mGzMOEiml9TQKCvEEyfKlsPOyT1NWa/joQIzQoetAC86Er1cQnJmJ4SYSQ3SYigh1lidsiI7n6+5jdzGBcRrc0RKwEw2NYZAK9URxq0Mys81pzHI6i7ADWAGoL/Z0cswZgKPg5oMAOtN6NIcy5ZF9UfLnpvaTkL0BqE0Bj8sGgur/zvca4BNVVk7rP8XVclGyIEM3mgsazoS/gYdvoRKgoPn5m7Bdt+97lxFAvIMw3ZGkYFt1IJ/kAQbtuX/snRy0VCd/fpX40g/yeFXHIgT5wGt/b1YZGskch+7DfkmEcQDFXxOCcS95+sxhwgHHo23p2rIBYmpFqdAaeUXLlbfYQo+xhwXIyMFaFDwOGuA/3web4OW5MEF3DbM+e6BWVKe64oCUzIML+3j68cfGHqsbb+6LIil9EgkP2eX5j7oa4HN0K/0A7mEgnuksOw+kkvhHaYP3bNaAWWDNmg+Dx1dghVx1lav3fX5R kzcJuFA1 6hdfdVJpaN4tYDHu1g9kYsNv0uRqNto4DVDHEWyTgQ0B41W9FcNFQqLuQpl/qiUP4ivlqxC2nlQDr782iuafp9njPwxL5d5J1lplgfpZqgrQA9gmO7SQo2aU11aIJz34RVzv6OdihgTHvpcfYRPY7ze3MCajIeNyeIRs6ECrxmpZhg/Iy6jxxm0iAi8VmI+Q9Ib+ISdnVndSfPl5K+1lIAGknOL57uIaZBYdc4liOSVDJ4HpfaLBoGj30RcQZkGpNXd850DojxX5+rc0kAMGh8u2PNaCGzQPM5J5gj3dG1wwW1z7nv9zJJVou8P9dlrVxcv8o X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Nov 20, 2024 at 03:44:29PM -0800, Suren Baghdasaryan wrote: > On Wed, Nov 20, 2024 at 3:33 PM Shakeel Butt wrote: > > > > On Tue, Nov 19, 2024 at 04:08:23PM -0800, Suren Baghdasaryan wrote: > > > Back when per-vma locks were introduces, vm_lock was moved out of > > > vm_area_struct in [1] because of the performance regression caused by > > > false cacheline sharing. Recent investigation [2] revealed that the > > > regressions is limited to a rather old Broadwell microarchitecture and > > > even there it can be mitigated by disabling adjacent cacheline > > > prefetching, see [3]. > > > Splitting single logical structure into multiple ones leads to more > > > complicated management, extra pointer dereferences and overall less > > > maintainable code. When that split-away part is a lock, it complicates > > > things even further. With no performance benefits, there are no reasons > > > for this split. Merging the vm_lock back into vm_area_struct also allows > > > vm_area_struct to use SLAB_TYPESAFE_BY_RCU later in this patchset. > > > Move vm_lock back into vm_area_struct, aligning it at the cacheline > > > boundary and changing the cache to be cacheline-aligned as well. > > > With kernel compiled using defconfig, this causes VMA memory consumption > > > to grow from 160 (vm_area_struct) + 40 (vm_lock) bytes to 256 bytes: > > > > > > slabinfo before: > > > ... : ... > > > vma_lock ... 40 102 1 : ... > > > vm_area_struct ... 160 51 2 : ... > > > > > > slabinfo after moving vm_lock: > > > ... : ... > > > vm_area_struct ... 256 32 2 : ... > > > > > > Aggregate VMA memory consumption per 1000 VMAs grows from 50 to 64 pages, > > > which is 5.5MB per 100000 VMAs. Note that the size of this structure is > > > dependent on the kernel configuration and typically the original size is > > > higher than 160 bytes. Therefore these calculations are close to the > > > worst case scenario. A more realistic vm_area_struct usage before this > > > change is: > > > > > > ... : ... > > > vma_lock ... 40 102 1 : ... > > > vm_area_struct ... 176 46 2 : ... > > > > > > Aggregate VMA memory consumption per 1000 VMAs grows from 54 to 64 pages, > > > which is 3.9MB per 100000 VMAs. > > > This memory consumption growth can be addressed later by optimizing the > > > vm_lock. > > > > > > [1] https://lore.kernel.org/all/20230227173632.3292573-34-surenb@google.com/ > > > [2] https://lore.kernel.org/all/ZsQyI%2F087V34JoIt@xsang-OptiPlex-9020/ > > > [3] https://lore.kernel.org/all/CAJuCfpEisU8Lfe96AYJDZ+OM4NoPmnw9bP53cT_kbfP_pR+-2g@mail.gmail.com/ > > > > > > Signed-off-by: Suren Baghdasaryan > > > Reviewed-by: Lorenzo Stoakes > > > > Reviewed-by: Shakeel Butt > > Thanks! > > > > > > > One question below. > > > > > --- a/include/linux/mm_types.h > > > +++ b/include/linux/mm_types.h > > > @@ -716,8 +716,6 @@ struct vm_area_struct { > > > * slowpath. > > > */ > > > unsigned int vm_lock_seq; > > > - /* Unstable RCU readers are allowed to read this. */ > > > - struct vma_lock *vm_lock; > > > #endif > > > > > > /* > > > @@ -770,6 +768,10 @@ struct vm_area_struct { > > > struct vma_numab_state *numab_state; /* NUMA Balancing state */ > > > #endif > > > struct vm_userfaultfd_ctx vm_userfaultfd_ctx; > > > +#ifdef CONFIG_PER_VMA_LOCK > > > + /* Unstable RCU readers are allowed to read this. */ > > > + struct vma_lock vm_lock ____cacheline_aligned_in_smp; > > > +#endif > > > } __randomize_layout; > > > > Do we just want 'struct vm_area_struct' to be cacheline aligned or do we > > want 'struct vma_lock vm_lock' to be on a separate cacheline as well? > > We want both to minimize cacheline sharing. > For later, you will need to add a pad after vm_lock as well, so any future addition will not share the cacheline with vm_lock. I would do something like below. This is a nit and can be done later. diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 7654c766cbe2..5cc4fff163a0 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -751,10 +751,12 @@ struct vm_area_struct { #endif struct vm_userfaultfd_ctx vm_userfaultfd_ctx; #ifdef CONFIG_PER_VMA_LOCK + CACHELINE_PADDING(__pad1__); /* Unstable RCU readers are allowed to read this. */ - struct vma_lock vm_lock ____cacheline_aligned_in_smp; + struct vma_lock vm_lock; + CACHELINE_PADDING(__pad2__); #endif -} __randomize_layout; +} __randomize_layout ____cacheline_aligned_in_smp; #ifdef CONFIG_NUMA #define vma_policy(vma) ((vma)->vm_policy)