From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9EF42D711BC for ; Wed, 20 Nov 2024 15:54:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 004596B0088; Wed, 20 Nov 2024 10:54:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id ECF406B008A; Wed, 20 Nov 2024 10:54:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D48C36B0093; Wed, 20 Nov 2024 10:54:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id AE4836B0088 for ; Wed, 20 Nov 2024 10:54:30 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 3F6741C7502 for ; Wed, 20 Nov 2024 15:54:30 +0000 (UTC) X-FDA: 82806918054.01.3029D69 Received: from mail-qt1-f173.google.com (mail-qt1-f173.google.com [209.85.160.173]) by imf27.hostedemail.com (Postfix) with ESMTP id 8A8D840016 for ; Wed, 20 Nov 2024 15:53:33 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Mctkc69k; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of surenb@google.com designates 209.85.160.173 as permitted sender) smtp.mailfrom=surenb@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732117883; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7aS8lSIRxg0WP9jRTUfXo3i0w6S60egDeEb+mK1HEyI=; b=g1RC/zIc+NZxT00d4LL4tKGbSk9B4Dvit/R9a4jkMPLKu/O8WSrPMsGUbg2FLCPVjQyUhN 9StXhxrhu/fhVMM6iPVNxQow/AsH0nPW2kxZs/ou4Pq4OpK/p5owQDleq2/J+PbZY621rm rwZwXU9dtCWTHD63csGdzMi/1vBP/So= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Mctkc69k; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of surenb@google.com designates 209.85.160.173 as permitted sender) smtp.mailfrom=surenb@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732117883; a=rsa-sha256; cv=none; b=ecCv6mfmSvJj5mWiKevL7sR7iOkoK1YnRAyaqqMz110otp/BwhdkYfxKOFdDecIvKj6ifY PNWb2dX4Ix1aDegnCdIUN9tiDOmqIHbGC+PT5rnRLU1Jse4MGU77fBYWwmdwPlHvZ4IE7e N7zgukGphDAKe8b0OURAysl77JAjgxk= Received: by mail-qt1-f173.google.com with SMTP id d75a77b69052e-4635b03a3cdso507871cf.1 for ; Wed, 20 Nov 2024 07:54:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732118067; x=1732722867; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=7aS8lSIRxg0WP9jRTUfXo3i0w6S60egDeEb+mK1HEyI=; b=Mctkc69kcQgRKnrXGpkDPcSDLTCwTXupUdS7Do2Z7W9ApT03CFtZGLY7AojXMLUfxq ZILTsmXFF0MTEPSG9cR9cKPm6Xm4ZdjbBcI5cGFIprW9QFD2ALeCPUgzWz9kYyTIOdyp 9DPTRhvdmBltdcWVoo2AqRRKIAr/JjmmcpvJ+ewps/dQJ6vNE8F+tRE3Tg51nnEMCbMT zFhHBLvse5bWrTVaKXSGeMMMYyDVJ/TiGHjDifWwxNtIzNUHwE/CsQU5/+vyJKqC9wYt xMQHeMnyEclwvJ7xjPpedl48ntV4DCh7fI6fGXokZ1ZszC9sCpEV5GOh8vELDyuEtIxl TnmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732118067; x=1732722867; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7aS8lSIRxg0WP9jRTUfXo3i0w6S60egDeEb+mK1HEyI=; b=VQXWxaCUm2hcvh5RL6Y/q0f2sfDrog5DSAf3pOesb2u+FZ0W/qitcpPIUQ93kM4yKw vnKVZdjZkdCzCaPer9NndPAHg+t6tw3S2rDMN+a/bPWY0nIbF3/O280KeCev9J3nuj7l rA+T5Y+B0ordZLoqFxR5BQDhMh7vSIyqYFSLNfrCRB9csK2RSp4/qiysd/9jqfgWlaEy cTGIJhYwm9m+GZIexDbD8Ch9wdetXBOzm3Iqm4eKiSE4LEyBWso2V4r5IdgLv+23baTM k8vvIuTcMcrF5XCRJ76/EqQaWRTTuY98F6VUxHatop7RfPgadBP69BhjPWJyLobTA7bt BQ5Q== X-Forwarded-Encrypted: i=1; AJvYcCWCrkFfelbXPclDU9yYE48L46JOIVkiUymJDXxSH7eNKjBcbbuG5PnjEgXG/Rk+86RKIypPTTvtlA==@kvack.org X-Gm-Message-State: AOJu0YxutN3nmL2/Nd1zEbbUrip6E4H6OamfKhMvtowmi24/J+vTzD3P bb5ssUZW2Tc+clSSlnlKiU3Q1qF8M9K1bxZ3Uxd9vLNyz0m3jF5SfUeWzKoGLT7TCck+Euzsc48 ZvLz8WI6XMow4CtOjDbYAnvR/ERWZYlTIw8Mn X-Gm-Gg: ASbGncun6ccfMdN5AIXORFrwjUJtfJnYIp+bQX/c5EVcmShIm/Y6eE5KVarU4ayidEw pQAGcR6vme344UgE0b1bEhaCUajV+rwh8O44v2XJ1kdXD9NIn8lFrlITtN9BJqA== X-Google-Smtp-Source: AGHT+IHjkarCx+F0hcYKxhgZhANOrHBhOqvAWG4Bxw406DhR0f2zkhheZsfKiSDdM6g4dVF7Q5o38nXkwWZlAmq4Kt4= X-Received: by 2002:a05:622a:1823:b0:461:4898:8614 with SMTP id d75a77b69052e-464d36ed8f0mr3163811cf.27.1732118067061; Wed, 20 Nov 2024 07:54:27 -0800 (PST) MIME-Version: 1.0 References: <20241120000826.335387-1-surenb@google.com> <20241120000826.335387-5-surenb@google.com> In-Reply-To: From: Suren Baghdasaryan Date: Wed, 20 Nov 2024 07:54:16 -0800 Message-ID: Subject: Re: [PATCH v4 4/5] mm: make vma cache SLAB_TYPESAFE_BY_RCU To: Vlastimil Babka Cc: akpm@linux-foundation.org, willy@infradead.org, liam.howlett@oracle.com, lorenzo.stoakes@oracle.com, mhocko@suse.com, hannes@cmpxchg.org, mjguzik@gmail.com, oliver.sang@intel.com, mgorman@techsingularity.net, david@redhat.com, peterx@redhat.com, oleg@redhat.com, dave@stgolabs.net, paulmck@kernel.org, brauner@kernel.org, dhowells@redhat.com, hdanton@sina.com, hughd@google.com, minchan@google.com, jannh@google.com, shakeel.butt@linux.dev, souravpanda@google.com, pasha.tatashin@soleen.com, corbet@lwn.net, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@android.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 8A8D840016 X-Stat-Signature: biw5t5jakr53r5jxqmncmok5qgspm58c X-Rspam-User: X-HE-Tag: 1732118013-346861 X-HE-Meta: U2FsdGVkX1/xOCqvP4/1jGfL7N35fVkXK+6GKYWprj8BOOhN0lUNflvThF+/VJCNY+a/AX13Puv1ZYLemIaaPtq/efjJk4sjEezqPAW+FJ/OCQZ1cBtlJ8NIFQMYiRDAcY8pc9EYmuAgap5TYl12PP02mXV1BMGshSmcihwbaVE50uAYS/M9K+TTyWGrMvcwB/PZq/PP1VS64Z5YK4y4InPqYLvM+rll9SXm25Ti66oWZTRClj5yZFvd4AsiHymHA5yjfa9KWFaIalNaFiyb2iWJBMpXijSNB/o0B2H7REuu31U5cTWszfX0Lb4z5FKVQnQ/tK66wE2iXTWb0O5CHdWXquDstxDBFYwH0Uv3NDpXhA/zJLM3OUF6dAcsFjHBetrqxVKlqMNTDDjmRbYesSpZkWNARGpOG1nPAEuGiTIxxbWSupSi/NTltggkV68hj795mPrcf+aLJpqJX2d3ccmYXBHGCdz644NqzfMNzTmyilIaUZL1cYQ8IOAPut5a4DjFL8o4arokQmBP0DD7WNgZtdJCpW2/qtu/JEfEYIPpt+fJMB87qPFSc32MhI0Tp3oii1nNFO/MkSl15usfPEuzbKIscm+GyMwx3J5exOk4DDI3LXIcDiMwhumtXZBw5CiolMrr4Tzm61ODSoTwMYaWewx2IoGf5cEoULYlxQmBF1lXwsrs/3tupQH7sWLW9wcgw/H+mUxlcJu6JzGq+Ro6c5RNcgwklnZK3dMrIFNyj+obglxs+oNrYmhd4DcpIGbMj9kKeACwSFxxABy2zNj/oa2FC9zA9v6ckttuYHFOxSw7P1yu0Fm/jepMBRtXh0NKYjuRVSWP+b7SSH+poLYxzmHFGbWX0isJX3tefMYmmPMYhvh+OFI2mgrxCmFofoc/3tNVDm31WBb8a+QiqhcIeRHqEVOiHphQdQzuwRpAELQ0+ttfae734JAWel9nyxoZNgePN/JtYk4Dz2h GZ11DJko f4qqFnBlSxxFPEHlFkj7qBGlJTrAJnc1Lk3e+whCJLNUUo3ZEx8zoVouQa/1+X+7d57yCB7jpa8XGQ8dSVl8YNCnAQ6y/SI9GJSTMyw7BkbD0OqanHkbnTloRrRt1+0EUCxu1LEcgltnAXf1qsMxP1Oa6y+84t0rkvUiSLZOILrWAOxIb9BewnWzNtUHT+O7QIR/daCrKxHHytQ/y9l3K8kn5Nijh/brta2huyT1/10MgYl3zYPqEO2xD67mDz/WJ26gtQ30gI8HwA94= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Nov 20, 2024 at 2:16=E2=80=AFAM Vlastimil Babka wr= ote: > > On 11/20/24 01:08, Suren Baghdasaryan wrote: > > To enable SLAB_TYPESAFE_BY_RCU for vma cache we need to ensure that > > object reuse before RCU grace period is over will be detected inside > > lock_vma_under_rcu(). > > lock_vma_under_rcu() enters RCU read section, finds the vma at the > > given address, locks the vma and checks if it got detached or remapped > > to cover a different address range. These last checks are there > > to ensure that the vma was not modified after we found it but before > > locking it. > > vma reuse introduces several new possibilities: > > 1. vma can be reused after it was found but before it is locked; > > 2. vma can be reused and reinitialized (including changing its vm_mm) > > while being locked in vma_start_read(); > > 3. vma can be reused and reinitialized after it was found but before > > it is locked, then attached at a new address or to a new mm while being > > read-locked; > > For case #1 current checks will help detecting cases when: > > - vma was reused but not yet added into the tree (detached check) > > - vma was reused at a different address range (address check); > > We are missing the check for vm_mm to ensure the reused vma was not > > attached to a different mm. This patch adds the missing check. > > For case #2, we pass mm to vma_start_read() to prevent access to > > unstable vma->vm_mm. > > So we may now be looking at different mm's mm_lock_seq.sequence and retur= n a > false unlocked result, right? I guess the mm validation in > lock_vma_under_rcu() handles that, but maybe the comment of vma_start_rea= d() > needs updating. Correct. I'll add a comment about this. > > > For case #3, we ensure the order in which vma->detached flag and > > vm_start/vm_end/vm_mm are set and checked. vma gets attached after > > vm_start/vm_end/vm_mm were set and lock_vma_under_rcu() should check > > vma->detached before checking vm_start/vm_end/vm_mm. This is required > > because attaching vma happens without vma write-lock, as opposed to > > vma detaching, which requires vma write-lock. This patch adds memory > > barriers inside is_vma_detached() and vma_mark_attached() needed to > > order reads and writes to vma->detached vs vm_start/vm_end/vm_mm. > > After these provisions, SLAB_TYPESAFE_BY_RCU is added to vm_area_cachep= . > > This will facilitate vm_area_struct reuse and will minimize the number > > of call_rcu() calls. > > Adding a freeptr_t into vm_area_struct (unioned with vm_start/vm_end) > > could be used to avoids bloating the structure, however currently > > custom free pointers are not supported in combination with a ctor > > (see the comment for kmem_cache_args.freeptr_offset). > > I think there's nothing fundamental preventing to support that, there was > just no user of it. We can do it later. Oh, ok. I can add it back so that we have one user and then when the mechanism is implemented it can be used for testing. Adding freeptr_t has no negative effects and will reduce later churn. > > > Signed-off-by: Suren Baghdasaryan > > --- a/kernel/fork.c > > +++ b/kernel/fork.c > > @@ -436,6 +436,11 @@ static struct kmem_cache *vm_area_cachep; > > /* SLAB cache for mm_struct structures (tsk->mm) */ > > static struct kmem_cache *mm_cachep; > > > > +static void vm_area_ctor(void *data) > > +{ > > + vma_lock_init(data); > > +} > > + > > struct vm_area_struct *vm_area_alloc(struct mm_struct *mm) > > { > > struct vm_area_struct *vma; > > @@ -462,8 +467,7 @@ struct vm_area_struct *vm_area_dup(struct vm_area_s= truct *orig) > > * orig->shared.rb may be modified concurrently, but the clone > > * will be reinitialized. > > */ > > - data_race(memcpy(new, orig, sizeof(*new))); > > - vma_lock_init(new); > > + vma_copy(new, orig); > > INIT_LIST_HEAD(&new->anon_vma_chain); > > #ifdef CONFIG_PER_VMA_LOCK > > /* vma is not locked, can't use vma_mark_detached() */ > > Here we mark it detached but we might have already copied it as attached = and > confused a reader? Very true. Thanks for catching this one! > > I think this will be covered by what you said in reply to willy: > "vma_copy() will have to also copy vma members individually." Yes, I think so. vma_copy() will need to copy most but not all members. vma->detached will be among those not copied. Thanks! > > > @@ -475,32 +479,37 @@ struct vm_area_struct *vm_area_dup(struct vm_area= _struct *orig) > > return new; > > } > >