From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56B3DE7719A for ; Sat, 11 Jan 2025 03:38:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D855D6B0089; Fri, 10 Jan 2025 22:38:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D36736B008C; Fri, 10 Jan 2025 22:38:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BD7006B0092; Fri, 10 Jan 2025 22:38:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9DB2E6B0089 for ; Fri, 10 Jan 2025 22:38:00 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 590F11C809E for ; Sat, 11 Jan 2025 03:38:00 +0000 (UTC) X-FDA: 82993762320.11.AA54FF4 Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) by imf13.hostedemail.com (Postfix) with ESMTP id 83EF320002 for ; Sat, 11 Jan 2025 03:37:58 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=1l3Yqy3g; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf13.hostedemail.com: domain of surenb@google.com designates 209.85.160.176 as permitted sender) smtp.mailfrom=surenb@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736566678; a=rsa-sha256; cv=none; b=O4as2fmoq26szdLxuf2CbM4SxjlnEzZFqj+XBuHQXodbdJRH7D9tnn0jKW730XF66XY0NV Ngpde6vy9L1rYbTNyFoV3yDLNX3o9YCp+y7XJJynmLV9q3X+F7aTwtZ78jtOWrNwfBI/Ln uO4JfmmHzsLYOWp6n/vS4AdTT/af4Gs= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=1l3Yqy3g; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf13.hostedemail.com: domain of surenb@google.com designates 209.85.160.176 as permitted sender) smtp.mailfrom=surenb@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736566678; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tjCNJifdnaYvSi1H/hMETBLfTiB5LfDbMbI0O7SYEXw=; b=wWcx9iahtA5mUqTKyjr2cfrOO2SNSVlUkwmmEud+GLSFzO4aHmSoP7VN45F9duImK2QXNi b4shcULMmJAYcrXoDxTD6M6oTnY67RYveVyiwRrREjVZk/GKZERPy2mtgj5jhjToguqIrw ih1eIwpiBvCbEOetKlkyThruyvvyERk= Received: by mail-qt1-f176.google.com with SMTP id d75a77b69052e-4678c9310afso105871cf.1 for ; Fri, 10 Jan 2025 19:37:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736566677; x=1737171477; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=tjCNJifdnaYvSi1H/hMETBLfTiB5LfDbMbI0O7SYEXw=; b=1l3Yqy3g0gQ6cRhSEj945wqA2ZFNvqXOwlTk6v39I3Jwty8dd0bqvTj2FC1PiuBCy2 Oe2CVIQWf8xd7obwuqtwpdJ60IcqLHP43tR4eZdnWa5ktVZaUt0CEqUMbwsXLui9WwrN vQyJ07z3O47V2cp5wJNhSTIEZoHn+uabe51d7skGf49uBgKOa0JoHgwImHY0EJxrmnt2 EUDhq5Dlw+UJtsJqUdGTL2NRpyLaDH8NyKypgS9t24h3d5aqU+5Qor/3RyJ5rQCgKS5d B8PLuaqwSiGqgS4B6ND82kmd3+Tnsv/bsL/q3yTzMA/jDEGVQetQRkZ3HNYmFj8ep+0B dUBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736566677; x=1737171477; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tjCNJifdnaYvSi1H/hMETBLfTiB5LfDbMbI0O7SYEXw=; b=PyHNptUPq3tisNleM5tYzEMUxs4W+w0pJ66+ec0OTJS7vfNb9K7WmhnG4T3xlHHW6C Sc6c3dC8OxXOCkUgRPDEio6ppjPt6O9ZR6bSFZfb08pTIewAdLJfFpw3vF0nx5fGp7bP e7cB/w5NNVRLpmrXGjkhASzPxQ0R5EaORVQR9yZGdNl3Thok01MGcOJP3B2wcet9p1Pi PBbkKgap7sdckgL0j683QXWQ3p6sVBjLo+9TSxwGf9GwXBXktsdFaAw385i2lTI+YosT cd7J8g4rgPquI2oXyO4jXvZgi4gogN2afOYdJFAyipykQbZKhDu3VsjJJAu31rrtFIZt t5uw== X-Forwarded-Encrypted: i=1; AJvYcCXaf4U6SIUjmp19PSY+ZgoAWM+lhyqm5gg7at7iT9Dry4XsV+LwB3iYHftAiMXZDkdTBrTULjldCw==@kvack.org X-Gm-Message-State: AOJu0YzmnpN6K6Q4qtBckhgKJB7YMVwSML2LfmtxyYVJ9ZEpIY3FgrbE rOJoQ/tBKyfMVTMA2fiuHOn4AC9V+52mcq5x3dhO9qqCX7bnxxuK9zcOd1sss3Bql0i+7C2Fc6o poiqplfAgzUgLlWgvS9Kt6AHNG5Muj7IKM2Vb X-Gm-Gg: ASbGncuGlcCqEDsYEIr/nDtD8aR+pgQflu4pV8xVE0eSz6bZ0tyg6aUqOPuP/xuv9Xw E3BvpRryX8APfo5hMBAQBwfR9dGWzXbW3Sn8C+MfArMtcpu8GZCOwiAj9wtMKnhNUPUQG X-Google-Smtp-Source: AGHT+IGLyqgJQvquEkjx2CpcGrG6fCsPbHEiuikqAXFIL/tAAzxHeh2iJ8qWzA3jd+rl4Cj1/HNIGhOyQa4PVfwnuQM= X-Received: by 2002:ac8:5dcb:0:b0:460:463d:78dd with SMTP id d75a77b69052e-46c87e0774bmr6072891cf.4.1736566677375; Fri, 10 Jan 2025 19:37:57 -0800 (PST) MIME-Version: 1.0 References: <20250109023025.2242447-1-surenb@google.com> <20250109023025.2242447-16-surenb@google.com> <41ebce1a-9cc0-471e-ac20-25efea9a933a@suse.cz> In-Reply-To: From: Suren Baghdasaryan Date: Fri, 10 Jan 2025 19:37:46 -0800 X-Gm-Features: AbW1kvYfbkvGmBZBVy4z_mtv76DnzMFiEmUC36kvKBERuiI_7frFWhcsZc7ZwvA Message-ID: Subject: Re: [PATCH v8 15/16] mm: make vma cache SLAB_TYPESAFE_BY_RCU To: Vlastimil Babka Cc: akpm@linux-foundation.org, peterz@infradead.org, willy@infradead.org, liam.howlett@oracle.com, lorenzo.stoakes@oracle.com, mhocko@suse.com, hannes@cmpxchg.org, mjguzik@gmail.com, oliver.sang@intel.com, mgorman@techsingularity.net, david@redhat.com, peterx@redhat.com, oleg@redhat.com, dave@stgolabs.net, paulmck@kernel.org, brauner@kernel.org, dhowells@redhat.com, hdanton@sina.com, hughd@google.com, lokeshgidra@google.com, minchan@google.com, jannh@google.com, shakeel.butt@linux.dev, souravpanda@google.com, pasha.tatashin@soleen.com, klarasmodin@gmail.com, richard.weiyang@gmail.com, corbet@lwn.net, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@android.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 83EF320002 X-Stat-Signature: iew1tf8ndzn7865i5fezcnnyxjs81cnr X-HE-Tag: 1736566678-95093 X-HE-Meta: U2FsdGVkX19VgdvVkFtOM+iQj+AsE7eowlJsoVOo9i5BbIAUi80oc914qmYZPLtDzuU0846D9qeNctQu+gGnQ9s36zAek23mPjgs/55iK6shi5hFpII58bK6RJpehxfuQ0uqzCb2Nad9PH/74AH7f7mGszRzlTpzX0BVsRqtDL0QkmT9JrReSSlSE6oClB1Ss4d/PajLPXYDp/Vtv4P9QnVVzvybnya2gIG5BfI56ax2eaR4j6Z3xOlUHBdMD65IlwG28QOxoce8PWZdgx+OPXGNFd+7dgfBUGgNHkC6fCGwRgltm578zw7N0ma50ZT2cQdVe01qdUaFTDSeYAPb84DqxvetPKAtvcwwRFIohq6ONk5tkgV+CJQslEUcIhkBKt01zR7r0k7PEpefzl7QeyPzR9IEegJyFlvi7I+SKfZYxNmByv0JFIqVch46Ka/tC93pk1lICulIHLR7H0zMzDmjeuGx6ezwEfkgBP/oYb6Gy+/QHn2p37Sh8Ha5FXADhZ6Lldes/15AyPpKMTYtrB/UwDieCKO52+t0/e74T+Yxtjx3GTpLHJomHu7D3rQgeD3xp867EwlZs4Lmo0OLODtjZEyS+onPIpgDb18Qp1ZmRzUo+pC9yHDCUa7uzC8jWQmkhzjBT03Wnbs6t0UsBHKzFYvPsl+10IQGy1muk3hxlUAgFPDFmf897PJa+FXVrc35jWI5Bg5PS2ct/2sQKK7bXEgtJsfvCwimGWKoZYgaM/3AGaepNVHErM4GzIx/0KeFKpo5LsxG4iMdhCvQBAt/PjZRuPuYi5R55wypCxXOV8PBtSFjiJB0Xt6gv2NnEHCRtWxFi1TUnxwMjR4vKjFZrLSOm63+DNcPfE4aIfg5R7uKL7po7y0lpklvh4INTe2lhPlVc04UolFHcUmaQ9mBg3vJGOpCCM0MhaPKvHjHEjEVV54t7xWSfVAFKkzOU4I/RYdUg3NubH4LnTj um19XvHZ kXHmd+ixjtMVeCYZnCduSeX+ij9DeyZH9krAtlNWAvUUYQxQ8Ucw3ePE8Skksgcr5IFQnQWpHN4AAZk50iixECGxR6nE4MEtNAFMBw7oVTGAv9hM9pE0zf2orjE61YsqJtjs8vA2aomjd4MoEYq1PCULJE8+5ps9TZTLMR2mSObOFAlN5z29LUUNY92kvTThPgp6G0rsC8LSQBYqPs7nC4saH4vR9QttBGMZ884MbugbNRYx4GvtahVcFBNGdyc3KYGmwUHGl6H6Bhc78OFFrxYO6SnqvJSgluelplvI0CwSlaHCSApR5qqhGWxiLScJ+61wSmmNqYpVV7yNj6M387T+yph+WElG8rlKH X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Jan 10, 2025 at 8:07=E2=80=AFAM Suren Baghdasaryan wrote: > > On Fri, Jan 10, 2025 at 7:31=E2=80=AFAM Vlastimil Babka = wrote: > > > > On 1/9/25 3:30 AM, Suren Baghdasaryan wrote: > > > To enable SLAB_TYPESAFE_BY_RCU for vma cache we need to ensure that > > > object reuse before RCU grace period is over will be detected by > > > lock_vma_under_rcu(). > > > Current checks are sufficient as long as vma is detached before it is > > > freed. The only place this is not currently happening is in exit_mmap= (). > > > Add the missing vma_mark_detached() in exit_mmap(). > > > Another issue which might trick lock_vma_under_rcu() during vma reuse > > > is vm_area_dup(), which copies the entire content of the vma into a n= ew > > > one, overriding new vma's vm_refcnt and temporarily making it appear = as > > > attached. This might trick a racing lock_vma_under_rcu() to operate o= n > > > a reused vma if it found the vma before it got reused. To prevent thi= s > > > situation, we should ensure that vm_refcnt stays at detached state (0= ) > > > when it is copied and advances to attached state only after it is add= ed > > > into the vma tree. Introduce vma_copy() which preserves new vma's > > > vm_refcnt and use it in vm_area_dup(). Since all vmas are in detached > > > state with no current readers when they are freed, lock_vma_under_rcu= () > > > will not be able to take vm_refcnt after vma got detached even if vma > > > is reused. > > > Finally, make vm_area_cachep SLAB_TYPESAFE_BY_RCU. This will facilita= te > > > vm_area_struct reuse and will minimize the number of call_rcu() calls= . > > > > > > Signed-off-by: Suren Baghdasaryan > > > > Reviewed-by: Vlastimil Babka > > > > You could also drop the reset_refcnt parameter of vma_lock_init() now, > > as the usage in vm_area_dup() should now be just setting 0 over 0. Mayb= e > > a VM_WARN_ON if it's not 0 already? > > Yeah, that's a good idea. Will do. Ugh, once I made this change, the newly added VM_WARN_ON() immediately triggered because vm_area_dup() does not memset(0) the entire vma and kmem_cache_alloc(vm_area_cachep) does not always return a reused vma. I could add a vm_area_cachep constructor to always initialize vm_refcnt to 0 but that would lead to more changes. I think I'll keep reset_refcnt for now and will add vm_area_cachep constructor as a follow-up optimization after this patchset is merged. > > > And a comment in vm_area_struct definition to consider vma_copy() when > > adding any new field? > > Sure, will add. > > > > > > + /* > > > + * src->shared.rb may be modified concurrently, but the clone > > > + * will be reinitialized. > > > + */ > > > + data_race(memcpy(&dest->shared, &src->shared, sizeof(dest->shar= ed))); > > > > The comment makes it sound as if we didn't need to do it at all? But I > > didn't verify. If we do need it in some cases (i.e. the just allocated > > vma might have garbage from previous lifetime, but src is well defined > > and it's a case where it's not reinitialized afterwards) maybe the > > comment should say? Or if it's either reinitialized later or zeroes at > > src, we could memset() the zeroes instead of memcpying them, etc. > > I see vm_area_dup() being used in dup_mmap() and I think this comment > is about this usage in case the src vma changes from under us. > However, vm_area_dup() is also used when we simply duplicate an > existing vma while holding an mmap_write_lock, like in __split_vma(). > In these cases there is no possibility of a race and copied value > should hold. Maybe I should amend this comment like this: > > /* > * src->shared.rb may be modified concurrently when called from dup_mmap(= ), > * but the clone will reinitialize it. > */ > > WDYT? > > > > >