From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4AA6C77B7F for ; Tue, 24 Jun 2025 19:34:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C285A6B00BA; Tue, 24 Jun 2025 15:34:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BD8876B00BB; Tue, 24 Jun 2025 15:34:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AC7F66B00BC; Tue, 24 Jun 2025 15:34:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 98C9D6B00BA for ; Tue, 24 Jun 2025 15:34:20 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 457BD105B94 for ; Tue, 24 Jun 2025 19:34:20 +0000 (UTC) X-FDA: 83591295480.17.1098B2D Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) by imf17.hostedemail.com (Postfix) with ESMTP id 6A4AF4000C for ; Tue, 24 Jun 2025 19:34:18 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=yoPdQ49T; spf=pass (imf17.hostedemail.com: domain of 3uf1aaAYKCGoacZMVJOWWOTM.KWUTQVcf-UUSdIKS.WZO@flex--surenb.bounces.google.com designates 209.85.210.202 as permitted sender) smtp.mailfrom=3uf1aaAYKCGoacZMVJOWWOTM.KWUTQVcf-UUSdIKS.WZO@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1750793658; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1BERKeqLlcPDyrSUuZIHMWPi0iaWreXsjzz5dAn8mkk=; b=sgoyGBloUKtdirtf72orzKNj9QRCz4Uvmh9JC4yC0Og1ezwolOtChZg7H82RS45uAYwp39 iGBrgQkxctU8wr8hJPeXr9BMruzk/2SLdMnzFJT22h6zjZPiLX44RjPo662F+o684/+akz 385k8dpzajSsYedGwqUdIA8cquUeQeg= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=yoPdQ49T; spf=pass (imf17.hostedemail.com: domain of 3uf1aaAYKCGoacZMVJOWWOTM.KWUTQVcf-UUSdIKS.WZO@flex--surenb.bounces.google.com designates 209.85.210.202 as permitted sender) smtp.mailfrom=3uf1aaAYKCGoacZMVJOWWOTM.KWUTQVcf-UUSdIKS.WZO@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1750793658; a=rsa-sha256; cv=none; b=Q0dMIJdWLvjBNn7R2xnCsfq+1VFBiU6mBDhmx1Dmw5qJOWlOtKSVpQ0/mO46l1XGagPKZt tjd2vuZVV18QRn7HZ0wDoUByE/LbmPSsfvgN3NgRGdQ1+HVsQy5IKg+s0pc3org6VJho8S SNGwpPhACbG0ymBogmOjlCX6lLrPAOc= Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-747dd44048cso610443b3a.3 for ; Tue, 24 Jun 2025 12:34:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1750793657; x=1751398457; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=1BERKeqLlcPDyrSUuZIHMWPi0iaWreXsjzz5dAn8mkk=; b=yoPdQ49TzdXmNo0MlwxiQfdu0isi49pCQ34oWjeSDncJjEXnkLCOS4XQ1kZ7NerQjD dzCcq8xDkA3yNzybKj4tEJ/xML+YnA35tP3KO/M2Bbuaos6LsyM2QTaN9qC0n/oSMXYO LRAQ0vqwRO+RdXlgrEILlI333BRuHPRer3CFatH6DaKz//aPKh3LNRSKD1GEd6xTZ+o4 xEA45VjJqeIpcgY3CeSP33aQXY1ZWmLSeXxxYgl7+MriVoTJtQXq+Pa+SdgE+cWgfF3f jfrfmIdWr6WUFJS24o7k2RsyiD/+R1XrDlt/nzamgScvCN6yXVECkTKOUxsYZEVlUcSh d8KQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750793657; x=1751398457; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1BERKeqLlcPDyrSUuZIHMWPi0iaWreXsjzz5dAn8mkk=; b=QHvdyjKWTEbeqX83gDT6A5AkwzbT0IvANYGwpEHWU5AiUeKFdkA6Z/I/W485cHqJpw JKjPCHlqEW6QEBhq01Ycd/pLDAeeHHA2hIsg+yeOmOVxiEn8k+hPyHzvgNXJ0jvGWxzW jNspLs2UJgbBl5PwOqxipXzthsuOTGvzRbacr0q8ZHxii3mya4Sasop/BYbFlQb8wgqQ oVRhZB6R4hrlDjmxIotKPAndw21elJqkUkx/IY3n0ycH0J+H7W6vuD5sQfOvAl9Qcwi5 y/drHw1hVjBlP8AQanSq3Rs3xcrToWOprpks0iQF6vg9tt73krMKotOeDAl9sokEH5Le Dmpw== X-Forwarded-Encrypted: i=1; AJvYcCXPDs0sjS4GDtyQM9OZV9xlVl7bWztp6gR9UlDCdjS/HcVFntVhPjA8DaIaxZFy9kbLPh93muFGsQ==@kvack.org X-Gm-Message-State: AOJu0YxITPjkfbE3i8MfvlfObNkQYC9exr4wtB61+V3z9vd5Qp54TeeD 9U8DPxlK8NdmvxkZJ5BtjEkG8Ysxkabc33bbn0ucM+gUvtELPMamWa7VTYp0McVmecgKLnbvPNf yutecdA== X-Google-Smtp-Source: AGHT+IHtLljtrcUhHYC7Wt+7WKI/a5PDJ4OrDvfWNeRaq5sGEKU3TQKgRzLNMXpoHYXyrtGwyT6oPA3hh2M= X-Received: from pfrh7.prod.google.com ([2002:aa7:9f47:0:b0:748:4f7c:c605]) (user=surenb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:6f8b:b0:220:7b2e:5b3f with SMTP id adf61e73a8af0-2207f27d61amr271750637.19.1750793657223; Tue, 24 Jun 2025 12:34:17 -0700 (PDT) Date: Tue, 24 Jun 2025 12:33:59 -0700 In-Reply-To: <20250624193359.3865351-1-surenb@google.com> Mime-Version: 1.0 References: <20250624193359.3865351-1-surenb@google.com> X-Mailer: git-send-email 2.50.0.714.g196bf9f422-goog Message-ID: <20250624193359.3865351-8-surenb@google.com> Subject: [PATCH v5 7/7] mm/maps: execute PROCMAP_QUERY ioctl under per-vma locks From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: Liam.Howlett@oracle.com, lorenzo.stoakes@oracle.com, david@redhat.com, vbabka@suse.cz, peterx@redhat.com, jannh@google.com, hannes@cmpxchg.org, mhocko@kernel.org, paulmck@kernel.org, shuah@kernel.org, adobriyan@gmail.com, brauner@kernel.org, josef@toxicpanda.com, yebin10@huawei.com, linux@weissschuh.net, willy@infradead.org, osalvador@suse.de, andrii@kernel.org, ryan.roberts@arm.com, christophe.leroy@csgroup.eu, tjmercier@google.com, kaleshsingh@google.com, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, surenb@google.com Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 6A4AF4000C X-Rspam-User: X-Rspamd-Server: rspam07 X-Stat-Signature: oo9t1c8jxdqs3wkjj39ddgajeaqto777 X-HE-Tag: 1750793658-223611 X-HE-Meta: U2FsdGVkX1+OHi/Z0Ob1A2Y2ZbYEy7QfRrt+M9+d8mj4CSku7fQ8sLsionp7fCotc0CNW5dyMH8rPC507s2ajWl0/PdKDxNwYzeCes9Py8kZ7J+4YcfgngLhgq7jgI50drLHcSul8MPrEoE6aRW2H1K6mJ9YpsupzhMxS5tfYjcKWGAHYvpXOCpx0bL2CLqKH6idt1NF1O002odlenTYdtLYTIiuomO5ViRkaCrPNuyZHqDliIVdE6l1oX3r/1pu1yt3HjKBz2l5LM1igsE3mF2yFZYGYkXkmkcsadZr1xlafhKqnj2ZverWVIsTdv2OqtHdVqBUZ+v/VgmhS/T+Grn9UiCyXt16pTHpx7d5oHVOtOHIMX0cOuH9/OGK32WrpO7jMg4b2FZyhQ/26CM7cW6NF5O2P0oIBhu//pOJRqTVYwvAnsN0dv8KhpX286z15+QUAssg6BuNs39QhJBxj/GoPNvnDp1+yVAQhlqHFTnt2ItUVstdsrO86qwR64VagcySWiyOHcbPry907535YoduDrv5FrrSyECj6joM6UidCkq4aHt1JowJ9sTrRZXIwN5cp7AmLg+agJBhf7GTXKsTU4cVJr7xn/UDj77LukEU7GgMQzIUZLhj6nxi+Rg3TxMBNa24eNe8qQHR7HXpvK31rOIYf5g2ns33LuJgxQ/Vai37xeOJFV7LgG45v0a7pjkcsKHNGeBSlUOl/sKY60WdCl9HpyKBa+ML7wi/xQHzmve8akMCiBzik3e7gkfyvoKE6JWsNSPMr3gG0L/Q108sRIFfYjj+5/V42e0g0dVR4uOt4y42v+xhxDVI9ZGQPF03SsVdrThud9XQgmeIN3oQ3Dn5/cZatYvYGgT5Ub8Ihi9++l7ujOtv58AlDOoVODkufB6Mut1Eb28Fh7vu6/O2cpxg1LDq7FMlIovTP4c1VDaCHPcq2FfqZ4RaaPDOUSU9ovL8JRuzSjaYcJk l4avJ7wv k7CCUvjri058AUEdrw0ykscEeU25aDrL3kdnodQ+f/EFn4R8UfGo/GEcRcAJvvrOUUBCot+RXV5SUHl04Nj1+vLeftqyCq2GnhsIjKiop+Rk35pHqLk3e/zuDpd56P265ykLPOifmoRT0q+lKwjnNC0cvwp1JhknEO/aXAP4zExTY7YQdwRrVuIEMBoeII45nOioMAW6LWbupuzjrVY8xk4j1Cs0Nj6IlMvhuJGTg4Qn7oJPuPHN/dv/HS57LI6p8eoEzBX0rMr0Ts2YIGfEcfx6gvn4IL9d8R9JLM2VvKuhVz5dvo25rjF7sPfGY9sFFeBCAc22rx6pCgF1s8sSNfO7Ym1lq5QfGjUpexT3XpXCf/Dc8fkoOMwgliBcmQEcPwq5VsrE6praEk8shJUTBBHKQcEKHNvL/FG8N7JzASnXUynfStrcUCFELQsh6cTxCzR9yBFPHLs4WOvPdpjPasqQLzfZyvEoKc2inRpg9E23cmdT8Czh0+IRuSRrAEZ5d+/RsqqrNJmYF8KiRQGn1D1CtB4ifxapf0RVSUC8tU3c6rDvrKj7VFasyjiam4Ar1Xz074V92HUi4582RNi6B1xVOcLkbJb2RBJmXlEYTRqtKW5Qo7aFl26DFhWcjMP6YrmlgR54yW8GCQ3ykFNE15nYTCQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Utilize per-vma locks to stabilize vma after lookup without taking mmap_lock during PROCMAP_QUERY ioctl execution. While we might take mmap_lock for reading during contention, we do that momentarily only to lock the vma. This change is designed to reduce mmap_lock contention and prevent PROCMAP_QUERY ioctl calls from blocking address space updates. Signed-off-by: Suren Baghdasaryan Acked-by: Andrii Nakryiko --- fs/proc/task_mmu.c | 56 ++++++++++++++++++++++++++++++++++++---------- 1 file changed, 44 insertions(+), 12 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 33171afb5364..f3659046efb7 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -492,28 +492,60 @@ static int pid_maps_open(struct inode *inode, struct file *file) PROCMAP_QUERY_VMA_FLAGS \ ) -static int query_vma_setup(struct mm_struct *mm) +#ifdef CONFIG_PER_VMA_LOCK + +static int query_vma_setup(struct proc_maps_private *priv) { - return mmap_read_lock_killable(mm); + rcu_read_lock(); + priv->locked_vma = NULL; + priv->mmap_locked = false; + + return 0; } -static void query_vma_teardown(struct mm_struct *mm, struct vm_area_struct *vma) +static void query_vma_teardown(struct proc_maps_private *priv) { - mmap_read_unlock(mm); + unlock_vma(priv); + rcu_read_unlock(); +} + +static struct vm_area_struct *query_vma_find_by_addr(struct proc_maps_private *priv, + unsigned long addr) +{ + vma_iter_init(&priv->iter, priv->mm, addr); + return get_next_vma(priv, addr); +} + +#else /* CONFIG_PER_VMA_LOCK */ + +static int query_vma_setup(struct proc_maps_private *priv) +{ + return mmap_read_lock_killable(priv->mm); } -static struct vm_area_struct *query_vma_find_by_addr(struct mm_struct *mm, unsigned long addr) +static void query_vma_teardown(struct proc_maps_private *priv) { - return find_vma(mm, addr); + mmap_read_unlock(priv->mm); } -static struct vm_area_struct *query_matching_vma(struct mm_struct *mm, +static struct vm_area_struct *query_vma_find_by_addr(struct proc_maps_private *priv, + unsigned long addr) +{ + return find_vma(priv->mm, addr); +} + +#endif /* CONFIG_PER_VMA_LOCK */ + +static struct vm_area_struct *query_matching_vma(struct proc_maps_private *priv, unsigned long addr, u32 flags) { struct vm_area_struct *vma; next_vma: - vma = query_vma_find_by_addr(mm, addr); + vma = query_vma_find_by_addr(priv, addr); + if (IS_ERR(vma)) + return vma; + if (!vma) goto no_vma; @@ -589,13 +621,13 @@ static int do_procmap_query(struct proc_maps_private *priv, void __user *uarg) if (!mm || !mmget_not_zero(mm)) return -ESRCH; - err = query_vma_setup(mm); + err = query_vma_setup(priv); if (err) { mmput(mm); return err; } - vma = query_matching_vma(mm, karg.query_addr, karg.query_flags); + vma = query_matching_vma(priv, karg.query_addr, karg.query_flags); if (IS_ERR(vma)) { err = PTR_ERR(vma); vma = NULL; @@ -680,7 +712,7 @@ static int do_procmap_query(struct proc_maps_private *priv, void __user *uarg) } /* unlock vma or mmap_lock, and put mm_struct before copying data to user */ - query_vma_teardown(mm, vma); + query_vma_teardown(priv); mmput(mm); if (karg.vma_name_size && copy_to_user(u64_to_user_ptr(karg.vma_name_addr), @@ -700,7 +732,7 @@ static int do_procmap_query(struct proc_maps_private *priv, void __user *uarg) return 0; out: - query_vma_teardown(mm, vma); + query_vma_teardown(priv); mmput(mm); kfree(name_buf); return err; -- 2.50.0.714.g196bf9f422-goog