From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71E13C5B549 for ; Wed, 4 Jun 2025 23:12:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 123268D0055; Wed, 4 Jun 2025 19:12:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0D2298D0007; Wed, 4 Jun 2025 19:12:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F03858D0055; Wed, 4 Jun 2025 19:12:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id CE9CE8D0007 for ; Wed, 4 Jun 2025 19:12:31 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 942B5BDBBB for ; Wed, 4 Jun 2025 23:12:31 +0000 (UTC) X-FDA: 83519269302.01.6E34CA1 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf14.hostedemail.com (Postfix) with ESMTP id CC7B0100003 for ; Wed, 4 Jun 2025 23:12:29 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=QxWBG8QO; spf=pass (imf14.hostedemail.com: domain of 33NJAaAYKCM0BDAx6uz77z4x.v75416DG-553Etv3.7Az@flex--surenb.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=33NJAaAYKCM0BDAx6uz77z4x.v75416DG-553Etv3.7Az@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1749078749; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5S3jZ35KMoBO4RnXYNxi81hGE/ddSIH7so+ydU69UE8=; b=Zy9p05mOu7AgSPlrmCFxsv9oyTV24xH5zbTod7kWQ1RqCzL90KlKpBW+Rh+pa7pi7pByfr ZBF75fkX9WeCI+C1icf2iwjuL+dQltpNADsqEfQN8KNw3BBgt8iY23rVGBIDInbQA1yVtU PUj9otRo1QPZmqfeqkoJzYvL+e+8stc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1749078749; a=rsa-sha256; cv=none; b=Hd/lQHhQzw2od8hQEPug6s8tOYjm8CQE9OObNTjH0I2EztqqWRbgEwlYzkGPcFn7F/SqNc s1zvUSdPM3ClfacRX+XBk7dyCxs5Ey8FMZ4R1tci2bI4ofAlIvLhaMO1ki6fz0ojFNinY0 xADA+zLDYfRprBmpQdyyB1No1Bx9bZg= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=QxWBG8QO; spf=pass (imf14.hostedemail.com: domain of 33NJAaAYKCM0BDAx6uz77z4x.v75416DG-553Etv3.7Az@flex--surenb.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=33NJAaAYKCM0BDAx6uz77z4x.v75416DG-553Etv3.7Az@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-23507382e64so2847685ad.2 for ; Wed, 04 Jun 2025 16:12:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1749078749; x=1749683549; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=5S3jZ35KMoBO4RnXYNxi81hGE/ddSIH7so+ydU69UE8=; b=QxWBG8QO+Jz/gSkSpiEI6K4HtTvKQOwOSUEnoBob7VN6IdTpoFGOUMvkDVVi7jRCTK vJEDbfuQiVK1z0i30IiPKxnkEkRt/EcxYpE5idGvzpLpx2l/tfBRemSJlkHWFK+qeUUy zLv/EckewBntFRuemO1jkTQwfHLq53TTtL0+LVeyk/G2A4jIjThsmALSH5KSt6y4s8pO olGW7dmMQzHoALs0lnnVKZ4TBDeasNKB1j/XjmAs7CePu+0BFXX9nXwmwFkJ7Dc/6e8/ pR8vKnFLmBfH42aiyQ0HcViBB/0m3EeZYuzuQCXOQg30cWK0ZpdG73NO2F2jsr0m4Gs+ ZlqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749078749; x=1749683549; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=5S3jZ35KMoBO4RnXYNxi81hGE/ddSIH7so+ydU69UE8=; b=V2WS0vP/JjNP9KiUXvSrhjMPoHuc7MF3pwT/b9OCPFlHEeUbCGrhdVIvgCGWk/cKZ9 /FeLS0sOdjaG0lvQ3Eh/ao+T5Hs9M88OftpAbbZVOBc+m88S7erEgu7sgo2XzScvfhpD tbkhgQRYaYprh+Sa7t3SvvAzgNJoNJdHf5mVNF6nBc7HW+93/DP2IdkuvjRGkdcUD7An 18c4LIcQ6HpZNeRC0UxNsi8LJ4eeB0HY11Q9yr9HKrKuCZM35eNvkeLytWDEGV0g/wyQ LGQYgCncD7KqdeQZVRNMnkhVpg0tOvC8CDbJi50fR4zc9JBMUcnf10KQYN9xsNIoWaOq Ykpw== X-Forwarded-Encrypted: i=1; AJvYcCVEf0toCsorfHYCKks2ApCq4qWRmEFSPEo01R7pPuvaSkNj3IPCArdaE9fQ3ikd7IBPnfEXdpwmiA==@kvack.org X-Gm-Message-State: AOJu0YxK6UtwYF1bORx5R1qoZFSHsKKWTUHyonXXoZKuq+rVsFkN1lEt IGDMnhpVxSSOaSlRExkmdvaO4EWDAdD+cOcEEQlp4CODxnDD9O85IBSbpsp/9SNCzaf9Q2ugZ/Y s7R93Xw== X-Google-Smtp-Source: AGHT+IGoGwd/Y8vsxaXkZq3lcq5gV0uAcPoWJqxGcCejiu9rQZwPdtbf9qfiVIzoSAsbfmD8Xw6SbN6aqVM= X-Received: from plnz4.prod.google.com ([2002:a17:902:8344:b0:231:e2fb:ebf]) (user=surenb job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ccd0:b0:234:e655:a62e with SMTP id d9443c01a7336-235e101356bmr63542455ad.9.1749078748733; Wed, 04 Jun 2025 16:12:28 -0700 (PDT) Date: Wed, 4 Jun 2025 16:11:51 -0700 In-Reply-To: <20250604231151.799834-1-surenb@google.com> Mime-Version: 1.0 References: <20250604231151.799834-1-surenb@google.com> X-Mailer: git-send-email 2.49.0.1266.g31b7d2e469-goog Message-ID: <20250604231151.799834-8-surenb@google.com> Subject: [PATCH v4 7/7] mm/maps: execute PROCMAP_QUERY ioctl under per-vma locks From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: Liam.Howlett@oracle.com, lorenzo.stoakes@oracle.com, david@redhat.com, vbabka@suse.cz, peterx@redhat.com, jannh@google.com, hannes@cmpxchg.org, mhocko@kernel.org, paulmck@kernel.org, shuah@kernel.org, adobriyan@gmail.com, brauner@kernel.org, josef@toxicpanda.com, yebin10@huawei.com, linux@weissschuh.net, willy@infradead.org, osalvador@suse.de, andrii@kernel.org, ryan.roberts@arm.com, christophe.leroy@csgroup.eu, tjmercier@google.com, kaleshsingh@google.com, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, surenb@google.com Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: CC7B0100003 X-Stat-Signature: gz4uxkmboc4s5fsk3o57t79p5jrehrtu X-Rspam-User: X-HE-Tag: 1749078749-247199 X-HE-Meta: U2FsdGVkX1+SB94EAyPRQQdRmVZ2ZB+E+uYdSdKmFHFWHFBFrpAHNsNQDGDt1kw6kMyINrzLOdYhEyi7UDiTsiyBUnMAXSRm8bcy0c9npSnihQOSIRrfkh3hXYG6EsvaRKUILcx5A8/puDoa95bQeQQnyZWN0AFB3ZOBktC6fmPgdjgq+Ltn+Ip5We2k5Wk6r3ttJAESfHI4xMCcaqGcuFOZEP5FK0e1VwMparm53++lFZc4i4EgBsmrDOCeWRdmJbMheoecUj3x/o8swFfMtLNDWpRPEHxAovAYqi4aSXIq6z5lyKWWXhpGWxfL6gMhGiVH37t8epaFqylPZK8ngRMHwEdVOeslAHCRi4wcUm1RDJF7L5q3VOa5AlM+XpEJvww6tyiZb/HtlVg3gUP53w6WSeP1wVbbs8IGHKwFcNAWdk9oCk9cTA4lWjO1+cSzTm8P+vxTPk6ePcNmOrvLSiBe5BWg5dYLQ9ieF7k5K8c79lvgK60wIIH/iXEV+rIMjVQGjefVubt1zQFKyhyjVhmTxm1xn+2PWuvTBA5hMzoMt9bEEvXfmOdtkpti7eXKuif5WnpsC7M9SyDQSFtSj1bgTiPUcgjM7VNTHzJmKikmhNU+J0SesBpDf5BMRxReLXV/xZsZEUmHI+hspt3jC1eB5ZvgqRxVTq3Uzm4BO1uP/fBz0n+ztQh/7B7O+hR+yhY4YWdek0aygFYYMwtYnx+R2Cci5PkbtuKV1uUrGGCD+NOjZ189B4ccjWi6JsF3tq4Usdcqna7jEk4nkxO6c8KzyoLwhyU30/CJPo36ObK1CrINRgyqoQ82lUl/9EVnQjawXk3xNg4e28u5B19KLUC45Y7aH5EGy+K1rEUo7idjd4XIp3pM0yL14mzfcykWHdDvh+zZXYy65ze4eMp+ZW/ffYdxlfSyp77A2Ztbk+L3mlgzQq+L/vGRhYSLoDKn4i1p36/nIjxEoAxKkTF xmZrBDcE 0APmsX38asR4xbr1285Bm+vZ6SE3EhJDW0PW3lc23W1od9ESlEQvPCNTLbW3kNbM6hwQK5Thw4OHoz6L6oov4V2zBbaKpcSG1iCU58NJcZIWe398NfG1jwQOKw1e+jfv1e9KzTpdz84+H6Y6+JDEYOHSNnCpRQx0HYFpyHV7kU1YJ/4rcFTlyWzLqN8Gnxk+E1vUuAAlZB+f1/THcUPK99G6wm5Phyh06PC7fyk+gnY/GhOeneLh6G1VtKa9eZhWnigvFG8XgPOhq41L1xaRpxH9aMGtssCy3xDomE7sXCEu7fZTGRuhJbA9zu+v6gRM+xE98z9vU0P6oV9vQs7hNPUDoIXvEqX37DCQQptx0zhaN/V0qVWLPOPQQm2BT05fBSQ1fu6aUdD8KiNtfOMosZhOmdRCCNsuR6AcWlWh8u0YUlYSBUKnkmX067Q7b8/TWQrGRrdH7z2Tyr7nPiesYvcfvfpyY0zLI2nVwguC1RlhbcjxyTSNG033zjvCh5lT83t28yfrlqb8QnABlZhLRtQkxUevCvyIzZcfrt1i/q72qj/GLmPALsWiwj9n+Mjp8+zXXcvTmqapW33XlW2jsAjbCfQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Utilize per-vma locks to stabilize vma after lookup without taking mmap_lock during PROCMAP_QUERY ioctl execution. While we might take mmap_lock for reading during contention, we do that momentarily only to lock the vma. This change is designed to reduce mmap_lock contention and prevent PROCMAP_QUERY ioctl calls from blocking address space updates. Signed-off-by: Suren Baghdasaryan --- fs/proc/task_mmu.c | 56 ++++++++++++++++++++++++++++++++++++---------- 1 file changed, 44 insertions(+), 12 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 36d883c4f394..93ba35a84975 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -550,28 +550,60 @@ static int pid_maps_open(struct inode *inode, struct file *file) PROCMAP_QUERY_VMA_FLAGS \ ) -static int query_vma_setup(struct mm_struct *mm) +#ifdef CONFIG_PER_VMA_LOCK + +static int query_vma_setup(struct proc_maps_private *priv) { - return mmap_read_lock_killable(mm); + rcu_read_lock(); + priv->locked_vma = NULL; + priv->mmap_locked = false; + + return 0; } -static void query_vma_teardown(struct mm_struct *mm, struct vm_area_struct *vma) +static void query_vma_teardown(struct proc_maps_private *priv) { - mmap_read_unlock(mm); + unlock_vma(priv); + rcu_read_unlock(); +} + +static struct vm_area_struct *query_vma_find_by_addr(struct proc_maps_private *priv, + unsigned long addr) +{ + vma_iter_init(&priv->iter, priv->mm, addr); + return get_next_vma(priv, addr); +} + +#else /* CONFIG_PER_VMA_LOCK */ + +static int query_vma_setup(struct proc_maps_private *priv) +{ + return mmap_read_lock_killable(priv->mm); +} + +static void query_vma_teardown(struct proc_maps_private *priv) +{ + mmap_read_unlock(priv->mm); } -static struct vm_area_struct *query_vma_find_by_addr(struct mm_struct *mm, unsigned long addr) +static struct vm_area_struct *query_vma_find_by_addr(struct proc_maps_private *priv, + unsigned long addr) { - return find_vma(mm, addr); + return find_vma(priv->mm, addr); } -static struct vm_area_struct *query_matching_vma(struct mm_struct *mm, +#endif /* CONFIG_PER_VMA_LOCK */ + +static struct vm_area_struct *query_matching_vma(struct proc_maps_private *priv, unsigned long addr, u32 flags) { struct vm_area_struct *vma; next_vma: - vma = query_vma_find_by_addr(mm, addr); + vma = query_vma_find_by_addr(priv, addr); + if (IS_ERR(vma)) + return vma; + if (!vma) goto no_vma; @@ -647,13 +679,13 @@ static int do_procmap_query(struct proc_maps_private *priv, void __user *uarg) if (!mm || !mmget_not_zero(mm)) return -ESRCH; - err = query_vma_setup(mm); + err = query_vma_setup(priv); if (err) { mmput(mm); return err; } - vma = query_matching_vma(mm, karg.query_addr, karg.query_flags); + vma = query_matching_vma(priv, karg.query_addr, karg.query_flags); if (IS_ERR(vma)) { err = PTR_ERR(vma); vma = NULL; @@ -738,7 +770,7 @@ static int do_procmap_query(struct proc_maps_private *priv, void __user *uarg) } /* unlock vma or mmap_lock, and put mm_struct before copying data to user */ - query_vma_teardown(mm, vma); + query_vma_teardown(priv); mmput(mm); if (karg.vma_name_size && copy_to_user(u64_to_user_ptr(karg.vma_name_addr), @@ -758,7 +790,7 @@ static int do_procmap_query(struct proc_maps_private *priv, void __user *uarg) return 0; out: - query_vma_teardown(mm, vma); + query_vma_teardown(priv); mmput(mm); kfree(name_buf); return err; -- 2.49.0.1266.g31b7d2e469-goog