From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 556B2C27C54 for ; Thu, 6 Jun 2024 16:52:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D3E216B00A6; Thu, 6 Jun 2024 12:52:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CC7746B00A7; Thu, 6 Jun 2024 12:52:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B40C46B00A8; Thu, 6 Jun 2024 12:52:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 937736B00A6 for ; Thu, 6 Jun 2024 12:52:05 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 4BD1EA0FD4 for ; Thu, 6 Jun 2024 16:52:05 +0000 (UTC) X-FDA: 82201056210.09.7F8C323 Received: from mail-pg1-f174.google.com (mail-pg1-f174.google.com [209.85.215.174]) by imf24.hostedemail.com (Postfix) with ESMTP id 78073180018 for ; Thu, 6 Jun 2024 16:52:03 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OIJONbED; spf=pass (imf24.hostedemail.com: domain of andrii.nakryiko@gmail.com designates 209.85.215.174 as permitted sender) smtp.mailfrom=andrii.nakryiko@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717692723; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bPKhCgxXa/CNr7U5QI54wZCLdvYy/KVKpQ3JKTtNoA4=; b=wO0iEwFJKb/nKROXDHRzR5JUB0Vnos2YpezYNM/mRhQIuTtsh5clMK/o8peBGWlDL99X6W ultR3qLIBZJP9GHgQjtFp72kLPXSyAkXElX6pfB3NMjze0KRtbHaBsJWAzNhCdzjFvR/Nd Hd0cBpMR1FMzvoNelv5Cr5kam+grUzo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717692723; a=rsa-sha256; cv=none; b=l2zlCPchSygVB7vPDFpvxdt2ycb30aQx8snZtSfGs9jZ2vWIv3+EZA0DZUW1QEGUNQQ6EI WGZW2cGKVmhcFABRMRv7tipcbJNXxPMlAa52WYPOMdH9ch/uPmEnYaxM8xHqmVB1mVNgv4 c1ieNZ5RgXxCK6s3PuTAPsuopazi1Yw= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OIJONbED; spf=pass (imf24.hostedemail.com: domain of andrii.nakryiko@gmail.com designates 209.85.215.174 as permitted sender) smtp.mailfrom=andrii.nakryiko@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pg1-f174.google.com with SMTP id 41be03b00d2f7-6d4bf8eb8e2so914081a12.3 for ; Thu, 06 Jun 2024 09:52:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1717692722; x=1718297522; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=bPKhCgxXa/CNr7U5QI54wZCLdvYy/KVKpQ3JKTtNoA4=; b=OIJONbEDshk3axK2vnkmK1Fa/k3WuDX9MjxyDDciPRLdkS5K0XuIyP4ST1WOfXUOjO YdDz1NJRJr1oJ5/atT0iGDKa9AVZualgcERZWJQTk3KYCRXSw1P4d7LVApmZwXHVqxmN Dhz00vYGXpnybz4kmmZ71gSSRvRHiW10zZa6ZkSr0oNeeoVhtxjpdEX25wf4QazsWWYd ZBHCyq+KnfkIjlEDL5N3iK6dJapiymQomMJuUUieczKnGJqhj2nwFghTesYBE3SwaYOU qq86+MdpkYUgWcWT3rh3h95tZwQIfwOZ5gWUT+eFUrl4LBn25twh68ifTYO1Vm0mpsV2 Iqvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717692722; x=1718297522; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bPKhCgxXa/CNr7U5QI54wZCLdvYy/KVKpQ3JKTtNoA4=; b=GHp9uarfxgImnUeYAL83sUNg86JXZ9v4Lwpn4qJgCFfukToqfMq9cvtV+ppT8vr2+l xVDbjyCPbHXtPPPsxmaZY3ylY7HP7jvl8TuoCdxOzpwJnuhk9B7M2JqftOvXmV+mo/Dp ZFudQhNkHFbV3cP4IJu0lc5hzTW/k6Ye7aj8ipuydOA4jWE2AMdPwF9LvY2q1BPY0FJN kN506u3ieX/RE6DSruYjxi6OjZ2+4qPwo86EIFQOELTNit4zRPgnwpm1CnRFJ197ZauE 6xqNBiHF08OZN2GnqG7CPCHDEg9ALqr3r+EKmGO8BnFZUjIBXKT5oVI4/i0rZiS0KiQe N/TA== X-Forwarded-Encrypted: i=1; AJvYcCW9GWl95PTiIF6RciAxRtAUN1s/2h9fDZfK6P/wBPlBnThVTXncRV/Y3rIQ8jbocblEiJVpKQrHA5TO5END97SJOZY= X-Gm-Message-State: AOJu0YwRPAgUSkFMCZxK6CFeIRaVH3nl6DDklV8ZxBmb/8tsc+Xe3gch iE6bFErAjB/q7lSKOV6fBnJVaMRuj4zPJ0SPip8BOY94mxyuaYALVI+6FD2zO7uNiFSLThei/9T u9GHsoggQCuzwMfRmRsICxgBkgv8= X-Google-Smtp-Source: AGHT+IEB6OolDSrP3Rnv3sb7vP6n8o1kGoB493b6UbHnJIs0/RiYGEy3CxkFieHAInZhO251MHV/iElJCcjo/8r8NX0= X-Received: by 2002:a17:90a:5588:b0:2bf:6a1b:fcb8 with SMTP id 98e67ed59e1d1-2c2bcaf9250mr77250a91.24.1717692722316; Thu, 06 Jun 2024 09:52:02 -0700 (PDT) MIME-Version: 1.0 References: <20240605002459.4091285-1-andrii@kernel.org> <20240605002459.4091285-2-andrii@kernel.org> <5fmylram4hhrrdl7vf6odyvuxcrvhipsx2ij5z4dsfciuzf4on@qwk7qzze6gbt> In-Reply-To: From: Andrii Nakryiko Date: Thu, 6 Jun 2024 09:51:50 -0700 Message-ID: Subject: Re: [PATCH v3 1/9] mm: add find_vma()-like API but RCU protected and taking VMA lock To: Suren Baghdasaryan Cc: "Liam R. Howlett" , Matthew Wilcox , Andrii Nakryiko , linux-fsdevel@vger.kernel.org, brauner@kernel.org, viro@zeniv.linux.org.uk, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, gregkh@linuxfoundation.org, linux-mm@kvack.org, rppt@kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 78073180018 X-Rspam-User: X-Stat-Signature: 573n196ttux7pjfd83un4snb3r1xm84q X-HE-Tag: 1717692723-545072 X-HE-Meta: U2FsdGVkX1/rypi4HBYx+/MtUPMpU+TpAwnpeU7SkwkrpebR/emVGYc2Ws9Ldo3zfnkb8O+Z/ph44GzLtIy7uQ7J3+SfhuZvb3ZC/UK9xixxaPhdV2WgOF0RIhOVxi/f4wDcjGOVNNocyCW2QyXl9lGSZm9KWOmCFYxQshKIN9p6iqxvQVArrnFIOnEF/Iyu6I791o8qcQFmEqc6O3HuG2DdqkhZC9AkOmZhb2cLw0HwBXyCjj5i8XKi5B6AIZ7aqSL17hB8K2mO3lWMVxbxGoP2D8dlXfjd6L5mksGluBXTHQXojNG/qgzFMMcTDlW/WsmC7zmrA5AU3PZrmybIW9GvbOCT7q4y/2q6ec+mncZPOmiyBEyWdZvvqleySV/nIEBARdoVo91PREGb94c6KPMnoSUVkus9fWlNL+XtF63K76lErknBq59dZF+YhAZwhxqHglhzaVzPDaynT9BA9aB5UwMyfRelpW3YnuKPBvVIuojezKsaRAtV2yLzPXb0MHOS3FuPBGt5H1fnmC5SG6rMRVD9k1HvPbOfN8ctdyABfxsKQe/o1AukVPJgVoP3lgq92376A6tBB5/q0fX907SpXxOmClSzL2ZLnUXEnmRvWB3lXhnHFphppgblj2AeYURM+tlE+tqA60YVVea2B2h4q5hyeeMctuknVu0gdswPROmBLY0VsKyOs9Wb6tFx3q4pGVDqRi/2G2jkT8SSM+L1k6yfiJvPgj56MQ0M2Sum0+RkMueF8em43nOWdG7DXw2sc6pMpK85ygYh58HENQZXrWalW/OH0lw9e6twIQBuwczIR5csewOBX/MyLK91/fVRU+FED3Pk2VYq4YJ+WwTwTEj6LFTuGpEKwV+JIVcPjLTUJyap3iKvYftc/rhb/1LSMPmxkbzXbay9IqU+C129a8XIuanpwH3hQjeCRjaTa7pk7j7om0r7eO56KQ+bOtPIek0JoRUXbDAHdbw 9MVT02lF O0gsOA52N9G3cj9fyxz8dNrPkJSgnrOwGVaEd7Sl/DT5Ou4DZi/d/MrsPkvTPYoUGF3gsUJMUsJQBzPxznaGHF9civttF+GsB7qQrIjkdOcb8KW+JFugbv+QQ5m4coSQAIKVMwn4/7jRlnowmzRbL8AN+5MtUvEd5IqR7n8Wi5w59uXqNmom0W0tIJT9WJKxquGMBHnnPCaOBLQLqXbURUqQlXRy3MddzfECgoCr4Hc8teTiWh7Hn8PyVl2FMbakobnzHK7k40wcK0Y2HmIq0z9IeRDIacYh3YEB3Gdl7D11JzgBwuTlXysaXDiGfLUpKhT94zVzZ57LY/x3I9VILnOvYQn9xZlh0hYPxvzGpiC2I+CWPrdYV9v3qFJrB1oL1pNgfq5rchYNyc1ailrf8iNSGVfDkBdVtUzP/lzO67ARHoB97H9pgqSUzRi7xpxNFhN1ZYza+F7JtM6Ss2grTgH5NXLc97Bg2sPImtRFIQ52xVzxSwSZ1CJ7OkUvilWPGfBgJLf7N07hIwZI8Peg193paDg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jun 5, 2024 at 4:22=E2=80=AFPM Suren Baghdasaryan wrote: > > On Wed, Jun 5, 2024 at 10:03=E2=80=AFAM Liam R. Howlett wrote: > > > > * Andrii Nakryiko [240605 12:27]: > > > On Wed, Jun 5, 2024 at 9:24=E2=80=AFAM Andrii Nakryiko > > > wrote: > > > > > > > > On Wed, Jun 5, 2024 at 9:13=E2=80=AFAM Andrii Nakryiko > > > > wrote: > > > > > > > > > > On Wed, Jun 5, 2024 at 6:33=E2=80=AFAM Liam R. Howlett wrote: > > > > > > > > > > > > * Matthew Wilcox [240604 20:57]: > > > > > > > On Tue, Jun 04, 2024 at 05:24:46PM -0700, Andrii Nakryiko wro= te: > > > > > > > > +/* > > > > > > > > + * find_and_lock_vma_rcu() - Find and lock the VMA for a g= iven address, or the > > > > > > > > + * next VMA. Search is done under RCU protection, without = taking or assuming > > > > > > > > + * mmap_lock. Returned VMA is guaranteed to be stable and = not isolated. > > > > > > > > > > > > > > You know this is supposed to be the _short_ description, righ= t? > > > > > > > Three lines is way too long. The full description goes betwe= en the > > > > > > > arguments and the Return: line. > > > > > > > > > > Sure, I'll adjust. > > > > > > > > > > > > > > > > > > > > + * @mm: The mm_struct to check > > > > > > > > + * @addr: The address > > > > > > > > + * > > > > > > > > + * Returns: The VMA associated with addr, or the next VMA. > > > > > > > > + * May return %NULL in the case of no VMA at addr or above= . > > > > > > > > + * If the VMA is being modified and can't be locked, -EBUS= Y is returned. > > > > > > > > + */ > > > > > > > > +struct vm_area_struct *find_and_lock_vma_rcu(struct mm_str= uct *mm, > > > > > > > > + unsigned long addr= ess) > > > > > > > > +{ > > > > > > > > + MA_STATE(mas, &mm->mm_mt, address, address); > > > > > > > > + struct vm_area_struct *vma; > > > > > > > > + int err; > > > > > > > > + > > > > > > > > + rcu_read_lock(); > > > > > > > > +retry: > > > > > > > > + vma =3D mas_find(&mas, ULONG_MAX); > > > > > > > > + if (!vma) { > > > > > > > > + err =3D 0; /* no VMA, return NULL */ > > > > > > > > + goto inval; > > > > > > > > + } > > > > > > > > + > > > > > > > > + if (!vma_start_read(vma)) { > > > > > > > > + err =3D -EBUSY; > > > > > > > > + goto inval; > > > > > > > > + } > > > > > > > > + > > > > > > > > + /* > > > > > > > > + * Check since vm_start/vm_end might change before we l= ock the VMA. > > > > > > > > + * Note, unlike lock_vma_under_rcu() we are searching f= or VMA covering > > > > > > > > + * address or the next one, so we only make sure VMA wa= sn't updated to > > > > > > > > + * end before the address. > > > > > > > > + */ > > > > > > > > + if (unlikely(vma->vm_end <=3D address)) { > > > > > > > > + err =3D -EBUSY; > > > > > > > > + goto inval_end_read; > > > > > > > > + } > > > > > > > > + > > > > > > > > + /* Check if the VMA got isolated after we found it */ > > > > > > > > + if (vma->detached) { > > > > > > > > + vma_end_read(vma); > > > > > > > > + count_vm_vma_lock_event(VMA_LOCK_MISS); > > > > > > > > + /* The area was replaced with another one */ > > > > > > > > > > > > > > Surely you need to mas_reset() before you goto retry? > > > > > > > > > > > > Probably more than that. We've found and may have adjusted the > > > > > > index/last; we should reconfigure the maple state. You should = probably > > > > > > use mas_set(), which will reset the maple state and set the ind= ex and > > > > > > long to address. > > > > > > > > > > Yep, makes sense, thanks. As for the `unlikely(vma->vm_end <=3D > > > > > address)` case, I presume we want to do the same, right? Basicall= y, on > > > > > each retry start from the `address` unconditionally, no matter wh= at's > > > > > the reason for retry. > > > > > > > > ah, never mind, we don't retry in that situation, I'll just put > > > > `mas_set(&mas, address);` right before `goto retry;`. Unless we sho= uld > > > > actually retry in the case when VMA got moved before the requested > > > > address, not sure, let me know what you think. Presumably retrying > > > > will allow us to get the correct VMA without the need to fall back = to > > > > mmap_lock? > > > > > > sorry, one more question as I look some more around this (unfamiliar > > > to me) piece of code. I see that lock_vma_under_rcu counts > > > VMA_LOCK_MISS on retry, but I see that there is actually a > > > VMA_LOCK_RETRY stat as well. Any reason it's a MISS instead of RETRY? > > > Should I use MISS as well, or actually count a RETRY? > > > > > > > VMA_LOCK_MISS is used here because we missed the VMA due to a write > > happening to move the vma (rather rare). The VMA_LOCK missed the vma. > > > > VMA_LOCK_RETRY is used to indicate we need to retry under the mmap lock= . > > A retry is needed after the VMA_LOCK did not work under rcu locking. > > Originally lock_vma_under_rcu() was used only inside page fault path, > so these counters helped us quantify how effective VMA locking is when > handling page faults. With more users of that function these counters > will be affected by other paths as well. I'm not sure but I think it > makes sense to use them only inside page fault path, IOW we should > probably move count_vm_vma_lock_event() calls outside of > lock_vma_under_rcu() and add them only when handling page faults. Alright, seems like I should then just drop count_vm_vma_lock_event() from the API I'm adding. > > > > > Thanks, > > Liam