From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25CB3E95A67 for ; Sun, 8 Oct 2023 22:00:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A3D486B0193; Sun, 8 Oct 2023 18:00:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9C59B6B0194; Sun, 8 Oct 2023 18:00:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 867C56B0195; Sun, 8 Oct 2023 18:00:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 6EA8C6B0193 for ; Sun, 8 Oct 2023 18:00:56 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 410B9801E8 for ; Sun, 8 Oct 2023 22:00:56 +0000 (UTC) X-FDA: 81323664912.07.E266DBC Received: from mail-yw1-f181.google.com (mail-yw1-f181.google.com [209.85.128.181]) by imf21.hostedemail.com (Postfix) with ESMTP id 771021C0018 for ; Sun, 8 Oct 2023 22:00:54 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=aX2wTx7L; spf=pass (imf21.hostedemail.com: domain of surenb@google.com designates 209.85.128.181 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696802454; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ztOTGJqPQRfD81PuI09dHiiKzvN89O3gl+AaTLjxc7M=; b=W1JFqih9jLEAKhwZyrjYWS5xt6cjChfhJIpNdq5MQ7jeR2x5+p6+e2YQPlxbPTWDSDe/tM FDwL4Br0+vQwyf9Sl//ZbYMZmjSZ4OG6GVXzZAeMqpvugx1I5pEouTfWW5PzHvVw1K7CXo kIKwSDdddE0Fr0RyZBAH0PoGJ/ZqOVk= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=aX2wTx7L; spf=pass (imf21.hostedemail.com: domain of surenb@google.com designates 209.85.128.181 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696802454; a=rsa-sha256; cv=none; b=qvKotpviGcSNsvvjyY593xY9De/CdKVZynQaz1YB9ZR3R6Mxb1z7cIijxuhm0p484uUE3t Wqczhp5TdeZarxC1HcS6mbgfhmwVF1yJKDGYz6hH7Pn4rz0IeaMFK6u8HAUNfTYzj1Lp/C vDQLasdh7ZEZ4KRveUXfIERhgYbuH6g= Received: by mail-yw1-f181.google.com with SMTP id 00721157ae682-59f1dff5298so47693087b3.3 for ; Sun, 08 Oct 2023 15:00:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1696802453; x=1697407253; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=ztOTGJqPQRfD81PuI09dHiiKzvN89O3gl+AaTLjxc7M=; b=aX2wTx7LMVf2qnJxs3oyDpqN9myB3zBdK1PMuLOvqW2Yh7Q1bqssBQk/pzjkSWKiNA hOuy2JQHKJbMs3grZJ7jMikfjXtTl2mRt046l8J3BifAjQvx/gX6MUbi+bT/VbqezW+W I7z0Ah262V2dZOejdPZyKMzMO/rhdBsp+/+ZO2QRefUeIon33OBBZnvz6lw7PKT2wYKu ciq/Y+vGkhXIaBe4xCUdb4iXtevYgLdPGiA+G21YHx5kzbFxHtz4yoKhtDRmw1E72QM8 AQalS/MbSDWLEYP1UBbAOphlYQIOvs1qmqMFTKa+V6lcZKHr5J+EwSArcqRtzmkoTkWp va8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696802453; x=1697407253; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ztOTGJqPQRfD81PuI09dHiiKzvN89O3gl+AaTLjxc7M=; b=I8QByCx9DKaFtjFWRZnJ00xylfnOyt0FUBAjoixyN06nsQ6OLVWMvQ4PebqzHKBlI9 g+rhT0cLk0qkqujpAPdl8wji4i5bcYmqrCtvl3bt0pmZaidIqwb97YMhI65yihwZf7rB hYu5rYtPmsywt9BIz+11r0AO8O6dbEw/mi1zs4G3otYNs/K+AnDcdPGW5UnrhB7/W6Rd ZZG2mqipMCwmCjC9JS9hO9B3Umw9OxuepT2UwRa7nFo1cp740PUlIwLFY70rK1ekTdJu 04+WDFgQAuK+ty8uVnchlWwvXWQck1hTBNikgzx7mVRxwGqAE9vEYJ+t7m06YDHnaqdp i36g== X-Gm-Message-State: AOJu0YyNM98pMKMu8oG7qFZMIoxPeHeWFFQ5x0ZsNwjnWriFfqN2sVim rbBViO3RAgnP3/q/khl2qYUaaciY6E5S1THwHiJIuk+E+MI9SbLFTSqnMA== X-Google-Smtp-Source: AGHT+IEctVSBTrQGzPhx59mgq/ctDkcL/ZE/bnr63qD3tFjOeTuy1o5XNnGJzegIxczOFHrSC6EigKKtJe/MVhFrd6U= X-Received: by 2002:a25:6dc4:0:b0:d84:e7a6:fc09 with SMTP id i187-20020a256dc4000000b00d84e7a6fc09mr11468337ybc.17.1696802453394; Sun, 08 Oct 2023 15:00:53 -0700 (PDT) MIME-Version: 1.0 References: <20231006195318.4087158-1-willy@infradead.org> <20231006195318.4087158-3-willy@infradead.org> In-Reply-To: <20231006195318.4087158-3-willy@infradead.org> From: Suren Baghdasaryan Date: Sun, 8 Oct 2023 15:00:42 -0700 Message-ID: Subject: Re: [PATCH v2 2/6] mm: Call wp_page_copy() under the VMA lock To: "Matthew Wilcox (Oracle)" Cc: Andrew Morton , linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 771021C0018 X-Rspam-User: X-Stat-Signature: ye5eu6b19cyans9ia83uano9b6zwatwu X-Rspamd-Server: rspam01 X-HE-Tag: 1696802454-92902 X-HE-Meta: U2FsdGVkX1/TcRpUUGPxtXMsMFYrnnBW4wVUj/lSwzs5ipIUVeIft2998zJYdadyCdrnz46xqDPAPux46W3ad8es54rDjMt5d4UrC2JwBfL6JKeKAKyXY9vEWGE3jsJ40rurrd6Oxg0ImkgmIQUs9lRTC/4pkwGoanKNxdQ3KB/u6HYe61fUx9EM9178C6+4ASgnNtLZG4hSnXuuzPg67UrMSees7lhQWI3fVl+JO5L4spw27Dt+cQHA5Z14o5256s4Xis68rJfJiy9m1k8Zfr23wkWCQs8pJ7ImqK+n0anXeBF4h6qmfswpV4B0olDebJeT90xtwQvH5/ZHrigVgBOy+lYRcM6LdRUTXkPhcgG7feA0ayKSpuOeOBbEd2pP9NSbV/zOfXEAez3rqZmxi6cRaJehE91nYGiLHmD6/DmvxkWaGCf+7Xfv5vzvW/s/u+mgPwd1Zxt7lMaBazOnKvV4PWsdxBuo+WuwMyGn/hB/6e28IFUrsQag5yxcNgj/PgMdfss5tx/9N9DK8/yGl3YHA5JAor8q64IWnvtUjz2Fn35lpUtwFSoNXWNqS16FP1WQddRaZ0/BgXKbMK2EFYIWMXwhYaacngFJnQILonZG6+ZLMdObJheemYb8feLz3+7ubuZBOo4s/kR/FbNqsn1sl7V81TzcgynxdFjWv4+jZzmxSoSmrFviq/CVmeSxrpXX/UvF+VPuSiOu4pKV271y/P9NhwyPGbKUW6/bdpUjVjHEd+SIgT9lvBotDqUtklAQuB177WQkUi69q2KzsQ4kKQ11YwlfiFlbROJM5oC9djZVtrJo79Z8kjHE75R+0FRMqszPZ2zR6awp7IHjYUEDFqHE2BGJUqWgndu1n8lC1Hb4lFvLkYKGG1aPBuOqwKiCJOl55dG+YqYUT1vCpEeYzgeD/9/j2bHiT2kyHBh+rRxk8Kp0MYYKCSv8Sn2TtBJ66ePXUYZ/V5XL05Y RaGXAS+V g0+PW90Bxm60hM+MuqBaN4J/A85HmHnsLiSxGbZZGZVFjbTcx9K03wJE6qmtbulVWFzCsWpyJ1cZ9NR0HxQW1dRNepl2xjFzRNTBslCpkqg/QoaFFvY0CaCmHAhqACK1unRgTME7N4JBMSZqBycxFW8jnDJFaDFkKRhIHi6vGoN2VRNTTTJOSPfZfx8vkv7jcMnudXQriAVntj0AnPyiM0E4OW/LcqCnBAH8mOQ9Yfbzq3b4CDMIIJHO1IZGMLTU327xJ1XRJS2pGyGtutFJ5MOnWIE9faEsY2wT2yvIOvn1mqpL9rxNFd0Tn6VFr6agG6zaPKbyDxtnhvpWBy/z2VCWueXmgCJWO6mJt X-Bogosity: Ham, tests=bogofilter, spamicity=0.000176, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Oct 6, 2023 at 12:53=E2=80=AFPM Matthew Wilcox (Oracle) wrote: > > It is usually safe to call wp_page_copy() under the VMA lock. The only > unsafe situation is when no anon_vma has been allocated for this VMA, > and we have to look at adjacent VMAs to determine if their anon_vma can > be shared. Since this happens only for the first COW of a page in this > VMA, the majority of calls to wp_page_copy() do not need to fall back > to the mmap_sem. > > Add vmf_anon_prepare() as an alternative to anon_vma_prepare() which > will return RETRY if we currently hold the VMA lock and need to allocate > an anon_vma. This lets us drop the check in do_wp_page(). > > Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Suren Baghdasaryan > --- > mm/memory.c | 39 ++++++++++++++++++++++++++------------- > 1 file changed, 26 insertions(+), 13 deletions(-) > > diff --git a/mm/memory.c b/mm/memory.c > index 97f860d6cd2a..cff78c496728 100644 > --- a/mm/memory.c > +++ b/mm/memory.c > @@ -3042,6 +3042,21 @@ static inline void wp_page_reuse(struct vm_fault *= vmf) > count_vm_event(PGREUSE); > } > > +static vm_fault_t vmf_anon_prepare(struct vm_fault *vmf) > +{ > + struct vm_area_struct *vma =3D vmf->vma; > + > + if (likely(vma->anon_vma)) > + return 0; > + if (vmf->flags & FAULT_FLAG_VMA_LOCK) { > + vma_end_read(vma); > + return VM_FAULT_RETRY; > + } > + if (__anon_vma_prepare(vma)) > + return VM_FAULT_OOM; > + return 0; > +} > + > /* > * Handle the case of a page which we actually need to copy to a new pag= e, > * either due to COW or unsharing. > @@ -3069,27 +3084,29 @@ static vm_fault_t wp_page_copy(struct vm_fault *v= mf) > pte_t entry; > int page_copied =3D 0; > struct mmu_notifier_range range; > - int ret; > + vm_fault_t ret; > > delayacct_wpcopy_start(); > > if (vmf->page) > old_folio =3D page_folio(vmf->page); > - if (unlikely(anon_vma_prepare(vma))) > - goto oom; > + ret =3D vmf_anon_prepare(vmf); > + if (unlikely(ret)) > + goto out; > > if (is_zero_pfn(pte_pfn(vmf->orig_pte))) { > new_folio =3D vma_alloc_zeroed_movable_folio(vma, vmf->ad= dress); > if (!new_folio) > goto oom; > } else { > + int err; > new_folio =3D vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, vm= a, > vmf->address, false); > if (!new_folio) > goto oom; > > - ret =3D __wp_page_copy_user(&new_folio->page, vmf->page, = vmf); > - if (ret) { > + err =3D __wp_page_copy_user(&new_folio->page, vmf->page, = vmf); > + if (err) { > /* > * COW failed, if the fault was solved by other, > * it's fine. If not, userspace would re-fault on > @@ -3102,7 +3119,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf= ) > folio_put(old_folio); > > delayacct_wpcopy_end(); > - return ret =3D=3D -EHWPOISON ? VM_FAULT_HWPOISON = : 0; > + return err =3D=3D -EHWPOISON ? VM_FAULT_HWPOISON = : 0; > } > kmsan_copy_page_meta(&new_folio->page, vmf->page); > } > @@ -3212,11 +3229,13 @@ static vm_fault_t wp_page_copy(struct vm_fault *v= mf) > oom_free_new: > folio_put(new_folio); > oom: > + ret =3D VM_FAULT_OOM; > +out: > if (old_folio) > folio_put(old_folio); > > delayacct_wpcopy_end(); > - return VM_FAULT_OOM; > + return ret; > } > > /** > @@ -3458,12 +3477,6 @@ static vm_fault_t do_wp_page(struct vm_fault *vmf) > return 0; > } > copy: > - if ((vmf->flags & FAULT_FLAG_VMA_LOCK) && !vma->anon_vma) { > - pte_unmap_unlock(vmf->pte, vmf->ptl); > - vma_end_read(vmf->vma); > - return VM_FAULT_RETRY; > - } > - > /* > * Ok, we need to copy. Oh, well.. > */ > -- > 2.40.1 >