From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.3 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B3E9C2BA83 for ; Thu, 13 Feb 2020 16:52:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D0FE6206ED for ; Thu, 13 Feb 2020 16:52:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="QdJXs6EK" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D0FE6206ED Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 68B856B0575; Thu, 13 Feb 2020 11:52:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 63D506B0577; Thu, 13 Feb 2020 11:52:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 52A226B0578; Thu, 13 Feb 2020 11:52:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0064.hostedemail.com [216.40.44.64]) by kanga.kvack.org (Postfix) with ESMTP id 377AC6B0575 for ; Thu, 13 Feb 2020 11:52:48 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id C9A63249E for ; Thu, 13 Feb 2020 16:52:47 +0000 (UTC) X-FDA: 76485697974.09.back11_6fa2c53a9ec45 X-HE-Tag: back11_6fa2c53a9ec45 X-Filterd-Recvd-Size: 12977 Received: from mail-wm1-f68.google.com (mail-wm1-f68.google.com [209.85.128.68]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Thu, 13 Feb 2020 16:52:47 +0000 (UTC) Received: by mail-wm1-f68.google.com with SMTP id a9so7547269wmj.3 for ; Thu, 13 Feb 2020 08:52:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=rlZLwMr4NYmyD4yGsPowBBoh4UyAr/DFmJDENhHXGyg=; b=QdJXs6EK7WL8+GDWLBvvKXO2LTncJ+lmxQ+ltiWhZPfnGcN94q9M9TJY1NyCUb4pZK oYK3Znu7IbHdyj2bX91nykbqB9V3jZpCR70J1Bxc4DqHk7gLphPhsgiXZQPKtNIoa5TV rDcuPCCaAf6b8efGZAUBcQS3YB7GZkp0TfYqQehw1p4RVFFCiO9IDDHykHu3MUr85OHw aKO8oH4amy6T2MpU4VVctnNzwTYT6sczTFGaJpUM/uqCoDa23gviLyg2CZclro69V3Gr dtpXUdXjkNlG6cInbjwUI6gUoN2EaEYK9SVussu2x4a05KLdGOET4NZdr9lOY7DZGZk9 tAzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=rlZLwMr4NYmyD4yGsPowBBoh4UyAr/DFmJDENhHXGyg=; b=HCjm+i7apo36nPACUvvcaykaG5WaLHCZexdGHpA2kPYulSbebCXxelcsCg2jkSC5lo WksmB1nnUBB07JyOUTFhSEBGn6hnllJ11/4UKDSV+hQ6k58zkvgV7yzNDe8HdAd1UcK7 0C8gugiGL+BsaLklytNm88OXdDXFw0zhDfv6kwN9kQscWG8jL4mCsGjTxchySMj6+p/q 02QYGIBrx75kzgDplNhuqv2rTWUSfrCDnc9yAU2rjw/dnw61kLaRph5mwC4hLJ22VCrw 2+g4RPd+N0bTdzvG5g+wWmfO+2GvIa9lfNHNeeHrtNWsQOS7gK/HiFTgNU9K6NNMdLUe lHnw== X-Gm-Message-State: APjAAAW8duPSkPJ2jaoL70LxKrbDRI+3tzLvU3Q6ABhAk74vPrDVcIOe onXALNqhhTjtiU202CG1jDMlAeP4wfmT/HybkTXVew== X-Google-Smtp-Source: APXvYqwoFAZCN2rZ5pnHtAShATylf04Kf6JCpyl4f5v6/e4Ytep76dc0V+IwDH6u0wI2NCg+4xYBD8SBO0jeJrKMHKY= X-Received: by 2002:a1c:7419:: with SMTP id p25mr6439702wmc.129.1581612765627; Thu, 13 Feb 2020 08:52:45 -0800 (PST) MIME-Version: 1.0 References: <20200128025958.43490-1-arjunroy.kdev@gmail.com> <20200212184115.127c17c6b0f9dab6fcae56c2@linux-foundation.org> In-Reply-To: <20200212184115.127c17c6b0f9dab6fcae56c2@linux-foundation.org> From: Arjun Roy Date: Thu, 13 Feb 2020 08:52:34 -0800 Message-ID: Subject: Re: [PATCH resend mm,net-next 1/3] mm: Refactor insert_page to prepare for batched-lock insert. To: Andrew Morton Cc: Arjun Roy , davem@davemloft.net, netdev@vger.kernel.org, linux-mm@kvack.org, Eric Dumazet , Soheil Hassas Yeganeh Content-Type: multipart/alternative; boundary="00000000000011be2c059e77ebd2" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --00000000000011be2c059e77ebd2 Content-Type: text/plain; charset="UTF-8" The addition of page_has_type() looks good to me, thanks! -Arjun On Wed, Feb 12, 2020 at 6:41 PM Andrew Morton wrote: > On Mon, 27 Jan 2020 18:59:56 -0800 Arjun Roy > wrote: > > > Add helper methods for vm_insert_page()/insert_page() to prepare for > > vm_insert_pages(), which batch-inserts pages to reduce spinlock > > operations when inserting multiple consecutive pages into the user > > page table. > > > > The intention of this patch-set is to reduce atomic ops for > > tcp zerocopy receives, which normally hits the same spinlock multiple > > times consecutively. > > I tweaked this a bit for the addition of page_has_type() to > insert_page(). Please check. > > > > From: Arjun Roy > Subject: mm: Refactor insert_page to prepare for batched-lock insert. > > From: Arjun Roy > > Add helper methods for vm_insert_page()/insert_page() to prepare for > vm_insert_pages(), which batch-inserts pages to reduce spinlock > operations when inserting multiple consecutive pages into the user > page table. > > The intention of this patch-set is to reduce atomic ops for > tcp zerocopy receives, which normally hits the same spinlock multiple > times consecutively. > > Link: > http://lkml.kernel.org/r/20200128025958.43490-1-arjunroy.kdev@gmail.com > Signed-off-by: Arjun Roy > Signed-off-by: Eric Dumazet > Signed-off-by: Soheil Hassas Yeganeh > > Signed-off-by: Andrew Morton > --- > > mm/memory.c | 39 ++++++++++++++++++++++++--------------- > 1 file changed, 24 insertions(+), 15 deletions(-) > > --- > a/mm/memory.c~mm-refactor-insert_page-to-prepare-for-batched-lock-insert > +++ a/mm/memory.c > @@ -1430,6 +1430,27 @@ pte_t *__get_locked_pte(struct mm_struct > return pte_alloc_map_lock(mm, pmd, addr, ptl); > } > > +static int validate_page_before_insert(struct page *page) > +{ > + if (PageAnon(page) || PageSlab(page) || page_has_type(page)) > + return -EINVAL; > + flush_dcache_page(page); > + return 0; > +} > + > +static int insert_page_into_pte_locked(struct mm_struct *mm, pte_t *pte, > + unsigned long addr, struct page *page, pgprot_t > prot) > +{ > + if (!pte_none(*pte)) > + return -EBUSY; > + /* Ok, finally just insert the thing.. */ > + get_page(page); > + inc_mm_counter_fast(mm, mm_counter_file(page)); > + page_add_file_rmap(page, false); > + set_pte_at(mm, addr, pte, mk_pte(page, prot)); > + return 0; > +} > + > /* > * This is the old fallback for page remapping. > * > @@ -1445,26 +1466,14 @@ static int insert_page(struct vm_area_st > pte_t *pte; > spinlock_t *ptl; > > - retval = -EINVAL; > - if (PageAnon(page) || PageSlab(page) || page_has_type(page)) > + retval = validate_page_before_insert(page); > + if (retval) > goto out; > retval = -ENOMEM; > - flush_dcache_page(page); > pte = get_locked_pte(mm, addr, &ptl); > if (!pte) > goto out; > - retval = -EBUSY; > - if (!pte_none(*pte)) > - goto out_unlock; > - > - /* Ok, finally just insert the thing.. */ > - get_page(page); > - inc_mm_counter_fast(mm, mm_counter_file(page)); > - page_add_file_rmap(page, false); > - set_pte_at(mm, addr, pte, mk_pte(page, prot)); > - > - retval = 0; > -out_unlock: > + retval = insert_page_into_pte_locked(mm, pte, addr, page, prot); > pte_unmap_unlock(pte, ptl); > out: > return retval; > _ > > --00000000000011be2c059e77ebd2 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
The addition of page_has_type() looks good to me, thanks!<= div>
-Arjun

On Wed, Feb 12, 2020 at 6:41 PM Andrew Morto= n <akpm@linux-foundation.or= g> wrote:
On Mon, 27 Jan 2020 18:59:56 -0800 Arjun Roy <arjunroy.kdev@gmail.com> wrote:<= br>
> Add helper methods for vm_insert_page()/insert_page() to prepare for > vm_insert_pages(), which batch-inserts pages to reduce spinlock
> operations when inserting multiple consecutive pages into the user
> page table.
>
> The intention of this patch-set is to reduce atomic ops for
> tcp zerocopy receives, which normally hits the same spinlock multiple<= br> > times consecutively.

I tweaked this a bit for the addition of page_has_type() to
insert_page().=C2=A0 Please check.



From: Arjun Roy <arjunroy.kdev@gmail.com>
Subject: mm: Refactor insert_page to prepare for batched-lock insert.

From: Arjun Roy <arjunroy@google.com>

Add helper methods for vm_insert_page()/insert_page() to prepare for
vm_insert_pages(), which batch-inserts pages to reduce spinlock
operations when inserting multiple consecutive pages into the user
page table.

The intention of this patch-set is to reduce atomic ops for
tcp zerocopy receives, which normally hits the same spinlock multiple
times consecutively.

Link: http://lkml.kernel.org/= r/20200128025958.43490-1-arjunroy.kdev@gmail.com
Signed-off-by: Arjun Roy <arjunroy@google.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>

Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

=C2=A0mm/memory.c |=C2=A0 =C2=A039 ++++++++++++++++++++++++---------------<= br> =C2=A01 file changed, 24 insertions(+), 15 deletions(-)

--- a/mm/memory.c~mm-refactor-insert_page-to-prepare-for-batched-lock-inser= t
+++ a/mm/memory.c
@@ -1430,6 +1430,27 @@ pte_t *__get_locked_pte(struct mm_struct
=C2=A0 =C2=A0 =C2=A0 =C2=A0 return pte_alloc_map_lock(mm, pmd, addr, ptl);<= br> =C2=A0}

+static int validate_page_before_insert(struct page *page)
+{
+=C2=A0 =C2=A0 =C2=A0 =C2=A0if (PageAnon(page) || PageSlab(page) || page_ha= s_type(page))
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -EINVAL;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0flush_dcache_page(page);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0return 0;
+}
+
+static int insert_page_into_pte_locked(struct mm_struct *mm, pte_t *pte, +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0unsigned long addr, struct page *page, pgprot_t prot)
+{
+=C2=A0 =C2=A0 =C2=A0 =C2=A0if (!pte_none(*pte))
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return -EBUSY;
+=C2=A0 =C2=A0 =C2=A0 =C2=A0/* Ok, finally just insert the thing.. */
+=C2=A0 =C2=A0 =C2=A0 =C2=A0get_page(page);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0inc_mm_counter_fast(mm, mm_counter_file(page));=
+=C2=A0 =C2=A0 =C2=A0 =C2=A0page_add_file_rmap(page, false);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0set_pte_at(mm, addr, pte, mk_pte(page, prot));<= br> +=C2=A0 =C2=A0 =C2=A0 =C2=A0return 0;
+}
+
=C2=A0/*
=C2=A0 * This is the old fallback for page remapping.
=C2=A0 *
@@ -1445,26 +1466,14 @@ static int insert_page(struct vm_area_st
=C2=A0 =C2=A0 =C2=A0 =C2=A0 pte_t *pte;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 spinlock_t *ptl;

-=C2=A0 =C2=A0 =C2=A0 =C2=A0retval =3D -EINVAL;
-=C2=A0 =C2=A0 =C2=A0 =C2=A0if (PageAnon(page) || PageSlab(page) || page_ha= s_type(page))
+=C2=A0 =C2=A0 =C2=A0 =C2=A0retval =3D validate_page_before_insert(page); +=C2=A0 =C2=A0 =C2=A0 =C2=A0if (retval)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 goto out;
=C2=A0 =C2=A0 =C2=A0 =C2=A0 retval =3D -ENOMEM;
-=C2=A0 =C2=A0 =C2=A0 =C2=A0flush_dcache_page(page);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 pte =3D get_locked_pte(mm, addr, &ptl);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (!pte)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 goto out;
-=C2=A0 =C2=A0 =C2=A0 =C2=A0retval =3D -EBUSY;
-=C2=A0 =C2=A0 =C2=A0 =C2=A0if (!pte_none(*pte))
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0goto out_unlock; -
-=C2=A0 =C2=A0 =C2=A0 =C2=A0/* Ok, finally just insert the thing.. */
-=C2=A0 =C2=A0 =C2=A0 =C2=A0get_page(page);
-=C2=A0 =C2=A0 =C2=A0 =C2=A0inc_mm_counter_fast(mm, mm_counter_file(page));=
-=C2=A0 =C2=A0 =C2=A0 =C2=A0page_add_file_rmap(page, false);
-=C2=A0 =C2=A0 =C2=A0 =C2=A0set_pte_at(mm, addr, pte, mk_pte(page, prot));<= br> -
-=C2=A0 =C2=A0 =C2=A0 =C2=A0retval =3D 0;
-out_unlock:
+=C2=A0 =C2=A0 =C2=A0 =C2=A0retval =3D insert_page_into_pte_locked(mm, pte,= addr, page, prot);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 pte_unmap_unlock(pte, ptl);
=C2=A0out:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 return retval;
_

--00000000000011be2c059e77ebd2--