From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25DBDC25B75 for ; Mon, 3 Jun 2024 04:44:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7E4E66B009B; Mon, 3 Jun 2024 00:44:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 794786B009E; Mon, 3 Jun 2024 00:44:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 65C3E6B00A0; Mon, 3 Jun 2024 00:44:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 4799F6B009B for ; Mon, 3 Jun 2024 00:44:53 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 9EBCC1406D5 for ; Mon, 3 Jun 2024 04:44:52 +0000 (UTC) X-FDA: 82188337224.17.2841D41 Received: from mail-ed1-f52.google.com (mail-ed1-f52.google.com [209.85.208.52]) by imf28.hostedemail.com (Postfix) with ESMTP id CC0E7C000E for ; Mon, 3 Jun 2024 04:44:50 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=d9FnEn1A; spf=pass (imf28.hostedemail.com: domain of ioworker0@gmail.com designates 209.85.208.52 as permitted sender) smtp.mailfrom=ioworker0@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717389890; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=36DAp6c5wHQAG3f8vCocy5x73GV3ytJxTrWvAggCAeQ=; b=uEM5g4VbGya8ujc5UEChxeQPNGY8MxBvbpQ57zQ281vLdRloSLS7v+P7zHyMVTmWFRAvkl u+w2+3YJhcBQGCs6O8qd3sADvM8L7yrbryNoX7E5sXYgw3w5Y5AV9mS8tYoixs+1RcFGyQ dAod9hmajFFz4yKAM7Kj0LIWjsyjBx4= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=d9FnEn1A; spf=pass (imf28.hostedemail.com: domain of ioworker0@gmail.com designates 209.85.208.52 as permitted sender) smtp.mailfrom=ioworker0@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717389890; a=rsa-sha256; cv=none; b=Bkc3J9JVNwkcLKFel8hlHNwTcEF5qE7hkrRk05g3TfFgGS3nSBV06lLUEKRLSSTDoQ3vEX nQ+tm1h3SFnpexnBVvpQ6ZTjFZotmqLlVoWObGYWIROvt/E4CbR+cVQArB0njaxvRhwvHm frno25DtxrxDqa1BGmaZAp8IxPjY7bI= Received: by mail-ed1-f52.google.com with SMTP id 4fb4d7f45d1cf-57a2406f951so4018524a12.1 for ; Sun, 02 Jun 2024 21:44:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1717389889; x=1717994689; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=36DAp6c5wHQAG3f8vCocy5x73GV3ytJxTrWvAggCAeQ=; b=d9FnEn1AVLUqKqcIIX5FAxlEnBJ3Rk4n4v6ug97smar4oKpXkTZIJRDtheo9BMLEO7 JtgeLxNPNONLx7/nXKLkupMu7QkYuPrNxAkaMHhxfca9CD06uQ3wMyO/8qF8HW4m3wOh UtszGiA+Se/MVXSdHj1SRwpZ+gJrfHEKDNWHRksUkKvfIfj6rgIOEKPBW65wsVi7cxb/ aic0vTiumanSLAoflmuf7CzgBz86VU+MeXTSobA9psEDd2F+S5ZJLc6PeYbUqUEUB96/ J+Ue6Glies/tn/rkprwPwVLCELkvhzHBUrtZznvKcp/myCM2PwEEHBTVGb1VS40/Pung y04A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717389889; x=1717994689; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=36DAp6c5wHQAG3f8vCocy5x73GV3ytJxTrWvAggCAeQ=; b=wiSB5TXS2V5BZ099oSkGVuceJdMqLTvPHZ7GuTEzpGTLnt4cwaTya1uZELs4HGo92F z+6N3tozr7rJItHAfr05ARCwAnigPQGkzgG8ubp/nXg4wovEfG/K5m2E6VNGqSxIBAMe wL25VTgFWk2h3jEbqjWahsT2gT/um6QdQYMxnaU0rfBdJGrXzaZbXGhZQ/+9PkjfSMOU 9bwSfzcEIf0lDM8ZxbkwA1Lju9krbjZKagtqpEM/RorBzua96yHev2eFFjLm01NCkmSi xuqDXmPLDEhzH43BpBWyhk6Mh185V/xqUiQbfx7HMy08Dg4VCWP6HUoi1TZQmttgRsWt bkrg== X-Forwarded-Encrypted: i=1; AJvYcCUIp7ACruvZaRVUbQS+aUNfU8AJeLVSza5yKeqHYOJpYAZeaJwjEkhxLwuDt9ryQPv7oyODEzlBaPfV9aO1cKgG8Xw= X-Gm-Message-State: AOJu0YzgllLICSBUE1wunADFH9PBQ6x22zmsR5bKhdazUmB4BpxvvBZL gaYvw7j0ym4WUqe1icPOVkCOv0K+aLQnai/6N+kPIzuqcQ8Kqp+CGp1/Xd3LCnIoMdM2EvFEm8b iueijQzwfBa/uqWsXQupduRIzLqg= X-Google-Smtp-Source: AGHT+IFj0spnrgaVO4RVLb/0Xcf4qFJwEH01qjZDVGTeVkSMX7Tf+chVlk4bI9VroNgSzpkd5Pp9PKDbN4Ucg287zf0= X-Received: by 2002:a50:a415:0:b0:578:5851:b0da with SMTP id 4fb4d7f45d1cf-57a364e37dcmr4994453a12.36.1717389888891; Sun, 02 Jun 2024 21:44:48 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Lance Yang Date: Mon, 3 Jun 2024 12:44:35 +0800 Message-ID: Subject: Re: [PATCH v3 1/6] mm: memory: extend finish_fault() to support large folio To: Baolin Wang Cc: akpm@linux-foundation.org, hughd@google.com, willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, da.gomez@samsung.com, p.raghav@samsung.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Bang Li Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: CC0E7C000E X-Stat-Signature: 5cbnjafhpbqypokisftrbpz1kn5eabz6 X-HE-Tag: 1717389890-200718 X-HE-Meta: U2FsdGVkX1+5sHJ7RZ58ylKuHhoH+/gsscojaM7nJDjNaKILNyWjvvpal1LE0Rzl6czoea1gkxkT4YmUk33MacpGavi6cryivwHR7mhXJLRo43bxJC9A7w7KMjVY7PXD8qOqZiXYgSDBNhumfsfAx05JfRQ+SdQBQXw7ggexR+ygJNBq2bM7N0GVt5Ck+YMTdoErkFaV8eY9Q79xHG8j7Yr8krjCmA0AHGc4b4GU46jFBh1D8EFQncmxz2Ij1S/9bt5T1rGXcd3QpHjt2Ug7w3bBmlI3X34EUT4U1lHy1WigwyMKkH3SOdg8ROk/yFGPsOFQesE7L3svqgdZKIY5Ll4FdQWH+VOx314dYKO0XrOal3sU4dOPlXOAh9+yb9VPQKqjRA5TF7D4N35Yo7f88bqIU9Tw8vHOjQITvC4ViizQtX+dp6avbAk9fa6/yhV9eYfh41ORc6okpGMI6KqjUFpD7XPdUMm0bel47DKcxlvdaxNoIjd8AleNAl4rL9JPTDTn9lKKylrXl9kv+AoIBCimBWpcOMHE+m4i24Ji/xUP9T4MbGWVTKptXMGKrRYbHMyegs42DVas67h4lRB7Z490945RuAW8kFKGqeErHg5wO5Q78cNvMNxITqfnYtJaWGC59ASv/d//jHR/SAXLtwRK6w5aJ264tpWDz6JpHjhoQUidfWvIFeFJ+x0DtPXPXZvysXv8RwjqcVIlgxinLxKNJ3lZyyV0Si5Y9QEsN6QttSUmLQe62w1yQnN0owAvBMW1K0ssECszjYc+sL3r47mhvDKB7mARtrglGaH7rfYSyJh8rs9PUzWjXczV1fPiEjFEBy5mMSvTRRjyb4DMsaNeqw+2URuJh0jEPRj4ylVWzuFR3+Of/s5lw5FNoy9L4NbbOBtHZshMB3whKpg1Ji5Dd/0qsHfBTjB5N0dlPfcpP5OBZ6ruwmTd9BwMYtdncpKuD6C+qlyBJvlU32t +uBB3iZt 7mO8+Wwef+xGVgBXe8chjWz40mvBO8w3Oi3jn8PLUL1i+kMoy2ur85mzWx1HKj6nJD/qjiew2DpHUIt3jMKdtWGsIm8oQ9uvQaflRD5imQyRJzd3m03sSRG5+PEiAfhPuPuThdtG5wvxSHTtOeE4GOoTVywjX1/IBWx8LXYRQLtR8C6W7mbsPIobCSZqN53kUPlLdARUnKb/k2vUIjNcFSMSmd2CSHzGqoiUmmx/94THf1BwdsON4VeY5M4I9+vnetCKTe75v3LenPbK+hT/Wt23oL2irn76wOuwDq58KNKOB21r+2mSvxo+fcVSbsFVAykeEyg6AS3mwf7P2kQLojXuAoFBTMP1TrAKvDQnrlccTiYYl3xVQxoYhw6rdDuYSPvefUxAnES0NwWeQ4HZAV0FfUDmZ4U8dQ7RBRb8OV+41ShFrzhc3olimswj5rJLFsFd+mkWKnsBKu/nvRlThaZTD3JsFxg0Vcl638ATy60UOCwZtmWEIZj9pa8H9YxTkIAkyULInxRaigrd7MBto4NLLung5cwl+F5m6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, May 30, 2024 at 10:04=E2=80=AFAM Baolin Wang wrote: > > Add large folio mapping establishment support for finish_fault() as a pre= paration, > to support multi-size THP allocation of anonymous shmem pages in the foll= owing > patches. > > Signed-off-by: Baolin Wang > --- > mm/memory.c | 58 ++++++++++++++++++++++++++++++++++++++++++++--------- > 1 file changed, 48 insertions(+), 10 deletions(-) > > diff --git a/mm/memory.c b/mm/memory.c > index eef4e482c0c2..435187ff7ea4 100644 > --- a/mm/memory.c > +++ b/mm/memory.c > @@ -4831,9 +4831,12 @@ vm_fault_t finish_fault(struct vm_fault *vmf) > { > struct vm_area_struct *vma =3D vmf->vma; > struct page *page; > + struct folio *folio; > vm_fault_t ret; > bool is_cow =3D (vmf->flags & FAULT_FLAG_WRITE) && > !(vma->vm_flags & VM_SHARED); > + int type, nr_pages, i; > + unsigned long addr =3D vmf->address; > > /* Did we COW the page? */ > if (is_cow) > @@ -4864,24 +4867,59 @@ vm_fault_t finish_fault(struct vm_fault *vmf) > return VM_FAULT_OOM; > } > > + folio =3D page_folio(page); > + nr_pages =3D folio_nr_pages(folio); > + > + /* > + * Using per-page fault to maintain the uffd semantics, and same > + * approach also applies to non-anonymous-shmem faults to avoid > + * inflating the RSS of the process. > + */ > + if (!vma_is_anon_shmem(vma) || unlikely(userfaultfd_armed(vma))) = { > + nr_pages =3D 1; > + } else if (nr_pages > 1) { > + pgoff_t idx =3D folio_page_idx(folio, page); > + /* The page offset of vmf->address within the VMA. */ > + pgoff_t vma_off =3D vmf->pgoff - vmf->vma->vm_pgoff; > + > + /* > + * Fallback to per-page fault in case the folio size in p= age > + * cache beyond the VMA limits. > + */ > + if (unlikely(vma_off < idx || > + vma_off + (nr_pages - idx) > vma_pages(vma))= ) { > + nr_pages =3D 1; > + } else { > + /* Now we can set mappings for the whole large fo= lio. */ > + addr =3D vmf->address - idx * PAGE_SIZE; > + page =3D &folio->page; > + } > + } > + > vmf->pte =3D pte_offset_map_lock(vma->vm_mm, vmf->pmd, > - vmf->address, &vmf->ptl); > + addr, &vmf->ptl); > if (!vmf->pte) > return VM_FAULT_NOPAGE; > > /* Re-check under ptl */ > - if (likely(!vmf_pte_changed(vmf))) { > - struct folio *folio =3D page_folio(page); > - int type =3D is_cow ? MM_ANONPAGES : mm_counter_file(foli= o); > - > - set_pte_range(vmf, folio, page, 1, vmf->address); > - add_mm_counter(vma->vm_mm, type, 1); > - ret =3D 0; > - } else { > - update_mmu_tlb(vma, vmf->address, vmf->pte); > + if (nr_pages =3D=3D 1 && unlikely(vmf_pte_changed(vmf))) { > + update_mmu_tlb(vma, addr, vmf->pte); > ret =3D VM_FAULT_NOPAGE; > + goto unlock; > + } else if (nr_pages > 1 && !pte_range_none(vmf->pte, nr_pages)) { > + for (i =3D 0; i < nr_pages; i++) > + update_mmu_tlb(vma, addr + PAGE_SIZE * i, vmf->pt= e + i); Just a friendly reminder: Bang has added the update_mmu_tlb_range()[1] batc= h function to update TLB in batches, so we can use it instead of the update_mmu_tlb() loop. [1] https://lore.kernel.org/linux-mm/20240522061204.117421-1-libang.li@antg= roup.com/ Thanks, Lance > + ret =3D VM_FAULT_NOPAGE; > + goto unlock; > } > > + folio_ref_add(folio, nr_pages - 1); > + set_pte_range(vmf, folio, page, nr_pages, addr); > + type =3D is_cow ? MM_ANONPAGES : mm_counter_file(folio); > + add_mm_counter(vma->vm_mm, type, nr_pages); > + ret =3D 0; > + > +unlock: > pte_unmap_unlock(vmf->pte, vmf->ptl); > return ret; > } > -- > 2.39.3 >