From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6A90EB64DC for ; Mon, 17 Jul 2023 16:30:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 28D6E6B0072; Mon, 17 Jul 2023 12:30:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 23DDF6B0074; Mon, 17 Jul 2023 12:30:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0DE698D0001; Mon, 17 Jul 2023 12:30:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id EF46E6B0072 for ; Mon, 17 Jul 2023 12:29:59 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id BF480A0471 for ; Mon, 17 Jul 2023 16:29:59 +0000 (UTC) X-FDA: 81021640518.26.C6344D6 Received: from mail-qt1-f172.google.com (mail-qt1-f172.google.com [209.85.160.172]) by imf04.hostedemail.com (Postfix) with ESMTP id BA37B40026 for ; Mon, 17 Jul 2023 16:29:57 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=iNg0y5UH; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf04.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.172 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689611397; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hbeYNfnz8jWi/nxzcYBnRg/O53FAY/0l+xGf/60VJq4=; b=ScT0uGTrYNh/QyVxpv+WUTjRXJBo2VASJXLQ9Ud0TL/4bJ+NKyP4D6tOteVfnPrQrjCy0m y+GtZaFkM3gX78Ijc6mxxFb3d3TBPwtIr4iTcObXaB6GH0REdOVtOaI4UGGcL9hnL+DL8L 36m5yJZrAX4Umnaq0UjAU8gZ+LYuHYQ= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=iNg0y5UH; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf04.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.172 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689611397; a=rsa-sha256; cv=none; b=OGonTDXwrJeYj9iXLWg0LATWJpaSn2lv0cmlJvs16WTcnaDlbVlDXUyb12NhzFAwBLYGSI cPVt6Ojmc4PxE9xz83Lzvi+yIck9OsY9WgUkORtMYDU72JH8scQvDGaAFu8eU+UBvAXDSF JB8UzH/wWsgapbrQE6TFgUrZYlygWZM= Received: by mail-qt1-f172.google.com with SMTP id d75a77b69052e-4036bd4fff1so1671cf.0 for ; Mon, 17 Jul 2023 09:29:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689611397; x=1692203397; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=hbeYNfnz8jWi/nxzcYBnRg/O53FAY/0l+xGf/60VJq4=; b=iNg0y5UHik6PTw5tTqzzM1OZ3sVQWkF5RazYr18+fPVe12rD97ZxOYHsOBawjYSlNT u9l3rNrobP4MMlTldiFRuU01eo9zjrN05kfosfIqBP0wFN9Gas5ENAiNBXUrgwWRihRW II5xp8g/qPz4qeJNQ+VcwG2v/Fkf+zZnzUPAF4NqYz3mhN6Tn7QTegi7xeMflrrKv1JH W/Guh/M1x0L5Z3K7XTC98Ilo3Qb6ALuaJgTop6D7RvwJzcE9TQWJaTO1I1ir9vjY6rK2 BeQ0vTsACv7zUJxo71xkRl8TerGRnbLXarBGxWHcDtjFjAsSqTFjJBFURbovDR9pDtPZ 1WIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689611397; x=1692203397; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hbeYNfnz8jWi/nxzcYBnRg/O53FAY/0l+xGf/60VJq4=; b=FfgIdYWKcgylWVj+AHC8PtLUh0B4B7HXHoh9dMfqy9njMfahgqUmZuw7a3UkwwbPdX 4uq/wFhP8V4ghsPafnkXXuSikVtxh3CL2bZm3sy1YSeEQPNzb9L0t+qJAKxTRWOGjX0Q AM3GEehCQbMO4pyQkM8axZH2bdoWPfkZEOt20pJPpwuHpw6Z3uZ1jFvQDU51pJ0vi8an S0HadGE1bZgJUCN1B7+Fk58HdPevmsuX6bCc5RnrqTXyUUI6S6y/KPiw4cQukJhI4kWc oWzQUVkqrRujBdyok7qb8jf7F0JMa4O87IXYYPuXK3ONmrynGjXSBKhMZvwnKS4GWVXx qmLA== X-Gm-Message-State: ABy/qLa9ni73vMRzcKpg3rJMd2iQL+58PkG1HmwxWI6QHGbOnXfSgUZe OI4WAHL0SpviTmjzGRVk2rfQJHD6saVJ5dgpWCQ9yg== X-Google-Smtp-Source: APBJJlG1+6eKjX0YOpNCtGwodJwcO3myqjLPem3EMPl/avx3O/U/0g2LQcKCKWUqFDr3r+FtaFzDoUXIUWKadgsrCLI= X-Received: by 2002:ac8:59ce:0:b0:3f8:8c06:c53b with SMTP id f14-20020ac859ce000000b003f88c06c53bmr1509542qtf.0.1689611396666; Mon, 17 Jul 2023 09:29:56 -0700 (PDT) MIME-Version: 1.0 References: <20230713150558.200545-1-fengwei.yin@intel.com> <7e09d69e-50a7-04c5-0a40-065e2d6fd658@intel.com> In-Reply-To: From: Yu Zhao Date: Mon, 17 Jul 2023 10:29:20 -0600 Message-ID: Subject: Re: [RFC PATCH] madvise: make madvise_cold_or_pageout_pte_range() support large folio To: "Yin, Fengwei" Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, willy@infradead.org, david@redhat.com, ryan.roberts@arm.com, shy828301@gmail.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: BA37B40026 X-Stat-Signature: msd385zbdjm45pxe4bagjphxmaof5sm7 X-HE-Tag: 1689611397-482845 X-HE-Meta: U2FsdGVkX18vV8oSPvAmYIxqzh4OCFBjb065CV5K1SklPTAEwWsA5aupTbVVGC9Iwa80YyD3ShRqLtx1TRlgXFKcB7MT4CFiRILIjV8P9e+AVdbzPLJuucWLmXdI2yRIMXwtjZB63cn14jOhEaBvu37xIyrZGWerk7PTB03hEr81LkWYTbUoc6rtoLhnxlyJVVNPePPCCC+hQhWfeOXf3/eDEwlCLrUwlcTO6cdzTk2sCDHVhxoGgXvFzW1Ge0k3VNVz+3FMT4HASFuS+kz4401y7+4lNUQ7kaYePmQ520bx4oh+Jt09OrtORyMMqnDvjbzDYYn0HKA3BxF83D4wqWkscWje8o8mfs0DoJtPQycwyPJf3FE+0evI6b2gbpCPwI0ZhL2GxA4Y29b+MYgsF8E9Feck1NxOh6qY3xg2zcRJY6bcNfXcvqEawYXgfXgq3SLnG2gdxFSyxbtkaEKyNKyKCgIz3Ph65gdq8Vz4iiBaF+p0Ku4yN+lZgBKefcCVH5WHJg2VJybG90oguyzzrUbiu3O6Fg0JKDb6grPH7JZdFuYnCW+yg/+fXNrgI5hZsFqtRqI2Vt7Ph56VZi4pOvoX6PJ1+lYIXLfRtVeipvGGN5uUpNH8sX8sO1+LrkmTBX5a2VR00Y1GAr9V0F2uSXAQlSAsUhmmfWzX+kxn6fPFzZERAkPPAcn2MjbSBcnWJs77ac0C5V/wWwR0u34mqCrN4l+Nm91sS7yv3RqWY5Zx7p5zDFB8SslbVOCDAAoiWoYt3hmUNqYwCKXKAUr1GkOI0eZRPgKEgRKwSSkl4J9+03esuo8L/aiOdLmcV1Fa+4ne9rNMunni6cctYiRl1dLmgekfMwk4ybxWAIt3dd3kRE4nOCPq5Hs0yM3QcO+xxKJpvYdBKqXppdTZPm/ydFzKkXIROCysdYBmWW+Mq39yQ9On2okcB0t0ZTMnveR34HdvVmitqmTavNunpA5 /ufrvELV ScytGCYmnqtIyp0aZ4moQevxHHqCjAGkHdxpUB1yEiBLsSqBokK5aX8QKk88X2Ek+CkEbJlnCI3Lspmk/+UJXZaI0wWzg9E/9/4mLrsxI1+Qy61T5FI9J40giG7KrrDZI9oBb9ubNSYLvfwB+k8vucfof4pou9QTNBL1IrVhQp7eu8Th6619xUmKU11S58X0hbu9rtwAj5KujBIcudnP0dkLsZNdG04srmNrm9fHLTnY4RP8SWUBSFcrrXjPMMpK43Y8dTAA4aHtJVEyPRQR+IVt9rjdMIqazUdr0Zg6oDm1BCEE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sun, Jul 16, 2023 at 5:52=E2=80=AFPM Yin, Fengwei wrote: > > On 7/14/2023 11:41 PM, Yu Zhao wrote: > > On Thu, Jul 13, 2023 at 11:57=E2=80=AFPM Yin, Fengwei wrote: > >> > >> > >>>> - if (pageout_anon_only_filter && !folio_test_anon(fol= io)) > >>>> + /* Do not interfere with other mappings of this foli= o */ > >>>> + if (folio_mapcount(folio) !=3D 1) > >>>> continue; > >>>> > >>>> - VM_BUG_ON_FOLIO(folio_test_large(folio), folio); > >>>> - > >>>> - if (pte_young(ptent)) { > >>>> - ptent =3D ptep_get_and_clear_full(mm, addr, = pte, > >>>> - tlb->fullmm)= ; > >>>> - ptent =3D pte_mkold(ptent); > >>>> - set_pte_at(mm, addr, pte, ptent); > >>>> - tlb_remove_tlb_entry(tlb, pte, addr); > >>>> - } > >>>> - > >>>> - /* > >>>> - * We are deactivating a folio for accelerating recl= aiming. > >>>> - * VM couldn't reclaim the folio unless we clear PG_= young. > >>>> - * As a side effect, it makes confuse idle-page trac= king > >>>> - * because they will miss recent referenced history. > >>>> - */ > >>>> - folio_clear_referenced(folio); > >>>> - folio_test_clear_young(folio); > >>>> - if (folio_test_active(folio)) > >>>> - folio_set_workingset(folio); > >>>> +pageout_cold_folio: > >>>> if (pageout) { > >>>> if (folio_isolate_lru(folio)) { > >>>> if (folio_test_unevictable(folio)) > >>>> @@ -529,8 +542,30 @@ static int madvise_cold_or_pageout_pte_range(pm= d_t *pmd, > >>>> arch_leave_lazy_mmu_mode(); > >>>> pte_unmap_unlock(start_pte, ptl); > >>>> } > >>>> - if (pageout) > >>>> - reclaim_pages(&folio_list); > >>>> + > >>>> + if (pageout) { > >>>> + LIST_HEAD(reclaim_list); > >>>> + > >>>> + while (!list_empty(&folio_list)) { > >>>> + int refs; > >>>> + unsigned long flags; > >>>> + struct mem_cgroup *memcg =3D folio_memcg(fol= io); > >>>> + > >>>> + folio =3D lru_to_folio(&folio_list); > >>>> + list_del(&folio->lru); > >>>> + > >>>> + refs =3D folio_referenced(folio, 0, memcg, &= flags); > >>>> + > >>>> + if ((flags & VM_LOCKED) || (refs =3D=3D -1))= { > >>>> + folio_putback_lru(folio); > >>>> + continue; > >>>> + } > >>>> + > >>>> + folio_test_clear_referenced(folio); > >>>> + list_add(&folio->lru, &reclaim_list); > >>>> + } > >>>> + reclaim_pages(&reclaim_list); > >>>> + } > >>> > >>> i overlooked the chunk above -- it's unnecessary: after we split the > >>> large folio (and splice the base folios onto the same LRU list), we > >>> continue at the position of the first base folio because of: > >>> > >>> pte--; > >>> addr -=3D PAGE_SIZE; > >>> continue; > >>> > >>> And then we do pte_mkold(), which takes care of the A-bit. > >> This patch moves the A-bit clear out of the folio isolation loop. So > >> even the folio is split and loop restarts from the first base folio, > >> the A-bit is not cleared. A-bit is only cleared in reclaim loop. > >> > >> There is one option for A-bit clearing: > >> - clear A-bit of base 4K page in isolation loop and leave large foli= o > >> A-bit clearing to reclaim loop. > >> > >> This patch didn't use it because don't want to introduce A-bit clearin= g > >> in two places. But I am open about clearing base 4K page A-bit cleared= in > >> isolation loop. Thanks. > > > > Sorry but why are we trying to do multiple things in one patch that I > > assumed is supposed to simply fix madvise() for large anon folios? And > > none of those things seems to have a clear rationale behind it. > > > > The only patch that makes sense at the moment (or the first patch of a > > series) is what I said before: > > > > - if (folio_mapcount(folio) !=3D 1) > > + if (folio_estimated_sharers(folio) !=3D 1) > Definitely. As I replied to you, I will split the patch to two parts: > - just bug fixing. Include the filio_mapcount() -> folio_estimated_shar= es(). I'm onboard with this fix. > And using ptep_clear_flush_young_notify() to clear the young of PTEs. This is another fix (if it's a real problem), hence a separate patch. > - refactor for large folio. Minchan will look at the last two. > Let me know if this is OK. Thanks. SGTM. Thanks.