From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C543EB64DC for ; Sat, 15 Jul 2023 06:07:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0AA0E6B0071; Sat, 15 Jul 2023 02:07:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 059216B0072; Sat, 15 Jul 2023 02:07:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E62BF6B0074; Sat, 15 Jul 2023 02:07:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id D7F0E6B0071 for ; Sat, 15 Jul 2023 02:07:17 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 79DFF1C79A9 for ; Sat, 15 Jul 2023 06:07:17 +0000 (UTC) X-FDA: 81012813714.11.C133BEE Received: from mail-qt1-f178.google.com (mail-qt1-f178.google.com [209.85.160.178]) by imf28.hostedemail.com (Postfix) with ESMTP id BB729C001A for ; Sat, 15 Jul 2023 06:07:15 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=FjpzhPEP; spf=pass (imf28.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.178 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689401235; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WfySVCw+dpSTsc2B4oZPue0u3zK7+gWWCOiPp0wEcv4=; b=pt86cYcmbCmfORpXObHgj9rW/Q6YjY4POeefwL4+eioqWSefXiuuG2pdwJ0a34v1J49tAS Xz5UYpbZxzTqm4+C6UiXjk+KrBhAuaeNJCa4RgRKFh/27a1VHh/Bk0L5ykSx9P2JoGsSXL ATvrq8BX3gJiJm2IUdrraTg3RyP5JUg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689401235; a=rsa-sha256; cv=none; b=lCYF0j40dkAAasHroYQUM9Yurpje7yLUmxe95zw/Be6Htm/JYDHsftSxg8+7R3xKSTul3S x3B18TsbTOiGsDCJ6nEBnAFhearOcgh+GnqqKgJDmBpP54zHpTePN+kXbBlQMqWu/el4H4 9Azup55l8ca3OM5PMttuOdPYUuxijQI= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=FjpzhPEP; spf=pass (imf28.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.178 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-qt1-f178.google.com with SMTP id d75a77b69052e-403b622101bso103131cf.1 for ; Fri, 14 Jul 2023 23:07:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689401235; x=1691993235; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=WfySVCw+dpSTsc2B4oZPue0u3zK7+gWWCOiPp0wEcv4=; b=FjpzhPEP83eaD5w6Z6mPAwSFgtQ/CSnpow4tvOH87Li3FnGeF8BbVB7frhWkwJHzCa vehXELES21VPBnrYq/800TmG7L6GNqAtfplXqWCJnGvzuqRBYnyWoOsUM1ED5OWMnJbR n8kWVM66vx6J9HIk9c98Cnf79Ls02VE5TcRUh7UwU6tzIfWUfTQ8GuLuyqmW/rMaXruP ONdPTNsFs5qyhE8vsmV8TRXuOqKP/aEFtbbfwcoGlI8hzM1LgjPITxIgw6SDX+5IofEd i2TUxPkw6izO03Y+4LYq9fUFWDlrSzjnbquk1lEbfRDlfxLMfXzqBV92nBynl8kJ78BS pfaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689401235; x=1691993235; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WfySVCw+dpSTsc2B4oZPue0u3zK7+gWWCOiPp0wEcv4=; b=fZz1YYerHKFUXJgUF7sz75w16JTMq3AyJarpwDPUkiYzDueZWBQYck9m9dL9j+mH8W RL3Jb5NkA9sH2A1O88fKNCpq0rR3dDW/t2mdGVRuL8+VekLbJkrVTXaEfyI/Xz7dSGdk HZAcz+vMAvaPHUDuCdinNGmoj7PsZYo5hODATUC5Tk+31M1PF8rXVw1P9yDS/OOt5egj annP9OUzuoGr+Ea3iUDWf65m+6aifZkAdsQCHrQc6iuEUYtzJIej4X5w2ic7F32l2lBv CpoSrj+FklHIQqktg3KL33hXiL8QBsEkQYrQth3hYNa7Tsp6U3JsUoctAR/s4d92H+1e WzpQ== X-Gm-Message-State: ABy/qLYWmPhZcsY9WLobShv/SWpR16IxUFBA2MKP9yyujqwUzBx6LYN4 b7XME+FgfM6vphM7DfIxCI7Ednc5aJ0yY18Xs/+wDQ== X-Google-Smtp-Source: APBJJlEN3scYa9/MXXc3sIsMXw+QEoBrMEBa9G2/rGleo/rVI4DffDqmiXH4E7i9LoBBSPrF9tdqCFEumTzco4s7Q9g= X-Received: by 2002:ac8:5814:0:b0:3ef:404a:b291 with SMTP id g20-20020ac85814000000b003ef404ab291mr991086qtg.7.1689401234707; Fri, 14 Jul 2023 23:07:14 -0700 (PDT) MIME-Version: 1.0 References: <20230712060144.3006358-1-fengwei.yin@intel.com> <20230712060144.3006358-4-fengwei.yin@intel.com> In-Reply-To: From: Yu Zhao Date: Sat, 15 Jul 2023 00:06:38 -0600 Message-ID: Subject: Re: [RFC PATCH v2 3/3] mm: mlock: update mlock_pte_range to handle large folio To: Yin Fengwei Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, willy@infradead.org, david@redhat.com, ryan.roberts@arm.com, shy828301@gmail.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: 1go8k7gjmganu73xn3fu1gjmtcr4etnw X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: BB729C001A X-Rspam-User: X-HE-Tag: 1689401235-422178 X-HE-Meta: U2FsdGVkX19SHAGNZlzsb/qS+Jw8W3Ee/2DZ1V+7ZwLiD4CL4tsB4jTigo/uuU7SNDQbFJ4pBJ/S0XftcnDT2A66tf8Ywi/EEgd5yEmCvpdnlgfUgbZoOVg/vwj/3ZU0CawdN0f4ah+SqyAEf9H125C+GaHCvk+aZ94egeMYYBNqh7saO5XC5DZGmWmKisLQ1ge0sqrbN9ktbM97PVnkW8/nRV1dhmNEvqYSqMdfYVap/w3FQBL5PIRF26A7u9ldQzgLmEMyuomRyn5OUgGi83lifGhAf6ztn9nCe2bM3eDrDh+GyvmqjhWlFiVn0KudwwHtWX1h8ABbWlbQth1MyvLae0H8iXskGqrE5vWShcvld89j7F1W+N3ZEHXgAkjMvnYOeW7ZogsgYWVlPTYgjigY8d0Jw1oBs9wH5j/l9IVdvSd4FWZD3g+yZwSgoDRV3+Inie88Slegp7VPQP07XIEZMw5UxoufHeGVc4x9sp41wOEsYUC9Huf3eL/xTTsaOoufa8yTPVKHH8atMxz6MmSReVgAmuKFdJdorBLc86z9ocATQmm58B7PpS6HDOyhN7Bt6ZWXkCf8e77V07foZtcmi+BQ0WTRG6pDlz/qtFZSJXBrYC39XVttP6kXTyA5qM2UAI5geVjdsJC7XMhQrsjhsAl9h5NwNS7zc0DvcUcji8xVRD6f2Yt8iLepJbB3a07MFzDSR8koMcGYGUtDqHv5Ul8bYuYJBW1n65LHSLus3GS4v8+5Jq20gYkQxSVXcI4Q5wUhdz+SHMOs0Ap4naehlQuk8SJu6bXEBkUt2+QtOs4rY9bMQlN27ZYDb+O9khz8iwHDZXPhSoQc1RXX2lV5fWaYTyztA1uJAk+BYVQK+T88XUpfeHbrxPrcYqZmLoxKzGPvcjofvoeEt4XJKAYZDoHnpJ0MuOlnC6S9UTst4nkTzDf4TIQUDvJOuqZd7OInI3TyLiKuENsWKn3 AZAoeDvd YV3JjMSkPB7HuAp+MjrYcBLF70cfYpVtt2sPRR+qHLwhWAkGJ3Qx/NrkGtj0NHCHjQrvpUkIgoZmfHG5yfQp5ZmZdyp5iZ7pkT9/TfPaSREP0A9yHkfoI53zdbFTNjBC/P/uPLRnKK6F+1ErZ6ioFJx/sMaS/K+HZc/xMz0/4/F7hMk9u2oYPAlUVGQgH2bepLRZnhV+Ry7q4fvm8En109wtxx3R4Ce7Mh5GLlE0egaD9a0xvPHX6ZWKX/FVs8yUQDBsKBYQCRbDnud109Yck0aCKM8De1+iV+oxQO2/YYgomM/GHhK3hPNHCy+5xRrUJnAc7 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Jul 12, 2023 at 12:31=E2=80=AFAM Yu Zhao wrote: > > On Wed, Jul 12, 2023 at 12:02=E2=80=AFAM Yin Fengwei wrote: > > > > Current kernel only lock base size folio during mlock syscall. > > Add large folio support with following rules: > > - Only mlock large folio when it's in VM_LOCKED VMA range > > > > - If there is cow folio, mlock the cow folio as cow folio > > is also in VM_LOCKED VMA range. > > > > - munlock will apply to the large folio which is in VMA range > > or cross the VMA boundary. > > > > The last rule is used to handle the case that the large folio is > > mlocked, later the VMA is split in the middle of large folio > > and this large folio become cross VMA boundary. > > > > Signed-off-by: Yin Fengwei > > --- > > mm/mlock.c | 104 ++++++++++++++++++++++++++++++++++++++++++++++++++--- > > 1 file changed, 99 insertions(+), 5 deletions(-) > > > > diff --git a/mm/mlock.c b/mm/mlock.c > > index 0a0c996c5c214..f49e079066870 100644 > > --- a/mm/mlock.c > > +++ b/mm/mlock.c > > @@ -305,6 +305,95 @@ void munlock_folio(struct folio *folio) > > local_unlock(&mlock_fbatch.lock); > > } > > > > +static inline bool should_mlock_folio(struct folio *folio, > > + struct vm_area_struct *vma) > > +{ > > + if (vma->vm_flags & VM_LOCKED) > > + return (!folio_test_large(folio) || > > + folio_within_vma(folio, vma)); > > + > > + /* > > + * For unlock, allow munlock large folio which is partially > > + * mapped to VMA. As it's possible that large folio is > > + * mlocked and VMA is split later. > > + * > > + * During memory pressure, such kind of large folio can > > + * be split. And the pages are not in VM_LOCKed VMA > > + * can be reclaimed. > > + */ > > + > > + return true; > > Looks good, or just > > should_mlock_folio() // or whatever name you see fit, can_mlock_folio()? > { > return !(vma->vm_flags & VM_LOCKED) || folio_within_vma(); > } > > > +} > > + > > +static inline unsigned int get_folio_mlock_step(struct folio *folio, > > + pte_t pte, unsigned long addr, unsigned long en= d) > > +{ > > + unsigned int nr; > > + > > + nr =3D folio_pfn(folio) + folio_nr_pages(folio) - pte_pfn(pte); > > + return min_t(unsigned int, nr, (end - addr) >> PAGE_SHIFT); > > +} > > + > > +void mlock_folio_range(struct folio *folio, struct vm_area_struct *vma= , > > + pte_t *pte, unsigned long addr, unsigned int nr) > > +{ > > + struct folio *cow_folio; > > + unsigned int step =3D 1; > > + > > + mlock_folio(folio); > > + if (nr =3D=3D 1) > > + return; > > + > > + for (; nr > 0; pte +=3D step, addr +=3D (step << PAGE_SHIFT), n= r -=3D step) { > > + pte_t ptent; > > + > > + step =3D 1; > > + ptent =3D ptep_get(pte); > > + > > + if (!pte_present(ptent)) > > + continue; > > + > > + cow_folio =3D vm_normal_folio(vma, addr, ptent); > > + if (!cow_folio || cow_folio =3D=3D folio) { > > + continue; > > + } > > + > > + mlock_folio(cow_folio); > > + step =3D get_folio_mlock_step(folio, ptent, > > + addr, addr + (nr << PAGE_SHIFT)); > > + } > > +} > > + > > +void munlock_folio_range(struct folio *folio, struct vm_area_struct *v= ma, > > + pte_t *pte, unsigned long addr, unsigned int nr) > > +{ > > + struct folio *cow_folio; > > + unsigned int step =3D 1; > > + > > + munlock_folio(folio); > > + if (nr =3D=3D 1) > > + return; > > + > > + for (; nr > 0; pte +=3D step, addr +=3D (step << PAGE_SHIFT), n= r -=3D step) { > > + pte_t ptent; > > + > > + step =3D 1; > > + ptent =3D ptep_get(pte); > > + > > + if (!pte_present(ptent)) > > + continue; > > + > > + cow_folio =3D vm_normal_folio(vma, addr, ptent); > > + if (!cow_folio || cow_folio =3D=3D folio) { > > + continue; > > + } > > + > > + munlock_folio(cow_folio); > > + step =3D get_folio_mlock_step(folio, ptent, > > + addr, addr + (nr << PAGE_SHIFT)); > > + } > > +} > > I'll finish the above later. There is a problem here that I didn't have the time to elaborate: we can't mlock() a folio that is within the range but not fully mapped because this folio can be on the deferred split queue. When the split happens, those unmapped folios (not mapped by this vma but are mapped into other vmas) will be stranded on the unevictable lru. For that matter, we can't mlock any large folios that are being shared, unless you want to overengineer it by checking whether all sharing vmas are also mlocked -- mlock is cleared during fork. So the condition for mlocking large anon folios is 1) within range 2) fully mapped 3) not shared (mapcount is 1). The final patch should look like something like this: - if (folio_test_large(folio)) + if (folio_pfn(folio) !=3D pte_pfn(ptent)) + continue; + if (!the_aforementioned_condition()) There is another corner case I forgot to mention: for example, what if a folio spans two (the only two) adjacent mlocked vmas? No need to worry about this since it's not worth optimizing.