From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7ECCCC001B0 for ; Wed, 9 Aug 2023 06:12:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 812B36B0071; Wed, 9 Aug 2023 02:12:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7C4B66B0074; Wed, 9 Aug 2023 02:12:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68B168D0001; Wed, 9 Aug 2023 02:12:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 5637C6B0071 for ; Wed, 9 Aug 2023 02:12:30 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 1252AC0299 for ; Wed, 9 Aug 2023 06:12:30 +0000 (UTC) X-FDA: 81103546860.26.2ADC95A Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.151]) by imf16.hostedemail.com (Postfix) with ESMTP id A89A8180004 for ; Wed, 9 Aug 2023 06:12:27 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="FogqE/TA"; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf16.hostedemail.com: domain of fengwei.yin@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=fengwei.yin@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1691561548; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=fZ1ATqiHZoq5dPzW1daeieQbJsADEPSAXMhZZPx0hhA=; b=XMLzlznOkw24sV4IqoCnKaoGQo1XhdoihenYwa34a4HN0ET5TlSKABnc7EqaGOIBBK/NBZ Wb2hqHuabn/31tfDnXCJJPpRMRWSOA9KJPVH2B05RCecuYwhHcswEYYSqHm132bkWC2oJ1 F4ZFAhH8FHhnS+fuIo+5Apali1x2xvw= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="FogqE/TA"; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf16.hostedemail.com: domain of fengwei.yin@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=fengwei.yin@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1691561548; a=rsa-sha256; cv=none; b=sabhAfyEZESKRY9HGahRyW6DcOl5ZdLzATYGauiqlD290aeoYmfhzbHMSxHA6esUMIugXf S+PRsRyxx+02CasiO/0Pfb8kn4RsVXKnBOEUEZAj6vibfe5PnyXmGtwo02UdntlRAmKMnc +RdO9VNfIyTpre1m/hjyCjTOp2abd5U= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691561547; x=1723097547; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=xtHRYNdtzfqEVZ/MBsxxW2ieL6E0Nq0jjS7kmoGVOpk=; b=FogqE/TAYmid0e91NAb0nnKuCHdcBCXbZ4wG++Rlqs1Vuasv5vMQDVvs 14MayBrGieblVm43izv/ypiCOBpUE6goPI9kSNAjokPQlDmtljUWKwySR F/IDLZEBHIQKw2wE0GD3aP3tA3OB5diOS9SMgttlfjZn2dRZNNAHQRIiY xQxEEg9nIjDrs99kIoef8T/ctgTJUCjjnD6v4wmWq8Oi9fqHPrQSlD70l JZjh3fWjYCWDPuCoKFL80rj8S61K5spoFWChDCzsehPQFf3Pum5Kc13Fy CRYAc2q1HsNNfbHgc9T/9kGfQfEH1UeZ16n6lD+pe5RY2upXamkpiodcN g==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="351341580" X-IronPort-AV: E=Sophos;i="6.01,158,1684825200"; d="scan'208";a="351341580" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 23:12:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="845810545" X-IronPort-AV: E=Sophos;i="6.01,158,1684825200"; d="scan'208";a="845810545" Received: from fyin-dev.sh.intel.com ([10.239.159.32]) by fmsmga002.fm.intel.com with ESMTP; 08 Aug 2023 23:12:22 -0700 From: Yin Fengwei To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, yuzhao@google.com, willy@infradead.org, hughd@google.com, yosryahmed@google.com, ryan.roberts@arm.com, david@redhat.com, shy828301@gmail.com Cc: fengwei.yin@intel.com Subject: [PATCH v2 0/3] support large folio for mlock Date: Wed, 9 Aug 2023 14:11:02 +0800 Message-Id: <20230809061105.3369958-1-fengwei.yin@intel.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: A89A8180004 X-Stat-Signature: b3fkrud74becr3ksgcsk9ddsedc7zatc X-HE-Tag: 1691561547-277115 X-HE-Meta: U2FsdGVkX1/Rq3PFFhHaJhIa3+eKJfFt5gFPO7hMZzALN3Qt6DAktAZAZ61QcfcRa/07m+vjSJ83ZerXYw+K5Yknn7+hRS5eXbmTsixHdfhu2OWTzFntO1Z9cVuZmQomu0lPcZ7ZrBJ07OchD6HjyLM7nAZdoN+FQ/SfWf357/c9ad74vlxAj9e1VQYTM7pL5yLGlEPW8sjBXduqAug1nLlDuUl8RRTUXSGv5vQOztK3zUhMBWZOER1zsLTyQN1om/+qT0cnKVzQtOXYx/U0GASV/wFrJjd/E6HneVtKCgupvP1gXGzZmuCv69T2A3cbQ87bk3x3UmS1XHRPaHCBd32eSmyI3ulWqJQ76TteHy+vgxOmSuONcD58PbZCKt0xl55Hsoi+/E1X7DUuCggFcfs0Z5+dkkw1q3P1BRYs1dwnDQxIDQ8iibbEyrfNppMuUN8ChIogk9Zd2ii3WBLNHJ5gyON6LUsOPGdu0aBGQ7PV7ep0yIWLwHXyuECXiSuqRrBmnH4/OnfbUsGtkAMnKPFSOT1toloG7F35cKg0+0/thdxfq2nfxcy/Jjid/KgRYtJkF9YjvJxAJgMTFK7oL6QX44j7F9eCz1yuFtppGzD+WJkL6gGj4DKMUcgj515JnKmu9XZM4DGCYR3f6YRNiHGVMKwJcQl6ufWn2HVoJIn0KH6LmYO/WBpmy+ZPTqmNtPhu14uipzzgvfUxunKsa9tkP3iDCBWr00LZd/CvVnvuPw6LPeOnHRTUYLQyJ+ZPayqcjlqn9tRZLJa+jvPk1IipIWNXu0jElOXQH6yatzczfXbxGksGWR1pCEx1jV9Ii19xrb438oSO/rVE4k/2xEJcP8QNAi9cYnQV6cLN9o+fNkTAxCGmtRWvG1QwHmI3+aFDu+iUYPuwvrjfAYJfXDoYWh+mRICJwXTIsjYDcZ9eQaHzpJDVtiogIVQ6O4Wa4hpSghRenCwzwFjtyMS FG5yaGTU CHGpO6KSdZqyg3w8rgQCcgrrSre1wi9OFm1owOyPXpxldIGPZwCoUVo1UMa4V/SwvbPkonHlWG9uk7yvlrmV1sbRCf9X+NTFuDXpFbk673T4YMdq/zbAo32S0wl6SPtOIzbSQChq4bxGGJj7RMlir5r8+3L7oCiQfhV8afRQesYJ2Yp2EYEHCVlavEdU9xnRh1QEMsKGeT8edyj/qdOhfbx2TV/3Z6i7Art3yJP9r4NCunWo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Yu mentioned at [1] about the mlock() can't be applied to large folio. I leant the related code and here is my understanding: - For RLIMIT_MEMLOCK related, there is no problem. Because the RLIMIT_MEMLOCK statistics is not related underneath page. That means underneath page mlock or munlock doesn't impact the RLIMIT_MEMLOCK statistics collection which is always correct. - For keeping the page in RAM, there is no problem either. At least, during try_to_unmap_one(), once detect the VMA has VM_LOCKED bit set in vm_flags, the folio will be kept whatever the folio is mlocked or not. So the function of mlock for large folio works. But it's not optimized because the page reclaim needs scan these large folio and may split them. This series identified the large folio for mlock to four types: - The large folio is in VM_LOCKED range and fully mapped to the range - The large folio is in the VM_LOCKED range but not fully mapped to the range - The large folio cross VM_LOCKED VMA boundary - The large folio cross last level page table boundary For the first type, we mlock large folio so page reclaim will skip it. For the second/third type, we don't mlock large folio. As the pages not mapped to VM_LOACKED range are mapped to none VM_LOCKED range, if system is in memory pressure situation, the large folio can be picked by page reclaim and split. Then the pages not mapped to VM_LOCKED range can be reclaimed. For the fourth type, we don't mlock large folio because locking one page table lock can't prevent the part in another last level page table being unmapped. Thanks to Ryan for pointing this out. To check whether the folio is fully mapped to the range, PTEs needs be checked to see whether the page of folio is associated. Which needs take page table lock and is heavy operation. So far, the only place needs this check is madvise and page reclaim. These functions already have their own PTE iterator. patch1 introduce API to check whether large folio is in VMA range. patch2 make page reclaim/mlock_vma_folio/munlock_vma_folio support large folio mlock/munlock. patch3 make mlock/munlock syscall support large folio. testing done: - kernel selftest. No extra failure introduced v1 was post here [2]. Yu also mentioned a race which can make folio unevictable after munlock during RFC v2 discussion [3]: We decided that race issue didn't block this series based on: - That race issue was not introduced by this series - We had a looks-ok fix for that race issue. Need to wait for mlock_count fixing patch as Yosry Ahmed suggested [4] ChangeLog from V1: - Remove the PTE check from folio_in_range() and reuse the page table iterator (in madvise and folio_referenced_one) to check whether fully mapped or not in callers - Avoid mlock the folio which cross last level page table. Thanks to Ryan for pointing this out. - Drop pte_none() check when iterate page table because we only care pte_present() case. - move folio_test_large() out of m(un)lock_vma_folio() ChangeLog from RFC v2: - Removed RFC - dropped folio_is_large() check as suggested by both Yu and Huge - Besides the address/pgoff check, also check the page table entry when check whether the folio is in the range. This is to handle mremap case that address/pgoff is in range, but folio can't be identified as in range. - Fixed one issue in page_add_anon_rmap() and page_add_anon_rmap() introdued by RFC v2. As these two functions can be called multiple times against one folio. And remove_rmap() may not be called same times. Which can bring imbalanced mlock_count. Fix it by skip mlock large folio in these two functions. [1] https://lore.kernel.org/linux-mm/CAOUHufbtNPkdktjt_5qM45GegVO-rCFOMkSh0HQminQ12zsV8Q@mail.gmail.com/ [2] https://lore.kernel.org/linux-mm/20230728070929.2487065-1-fengwei.yin@intel.com/ [3] https://lore.kernel.org/linux-mm/CAOUHufZ6=9P_=CAOQyw0xw-3q707q-1FVV09dBNDC-hpcpj2Pg@mail.gmail.com/ [4] https://lore.kernel.org/linux-mm/CAJD7tkZJFG=7xs=9otc5CKs6odWu48daUuZP9Wd9Z-sZF07hXg@mail.gmail.com/ Yin Fengwei (3): mm: add functions folio_in_range() and folio_within_vma() mm: handle large folio when large folio in VM_LOCKED VMA range mm: mlock: update mlock_pte_range to handle large folio mm/internal.h | 58 ++++++++++++++++++++++++++++++++++++-------- mm/mlock.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++++-- mm/rmap.c | 66 ++++++++++++++++++++++++++++++++++++++++++--------- 3 files changed, 167 insertions(+), 23 deletions(-) -- 2.39.2