From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89FC1C677F1 for ; Mon, 16 Jan 2023 19:28:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B8E996B0073; Mon, 16 Jan 2023 14:28:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id ABFD86B007B; Mon, 16 Jan 2023 14:28:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 863626B0073; Mon, 16 Jan 2023 14:28:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 728796B0072 for ; Mon, 16 Jan 2023 14:28:19 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 3CB32120608 for ; Mon, 16 Jan 2023 19:28:19 +0000 (UTC) X-FDA: 80361648318.20.8BE5479 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf26.hostedemail.com (Postfix) with ESMTP id 92B28140010 for ; Mon, 16 Jan 2023 19:28:17 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=oXpUTDz4; spf=none (imf26.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1673897297; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=USUc95np/oC5ESvdQAQk73Xl5bY25Yqt/iUL08/EgVo=; b=la3QcphWq9hKKLTARVs2GFf0yhRENDZ8gktA0R9fl6W/vAg8he3BEdyTFbrerGIZJ10fVC NFlM2H7vusjn0EBODpqWGi2ADo7GAKDyMfch6XLy6PweCBuZXUwEi3RV/tgg5MWMzn7uJj C/YH91Tkg0V56j1tzONwAwMXn3O1O1g= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=oXpUTDz4; spf=none (imf26.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1673897297; a=rsa-sha256; cv=none; b=L5z05Zdvj5WBf6+9eDsyw48O+el9ExAD0ktDNs4hkLK/ERlzR0C2UWLFkm6eHu8a52K9b7 c918Ry1W/3+DTFG33LLUmzcrK1puDzweP21Jjuk+yyKhB98feBDhP83+EReFCsCml+EJVm SXHtR37sHx1iHi6bMG3l7u/nG8H9McE= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=USUc95np/oC5ESvdQAQk73Xl5bY25Yqt/iUL08/EgVo=; b=oXpUTDz4qDthN1c8AvQAXgZtIE CnLalPWVToBMJBAvbilt0v3heHBQtFj+ja7aff0jcoPHxeMA9iQZtO0rqYMsGeI/VSUa4ecf9oR23 C5Z1GQVmrlSH5Epduvag9ZqYwAI77tKHmQN9JQ8J/ttRv9RYjeh1hzHHQIlB2+B5k/MY5aWHdIT7B EGgEF9trkJWfYIvlgPehCsK8Hool61nRzU5XLmoiRDwViz9utNM/674Y6bNNde51ZhJNOiDvVRTEO bnNFkPvCDk6Ikj/dnNsv834f8LiXkTXavZRhHnklr9cR1BDZGpE2rSlDxt7+InLqDnV/qL3hcD+ei JsiZ1hTg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1pHV9h-0090Su-72; Mon, 16 Jan 2023 19:28:29 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org, Andrew Morton Cc: "Matthew Wilcox (Oracle)" Subject: [PATCH 1/4] mm: Remove page_evictable() Date: Mon, 16 Jan 2023 19:28:24 +0000 Message-Id: <20230116192827.2146732-2-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230116192827.2146732-1-willy@infradead.org> References: <20230116192827.2146732-1-willy@infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 92B28140010 X-Stat-Signature: ue7xot14dphnw4cmkuc6r9ypd9y4f3hh X-HE-Tag: 1673897297-526579 X-HE-Meta: U2FsdGVkX1/xnLo9zrOIat5mCZgXjzFQ5jPNhjhrWSe/15riza8OY49ub9p8OjJknkQE9GJ1RN7FoDFGkBzBrTZOQPu+5brfJ9Olmg/HNlGmcxd04MoQFi+AY5U7T8ahxYu3zxXaC0voe8C+DlCIXWarNWaOQAKDLkmiCchdi366vajYUme5I2XwNfvyE0wxchienCc416HzF1G0LrzeEcADC8ybu06vsr9V8PRO3cYRm7DfFgO613bmi9CvVcPUm5MYydfCXHDQSEH9FoOdkFWihnsWRHOmzeSYn6XB9Fn3Eefg293JX04wYvb0xE9IAFqRgcMSCnuH1ioim2o65W0bXr86ViV9XWoi69ql91j8aMfGm/3Ny52qra84Tz238AknpC9aZiP9d/74kt3FtEa6e3MCSNYPOB3Lz4MygV/vSTgxjYjpSvpcvXL78A+EDAHU/sw80+dMOXiFpY2Pzh3nSqrdE8B3gQ4pZm1gFd7wqLca23TmyTVotDsN2SdQ8GXSqpcA/nRTDPdExKdhc8nrXGl9x5bUCkfnZOMaJkWnb+nyNvnmq4FlBsPEcLwXkzcj1YkNySN+K75wZsPsfdTVaICdeuoWe7exRXatw/nuINJwpLW+T04cozvwFZhRpfl0dNT23aqoaA4Jgkz8NBgrq52qa5o90veSUIHzuL7pzwzbrtSVAMbMh5ZyxNq+B5zZplI+L7DKYpBbozqfDNaoegPj4XEh1XZ8R5dF2rogNIW3kF+BKo3y5XDWvoNXfylXlJAo2fsnllBJZzGIBvmOZQh6BrNwfKuSuvB3Hk/dU7/87tdqs6qEocUm5sbD1emmiZtNsoBj8yM6rtlIMDEQvuiljXXNNm0ADi7oRmFnjQ10u1BvXe7icFuYEfbc4sdtv+yVOh6bGOgHgnZA28dhVoxTF2UK81e8HdJnRd8HNwxuutTlww== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This function now has no users. Also update the unevictable-lru documentation to discuss folios instead of pages (mostly). Signed-off-by: Matthew Wilcox (Oracle) --- Documentation/mm/unevictable-lru.rst | 89 ++++++++++++++-------------- mm/internal.h | 11 ---- 2 files changed, 46 insertions(+), 54 deletions(-) diff --git a/Documentation/mm/unevictable-lru.rst b/Documentation/mm/unevictable-lru.rst index 2a90d0721dd9..1972d37d97cf 100644 --- a/Documentation/mm/unevictable-lru.rst +++ b/Documentation/mm/unevictable-lru.rst @@ -12,7 +12,7 @@ Introduction This document describes the Linux memory manager's "Unevictable LRU" infrastructure and the use of this to manage several types of "unevictable" -pages. +folios. The document attempts to provide the overall rationale behind this mechanism and the rationale for some of the design decisions that drove the @@ -27,8 +27,8 @@ The Unevictable LRU =================== The Unevictable LRU facility adds an additional LRU list to track unevictable -pages and to hide these pages from vmscan. This mechanism is based on a patch -by Larry Woodman of Red Hat to address several scalability problems with page +folios and to hide these folios from vmscan. This mechanism is based on a patch +by Larry Woodman of Red Hat to address several scalability problems with folio reclaim in Linux. The problems have been observed at customer sites on large memory x86_64 systems. @@ -52,40 +52,41 @@ The infrastructure may also be able to handle other conditions that make pages unevictable, either by definition or by circumstance, in the future. -The Unevictable LRU Page List ------------------------------ +The Unevictable LRU Folio List +------------------------------ -The Unevictable LRU page list is a lie. It was never an LRU-ordered list, but a -companion to the LRU-ordered anonymous and file, active and inactive page lists; -and now it is not even a page list. But following familiar convention, here in -this document and in the source, we often imagine it as a fifth LRU page list. +The Unevictable LRU folio list is a lie. It was never an LRU-ordered +list, but a companion to the LRU-ordered anonymous and file, active and +inactive folio lists; and now it is not even a folio list. But following +familiar convention, here in this document and in the source, we often +imagine it as a fifth LRU folio list. The Unevictable LRU infrastructure consists of an additional, per-node, LRU list -called the "unevictable" list and an associated page flag, PG_unevictable, to -indicate that the page is being managed on the unevictable list. +called the "unevictable" list and an associated folio flag, PG_unevictable, to +indicate that the folio is being managed on the unevictable list. The PG_unevictable flag is analogous to, and mutually exclusive with, the -PG_active flag in that it indicates on which LRU list a page resides when +PG_active flag in that it indicates on which LRU list a folio resides when PG_lru is set. -The Unevictable LRU infrastructure maintains unevictable pages as if they were +The Unevictable LRU infrastructure maintains unevictable folios as if they were on an additional LRU list for a few reasons: - (1) We get to "treat unevictable pages just like we treat other pages in the + (1) We get to "treat unevictable folios just like we treat other folios in the system - which means we get to use the same code to manipulate them, the same code to isolate them (for migrate, etc.), the same code to keep track of the statistics, etc..." [Rik van Riel] - (2) We want to be able to migrate unevictable pages between nodes for memory + (2) We want to be able to migrate unevictable folios between nodes for memory defragmentation, workload management and memory hotplug. The Linux kernel - can only migrate pages that it can successfully isolate from the LRU + can only migrate folios that it can successfully isolate from the LRU lists (or "Movable" pages: outside of consideration here). If we were to - maintain pages elsewhere than on an LRU-like list, where they can be - detected by isolate_lru_page(), we would prevent their migration. + maintain folios elsewhere than on an LRU-like list, where they can be + detected by folio_isolate_lru(), we would prevent their migration. -The unevictable list does not differentiate between file-backed and anonymous, -swap-backed pages. This differentiation is only important while the pages are, -in fact, evictable. +The unevictable list does not differentiate between file-backed and +anonymous, swap-backed folios. This differentiation is only important +while the folios are, in fact, evictable. The unevictable list benefits from the "arrayification" of the per-node LRU lists and statistics originally proposed and posted by Christoph Lameter. @@ -158,7 +159,7 @@ These are currently used in three places in the kernel: Detecting Unevictable Pages --------------------------- -The function page_evictable() in mm/internal.h determines whether a page is +The function folio_evictable() in mm/internal.h determines whether a folio is evictable or not using the query function outlined above [see section :ref:`Marking address spaces unevictable `] to check the AS_UNEVICTABLE flag. @@ -167,7 +168,7 @@ For address spaces that are so marked after being populated (as SHM regions might be), the lock action (e.g. SHM_LOCK) can be lazy, and need not populate the page tables for the region as does, for example, mlock(), nor need it make any special effort to push any pages in the SHM_LOCK'd area to the unevictable -list. Instead, vmscan will do this if and when it encounters the pages during +list. Instead, vmscan will do this if and when it encounters the folios during a reclamation scan. On an unlock action (such as SHM_UNLOCK), the unlocker (e.g. shmctl()) must scan @@ -176,41 +177,43 @@ condition is keeping them unevictable. If an unevictable region is destroyed, the pages are also "rescued" from the unevictable list in the process of freeing them. -page_evictable() also checks for mlocked pages by testing an additional page -flag, PG_mlocked (as wrapped by PageMlocked()), which is set when a page is -faulted into a VM_LOCKED VMA, or found in a VMA being VM_LOCKED. +folio_evictable() also checks for mlocked folios by calling +folio_test_mlocked(), which is set when a folio is faulted into a +VM_LOCKED VMA, or found in a VMA being VM_LOCKED. -Vmscan's Handling of Unevictable Pages +Vmscan's Handling of Unevictable Folios -------------------------------------- -If unevictable pages are culled in the fault path, or moved to the unevictable -list at mlock() or mmap() time, vmscan will not encounter the pages until they +If unevictable folios are culled in the fault path, or moved to the unevictable +list at mlock() or mmap() time, vmscan will not encounter the folios until they have become evictable again (via munlock() for example) and have been "rescued" from the unevictable list. However, there may be situations where we decide, -for the sake of expediency, to leave an unevictable page on one of the regular +for the sake of expediency, to leave an unevictable folio on one of the regular active/inactive LRU lists for vmscan to deal with. vmscan checks for such -pages in all of the shrink_{active|inactive|page}_list() functions and will -"cull" such pages that it encounters: that is, it diverts those pages to the +folios in all of the shrink_{active|inactive|page}_list() functions and will +"cull" such folios that it encounters: that is, it diverts those folios to the unevictable list for the memory cgroup and node being scanned. -There may be situations where a page is mapped into a VM_LOCKED VMA, but the -page is not marked as PG_mlocked. Such pages will make it all the way to -shrink_active_list() or shrink_page_list() where they will be detected when -vmscan walks the reverse map in folio_referenced() or try_to_unmap(). The page -is culled to the unevictable list when it is released by the shrinker. +There may be situations where a folio is mapped into a VM_LOCKED VMA, +but the folio does not have the mlocked flag set. Such folios will make +it all the way to shrink_active_list() or shrink_page_list() where they +will be detected when vmscan walks the reverse map in folio_referenced() +or try_to_unmap(). The folio is culled to the unevictable list when it +is released by the shrinker. -To "cull" an unevictable page, vmscan simply puts the page back on the LRU list -using putback_lru_page() - the inverse operation to isolate_lru_page() - after -dropping the page lock. Because the condition which makes the page unevictable -may change once the page is unlocked, __pagevec_lru_add_fn() will recheck the -unevictable state of a page before placing it on the unevictable list. +To "cull" an unevictable folio, vmscan simply puts the folio back on +the LRU list using folio_putback_lru() - the inverse operation to +folio_isolate_lru() - after dropping the folio lock. Because the +condition which makes the folio unevictable may change once the folio +is unlocked, __pagevec_lru_add_fn() will recheck the unevictable state +of a folio before placing it on the unevictable list. MLOCKED Pages ============= -The unevictable page list is also useful for mlock(), in addition to ramfs and +The unevictable folio list is also useful for mlock(), in addition to ramfs and SYSV SHM. Note that mlock() is only available in CONFIG_MMU=y situations; in NOMMU situations, all mappings are effectively mlocked. diff --git a/mm/internal.h b/mm/internal.h index 2d09a7a0600a..74bc1fe45711 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -159,17 +159,6 @@ static inline bool folio_evictable(struct folio *folio) return ret; } -static inline bool page_evictable(struct page *page) -{ - bool ret; - - /* Prevent address_space of inode and swap cache from being freed */ - rcu_read_lock(); - ret = !mapping_unevictable(page_mapping(page)) && !PageMlocked(page); - rcu_read_unlock(); - return ret; -} - /* * Turn a non-refcounted page (->_refcount == 0) into refcounted with * a count of one. -- 2.35.1