From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28647CD1284 for ; Fri, 29 Mar 2024 01:58:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6906C6B0083; Thu, 28 Mar 2024 21:58:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6190C6B0087; Thu, 28 Mar 2024 21:58:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 492976B0089; Thu, 28 Mar 2024 21:58:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 28C0F6B0083 for ; Thu, 28 Mar 2024 21:58:43 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id CB3B680917 for ; Fri, 29 Mar 2024 01:58:42 +0000 (UTC) X-FDA: 81948417684.12.A74D448 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) by imf14.hostedemail.com (Postfix) with ESMTP id 3723010000B for ; Fri, 29 Mar 2024 01:58:40 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=HT8odmWn; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf14.hostedemail.com: domain of ying.huang@intel.com designates 198.175.65.10 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711677520; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aAnUQNx3LiQGrAv/6xtRf6SPXh2A8tCdiNxsYwgvvoc=; b=xgikmBvmcAsO0n9EEqjUbLa0mROu3Wkc6zjEkCg2AHMsfiuN9tQRfa9u6iNsVfW+BpjrOP 2jW2ed27l4Eeb5e5avHEVjmezGmLSYK03rHZiPCTD8HXcNHwu3967HguEMZglsbjCdghNA zjAeuUoDAWDtHSXcU2Rt1VjWd+PDnNg= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=HT8odmWn; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf14.hostedemail.com: domain of ying.huang@intel.com designates 198.175.65.10 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711677520; a=rsa-sha256; cv=none; b=auaU2CB6CYwavBEn+tkVSBeQSqdb5wALu+OAp0kd6J07S5Qt0LktoDZ+Y+JGc86cq76vfZ m9Jzpz9xdGna0RY3vgux3CSS7eFSDM25DwswBsVGCMrfJ9BcEKaAtkwKhQ4ptU4sYFw3hN K7qOCmBxp8Xvda7XrVNzVXtUDfhJk9g= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1711677520; x=1743213520; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version; bh=AFhrIjc8JM02NWzsD55P+Fymar831ZCBgitc1/ahE2k=; b=HT8odmWnKvr2jX2A8OlafEruZQL8TJNmJca29/irX5VqH/lcYpNkIhLP Yq2CUWVC5jzM1e+IK7X+kAY1w+2Ubg/mGm8DS75H+y5hTX1IH7TNdyx9c ft7CwbqBbe/Ga3k0gC5YAu97lzCNh1AOrt8cMbWF/l9N7MTuIdv59+DOi J27mSF++kBtQAxYL/wqjb8+wAePdgMN8gFGEmFvbW5rBdgWrXsc26gHv9 2s2BrB39/Ie6KBjrygvGgWTowMPAnkzXfbVm/NIWNZli9hhzf/Nco9Kzb 6lq5/fHPo8l5xEcdcUCIze4NG90gSBIfklZU6f0fdIbSKeGfChGByu3tC g==; X-CSE-ConnectionGUID: BsFR5cc8QhqBi/9k8n8QpA== X-CSE-MsgGUID: xdghMqgFS1ikSjw1+eSo8w== X-IronPort-AV: E=McAfee;i="6600,9927,11027"; a="24314631" X-IronPort-AV: E=Sophos;i="6.07,162,1708416000"; d="scan'208";a="24314631" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2024 18:58:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,162,1708416000"; d="scan'208";a="21343371" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2024 18:58:34 -0700 From: "Huang, Ying" To: Ryan Roberts Cc: Andrew Morton , David Hildenbrand , Matthew Wilcox , Gao Xiang , Yu Zhao , Yang Shi , Michal Hocko , Kefeng Wang , Barry Song <21cnbao@gmail.com>, Chris Li , Lance Yang , , Subject: Re: [PATCH v5 1/6] mm: swap: Remove CLUSTER_FLAG_HUGE from swap_cluster_info:flags In-Reply-To: <20240327144537.4165578-2-ryan.roberts@arm.com> (Ryan Roberts's message of "Wed, 27 Mar 2024 14:45:32 +0000") References: <20240327144537.4165578-1-ryan.roberts@arm.com> <20240327144537.4165578-2-ryan.roberts@arm.com> Date: Fri, 29 Mar 2024 09:56:41 +0800 Message-ID: <871q7tls12.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspamd-Queue-Id: 3723010000B X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: m9wbx5ekrjyxz9phukz378aq8wz7aero X-HE-Tag: 1711677519-123335 X-HE-Meta: U2FsdGVkX19UcxSQZ/3kJgiy49p2chpeNUZA3P8vI3W2RSSouwN/Lwi3HIJ7VMdZYLZ5RSRhoLvc0wnb+7AUq/8urhChPYrTPqXBModFMxhc3K3P6cTaVYcvWdrtO4sIZtRPtd7rfoS/a6m+k5CK1qmk7UgXyH6YSJOovub2DbWn9emlvT5iT959WNYClwJY+Zu7ELyf5N86AP6jF6fYlEHc54GdkKzr2KLwTJeylXv/GVkr14sut8KDwaTaX5TR1d4Bv2lfI1zT/iiXtXl/ndDlx/d9D2G4yk+Alm+eNFmHaOOkPQb77+pZjv0P7cTr9KvfdaFzD3rnUdi/xbyCJulwDFRltia3z2lo9idbYBKAQK1LLrKKWg0wjurd0wA5bGxjKWMomU5/aIV0ybTdfq1y4bxuBACW2uVZNwpSqIHAsL9AM09QFdGWJ2tXsw4BgMP4vwLQtY0ZcJ/hdbiGImqIWF83eFstxtDr94VZ6x+02TYXiSAAkMqdVym9OoIIEqUIN6pbphUjE7h0FaWWrWlTPF6uKqELbSUl1a4pyT1m8Az/QgqQnpyiWP+iA7ejvNleqJ0AXiee+iuIRtKthEqax5oZyOk8FcxeALB9gl70Q5kyoohaYGS7d4OmDdJ2l6aM5blwUpqE2rLqsBwmt1tLAc+09j3haTDq/bbl2CJkqI9fFKEmlG8GC7NHleNzvntfa/RhH8G5eZhl4jPGBE7g4/IR7VIuxJca+DVGZnSdId2hyyVTw4e8xQVImVp0S/hbwzGoO6wnSOqxlRduvQkkRD4/MPv0qElS+eGaVhDP9yVGPLFjNgwywlRxLMJ06PLlX3HDU5CzP1ZczLTO3DrvdmqURBucOqxS2Apdg+BWufl7qYmQB2KycXodJqKKbwQGhk2klhz/ze5Cqfg3arLgOUUUK4BN17CelB0rEVSBMox4gk8wcTXkkcQLTY+B6swreJHZPzVmWRWN47Q hXA5M9KN 9d3HJBVxTj02doZCtC148KU0+0hxXEa++UlvXXHsXcKbUCQCoIO1AHLmfu+Bw1k7cDcRHA6pCdHoia7Lo4UdLDvWc+Z2Rbz/7Pc6Zkp0gDHKcXRfIT8mZ9ifrWRb7MPGz/ryFB3aGTU1Vz4O2Uid5Ah9FoJukbzkAA2SR0NeRpXOOQdIYKukN4mWvZGGc7qhoqm8R56sg85vyZWA+x+VI2KOp2ngYO/gjp2TF0+1OTnSIt2GtXhTOCb9dosD9A484Mga5IribNpRbWy8oa1UAZbS8bg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Ryan Roberts writes: > As preparation for supporting small-sized THP in the swap-out path, > without first needing to split to order-0, Remove the CLUSTER_FLAG_HUGE, > which, when present, always implies PMD-sized THP, which is the same as > the cluster size. > > The only use of the flag was to determine whether a swap entry refers to > a single page or a PMD-sized THP in swap_page_trans_huge_swapped(). > Instead of relying on the flag, we now pass in nr_pages, which > originates from the folio's number of pages. This allows the logic to > work for folios of any order. > > The one snag is that one of the swap_page_trans_huge_swapped() call > sites does not have the folio. But it was only being called there to > shortcut a call __try_to_reclaim_swap() in some cases. > __try_to_reclaim_swap() gets the folio and (via some other functions) > calls swap_page_trans_huge_swapped(). So I've removed the problematic > call site and believe the new logic should be functionally equivalent. > > That said, removing the fast path means that we will take a reference > and trylock a large folio much more often, which we would like to avoid. > The next patch will solve this. > > Removing CLUSTER_FLAG_HUGE also means we can remove split_swap_cluster() > which used to be called during folio splitting, since > split_swap_cluster()'s only job was to remove the flag. > > Signed-off-by: Ryan Roberts LGTM, Thanks! Reviewed-by: "Huang, Ying" > --- > include/linux/swap.h | 10 ---------- > mm/huge_memory.c | 3 --- > mm/swapfile.c | 47 ++++++++------------------------------------ > 3 files changed, 8 insertions(+), 52 deletions(-) > > diff --git a/include/linux/swap.h b/include/linux/swap.h > index a211a0383425..f6f78198f000 100644 > --- a/include/linux/swap.h > +++ b/include/linux/swap.h > @@ -259,7 +259,6 @@ struct swap_cluster_info { > }; > #define CLUSTER_FLAG_FREE 1 /* This cluster is free */ > #define CLUSTER_FLAG_NEXT_NULL 2 /* This cluster has no next cluster */ > -#define CLUSTER_FLAG_HUGE 4 /* This cluster is backing a transparent huge page */ > > /* > * We assign a cluster to each CPU, so each CPU can allocate swap entry from > @@ -590,15 +589,6 @@ static inline int add_swap_extent(struct swap_info_struct *sis, > } > #endif /* CONFIG_SWAP */ > > -#ifdef CONFIG_THP_SWAP > -extern int split_swap_cluster(swp_entry_t entry); > -#else > -static inline int split_swap_cluster(swp_entry_t entry) > -{ > - return 0; > -} > -#endif > - > #ifdef CONFIG_MEMCG > static inline int mem_cgroup_swappiness(struct mem_cgroup *memcg) > { > diff --git a/mm/huge_memory.c b/mm/huge_memory.c > index b49fcb8a16cc..8c1f3393994a 100644 > --- a/mm/huge_memory.c > +++ b/mm/huge_memory.c > @@ -2961,9 +2961,6 @@ static void __split_huge_page(struct page *page, struct list_head *list, > shmem_uncharge(folio->mapping->host, nr_dropped); > remap_page(folio, nr); > > - if (folio_test_swapcache(folio)) > - split_swap_cluster(folio->swap); > - > /* > * set page to its compound_head when split to non order-0 pages, so > * we can skip unlocking it below, since PG_locked is transferred to > diff --git a/mm/swapfile.c b/mm/swapfile.c > index 5e6d2304a2a4..0d44ee2b4f9c 100644 > --- a/mm/swapfile.c > +++ b/mm/swapfile.c > @@ -343,18 +343,6 @@ static inline void cluster_set_null(struct swap_cluster_info *info) > info->data = 0; > } > > -static inline bool cluster_is_huge(struct swap_cluster_info *info) > -{ > - if (IS_ENABLED(CONFIG_THP_SWAP)) > - return info->flags & CLUSTER_FLAG_HUGE; > - return false; > -} > - > -static inline void cluster_clear_huge(struct swap_cluster_info *info) > -{ > - info->flags &= ~CLUSTER_FLAG_HUGE; > -} > - > static inline struct swap_cluster_info *lock_cluster(struct swap_info_struct *si, > unsigned long offset) > { > @@ -1027,7 +1015,7 @@ static int swap_alloc_cluster(struct swap_info_struct *si, swp_entry_t *slot) > offset = idx * SWAPFILE_CLUSTER; > ci = lock_cluster(si, offset); > alloc_cluster(si, idx); > - cluster_set_count_flag(ci, SWAPFILE_CLUSTER, CLUSTER_FLAG_HUGE); > + cluster_set_count(ci, SWAPFILE_CLUSTER); > > memset(si->swap_map + offset, SWAP_HAS_CACHE, SWAPFILE_CLUSTER); > unlock_cluster(ci); > @@ -1365,7 +1353,6 @@ void put_swap_folio(struct folio *folio, swp_entry_t entry) > > ci = lock_cluster_or_swap_info(si, offset); > if (size == SWAPFILE_CLUSTER) { > - VM_BUG_ON(!cluster_is_huge(ci)); > map = si->swap_map + offset; > for (i = 0; i < SWAPFILE_CLUSTER; i++) { > val = map[i]; > @@ -1373,7 +1360,6 @@ void put_swap_folio(struct folio *folio, swp_entry_t entry) > if (val == SWAP_HAS_CACHE) > free_entries++; > } > - cluster_clear_huge(ci); > if (free_entries == SWAPFILE_CLUSTER) { > unlock_cluster_or_swap_info(si, ci); > spin_lock(&si->lock); > @@ -1395,23 +1381,6 @@ void put_swap_folio(struct folio *folio, swp_entry_t entry) > unlock_cluster_or_swap_info(si, ci); > } > > -#ifdef CONFIG_THP_SWAP > -int split_swap_cluster(swp_entry_t entry) > -{ > - struct swap_info_struct *si; > - struct swap_cluster_info *ci; > - unsigned long offset = swp_offset(entry); > - > - si = _swap_info_get(entry); > - if (!si) > - return -EBUSY; > - ci = lock_cluster(si, offset); > - cluster_clear_huge(ci); > - unlock_cluster(ci); > - return 0; > -} > -#endif > - > static int swp_entry_cmp(const void *ent1, const void *ent2) > { > const swp_entry_t *e1 = ent1, *e2 = ent2; > @@ -1519,22 +1488,23 @@ int swp_swapcount(swp_entry_t entry) > } > > static bool swap_page_trans_huge_swapped(struct swap_info_struct *si, > - swp_entry_t entry) > + swp_entry_t entry, > + unsigned int nr_pages) > { > struct swap_cluster_info *ci; > unsigned char *map = si->swap_map; > unsigned long roffset = swp_offset(entry); > - unsigned long offset = round_down(roffset, SWAPFILE_CLUSTER); > + unsigned long offset = round_down(roffset, nr_pages); > int i; > bool ret = false; > > ci = lock_cluster_or_swap_info(si, offset); > - if (!ci || !cluster_is_huge(ci)) { > + if (!ci || nr_pages == 1) { > if (swap_count(map[roffset])) > ret = true; > goto unlock_out; > } > - for (i = 0; i < SWAPFILE_CLUSTER; i++) { > + for (i = 0; i < nr_pages; i++) { > if (swap_count(map[offset + i])) { > ret = true; > break; > @@ -1556,7 +1526,7 @@ static bool folio_swapped(struct folio *folio) > if (!IS_ENABLED(CONFIG_THP_SWAP) || likely(!folio_test_large(folio))) > return swap_swapcount(si, entry) != 0; > > - return swap_page_trans_huge_swapped(si, entry); > + return swap_page_trans_huge_swapped(si, entry, folio_nr_pages(folio)); > } > > /** > @@ -1622,8 +1592,7 @@ int free_swap_and_cache(swp_entry_t entry) > } > > count = __swap_entry_free(p, entry); > - if (count == SWAP_HAS_CACHE && > - !swap_page_trans_huge_swapped(p, entry)) > + if (count == SWAP_HAS_CACHE) > __try_to_reclaim_swap(p, swp_offset(entry), > TTRS_UNMAPPED | TTRS_FULL); > put_swap_device(p);