From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3EEB8C2D0CD for ; Thu, 15 May 2025 09:41:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8E3F06B00C8; Thu, 15 May 2025 05:41:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8938B8D0001; Thu, 15 May 2025 05:41:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 735516B00CA; Thu, 15 May 2025 05:41:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 554666B00C8 for ; Thu, 15 May 2025 05:41:03 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 0033BE2EAE for ; Thu, 15 May 2025 09:41:03 +0000 (UTC) X-FDA: 83444648448.19.32C8A76 Received: from mail-lf1-f42.google.com (mail-lf1-f42.google.com [209.85.167.42]) by imf26.hostedemail.com (Postfix) with ESMTP id 10BAF14000C for ; Thu, 15 May 2025 09:41:01 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=HVI1ibv9; spf=pass (imf26.hostedemail.com: domain of klarasmodin@gmail.com designates 209.85.167.42 as permitted sender) smtp.mailfrom=klarasmodin@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1747302062; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=16TXtjUQOKhYTp8nb0jqguevt6aIASjENt+W1s/s8pA=; b=uyCobeZL14NZbw9dqrGijMVMkNE1UjqZSkbwZ4einmxudEHnDqtuLvphHPV0YFHxk3RWtv MAQrwyjPmrp++/FurFnIopJB5mJG5vqHbo5M4gpIygNly+61l7Lh5fZ66R/kppa9JW57v2 hdwt2814ydmD2Ncn3KMXTjK+4IpD2s8= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=HVI1ibv9; spf=pass (imf26.hostedemail.com: domain of klarasmodin@gmail.com designates 209.85.167.42 as permitted sender) smtp.mailfrom=klarasmodin@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1747302062; a=rsa-sha256; cv=none; b=aR85bYOP7vBzUYYIRdK5BEPrEVJ6v+A3wx/aRbWIzFkTv9FA0bYddguStQgAXkslOGB7R/ jkLliEoPkEGFhD9Phkzxqv0Z8DE6CJB/XDpZiETk823aVFeHZHRTzWCUXv1ZmuH4ZqQPSr p4nd6Ngcpr/izKeq6CrkH7mxAOOdWuc= Received: by mail-lf1-f42.google.com with SMTP id 2adb3069b0e04-550d09d4daaso672152e87.2 for ; Thu, 15 May 2025 02:41:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1747302060; x=1747906860; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=16TXtjUQOKhYTp8nb0jqguevt6aIASjENt+W1s/s8pA=; b=HVI1ibv9qi2Ugr8DvCFci6rE3AWmfG9r6CzKmwMx5tX2C8yT0Rcjr44qun0P3/xTMH orbtQZpJQAaB4YQwQBJ0TY83kqxNx4s1xuAmFxAjJr+2YWae53OlUeiIymPy3a0t2LYi Zy1RC46XSWYq4dJgRTRCQlA+JRj7dcXhKJBx97DZiUS2uUAyK0P27h+5Pl7ZtTF+S0G9 dDGQBxKcll7Jtz6/ghmdzXcxnCJjTLwnnVO8DqUCoQFBzpTxhzzWIKsvhfD/YarH8Nuy lDhYAxBq5tLnUnBEsDIKWwK4atXPCkObEQDcygdr+btlRFnAdxjTnEjEK008XgNTGchv jCqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747302060; x=1747906860; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=16TXtjUQOKhYTp8nb0jqguevt6aIASjENt+W1s/s8pA=; b=Oz+5oMSUzjSprfEguAhS7i7e1hQEE9Wf9Z+g8Yhg/ovAIpduOG2m9icNBwP1RGZuwa cuVHkIsSJLBQQpj5PMIvbs9C0lU8ekrp3UR+y+hqr3vKWyQ/qigmfxQbhHMzKksLWPzD e2f3sF4WabDXVoaodrPOMw5ZFp+yHqAHKDv79AvV/04zGkMU40yabyj+dZKBOC3Z+THg neEIbt1hFOGDnAx9P1jh870oUfA6JwkUfGZ49sm1ypHl47ORDeLQjkCO9e2MwuiCQC7V 0DRptF/vqbiTWfpmQmj+i1B1BA2i1uiWkRPUht/P2nAUpmLg0fSWOd8Son5jiqj8s9aZ J91Q== X-Gm-Message-State: AOJu0YydDRpgw6oHzw2szYvGvQS926GkmaKqohyuGSPWRJ7LLH1MZ5e5 mYGRIQQr1v6WWwWrBxSi3Ls6f4jSe4JhPaAo1amjX0KKCbGMsts8 X-Gm-Gg: ASbGncueys2uDiEIf/wLeYvAFFmvvvblcewr6D6sHnkGKAqqcRxuTeDDi8eK16yct7A 7+6zPgoaSzGW+xF/XNRVxdYc8ZSwuhEPKZwHZQ+1oyzpzKnT19mj4ZUolDu2QUzbk7dYYJaFSVA tcSpc4SMZA3xT411wUEjEW+csPrPF6cptF325e51Octquy9uA3+oCodHmPm+bIBTEmNH5lHEpIf 3MovxE0JK6O1rtJFG6/PSLxMIWTaRxas43VX/bdQ4O6oy6Lap3EdmUp6eP7bF0+xnmPCeR462FD JPX5oeAwuBijMwR2+Ngu2KfZxqQDfU8hMGQsxg5jlMJN2WWsysN3GElwUC3bdjlI7w== X-Google-Smtp-Source: AGHT+IGYzEDJNoKwsDOnBe/xP7z666VnKlBcvQah0C6jkJJ61IWyXuFBCMCgwXZ0OGtqJHzXvSsKiw== X-Received: by 2002:a05:6512:6804:b0:54e:9c6e:8636 with SMTP id 2adb3069b0e04-550d765196dmr2139025e87.47.1747302060226; Thu, 15 May 2025 02:41:00 -0700 (PDT) Received: from localhost (soda.int.kasm.eu. [2001:678:a5c:1202:4fb5:f16a:579c:6dcb]) by smtp.gmail.com with UTF8SMTPSA id 2adb3069b0e04-550d49f3eebsm744705e87.244.2025.05.15.02.40.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 May 2025 02:40:59 -0700 (PDT) Date: Thu, 15 May 2025 11:40:59 +0200 From: Klara Modin To: Kairui Song Cc: linux-mm@kvack.org, Andrew Morton , Matthew Wilcox , Hugh Dickins , Chris Li , David Hildenbrand , Yosry Ahmed , "Huang, Ying" , Nhat Pham , Johannes Weiner , Baolin Wang , Baoquan He , Barry Song , Kalesh Singh , Kemeng Shi , Tim Chen , Ryan Roberts , linux-kernel@vger.kernel.org Subject: Re: [PATCH 24/28] mm, swap: implement helpers for reserving data in swap table Message-ID: References: <20250514201729.48420-1-ryncsn@gmail.com> <20250514201729.48420-25-ryncsn@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250514201729.48420-25-ryncsn@gmail.com> X-Rspamd-Queue-Id: 10BAF14000C X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: wh6wecp58dq17yhkyzt56rgfen6gxu7o X-HE-Tag: 1747302061-971858 X-HE-Meta: U2FsdGVkX19/LjXrMf1VkLRJGBJ2SCoxmfDkBM5XPUGX8I+PtlE6bak4hJ8KYF9Ct/X8jgtXuO6AaXijG/qZd8h4QvLZtCHMcZnXSsChHFslwnsGHHGFi/jgN4j/6if76UdC6jWY+rtDkgP0J5RfDDkteJSmC9498u5kzLs4lRnYR/kA6xzmWL3gRsvSWs5DPvq4dmbm1BRAj2vpU+3fxokeykHAIqX3V79vfaDCQU8IhGwgT5YgaYyXHSulRoeTZcks/BL9E2ZJkq7hlih41MRNOYxrgDYjrGOOSKCcCMCNQqDp0h4Gwt87i/wwoSlmygvX/Cu9Xi19cRWGYzTxzYV6mcyFS8HutBcHTD+zYPL86o3dwlbdi4wHqlGQTfOSEkf0Nax5l1luu9TuoCtu/gr6HBDCK04947kH1lJuzjs9+Hjvr1gtoy0iVTSmi0l3tiaPHWTrqktHfZQx3751iZkYgXQTjvRHo2g7qJkJipnC2v2H9kfAiBagEXqfrP4YqdjO7nfChhUDgKWtFDV+D+17dHBgiEbTy5H28TnduHBjk07ex5FmZ39ekxVEfTYJpL5EdGBM3ouc2IHLreKcAoKgnM867QszVPYfTFSwacmJOWfd7erKrMoioB0Nrd/OuqpY8PYYAFrwDad2g01gdGO068RK350rLMI2PNmb8Mk4H1y5MTPoU8rpcVXHd4NSPGSVFsizJ/kCrFFxOBXcNzX6S2vcc3uSPZUoxTDeYA4izJTnmKvs8SKXYVGd8Ok480CfEPqVN6goYkmB7m3sTr+VzIXW3WmrG9x5IBZaA00or4Q2NDjaitR07rQbjxSiodbBPVOLmV8R0/10TGvems3ncVwBn0sxPAhicqgGO1CmzB/2+5cs7nqEThPypixBpL3T5bjWSo/Dd6QDB3x4XcJfat5W9seqMBe5L0sUucMv6YSgtFoZCSNBIseEPq7IliQv4qkJnJ5lkm33JO1 uAolLdiL w4TxduMJKjm1eN5ElkAkK0KSnxUE3S1f7jri6dE1s05vCcxoNUBdYDIYVvf+bzSzbYGgZPuZdUkhxDlqAJZFICyMYIzwI4bA1zZN3xVl8ZlKxTdWDvpnzKzXFbqXcNk73rRTMdL1wcnyteBY8PWhAoTQJ+TPbra+MXPv5Dk8USKuG4sztNFS6yd75sh0jdhu2+vMH8mnNT0zykGwabHDWTv9MgoAY6ncFexLjdAh68MicX7SRf4+5EbW0Wv/r8b1VmRt8l6GBYBHIzzOeNO+8r5COy1GFNJUNrVM1W8RNq0hZuJWwdWvonLW7yHIPr2LhDuchCqK/YzYmnxDovBABDLExpTdLxiDYb8/752tFwV5nyvdi60OJh9GI+4TiV1Ye9NF+fvMqUa45AzNIQYrsNo9aYOflVRFyuYww X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi, On 2025-05-15 04:17:24 +0800, Kairui Song wrote: > From: Kairui Song > > To prepare for using swap table as the unified swap layer, introduce > macro and helpers for storing multiple kind of data in an swap table > entry. > > Signed-off-by: Kairui Song > --- > mm/swap_table.h | 130 ++++++++++++++++++++++++++++++++++++++++++++---- > 1 file changed, 119 insertions(+), 11 deletions(-) > > diff --git a/mm/swap_table.h b/mm/swap_table.h > index 69a074339444..9356004d211a 100644 > --- a/mm/swap_table.h > +++ b/mm/swap_table.h > @@ -5,9 +5,41 @@ > #include "swap.h" > > /* > - * Swap table entry could be a pointer (folio), a XA_VALUE (shadow), or NULL. > + * Swap table entry type and bit layouts: > + * > + * NULL: | ------------ 0 -------------| > + * Shadow: | SWAP_COUNT |---- SHADOW_VAL ---|1| > + * Folio: | SWAP_COUNT |------ PFN -------|10| > + * Pointer: |----------- Pointer ----------|100| > + * > + * Usage: > + * - NULL: Swap Entry is unused. > + * > + * - Shadow: Swap Entry is used and not cached (swapped out). > + * It's reusing XA_VALUE format to be compatible with workingset > + * shadows. SHADOW_VAL part could be all 0. > + * > + * - Folio: Swap Entry is in cache. > + * > + * - Pointer: Unused yet. Because only the last three bit of a pointer > + * is usable so now `100` is reserved for potential pointer use. > */ > > +#define ENTRY_COUNT_BITS BITS_PER_BYTE > +#define ENTRY_SHADOW_MARK 0b1UL > +#define ENTRY_PFN_MARK 0b10UL > +#define ENTRY_PFN_LOW_MASK 0b11UL > +#define ENTRY_PFN_SHIFT 2 > +#define ENTRY_PFN_MASK ((~0UL) >> ENTRY_COUNT_BITS) > +#define ENTRY_COUNT_MASK (~((~0UL) >> ENTRY_COUNT_BITS)) > +#define ENTRY_COUNT_SHIFT (BITS_PER_LONG - BITS_PER_BYTE) > +#define ENTRY_COUNT_MAX ((1 << ENTRY_COUNT_BITS) - 2) > +#define ENTRY_COUNT_BAD ((1 << ENTRY_COUNT_BITS) - 1) /* ENTRY_BAD */ > +#define ENTRY_BAD (~0UL) > + > +/* For shadow offset calculation */ > +#define SWAP_COUNT_SHIFT ENTRY_COUNT_BITS > + > /* > * Helpers for casting one type of info into a swap table entry. > */ > @@ -19,17 +51,27 @@ static inline swp_te_t null_swp_te(void) > > static inline swp_te_t folio_swp_te(struct folio *folio) > { > - BUILD_BUG_ON(sizeof(swp_te_t) != sizeof(void *)); > - swp_te_t swp_te = { .counter = (unsigned long)folio }; > + BUILD_BUG_ON((MAX_POSSIBLE_PHYSMEM_BITS - PAGE_SHIFT) > > + (BITS_PER_LONG - ENTRY_PFN_SHIFT - ENTRY_COUNT_BITS)); MAX_POSSIBLE_PHYSMEM_BITS does not seem to be available on all arches/configs. E.g. zsmalloc seems to set it to MAX_PHYSMEM_BITS when this is the case but I don't know if that applies here. > + swp_te_t swp_te = { > + .counter = (folio_pfn(folio) << ENTRY_PFN_SHIFT) | ENTRY_PFN_MARK > + }; > return swp_te; > } > > static inline swp_te_t shadow_swp_te(void *shadow) > { > - BUILD_BUG_ON((BITS_PER_XA_VALUE + 1) != > - BITS_PER_BYTE * sizeof(swp_te_t)); > - VM_WARN_ON_ONCE(shadow && !xa_is_value(shadow)); > swp_te_t swp_te = { .counter = ((unsigned long)shadow) }; > + BUILD_BUG_ON((BITS_PER_XA_VALUE + 1) != BITS_PER_BYTE * sizeof(swp_te_t)); > + BUILD_BUG_ON((unsigned long)xa_mk_value(0) != ENTRY_SHADOW_MARK); > + VM_WARN_ON_ONCE(shadow && !xa_is_value(shadow)); > + swp_te.counter |= ENTRY_SHADOW_MARK; > + return swp_te; > +} > + > +static inline swp_te_t bad_swp_te(void) > +{ > + swp_te_t swp_te = { .counter = ENTRY_BAD }; > return swp_te; > } > > @@ -43,7 +85,7 @@ static inline bool swp_te_is_null(swp_te_t swp_te) > > static inline bool swp_te_is_folio(swp_te_t swp_te) > { > - return !xa_is_value((void *)swp_te.counter) && !swp_te_is_null(swp_te); > + return ((swp_te.counter & ENTRY_PFN_LOW_MASK) == ENTRY_PFN_MARK); > } > > static inline bool swp_te_is_shadow(swp_te_t swp_te) > @@ -51,19 +93,63 @@ static inline bool swp_te_is_shadow(swp_te_t swp_te) > return xa_is_value((void *)swp_te.counter); > } > > +static inline bool swp_te_is_valid_shadow(swp_te_t swp_te) > +{ > + /* The shadow could be empty, just for holding the swap count */ > + return xa_is_value((void *)swp_te.counter) && > + xa_to_value((void *)swp_te.counter); > +} > + > +static inline bool swp_te_is_bad(swp_te_t swp_te) > +{ > + return swp_te.counter == ENTRY_BAD; > +} > + > +static inline bool __swp_te_is_countable(swp_te_t ent) > +{ > + return (swp_te_is_shadow(ent) || swp_te_is_folio(ent) || > + swp_te_is_null(ent)); > +} > + > /* > * Helpers for retrieving info from swap table. > */ > static inline struct folio *swp_te_folio(swp_te_t swp_te) > { > VM_WARN_ON(!swp_te_is_folio(swp_te)); > - return (void *)swp_te.counter; > + return pfn_folio((swp_te.counter & ENTRY_PFN_MASK) >> ENTRY_PFN_SHIFT); > } > > static inline void *swp_te_shadow(swp_te_t swp_te) > { > VM_WARN_ON(!swp_te_is_shadow(swp_te)); > - return (void *)swp_te.counter; > + return (void *)(swp_te.counter & ~ENTRY_COUNT_MASK); > +} > + > +static inline unsigned char swp_te_get_count(swp_te_t swp_te) > +{ > + VM_WARN_ON(!__swp_te_is_countable(swp_te)); > + return ((swp_te.counter & ENTRY_COUNT_MASK) >> ENTRY_COUNT_SHIFT); > +} > + > +static inline unsigned char swp_te_try_get_count(swp_te_t swp_te) > +{ > + if (__swp_te_is_countable(swp_te)) > + return swp_te_get_count(swp_te); > + return 0; > +} > + > +static inline swp_te_t swp_te_set_count(swp_te_t swp_te, > + unsigned char count) > +{ > + VM_BUG_ON(!__swp_te_is_countable(swp_te)); > + VM_BUG_ON(count > ENTRY_COUNT_MAX); > + > + swp_te.counter &= ~ENTRY_COUNT_MASK; > + swp_te.counter |= ((unsigned long)count) << ENTRY_COUNT_SHIFT; > + VM_BUG_ON(swp_te_get_count(swp_te) != count); > + > + return swp_te; > } > > /* > @@ -87,17 +173,39 @@ static inline swp_te_t __swap_table_get(struct swap_cluster_info *ci, pgoff_t of > static inline void __swap_table_set_folio(struct swap_cluster_info *ci, pgoff_t off, > struct folio *folio) > { > - __swap_table_set(ci, off, folio_swp_te(folio)); > + swp_te_t swp_te; > + unsigned char count; > + > + swp_te = __swap_table_get(ci, off); > + count = swp_te_get_count(swp_te); > + swp_te = swp_te_set_count(folio_swp_te(folio), count); > + > + __swap_table_set(ci, off, swp_te); > } > > static inline void __swap_table_set_shadow(struct swap_cluster_info *ci, pgoff_t off, > void *shadow) > { > - __swap_table_set(ci, off, shadow_swp_te(shadow)); > + swp_te_t swp_te; > + unsigned char count; > + > + swp_te = __swap_table_get(ci, off); > + count = swp_te_get_count(swp_te); > + swp_te = swp_te_set_count(shadow_swp_te(shadow), count); > + > + __swap_table_set(ci, off, swp_te); > } > > static inline void __swap_table_set_null(struct swap_cluster_info *ci, pgoff_t off) > { > __swap_table_set(ci, off, null_swp_te()); > } > + > +static inline void __swap_table_set_count(struct swap_cluster_info *ci, pgoff_t off, > + unsigned char count) > +{ > + swp_te_t swp_te; > + swp_te = swp_te_set_count(__swap_table_get(ci, off), count); > + __swap_table_set(ci, off, swp_te); > +} > #endif > -- > 2.49.0 > Regards, Klara Modin