From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C17EE7716E for ; Thu, 5 Dec 2024 17:50:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5FC226B0180; Thu, 5 Dec 2024 12:50:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 131536B0189; Thu, 5 Dec 2024 12:50:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C06CD6B0196; Thu, 5 Dec 2024 12:50:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id CC5BA6B0184 for ; Thu, 5 Dec 2024 12:50:09 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 8286D40593 for ; Thu, 5 Dec 2024 17:50:06 +0000 (UTC) X-FDA: 82861643508.30.DFF47A8 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf19.hostedemail.com (Postfix) with ESMTP id 0E4471A001B for ; Thu, 5 Dec 2024 17:49:46 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="i7qSmtS/"; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733420997; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ESBy/PC7w3Ubzt0too5hvxQOmgaQszQL5ZR2Oy7o5YE=; b=RGA6Ia66nwiUA+3LXVU46DRbrfh4tDZxbmpH/1ZmvOkxidbY/By+xe2N2DyFpDYIGHsa2w HVnBKUKjZlKkdU2VmcvK7shXC4rHSiT8zMD1D5jWWTzhy+57claUSF+gkxnIvR5bm+1jpw lXml7Y5sZItCSKmr8FZXkvzYKosprvU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733420997; a=rsa-sha256; cv=none; b=yxTivoLi3u/X+raSSpJkOtZ7093KJxV4tSSWONhcKh1ztgB7/1/tz7CUbEkD6LYm/8C0ud H3lA/3mNJiH/hLxLvXag5xPtr32uSgBIFZakJvP1fO3kQXX7pExYT6lnAiUuHR+OFgGeYB 0LxiK5timvV5VJhJ+4XDpjLsYeizh4I= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="i7qSmtS/"; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=ESBy/PC7w3Ubzt0too5hvxQOmgaQszQL5ZR2Oy7o5YE=; b=i7qSmtS/Gp3ZPGdzl9iC1H1xZW EcfexljHS28KMCWTq2dLwHoVk1VBCV+OKAcRR6hdW3gg2mgA3omeCWyEchOjgal933PShzvFzFr/D wCRUikXyp+pTYTW182AWIgiXT9a08FfkP6YfFjPTC/fONPxrdbrYDmcFSdaaLWIXIve8LjCFeF2Zm g3+Z9N+y7Toz3eeyOKa4UMRML8RLPk0FGhFP1Y6WvaW/yEDwVI+QBh7W1R5NPZzK9HTOCLsfR9rnH 7vs474SBegWYZAQzkfNdXSVmIYhGGAWXAQIHjPfE0vfiU5wfpr+cOk/1cXhZlL8EMjQJev4BnzBrg NmbH+h8g==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tJFzG-0000000DN6s-1Z1C; Thu, 05 Dec 2024 17:50:02 +0000 From: "Matthew Wilcox (Oracle)" To: Minchan Kim , Sergey Senozhatsky Cc: Alex Shi , linux-mm@kvack.org, Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [PATCH v8 01/21] mm/zsmalloc: add zpdesc memory descriptor for zswap.zpool Date: Thu, 5 Dec 2024 17:49:38 +0000 Message-ID: <20241205175000.3187069-2-willy@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241205175000.3187069-1-willy@infradead.org> References: <20241205175000.3187069-1-willy@infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 0E4471A001B X-Stat-Signature: pr5z31z5hg3y7p711jc8y1uoheb8c4pi X-Rspam-User: X-HE-Tag: 1733420986-255665 X-HE-Meta: U2FsdGVkX19rlDn/QmUlQpmOmouO0sEBuf+NCX93amiX5mzz+nw4wvLanpqNSKzKE9V3xg8AO6cJFqrMMCK0qMlTSEx82JkFxdLZNj2+heNoYmWiZNJ3wZHZIN0FWfDcKM+PZUGRgwC6u/aWzSCMzrvAVglYbHb9TPEa0HQKvfMhZIcY9BGdnvfwcHDWtWHdPx4UVgf/0aIr+uhubeceOJG7vARPclAQlpjOeeAAPL3aNGKuxED9deVuNpLlxYNb2e2h+e6VHcTkwRrNEiXGcJYLsksPjB21zO1THhcWEJkKJ0h2gAvGBIddhIOYo+969SYIi6UILPiy0HggbOi16WZhC97BzvNKb+jnWLeKVD1bUTD3bn62ivkzRvgCCaSERCeMW1//tjeSWwjdIjHBiuX5Shnsga3aIH4+dZ5X3+PvjnCFg33VG5qXRcylLrxxvp+430DDSalVvWRs1jPcuzz0cTuSW4N/33P+P9MEGEnKqO5erKqYvTNZxZr1CNYhYxSM3Q0dcszvZ3CEWab+h2piDmuzShdR4Jl2kbSW9Nar84uPzmAFjDgAxzvB9yHKnFK/oQVx2pzS0qBXLNSQGAE3qDCIgswaJGT0wVJHbnjSzULk/R3rcJKlQEWOzWY2MYtz/4rZ0zCyJJ1PWRV3FIFS4HqgLLwqV+liHfKYePGWuTftFwofWCMp2LU31XDKTJqfQqJwDJle9NUg21Nwr7EEJ0k3kixG3zHQA3pY+zst/Qs0NUkbePACdNt7FamWt9ebLxO0mSF/wcqvWOKQZEgIQcHDtXfehIsEmyB21UZChhyo6y3gsXDiMvIUdDfkCqlYcs51Kt1kWhMWW2a0wqPN4+q4mFex6hWXGjjsOt2HM8LH9MilFNIai6bOqIzuatD2oP/A4wOPMYQ+a+EWoGSiWB0zLna8DYm3YGFta6oFp5jBO88CCxBUJmwA3sWA7F1Pk9v5QlwrED0dcu3 IUMcE1DK n2iwV+hJvaxzsfv4KJI+0u4DuXVAqfmEQ4Jh9VjK+YrQtdwv6I28S0YPmEnt2w738iVhUJXdgyB9tXNyhIW9grBpGTV+OqpjqKtM6+9M9jUkP3BR76eK4TQfh+dP8vRI1Ykq16rsqlnH3+qa5CI/V8vP5JxzXJPHaKXi1L9e/IqORPA9l37ArMxSfv9PR+d9PffVs88SoTPane/DNKei1+5CIZ23ZoymgtP+98yqs9JcgoYigZaWwLXe1Pr2S6HrOWihJu1Hy5Lfn4AVyZclHIgh6CX77X4BQDy2eHmtX7nbUBsI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi The 1st patch introduces new memory descriptor zpdesc and rename zspage.first_page to zspage.first_zpdesc, no functional change. We removed PG_owner_priv_1 since it was moved to zspage after commit a41ec880aa7b ("zsmalloc: move huge compressed obj from page to zspage"). Originally-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Alex Shi --- mm/zpdesc.h | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++ mm/zsmalloc.c | 9 ++++--- 2 files changed, 74 insertions(+), 4 deletions(-) create mode 100644 mm/zpdesc.h diff --git a/mm/zpdesc.h b/mm/zpdesc.h new file mode 100644 index 000000000000..9ad232774469 --- /dev/null +++ b/mm/zpdesc.h @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* zpdesc.h: zswap.zpool memory descriptor + * + * Written by Alex Shi + * Hyeonggon Yoo <42.hyeyoo@gmail.com> + */ +#ifndef __MM_ZPDESC_H__ +#define __MM_ZPDESC_H__ + +/* + * struct zpdesc - Memory descriptor for zpool memory + * @flags: Page flags, mostly unused. + * @lru: Indirectly used by page migration + * @movable_ops: Used by page migration + * @next: Next zpdesc in a zspage in zsmalloc zpool + * @handle: For huge zspage in zsmalloc zpool + * @zspage: Points to the zspage this zpdesc is a part of + * @first_obj_offset: First object offset in zsmalloc zpool + * @_refcount: Indirectly used by page migration + * @memcg_data: Memory Control Group data. + * + * This struct overlays struct page for now. Do not modify without a good + * understanding of the issues. In particular, do not expand into + * the overlap with memcg_data. + * + * Page flags used: + * * PG_private identifies the first component page + * * PG_locked is used by page migration code + */ +struct zpdesc { + unsigned long flags; + struct list_head lru; + unsigned long movable_ops; + union { + struct zpdesc *next; + unsigned long handle; + }; + struct zspage *zspage; + unsigned int first_obj_offset; + atomic_t _refcount; +}; +#define ZPDESC_MATCH(pg, zp) \ + static_assert(offsetof(struct page, pg) == offsetof(struct zpdesc, zp)) + +ZPDESC_MATCH(flags, flags); +ZPDESC_MATCH(lru, lru); +ZPDESC_MATCH(mapping, movable_ops); +ZPDESC_MATCH(index, next); +ZPDESC_MATCH(index, handle); +ZPDESC_MATCH(private, zspage); +ZPDESC_MATCH(page_type, first_obj_offset); +ZPDESC_MATCH(_refcount, _refcount); +#undef ZPDESC_MATCH +static_assert(sizeof(struct zpdesc) <= sizeof(struct page)); + +#define zpdesc_page(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct page *)(zp), \ + struct zpdesc *: (struct page *)(zp))) + +/* Using folio conversion to skip compound_head checking */ +#define zpdesc_folio(zp) (_Generic((zp), \ + const struct zpdesc *: (const struct folio *)(zp), \ + struct zpdesc *: (struct folio *)(zp))) + +#define page_zpdesc(p) (_Generic((p), \ + const struct page *: (const struct zpdesc *)(p), \ + struct page *: (struct zpdesc *)(p))) + +#endif diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 64b66a4d3e6e..469fda76ed8a 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -67,6 +67,7 @@ #include #include #include +#include "zpdesc.h" #define ZSPAGE_MAGIC 0x58 @@ -254,7 +255,7 @@ struct zspage { }; unsigned int inuse; unsigned int freeobj; - struct page *first_page; + struct zpdesc *first_zpdesc; struct list_head list; /* fullness list */ struct zs_pool *pool; rwlock_t lock; @@ -459,7 +460,7 @@ static inline void mod_zspage_inuse(struct zspage *zspage, int val) static inline struct page *get_first_page(struct zspage *zspage) { - struct page *first_page = zspage->first_page; + struct page *first_page = zpdesc_page(zspage->first_zpdesc); VM_BUG_ON_PAGE(!is_first_page(first_page), first_page); return first_page; @@ -952,7 +953,7 @@ static void create_page_chain(struct size_class *class, struct zspage *zspage, set_page_private(page, (unsigned long)zspage); page->index = 0; if (i == 0) { - zspage->first_page = page; + zspage->first_zpdesc = page_zpdesc(page); SetPagePrivate(page); if (unlikely(class->objs_per_zspage == 1 && class->pages_per_zspage == 1)) @@ -1318,7 +1319,7 @@ static unsigned long obj_malloc(struct zs_pool *pool, link->handle = handle | OBJ_ALLOCATED_TAG; else /* record handle to page->index */ - zspage->first_page->index = handle | OBJ_ALLOCATED_TAG; + zspage->first_zpdesc->handle = handle | OBJ_ALLOCATED_TAG; kunmap_local(vaddr); mod_zspage_inuse(zspage, 1); -- 2.45.2