From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38CABC25B7E for ; Fri, 31 May 2024 14:28:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B08E86B0088; Fri, 31 May 2024 10:28:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AB8B26B008A; Fri, 31 May 2024 10:28:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 97FFE6B0092; Fri, 31 May 2024 10:28:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 7BD7C6B0088 for ; Fri, 31 May 2024 10:28:03 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id DAEF2160507 for ; Fri, 31 May 2024 14:28:02 +0000 (UTC) X-FDA: 82178920404.03.AE17208 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf28.hostedemail.com (Postfix) with ESMTP id 8F447C000E for ; Fri, 31 May 2024 14:28:00 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=KBTH98bx; spf=none (imf28.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717165681; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sFxJyQ/io0aHDuu0zh2SmigHrUNsFJB5zun4ic5Vs/g=; b=EKHtEoJOOaIIaAwZH/P7Pzj458UTD4U/jNq2xANYz4OWbJ9Jje7IMmHiu2hB3RWAmMuF4F w9PrZJbUpKBxK0JWBqq15EPQ6VJOJRCIvs1PLbpa/aKbPBEKjaqcSF6pDzjMZw1riuYWWV Bj5WQkU3B7XhUEFomFer9ZJVgmuno1U= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717165681; a=rsa-sha256; cv=none; b=rjeYsptATSHiCE/oCOPa4U7EP1/80ipL33n03s+kvyKPQJA/0YzD1l0uEEsF5GqwR6huLj y970uaHHK/g1AzoqWMBQQqPdggKnICUi6U6N7d5xpcG0kOewHaOAnxw5HJH4dveV1sb9Sg rQbKA9gVCIMm0Zzm76nis9ORu+v5cOU= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=KBTH98bx; spf=none (imf28.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=sFxJyQ/io0aHDuu0zh2SmigHrUNsFJB5zun4ic5Vs/g=; b=KBTH98bxJtCBJ4ffi0DXzONGEt nkbR1Md7j1pE7pP1L0v0NS63ADdPT51alwW0QmJs1NHFrvvy4BenX/WdA6BIpbP3oClfXyg+ecyFh PNsGl6FRM9ajDYokoD7nWPpbKC7SW9lwK2tsjg1/DbTVe01d3N3gahiUvVRmCg1WGx35BEukirKtt eI2hw5EWF53rTreEtcYuysiMky73k+IVDspV4WxhIUg7a9ahcNrDTgqc9jedNZb9KnCEd0YXIT9lK Pp/nybQIGeFVbNtrPUazk3BhPBnmgbdd52vpFXM0OTXL/FK5+tnZM87xUUmhIcXzdusAJCiLF8Mmt X17VLJrg==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1sD3Eb-0000000BmJT-1wAY; Fri, 31 May 2024 14:27:57 +0000 Date: Fri, 31 May 2024 15:27:57 +0100 From: Matthew Wilcox To: Sergey Senozhatsky Cc: David Hildenbrand , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Mike Rapoport , Minchan Kim , Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: Re: [PATCH v2 3/6] mm/zsmalloc: use a proper page type Message-ID: References: <20240529111904.2069608-1-david@redhat.com> <20240529111904.2069608-4-david@redhat.com> <20240530050123.GA8400@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240530050123.GA8400@google.com> X-Rspam-User: X-Stat-Signature: 3ff1pq9prypnpzahsrsbuqd4nqx5orcz X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 8F447C000E X-HE-Tag: 1717165680-43954 X-HE-Meta: U2FsdGVkX19Hp09mjFU+64lEhAR94c1dA4BxDAGkeP5BaZdPzbYETfAf46f37hqFFU58NKmmPyha6Tx/uCXIeO+/2YGxLSN+LExaKu3Tac9KiLteDvA73Y7IcXqChaONIsoPfbl2GiH3U6WlfOKQ7e5zBP16c6WNYznQGzQQHY/3GFrxkXllD2RZTsK6Vyuk2+J3FeyG7QZO+Cy7jBc7JXKmKuuyzeOfii0bI9z62lDHZDdURIV1qesh/IUR6hem6hylZj/y0b5zdBpOfe0E6/D2F0qMd0o7RUOgqbYJLZH6Hz6xYuOroNMwxhBGswgouH34N8LCooRnhAR1r0tkeIDBhTWWemxW+WZhbtUyvbt8LK6XBhj2l3lFgzjBZNUQHx/O//n7KRkTP3Gtyv5j4DX1d37+4cjSZRwPhnTeGiKfRiNGJEbN9+S+6BCa4fVdSBxGxEDAUqJw1XosXVRyJ1xRj4zSrIfVo2bgt/5sQbwhmjWA+gz6wITZRTSHY5w7hLkBNieCaSdccX0iUpz3Goqq+Y2ifbbPT68OeWod1Q4KUIu47I8uk5wicBu0QTjF+yp2M9Qut89cv3vcxwldP5OspfDDC8/N4tEkWIbBbcL7lx9fJPvYoJW99JRVTOh/DaBfhZcu3O79YtXoxB4GgocXB5rzCDQJ35rt3ANYzc9elqY10SqDAFGivjaELgZHvITG+pTBWiq0kPfF0LcOjW3vNxVCUd2zd/fm0MlK6itET6sweXsLxKfolg7C7vzrmuBnNSbMwoohlf3KqyI8RK+XnODPkKQjrrHh2m4K7S2t+pvqWiLxEx/k5e5SN7RWSX906IWXih2rl3hlQOMU0CEDEMshQZM7VKmVr4JrcGp5Dr938JGNzzw/srfzW/mbtScCOuI60AZWFa9c9LMEaNFdDUB3Sx61CyPuNoxlbtbMXhVJF9XYiWLGbXl/UCx2ZDVO30G0ueX/SGQMaVE 0+gOepfQ R63a59L4D37iPR4Sb5HuYH9/XuGtuSeTBXKdrwU2q+W5J6M3tFiSuv+1hVYmQbbJu+Zhd+bdn1AqiKkfzbB4p834kYvLprTm3e7e1hiV9XLvAdfx2Si+H/c7rz1CgIs4zQPar7QYZq1iifgrEwqPRjzuQvFdwJ6na4W9dj/lABpsTFImxHSRLWUxWOJONeP4sqTKDgV3ZcwJuysdZceaw8b/ghQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, May 30, 2024 at 02:01:23PM +0900, Sergey Senozhatsky wrote: > On (24/05/29 13:19), David Hildenbrand wrote: > > We won't be able to support 256 KiB base pages, which is acceptable. > [..] > > +config HAVE_ZSMALLOC > > + def_bool y > > + depends on MMU > > + depends on PAGE_SIZE_LESS_THAN_256KB # we want <= 64 KiB > > Can't really say that I'm happy with this, but if mm-folks are > fine then okay. I have an idea ... We use 6 of the bits in the top byte of the page_type to enumerate a type (ie value 0x80-0xbf) and then the remaining 24 bits are available. It's actually more efficient: $ ./scripts/bloat-o-meter prev.o .build-debian/mm/filemap.o add/remove: 0/0 grow/shrink: 0/3 up/down: 0/-40 (-40) Function old new delta __filemap_add_folio 1102 1098 -4 filemap_unaccount_folio 455 446 -9 replace_page_cache_folio 474 447 -27 Total: Before=41258, After=41218, chg -0.10% (that's all from PG_hugetlb) before: 1406: 8b 46 30 mov 0x30(%rsi),%eax mapcount = atomic_read(&folio->_mapcount) + 1; 1409: 83 c0 01 add $0x1,%eax if (mapcount < PAGE_MAPCOUNT_RESERVE + 1) 140c: 83 f8 81 cmp $0xffffff81,%eax 140f: 7d 6c jge 147d 1411: 8b 43 30 mov 0x30(%rbx),%eax 1414: 25 00 08 00 f0 and $0xf0000800,%eax 1419: 3d 00 00 00 f0 cmp $0xf0000000,%eax 141e: 74 4e je 146e after: 1406: 8b 46 30 mov 0x30(%rsi),%eax mapcount = atomic_read(&folio->_mapcount) + 1; 1409: 83 c0 01 add $0x1,%eax if (mapcount < PAGE_MAPCOUNT_RESERVE + 1) 140c: 83 f8 81 cmp $0xffffff81,%eax 140f: 7d 63 jge 1474 if (folio_test_hugetlb(folio)) 1411: 80 7b 33 84 cmpb $0x84,0x33(%rbx) 1415: 74 4e je 1465 so we go from "mov, and, cmp, je" to just "cmpb, je", which must surely be faster to execute as well as being more compact in the I$ (6 bytes vs 15). Anyway, not tested but this is the patch I used to generate the above. More for comment than application. diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 5265b3434b9e..4129d04ac812 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -942,24 +942,24 @@ PAGEFLAG_FALSE(HasHWPoisoned, has_hwpoisoned) * mistaken for a page type value. */ -#define PAGE_TYPE_BASE 0xf0000000 -/* Reserve 0x0000007f to catch underflows of _mapcount */ -#define PAGE_MAPCOUNT_RESERVE -128 -#define PG_buddy 0x00000080 -#define PG_offline 0x00000100 -#define PG_table 0x00000200 -#define PG_guard 0x00000400 -#define PG_hugetlb 0x00000800 -#define PG_slab 0x00001000 - -#define PageType(page, flag) \ - ((page->page_type & (PAGE_TYPE_BASE | flag)) == PAGE_TYPE_BASE) -#define folio_test_type(folio, flag) \ - ((folio->page.page_type & (PAGE_TYPE_BASE | flag)) == PAGE_TYPE_BASE) +/* Reserve 0x0000007f to catch underflows of _mapcount */ +#define PAGE_MAPCOUNT_RESERVE -128 + +#define PG_buddy 0x80 +#define PG_offline 0x81 +#define PG_table 0x82 +#define PG_guard 0x83 +#define PG_hugetlb 0x84 +#define PG_slab 0x85 + +#define PageType(page, type) \ + (((page)->page_type >> 24) == type) +#define folio_test_type(folio, type) \ + (((folio)->page.page_type >> 24) == type) static inline int page_type_has_type(unsigned int page_type) { - return (int)page_type < PAGE_MAPCOUNT_RESERVE; + return ((int)page_type < 0) && (page_type < 0xc0000000); } static inline int page_has_type(const struct page *page) @@ -975,12 +975,12 @@ static __always_inline bool folio_test_##fname(const struct folio *folio)\ static __always_inline void __folio_set_##fname(struct folio *folio) \ { \ VM_BUG_ON_FOLIO(!folio_test_type(folio, 0), folio); \ - folio->page.page_type &= ~PG_##lname; \ + folio->page.page_type = PG_##lname << 24; \ } \ static __always_inline void __folio_clear_##fname(struct folio *folio) \ { \ VM_BUG_ON_FOLIO(!folio_test_##fname(folio), folio); \ - folio->page.page_type |= PG_##lname; \ + folio->page.page_type = 0xffffffff; \ } #define PAGE_TYPE_OPS(uname, lname, fname) \ @@ -992,12 +992,12 @@ static __always_inline int Page##uname(const struct page *page) \ static __always_inline void __SetPage##uname(struct page *page) \ { \ VM_BUG_ON_PAGE(!PageType(page, 0), page); \ - page->page_type &= ~PG_##lname; \ + page->page_type = PG_##lname << 24; \ } \ static __always_inline void __ClearPage##uname(struct page *page) \ { \ VM_BUG_ON_PAGE(!Page##uname(page), page); \ - page->page_type |= PG_##lname; \ + page->page_type = 0xffffffff; \ } /*