From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A703C5AD49 for ; Mon, 2 Jun 2025 11:02:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7CF906B0287; Mon, 2 Jun 2025 07:02:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 758896B0289; Mon, 2 Jun 2025 07:02:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 646826B028A; Mon, 2 Jun 2025 07:02:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 430206B0287 for ; Mon, 2 Jun 2025 07:02:35 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id A83908018A for ; Mon, 2 Jun 2025 11:02:34 +0000 (UTC) X-FDA: 83510172228.04.71D0693 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf04.hostedemail.com (Postfix) with ESMTP id 6A7E840012 for ; Mon, 2 Jun 2025 11:02:32 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="O/TPJgsh"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=mZEFNmrg; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=OLd6pquk; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=wmwQ15hB; spf=pass (imf04.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748862152; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZztYGn3J3Hw8d9RMjSfh9mOTW5hxG+HFPyUVr4o4Ux0=; b=IIp3SQkOln8Ut2GykMCHjn5TZmhtmdMj0c1sKsSVUpXAQ/Y4NfztPNrYOBuFgBoBPZ8IfO wqsBVb6WQy+ZiiPHta1Qf/8TIC9YFi0Z32XC6WKT+O+wSmI4YHAAWvf3bf8dFLwUal0GyG xSgMhtbk0hDHesbDNLk6tKRIJFJ950M= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="O/TPJgsh"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=mZEFNmrg; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=OLd6pquk; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=wmwQ15hB; spf=pass (imf04.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748862152; a=rsa-sha256; cv=none; b=PaJngv6CdK/XdxPvLNd/VKzPyPAjbKMjPgs1/FWQMi6Wz8tFkKF9RFf0mSwkUvr4KWzU94 p7bwE/2btLs5Pxw6ArW8Kz+JwKSfWEshry/xKep6QHGDPUdabU9PEHCs+MZx5HL2EhNJ+W bSJqX261vCcMDrVyCmXRr6XrvRNA+0E= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id CFC1B219EB; Mon, 2 Jun 2025 11:02:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1748862138; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZztYGn3J3Hw8d9RMjSfh9mOTW5hxG+HFPyUVr4o4Ux0=; b=O/TPJgshfUfq1fyDdxPU8Hq2bPUX3ek9sFMZ4FBcOXJ2QTeWj8pMfsGbCTIt6JHowuGDYv /J/XUwWDuUjlBbqAQd1ZD7OpeLEBxyKV0qurjjwkX4MJAqaldxz96aBdFUv4WvpbvGaMes TrkeW/bTqpKbskxbwflbnU8XQl2obYQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1748862138; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZztYGn3J3Hw8d9RMjSfh9mOTW5hxG+HFPyUVr4o4Ux0=; b=mZEFNmrgGtn8H93eZ9a5RDjIvBlqh1ZKnv2zyIf+iwRu+DKMsJKlmxMT1s2ZxDpsIrI1Q5 scg7maOjd/iq17AQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1748862137; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZztYGn3J3Hw8d9RMjSfh9mOTW5hxG+HFPyUVr4o4Ux0=; b=OLd6pqukHonKrFnlRtGUf+A9pMdztBM48Ym20EE7O8iczKWlybdZ27zgoxMR4zZrQHbXF+ oGQzo5yzGHDG/Xns9kn77vwGyYOQuxvz6ILaUwLGtnJfoVI7vGUHcQ2+I3VLQGwnZOe5sE hdiuQ11yh2utQ8lK5RWaNSyPgc/z5Po= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1748862137; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZztYGn3J3Hw8d9RMjSfh9mOTW5hxG+HFPyUVr4o4Ux0=; b=wmwQ15hBa42uBZfrPFKqZb0eK7nVeiHs3NC91AfoxBMvpXSuoKbWSf9Y1g523gm+Y/tk79 b/Pj3M76DNweDJBA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id BA29A13AE1; Mon, 2 Jun 2025 11:02:17 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id KCArLbmEPWggHAAAD6G6ig (envelope-from ); Mon, 02 Jun 2025 11:02:17 +0000 From: Vlastimil Babka Date: Mon, 02 Jun 2025 13:02:13 +0200 Subject: [PATCH v2 2/2] mm, slab: use frozen pages for large kmalloc MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20250602-frozen-pages-for-large-kmalloc-v2-2-84a21f2c3640@suse.cz> References: <20250602-frozen-pages-for-large-kmalloc-v2-0-84a21f2c3640@suse.cz> In-Reply-To: <20250602-frozen-pages-for-large-kmalloc-v2-0-84a21f2c3640@suse.cz> To: Christoph Lameter , David Rientjes Cc: Andrew Morton , Roman Gushchin , Harry Yoo , Matthew Wilcox , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.2 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 6A7E840012 X-Stat-Signature: gwp65hq3c45b6wbt6twsrqr56hoo7i4s X-Rspam-User: X-HE-Tag: 1748862152-151853 X-HE-Meta: U2FsdGVkX1+NKSbOt7L1HwKbbBZZ+ivWujrBAsOS6ujAHREKYmBuKjztNfT0O9cDvwMivWEetVUp9ueexWk60rgqgnh8lt/USSX8uvOo5HQzRhLwrRitWNJ4NigzxEhNmufhzlHefKFx4JWSyZcLw/w8/RjHjVsBP60ForPxgKeSx91xR91mgRU8RAYMorS2TX4ZDAOP41QIRQ3TcUVLSIk4zL+amwv0HsfM3ggK4zf6lzaT3B8XrLfTzTNhTC16XGnwFSClc5R5PSDwwwJJDqbETI2jG3uHwYuZo+AR/IgreJ8l3564J1mpyaoeC6cMekRbPA1JpePxkuPWYm8mKsD9fWqPzEtjhIUj5f3/1fakttn/1oKMsS+RsW7gr25LETiSoa9nNZbGmV1d0npQHdSvqa1RTWXbwbdE/0BvcsrJVIZRD5yg5e9/FdkoUm4Zn3AtnKwik1E+Qk8qzGkUyBbAvznymOTwNQ025NKXaokx7qKwpMROvz1cE0ozqD79kWlpAyNwGAeZF6J2csLlSyw1gYr0d2aKlScwfTcz9cqA/IiKWxN3fcsb69NOYdtxtr5KNHIOuUhMY/6TFPYKC1YErQS95pnZP92+vsELFU+zMzwFau3Z6Pn7PNgs86GaauzQ/jbbFlfFlRbWLK1OL6p/7tNOtdOU3eNP/SJTtrQrrz9rS9Wb6fdlktkyT7rGo313dQ9BMNh5trTRzsDO9Tj/Pkx3IASvDCd6oYCPnp9lamI2kU76vb013ybbNcD1HnW0CSTV1OyajvSdDLztfKX/BxB649S2aDVbkdBDjeYbpikDpMH9QjkMmoNOy6NeZNM4yp1D3BcMI9/Azpkqp3CsoEF/1lXlPsgwtuS4dSPiQtybTy/2u6GVnQvtUco83A1OREUhTmxcvaQa2uT6hRbXsjIDOjMbYhjlNOe1AYonG0ZRgOdLnVQ9wLGUAhM0JNOK0pEvCtTGVjL5/RB cZMYCFqF XUF1gRlLh7hFz8EiO++RAIM7htpHChU4+75m6lfxr+CEMChzsZaJMsrXRIzK1zvAHIjnU8eumfHznJ3HPXQEy9TfqIQ+GWvwPP88mSQmssIGw0gchfArqfYIeGBwib71ZZ1SeQVzzQ4C79pmLPnacXkj656bcfXGc8yO2q9GFTntPxu1SzVdonOjGKXoFCmmnUFEUTt9OIZCljQfMZh+syt612K3HoAu4JlNrvYEjpGtkiSwRBV76gCGVoJv9F1FyZ/t9H6GR6tEJBsg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Since slab pages are now frozen, it makes sense to have large kmalloc() objects behave same as small kmalloc(), as the choice between the two is an implementation detail depending on allocation size. Notably, increasing refcount on a slab page containing kmalloc() object is not possible anymore, so it should be consistent for large kmalloc pages. Therefore, change large kmalloc to use the frozen pages API. Because of some unexpected fallout in the slab pages case (see commit b9c0e49abfca ("mm: decline to manipulate the refcount on a slab page"), implement the same kind of checks and warnings as part of this change. Notably, networking code using sendpage_ok() to determine whether the page refcount can be manipulated in the network stack should continue behaving correctly. Before this change, the function returns true for large kmalloc pages and page refcount can be manipulated. After this change, the function will return false. Acked-by: Roman Gushchin Acked-by: Harry Yoo Signed-off-by: Vlastimil Babka --- include/linux/mm.h | 4 +++- mm/slub.c | 6 +++--- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index bf55206935c467f7508e863332063bb15f904a24..d3eb6adf9fa949fbd611470182a03c743b16aac7 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1549,6 +1549,8 @@ static inline void get_page(struct page *page) struct folio *folio = page_folio(page); if (WARN_ON_ONCE(folio_test_slab(folio))) return; + if (WARN_ON_ONCE(folio_test_large_kmalloc(folio))) + return; folio_get(folio); } @@ -1643,7 +1645,7 @@ static inline void put_page(struct page *page) { struct folio *folio = page_folio(page); - if (folio_test_slab(folio)) + if (folio_test_slab(folio) || folio_test_large_kmalloc(folio)) return; folio_put(folio); diff --git a/mm/slub.c b/mm/slub.c index 11356c701f9f857a2e8cf40bf963ac3abdb5e010..d87015fad2df65629050d9bcd224facd3d2f4033 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4284,9 +4284,9 @@ static void *___kmalloc_large_node(size_t size, gfp_t flags, int node) flags |= __GFP_COMP; if (node == NUMA_NO_NODE) - folio = (struct folio *)alloc_pages_noprof(flags, order); + folio = (struct folio *)alloc_frozen_pages_noprof(flags, order); else - folio = (struct folio *)__alloc_pages_noprof(flags, order, node, NULL); + folio = (struct folio *)__alloc_frozen_pages_noprof(flags, order, node, NULL); if (folio) { ptr = folio_address(folio); @@ -4783,7 +4783,7 @@ static void free_large_kmalloc(struct folio *folio, void *object) lruvec_stat_mod_folio(folio, NR_SLAB_UNRECLAIMABLE_B, -(PAGE_SIZE << order)); __folio_clear_large_kmalloc(folio); - folio_put(folio); + free_frozen_pages(&folio->page, order); } /* -- 2.49.0