From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C316AEE6456 for ; Fri, 15 Sep 2023 10:59:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 58F758D0001; Fri, 15 Sep 2023 06:59:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5410B6B0356; Fri, 15 Sep 2023 06:59:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 408098D0001; Fri, 15 Sep 2023 06:59:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 2F3276B0354 for ; Fri, 15 Sep 2023 06:59:54 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id E34F541015 for ; Fri, 15 Sep 2023 10:59:53 +0000 (UTC) X-FDA: 81238536666.19.8A331D1 Received: from mail-ej1-f73.google.com (mail-ej1-f73.google.com [209.85.218.73]) by imf12.hostedemail.com (Postfix) with ESMTP id 1BA8240003 for ; Fri, 15 Sep 2023 10:59:51 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=wzVXYWCX; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf12.hostedemail.com: domain of 3JjkEZQsKCNcF3MM7HKBSSH9HH9E7.5HFEBGNQ-FFDO35D.HK9@flex--matteorizzo.bounces.google.com designates 209.85.218.73 as permitted sender) smtp.mailfrom=3JjkEZQsKCNcF3MM7HKBSSH9HH9E7.5HFEBGNQ-FFDO35D.HK9@flex--matteorizzo.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694775592; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jWKqeflunudoGymWmoNHHzgvD8T7OkY46quv3dOXpy0=; b=2Qt3KJRYEB5GoxNfLQuy/cct674J2TbdGdq5eCsHdbgLdxdhL8Ia4CkA3c7iWefVf177+j ci48ULEVnZtU1DPaAZH+rjDpPUfQtj03PWrvu6MMpWqWubrz8dQVQGBrQ5sQBnbKcpMjnm TK+ES749Qr+BvFue+y6SbWexmZDzu+k= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=wzVXYWCX; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf12.hostedemail.com: domain of 3JjkEZQsKCNcF3MM7HKBSSH9HH9E7.5HFEBGNQ-FFDO35D.HK9@flex--matteorizzo.bounces.google.com designates 209.85.218.73 as permitted sender) smtp.mailfrom=3JjkEZQsKCNcF3MM7HKBSSH9HH9E7.5HFEBGNQ-FFDO35D.HK9@flex--matteorizzo.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694775592; a=rsa-sha256; cv=none; b=ftaq0a0HwTI43FtrQ8QTatGBvRrelioPlwNSv11MK6x4SodzUVMVe61QO8i6UQg+xdTS9B WVJr/V9xMb5QqISW0iQGxzmkleq5gODpw+0u0Z2Fkr341oDaZqsTpfPHVB9SPfZsDDKgW7 1pigWOhePymbXVDyhlE1mfyBHPUPA4A= Received: by mail-ej1-f73.google.com with SMTP id a640c23a62f3a-9adb6dd9e94so137301466b.2 for ; Fri, 15 Sep 2023 03:59:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1694775590; x=1695380390; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=jWKqeflunudoGymWmoNHHzgvD8T7OkY46quv3dOXpy0=; b=wzVXYWCX8X6bRPCagUG+DfhHu0ZgFTyVZpxW51l9KmB5kU03B2um1hSvLF++rAjV/L v4w4n7s//D8LnfAliyZoZMijcnik2aVMQJbrK3x3LD3hfCSAljYcrmAT053ncuGc9H1+ LlQud8tbikOHh0huexb2i1UJmcoYIipaFOuBRyPxvXms6LBGMxrIOWj2rJxOZSZ2DtwT Bnm5lY/fkqOZiTP/qucXeQGF0iC2s+rYQnOn/8neWAIAC19jjTbeFm3nOlRxMz7M+0+S K4qCdDN3rjejNq6ctIyrk5Cv1JhyYWxhkPZROsCxWRsP5RE6UoqfKHLnUSFHCAU3WWwE /1VQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694775590; x=1695380390; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=jWKqeflunudoGymWmoNHHzgvD8T7OkY46quv3dOXpy0=; b=EpdJODZdSfPL7Ce7XQHoicopj0qLNDXlrERqX9Cc3ps1Xc2Tcnyw9cR3OFUYZMjtTc OMbDglKhZp/tu3zS2zL9TiuKueU31KapYJrxjaL62FxHLbgcBw3NlXIp1GtV7EzVYzdM 8NSTaw6rHq+annxzoi7eRVp8svMGOpmpUofYMBLqvGDKrlIPdoekrPnM5KV+9TqHoimr fc3TZ5OLVoZ2sKWAXaW5cogugF/vveaLJ0yGN0vzj3lLQZub12cl0XkbD1hjQ4xkBjzT UCu/vGj/x7LQ/YLOLU/NjPCbV4jGNT9MLgAspLK+ZFt8R5WUrYcvT0avUkkNVcjruzgJ OeiQ== X-Gm-Message-State: AOJu0Yy3i7WxMJFqPFkYQH8rPfpvv3+jPuI5ZumhwtcS/CO/au+ChXKk +CEugaCnJ6P6PvLkrRdzbm90zzDIUD8Oy5Lmgw== X-Google-Smtp-Source: AGHT+IHib2bYQAS4qp2GAjeH5qpDFROM7WBawhFVE9nuJn7ed324ILtD5K90tbrPwgE6ihy4nV/s6H7sq+VLOkV6Ww== X-Received: from mr-cloudtop2.c.googlers.com ([fda3:e722:ac3:cc00:31:98fb:c0a8:2a6]) (user=matteorizzo job=sendgmr) by 2002:a17:907:890:b0:9ad:a751:2ea3 with SMTP id zt16-20020a170907089000b009ada7512ea3mr6741ejb.6.1694775590539; Fri, 15 Sep 2023 03:59:50 -0700 (PDT) Date: Fri, 15 Sep 2023 10:59:24 +0000 In-Reply-To: <20230915105933.495735-1-matteorizzo@google.com> Mime-Version: 1.0 References: <20230915105933.495735-1-matteorizzo@google.com> X-Mailer: git-send-email 2.42.0.459.ge4e396fd5e-goog Message-ID: <20230915105933.495735-6-matteorizzo@google.com> Subject: [RFC PATCH 05/14] mm/slub: create folio_set/clear_slab helpers From: Matteo Rizzo To: cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, akpm@linux-foundation.org, vbabka@suse.cz, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com, keescook@chromium.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-hardening@vger.kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, corbet@lwn.net, luto@kernel.org, peterz@infradead.org Cc: jannh@google.com, matteorizzo@google.com, evn@google.com, poprdi@google.com, jordyzomer@google.com Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 1BA8240003 X-Stat-Signature: ywjej3efd4wqdt417nrx4z4pfz5jox36 X-Rspam-User: X-HE-Tag: 1694775591-348244 X-HE-Meta: U2FsdGVkX18WqcvXe8QghPM5eo0EECnl492zX7i3NR2bxPw18Fdt00mtLxEeu3umLLB1OnnSmVQcRUAH4EuuDOk22BnjqWwe7zelzF63ZEdo4JTrhD0DyQdC+oDwyY8b6heLpYLW7bKrfdsNw6FYkYxdVaX1nacTzWUGErYy/+FmggYti2ZhI7rEc1VFOIG4jAbK4Zcypwim6NfQUReXRY29cnYZZwKIPxsPvrsnDW3XVo4AySTSVY3OMmuUvjoXcpsnBnvXwc9IDbCZPfkCS29Ox6pD+vppkYe7NPxZqeGZo3+FspbSZoGNFw4tNvcYWbZFY86rmEeZc663ogj4uMyKemgIV2Mq97hmx2MBZyddaRqwxNe+1AYdozdroulQjJOzPgZvnsHIa4978YYKneHHP8wkBKPEgo+fnXuRttjtYzK/KNlHIdPnxrQ6s/VXaSfsa2hgJnDD344X0YVQbuMZsf2MAe2iECqIwaESPi4MhF0UStZBjjUJnuD5Vy3sKXbrwNpYx6y/i3R2dPOFYA68RvXq/TwIq1FlVFTWtjd2yOL0YQ80FTe9A3Np8l6QieYIgc6yt6uhvtFdKRzKnIs3psShlwkHNM2bnZ+XjR3GaPtV0rOzagmVUE+ejv8w6kYrzGrniT75LM4bybWZ74mekyDnDncWAXdK6JdikwiRO+bFZ0Re5+WUeNDo6lsqQkqUQgnKe6FFAwYjvqnG/rm+zUYLMuQ0SI16wHRAiAChIeR16z3tvzmmNXyRihcbrs+QhgayDg4rFqcU7x3YdwmwoitDlQJpL4VH3Qw4UU0vXi0fyF7rc8+fjpc+DHshB1AT7ywf9JmtvL5K6PNCZozr7j6H/c5ovwCZXdR8+S9+XBG4zgone54jmJofG0PAxrc69dmzQOr3tNAZJmKHe049hi/k5hoOWvRALHI8DUHCcJFzyQNxcJ12dM7lXhrPgaXV83lxIkqcT4TVSAg TIAinHNV tg5vcsfzQ5prqqAHqnhr9XS9uysoHw1Cd7OcYymeKYAtMJuiCLMyaYiNfzAkR6A39+i77weOWjig/QLIJ4Z5PqlEJSDzCBnyQoHWwfgip/dVp3p13cx5AkMzBQ+AxEV/UHuMhQ1FN1Gt4UWGtaxNPYyCkaFuHtB7spbyYZOz9zYS9ve7BXwZt0hl697sZAIXpiQG70K03jtoRuyXfGlehYY2ySGbp3Kv1Avlx0fns1wsWfyWThRsbGDolujy5SZvQtUkHPP6lQnbPoEhS/Cel/G+cdkxyEVB4YUOyhIpUeK6YocgNuyjUCrkNRreMMTMPB9dskC6Nq9nYX1Jqw4Qa7IemLMLSOpAGhBmdjJX4umyKm/JNDWZLHwp53VHJ2UqpTU+hYmk2uj3UVUmA+rGmSGh6rt+LV4ESvPe1TWRkU3yhtZG0kYMccZ46paZ8AXp3Qa9jHxZ8qg9AstiGy2WSXjBDdvPP5PsJ2tNHxw5a7ZtpV4IUgKScXAaUAbUL5eW1ydxcUWy3BGWsrO1W6uPPxhT3AE3Nk4BYD/Pm8oOEeOzkzd85JCDGYSxl7XQTJbuBwYn72RgaqM45+iiQpFkIZY8Gt0IKTs0eel9rDFJC8NZm5BW7MIk1TodedUxZ/kCHnhCpOcFBaM5VX0UMP5Qfr3nBEraMALW2b7ydZUHlhjv+Oi4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jann Horn This is refactoring in preparation for SLAB_VIRTUAL. Extract this code to separate functions so that it's not duplicated in the code that allocates and frees page with SLAB_VIRTUAL enabled. Signed-off-by: Jann Horn Co-developed-by: Matteo Rizzo Signed-off-by: Matteo Rizzo --- mm/slub.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index ad33d9e1601d..9b87afade125 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1849,6 +1849,26 @@ static void *setup_object(struct kmem_cache *s, void *object) /* * Slab allocation and freeing */ + +static void folio_set_slab(struct folio *folio, struct slab *slab) +{ + __folio_set_slab(folio); + /* Make the flag visible before any changes to folio->mapping */ + smp_wmb(); + + if (folio_is_pfmemalloc(folio)) + slab_set_pfmemalloc(slab); +} + +static void folio_clear_slab(struct folio *folio, struct slab *slab) +{ + __slab_clear_pfmemalloc(slab); + folio->mapping = NULL; + /* Make the mapping reset visible before clearing the flag */ + smp_wmb(); + __folio_clear_slab(folio); +} + static inline struct slab *alloc_slab_page(gfp_t flags, int node, struct kmem_cache_order_objects oo) { @@ -1865,11 +1885,7 @@ static inline struct slab *alloc_slab_page(gfp_t flags, int node, return NULL; slab = folio_slab(folio); - __folio_set_slab(folio); - /* Make the flag visible before any changes to folio->mapping */ - smp_wmb(); - if (folio_is_pfmemalloc(folio)) - slab_set_pfmemalloc(slab); + folio_set_slab(folio, slab); return slab; } @@ -2067,11 +2083,7 @@ static void __free_slab(struct kmem_cache *s, struct slab *slab) int order = folio_order(folio); int pages = 1 << order; - __slab_clear_pfmemalloc(slab); - folio->mapping = NULL; - /* Make the mapping reset visible before clearing the flag */ - smp_wmb(); - __folio_clear_slab(folio); + folio_clear_slab(folio, slab); mm_account_reclaimed_pages(pages); unaccount_slab(slab, order, s); __free_pages(&folio->page, order); -- 2.42.0.459.ge4e396fd5e-goog