From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E65D5C4332F for ; Mon, 14 Nov 2022 11:50:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5BE706B0072; Mon, 14 Nov 2022 06:50:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 56E368E0002; Mon, 14 Nov 2022 06:50:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 436AE8E0001; Mon, 14 Nov 2022 06:50:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 352D76B0072 for ; Mon, 14 Nov 2022 06:50:16 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id CF6E11A0E12 for ; Mon, 14 Nov 2022 11:50:15 +0000 (UTC) X-FDA: 80131879590.15.5763BF4 Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) by imf02.hostedemail.com (Postfix) with ESMTP id 643A380006 for ; Mon, 14 Nov 2022 11:50:15 +0000 (UTC) Received: by mail-pj1-f46.google.com with SMTP id m14-20020a17090a3f8e00b00212dab39bcdso13560961pjc.0 for ; Mon, 14 Nov 2022 03:50:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=pxLw92CxNmXV7lUHwwNNh3pTOe6v+PoNrvkBfjZKYfc=; b=KRZeS7F7eWOmhgnb4D7Y4+WqRwo8eLME20CHDJIBZlfXI6X4le5Kor2WIa1YVw4LM7 znrdNlx3rbhxecE9Q883iLc6fL9Jr+Hr2//jJdJGgVusXzXvJ+FyKKsiwr28hy8bsi1l fqEM4oXGaIGnNPmocA7h0jot+GMhRXl9oF8B0Ny8L4UsLANYRDZG1D6LiaeAxeDvTdQd nIoaI+6GXagI3hJ9Po9/vsKKQ7/5LJFeCdRQFWklw2UvLYZJT5hkxFmgiHe2s/DDRCGk HZUXEwWGqYqOjM8TFuceqGQPQ5D1yYzFxPsMMqX89I7raHubUCEHlPV4W3DJG8nz5wrR fALA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=pxLw92CxNmXV7lUHwwNNh3pTOe6v+PoNrvkBfjZKYfc=; b=tlsQ4DCJBhNaBZUMsJbtIgWeqXCiCwfwPK3Ne4Ie8cabvOABTdxYnlUwVFQV4Vi534 MXPbh3ktdawIqYFQCLpLhtArKrASQRF2MKOixvKRrxhPH4y303eRNIeB8EUWdcRf4mg+ mRY6C6PZpa519Jchqr3J3plCt7DnRlWJzHk+FWbLzDFqIs0pjJRJQJ3cL5Z1kpbZWUPN q4ajmqkahY/zlYyCVCppKCzU5MTfhd21LtLlsHyeR3oc6MoN0My5786tmINaSyJJrYxy BKnmQJvdxtP/DgXvMHBI/xYXesqbjRnneEkXrkXe7pGDxdD7dNcA4WnHKnSQKZE7iEoz zO4g== X-Gm-Message-State: ANoB5pnck4mt01LEo0/ko8xvZuvERlvCvPBAO03CbI3RYORLt4QZQuvc phKGfkixcn+b/maAWqnD/VQ= X-Google-Smtp-Source: AA0mqf51MvSUlnnQ65C1Z0llSKT9K+dncVmWd3bQIUneAAbtt+mMxubD8JoJPEXzsVKPNOov/K3OIg== X-Received: by 2002:a17:902:7843:b0:187:282c:9b9c with SMTP id e3-20020a170902784300b00187282c9b9cmr13412451pln.29.1668426614110; Mon, 14 Nov 2022 03:50:14 -0800 (PST) Received: from hyeyoo ([114.29.91.56]) by smtp.gmail.com with ESMTPSA id x12-20020aa79acc000000b0056ddd2b5e9bsm6517901pfp.41.2022.11.14.03.50.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Nov 2022 03:50:13 -0800 (PST) Date: Mon, 14 Nov 2022 20:50:03 +0900 From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To: Vlastimil Babka Cc: Damien Le Moal , Conor Dooley , Pasha Tatashin , Christoph Lameter , David Rientjes , Joonsoo Kim , Pekka Enberg , Matthew Wilcox , Roman Gushchin , Linus Torvalds , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , Catalin Marinas , Rustam Kovhaev , Andrew Morton , Josh Triplett , Arnd Bergmann , Russell King , Alexander Shiyan , Aaro Koskinen , Janusz Krzysztofik , Tony Lindgren , Yoshinori Sato , Rich Felker , Jonas Bonn , Stefan Kristiansson , Stafford Horne , "linux-arm-kernel@lists.infradead.org" , openrisc@lists.librecores.org, linux-riscv@lists.infradead.org, linux-sh@vger.kernel.org, Geert Uytterhoeven , Conor.Dooley@microchip.com, Paul Cercueil Subject: Re: Deprecating and removing SLOB Message-ID: References: <93079aba-362e-5d1e-e9b4-dfe3a84da750@opensource.wdc.com> <44da078c-b630-a249-bf50-67df83cd8347@suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <44da078c-b630-a249-bf50-67df83cd8347@suse.cz> ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1668426615; a=rsa-sha256; cv=none; b=s1rhApx8Lr1SXtVU6lcPDg/nw3xssrK3esRCIAhrRfLfOiv2I83h9npQPnOSabEbgC0/N7 0jgH5aNaJbczJQNbb84Rblrmoi5c+ZXOM+bnfMa2NsaKeRsFy3GjUytcfqkAzNJgQ6k5CL ckEsUk8JEskbaGjaxlXv8EQw9KKBOx8= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=KRZeS7F7; spf=pass (imf02.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1668426615; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pxLw92CxNmXV7lUHwwNNh3pTOe6v+PoNrvkBfjZKYfc=; b=L+tFg1ijDbTL/wftY8Krokk9ZXMam7bcFOXas/lunp4kGKWG1De9YZP1S+UUigm1tDGZJ/ AkbSCmgiNk8PVjqqufX1gFHb0RN6EUNXpJvS1UhD0T7s35sPRCQhS3An21zxkwh5iF9VpA YkJijF1UlyIJZPkYnWZ2iwLQErQeSRo= X-Stat-Signature: barg7o73ju1pkbmgpg5demh3rwcuimuk X-Rspamd-Queue-Id: 643A380006 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=KRZeS7F7; spf=pass (imf02.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam04 X-Rspam-User: X-HE-Tag: 1668426615-70809 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Nov 14, 2022 at 10:36:31AM +0100, Vlastimil Babka wrote: > On 11/14/22 06:48, Damien Le Moal wrote: > > On 11/14/22 10:55, Damien Le Moal wrote: > >> On 11/12/22 05:46, Conor Dooley wrote: > >>> On Fri, Nov 11, 2022 at 11:33:30AM +0100, Vlastimil Babka wrote: > >>>> On 11/8/22 22:44, Pasha Tatashin wrote: > >>>>> On Tue, Nov 8, 2022 at 10:55 AM Vlastimil Babka wrote: > >>>>>> > >>>>>> Hi, > >>>>>> > >>>>>> as we all know, we currently have three slab allocators. As we discussed > >>>>>> at LPC [1], it is my hope that one of these allocators has a future, and > >>>>>> two of them do not. > >>>>>> > >>>>>> The unsurprising reasons include code maintenance burden, other features > >>>>>> compatible with only a subset of allocators (or more effort spent on the > >>>>>> features), blocking API improvements (more on that below), and my > >>>>>> inability to pronounce SLAB and SLUB in a properly distinguishable way, > >>>>>> without resorting to spelling out the letters. > >>>>>> > >>>>>> I think (but may be proven wrong) that SLOB is the easier target of the > >>>>>> two to be removed, so I'd like to focus on it first. > >>>>>> > >>>>>> I believe SLOB can be removed because: > >>>>>> > >>>>>> - AFAIK nobody really uses it? It strives for minimal memory footprint > >>>>>> by putting all objects together, which has its CPU performance costs > >>>>>> (locking, lack of percpu caching, searching for free space...). I'm not > >>>>>> aware of any "tiny linux" deployment that opts for this. For example, > >>>>>> OpenWRT seems to use SLUB and the devices these days have e.g. 128MB > >>>>>> RAM, not up to 16 MB anymore. I've heard anecdotes that the performance > >>>>>> SLOB impact is too much for those who tried. Googling for > >>>>>> "CONFIG_SLOB=y" yielded nothing useful. > >>>>> > >>>>> I am all for removing SLOB. > >>>>> > >>>>> There are some devices with configs where SLOB is enabled by default. > >>>>> Perhaps, the owners/maintainers of those devices/configs should be > >>>>> included into this thread: > >>>>> > >>>>> tatashin@soleen:~/x/linux$ git grep SLOB=y > >>> > >>>>> arch/riscv/configs/nommu_k210_defconfig:CONFIG_SLOB=y > >>>>> arch/riscv/configs/nommu_k210_sdcard_defconfig:CONFIG_SLOB=y > >>>>> arch/riscv/configs/nommu_virt_defconfig:CONFIG_SLOB=y > >>> > >>>> > >>>> Turns out that since SLOB depends on EXPERT, many of those lack it so > >>>> running make defconfig ends up with SLUB anyway, unless I miss something. > >>>> Only a subset has both SLOB and EXPERT: > >>>> > >>>>> git grep CONFIG_EXPERT `git grep -l "CONFIG_SLOB=y"` > >>> > >>>> arch/riscv/configs/nommu_virt_defconfig:CONFIG_EXPERT=y > >>> > >>> I suppose there's not really a concern with the virt defconfig, but I > >>> did check the output of `make nommu_k210_defconfig" and despite not > >>> having expert it seems to end up CONFIG_SLOB=y in the generated .config. > >>> > >>> I do have a board with a k210 so I checked with s/SLOB/SLUB and it still > >>> boots etc, but I have no workloads or w/e to run on it. > >> > >> I sent a patch to change the k210 defconfig to using SLUB. However... > > Thanks! > > >> The current default config using SLOB gives about 630 free memory pages > >> after boot (cat /proc/vmstat). Switching to SLUB, this is down to about > >> 400 free memory pages (CONFIG_SLUB_CPU_PARTIAL is off). > > Thanks for the testing! How much RAM does the system have btw? I found 8MB > somewhere, is that correct? > So 230 pages that's a ~920 kB difference. Last time we saw less dramatic > difference [1]. But that was looking at Slab pages, not free pages. IIRC overhead of s->min_partial (between 5 and 10) was pretty big because SLUB caches at most (s->min_partial) * (nr of caches) * (size of slab) bytes of unused memory. Passing slub_max_order=0 also may help a little bit. > The extra overhead could be also in percpu allocations, code etc. SLUB do not use large amount of percpu allocator I think, less than 30kB on such a small machine. Maybe also it would help reducing code size to disable CONFIG_MEMCG and CONFIG_TRACING, CONFIG_SLUB_DEBUG and CONFIG_SYSFS. I started from tinyconfig and enabled only necessary configs when testing in [1] (it's a bit laborious cuz pure tinyconfig does not even boot...). > >> This is with a buildroot kernel 5.19 build including a shell and sd-card > >> boot. With SLUB, I get clean boots and a shell prompt as expected. But I > >> definitely see more errors with shell commands failing due to allocation > >> failures for the shell process fork. So as far as the K210 is concerned, > >> switching to SLUB is not ideal. > >> > >> I would not want to hold on kernel mm improvements because of this toy > >> k210 though, so I am not going to prevent SLOB deprecation. I just wish > >> SLUB itself used less memory :) > > > > Did further tests with kernel 6.0.1: > > * SLOB: 630 free pages after boot, shell working (occasional shell fork > > failure happen though) > > * SLAB: getting memory allocation for order 7 failures on boot already > > (init process). Shell barely working (high frequency of shell command fork > > failures) > > * SLUB: getting memory allocation for order 7 failures on boot. I do get a > > shell prompt but cannot run any shell command that involves forking a new > > process. > > > > So if we want to keep the k210 support functional with a shell, we need > > slob. If we reduce that board support to only one application started as > > the init process, then I guess anything is OK. > > In [1] it was possible to save some more memory with more tuning. Some of > that required boot parameters and other code changes. In another reply [2] I > considered adding something like SLUB_TINY to take care of all that, so > looks like it would make sense to proceed with that. > > [1] > https://lore.kernel.org/all/Yg9xSWEaTZLA+hYt@ip-172-31-19-208.ap-northeast-1.compute.internal/ > [2] https://lore.kernel.org/all/eebc9dc8-6a45-c099-61da-230d06cb3157@suse.cz/ -- Thanks, Hyeonggon