From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 763C8C4332F for ; Tue, 13 Dec 2022 03:04:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E57728E0005; Mon, 12 Dec 2022 22:04:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E06C48E0002; Mon, 12 Dec 2022 22:04:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA7D48E0005; Mon, 12 Dec 2022 22:04:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id B71918E0002 for ; Mon, 12 Dec 2022 22:04:48 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 5DE96AAFBE for ; Tue, 13 Dec 2022 03:04:48 +0000 (UTC) X-FDA: 80235790656.24.E6D4265 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf08.hostedemail.com (Postfix) with ESMTP id CA780160012 for ; Tue, 13 Dec 2022 03:04:45 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=FN9uaAoT; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf08.hostedemail.com: domain of bhe@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=bhe@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1670900686; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/ToI7hZ2DBEkG61hu0i1tPmmNJiNnyQodEsagogpyA4=; b=U6HSpaNipich0KIYsr2hrDPBZrrXCUflXFvh9ExS46YFGHa6inbo9qpDtrAMK/UKQCiHNv lgP3QpsmrWvpHMvL7v7Y/49rGIY6e325tKO1iKrrPIIusAAxjLgHB//BOfWqzyURX6vN1N 9roQ2SwjzYZ79GGuRJwhBAigiphbobM= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=FN9uaAoT; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf08.hostedemail.com: domain of bhe@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=bhe@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1670900686; a=rsa-sha256; cv=none; b=Yd/V/QdtvpQPkGIPn7OxXMA6jZAqkyrRxV8zHlb3Bq/COUNSoHBrieibONnyRVZrgC3gjs j3mqonPQxVbceSbFsQJUriokUUOkPA2xPO3yiCf8kxLrVsFODLKXSsUFmlHTU/pMDYv423 QUcqIA+3InQCZMbYWtYPVxwQ8pseVq0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1670900685; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=/ToI7hZ2DBEkG61hu0i1tPmmNJiNnyQodEsagogpyA4=; b=FN9uaAoTOlLAhEcK5cALBc4h0hptnLW8XEFhYprcahouym+qK0llJd9Jgdpg3KvFfwKna5 FyujetYSPmvzQraVTvfeAT3lUWQ++j5E+Zh3Nt9xV1BOkLwHrvi9dFz6opHqMTHtWJwsim CvStIIvm8urwxbj8JNX+TxSsiLT7fIo= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-54-T9RFYZ6OPNKcz28DZtS-yA-1; Mon, 12 Dec 2022 22:04:39 -0500 X-MC-Unique: T9RFYZ6OPNKcz28DZtS-yA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BE79D1C05AE8; Tue, 13 Dec 2022 03:04:38 +0000 (UTC) Received: from localhost (ovpn-12-126.pek2.redhat.com [10.72.12.126]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6A3FE140E949; Tue, 13 Dec 2022 03:04:37 +0000 (UTC) Date: Tue, 13 Dec 2022 11:04:33 +0800 From: Baoquan He To: Dennis Zhou , Vlastimil Babka Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>, Christoph Lameter , David Rientjes , Joonsoo Kim , Pekka Enberg , Roman Gushchin , Andrew Morton , Linus Torvalds , Matthew Wilcox , patches@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 10/12] mm, slub: remove percpu slabs with CONFIG_SLUB_TINY Message-ID: References: <20221121171202.22080-1-vbabka@suse.cz> <20221121171202.22080-11-vbabka@suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Rspamd-Queue-Id: CA780160012 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: yi3gcdsmixjr7xzfoyb4qbawr3agxxgf X-HE-Tag: 1670900685-836316 X-HE-Meta: U2FsdGVkX18WqBQuLwTTkczYSlITXWuckuAmBhZZhdifwVY1dTeHzK+8wsDM/vKq8/Z1MyMh2p0Gy61G9omasJa8PvbKhJnVcbjqUgqXuq07gnPQMZZwf3JWMIhJxi5rmK0g4Na49KxzWtmdVHnv+awLWpQhF2m+Ko5BFdIN010xX9CI/2mHPWxJV1Pow8DLEFxryEgqkaR2RCFHYx3eHYUmCaDX0MWr3jq0HZBRj2gMV4SW9v2xtGpgfiCN3keW8wPpSOPJ+eivuEDFDW7rpKgxGzaiHEQSAJ+TvEn+uSelvGdZX3L6ctVAkm98BbyYyrQ2sUjjjsEBxFOpIffEzSjLyLL4ymS04vCEcG/BxLf61bQGMOdp6ItCPF3OurC+tSLr/7Z8Fh23chHAqtwzsYikdAUAK5b5h/2o30X4AsdMKniDN4ASgOVimpNPa+Y0IUIklHVqYzsKf7UBshvUK1kMHD+XxiDDCAnxJLB+/9tvXSBDYQjDR7m168rMyK3zgemDVGpKkPy3v3VVBWOnKMPakVpnQFMzgwoyfp5D+bXiGcyegO+wHwyDTkR98bZ/OA05y4cqxj+0iDifDwr+GNScy3KM9p0dkLEWVjyNFZeT2nJsl3mqyOh5mHal+9bCPhN6kmw409dfV2Y4cb9mE50VcgiAI383keUw7iDuDA9+cAqR5HwXeb8QUL5MIFyoYGQPLVn4ONPS7i3ZuolYZHgiW4FP9G715OoTd3tY5s3rmSYRrL4gdGO5SBIadm4TinG+2bdFPqa5zBYvN4RgqnMC38qONSoE4PBMuHlJCyzw7i/cwk5pi/lOId2lI6p2vfHi7gO6ZgoKEBQagqCtshw8QJfon0UJOlIzpRTboSFq4WHO4yv0yfrI6j9gVC+m23Kko8iT4uxxbFvhoW2PlwOa8ikkEEGiDq9uIxdsuw1uuWZ76M41IukKlup/NtTdkjyPHzL/vqQaXu/Xx/b ZZ1gL4C5 +iKazLvaIIsnilR/GmRrvrlVfLGeoYDyTm9wgaSB1+W6Rh7xh/xHbXP/YWkZoD5bWzQik X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 12/12/22 at 05:11am, Dennis Zhou wrote: > Hello, > > On Mon, Dec 12, 2022 at 11:54:28AM +0100, Vlastimil Babka wrote: > > On 11/27/22 12:05, Hyeonggon Yoo wrote: > > > On Mon, Nov 21, 2022 at 06:12:00PM +0100, Vlastimil Babka wrote: > > >> SLUB gets most of its scalability by percpu slabs. However for > > >> CONFIG_SLUB_TINY the goal is minimal memory overhead, not scalability. > > >> Thus, #ifdef out the whole kmem_cache_cpu percpu structure and > > >> associated code. Additionally to the slab page savings, this reduces > > >> percpu allocator usage, and code size. > > > > > > [+Cc Dennis] > > > > +To: Baoquan also. Thanks for adding me. > > > > > Wondering if we can reduce (or zero) early reservation of percpu area > > > when #if !defined(CONFIG_SLUB) || defined(CONFIG_SLUB_TINY)? > > > > Good point. I've sent a PR as it was [1], but (if merged) we can still > > improve that during RC series, if it means more memory saved thanks to less > > percpu usage with CONFIG_SLUB_TINY. > > > > [1] > > https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab.git/tag/?h=slab-for-6.2-rc1 > > The early reservation area not used at boot is then used to serve normal > percpu allocations. Percpu allocates additional chunks based on a free > page float count and is backed page by page, not all at once. I get > slabs is the main motivator of early reservation, but if there are other > users of percpu, then shrinking the early reservation area is a bit > moot. Agree. Before kmem_cache_init() is done, anyone calling alloc_percpu() can only get allocation done from early reservatoin of percpu area. So, unless we can make sure nobody need to call alloc_percpu() before kmem_cache_init() now and future. The only drawback of early reservation is it's not so flexible. We can only dynamically create chunk to increase percpu areas when early reservation is run out, but can't shrink early reservation if system doesn't need that much. So we may need weigh the two ideas: - Not allowing to alloc_percpu() before kmem_cache_init(); - Keep early reservation, and think of a economic value for CONFIG_SLUB_TINY. start_kernel() ->setup_per_cpu_areas(); ...... ->mm_init(); ...... -->kmem_cache_init(); __alloc_percpu() -->pcpu_alloc() --> succeed to allocate from early reservation or -->pcpu_create_chunk() -->pcpu_alloc_chunk() -->pcpu_mem_zalloc()