From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF55AEE8013 for ; Fri, 8 Sep 2023 14:53:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D07D86B00CE; Fri, 8 Sep 2023 10:53:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9CDC76B00D0; Fri, 8 Sep 2023 10:53:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7D21B6B00CE; Fri, 8 Sep 2023 10:53:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 3D85E6B00D1 for ; Fri, 8 Sep 2023 10:53:23 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 14D90A02CB for ; Fri, 8 Sep 2023 14:53:23 +0000 (UTC) X-FDA: 81213723486.28.ECE5107 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf05.hostedemail.com (Postfix) with ESMTP id 025E8100024 for ; Fri, 8 Sep 2023 14:53:20 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=yaoSVUcf; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=1F8uln1O; spf=pass (imf05.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694184801; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yGsbXXqcf4ORLaF4fFwqqZYAQFs76ztvW90bKyvUaBg=; b=xV/340XFDJY0fsDFCw6gYFD8f/zU5cFRFugaOzcpZrFUWVW0yBrniBkcvPJBiRJbh4R1qK YLtuvvjnlU7TsZ5kDnNle222HKcAMWiMzgCX49Lq6IvBIhroyoew0BQuVTI6AeB45MuW2J tp2cSnV31b0PXp80xMKzxo05XCHAYtQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694184801; a=rsa-sha256; cv=none; b=4cPVS4lKAJmj0x8bH4JVaMvSKQR2bb2wlDrAnjv/rrs4djJt0LeASssUyEXcFKlV9VZZN3 pVCqfivC6nhyYucQJIxzPxcrVO0kDTWfEIclSpM/hu+ua8EJTouxCnWILSTvH921jjjBL4 vSlEZwVPAcThHOuO3xN7qTIpbMX7Fm8= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=yaoSVUcf; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=1F8uln1O; spf=pass (imf05.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 57A6A21D26; Fri, 8 Sep 2023 14:53:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1694184799; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yGsbXXqcf4ORLaF4fFwqqZYAQFs76ztvW90bKyvUaBg=; b=yaoSVUcftdz9Z0pvCocl71s6a42GoguCtjRHXa0NVRtJyYOG6A+9KfluK9Nx3m9FsnxsoH nQhFs/wVHJNXuBKUYjlgy0YSvjKAfZz5BhBOE/d/6MtSEdQP0EBpF8i9JPxbt4GAX6uJVi y0ZD68CA0KFlChqiBb0LB0e5BbPQwtk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1694184799; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yGsbXXqcf4ORLaF4fFwqqZYAQFs76ztvW90bKyvUaBg=; b=1F8uln1OGPghmt/eKsFza2ILUJbA9ByJ69bS4pA/H55d4XBFwWCspcGJ6dS96kWwjv3Hpo 6+lv3dA372hQ5VCA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3169A13357; Fri, 8 Sep 2023 14:53:19 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id wGFkC181+2QaBQAAMHmgww (envelope-from ); Fri, 08 Sep 2023 14:53:19 +0000 From: Vlastimil Babka To: David Rientjes , Christoph Lameter , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Jay Patel Cc: Roman Gushchin , Pekka Enberg , Joonsoo Kim , linux-mm@kvack.org, patches@lists.linux.dev, linux-kernel@vger.kernel.org, Vlastimil Babka Subject: [PATCH 3/4] mm/slub: attempt to find layouts up to 1/2 waste in calculate_order() Date: Fri, 8 Sep 2023 16:53:06 +0200 Message-ID: <20230908145302.30320-9-vbabka@suse.cz> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230908145302.30320-6-vbabka@suse.cz> References: <20230908145302.30320-6-vbabka@suse.cz> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 025E8100024 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 1jpdh6fzh8dba15woxchi5dhdzzh7fxf X-HE-Tag: 1694184800-245353 X-HE-Meta: U2FsdGVkX1/FH8OLWtQ3yK7GQN0RbhJBF4HiI2PelISVEQN/jXRFtaxz6XKMzdTQ6+PGTzk9XFNVsdLYuFI7IPHGJERCp3fSHX93Lc/IjSexqvcXjAzSQjjTMU+7cPbwHaPTNbz9+vcipxuiWhI8rbFWC4FoPa/K55LPImdskJBr4qnWtzY0u1numIH2EkFCavi5e063fWT/SRJdKPnvGqlczEpMQe31wW4DQHKSvo2MLws9iWad72PYbnuQ3XDknWwAL9sEW5+ZStwKlWHqttH+nJLPW70b935gpzB0P5HedlzT5PTCR9bxcHVpBi9b56AsVfzOF+YhnR0nr9fPdUJG/p2ElUog+Jfu8qnXEgi81wMMWL5LdleKIckxi4/qTkoGropgS0WSW85fa60nyAQLpxT/Bq/Ur/krjwynKVa/4wOU8Gxg0co/tpS9zEthG90+L6J2FFPHfg1HKltBlUtr61kD4aFHVXccpC8ttK+iHUHJuib6GZSVnBf/jtLyfz79IJOxt6vcUyIhLpSwmhl6F7ycNgNu4EaaFvWR5Y/qZOzbIjzb/SEeA3CaI3+ViS8QOUsqr++zTFFTfzPFivuMCPE4tq1nCsWUgu4oaRbjpZqlvnOa+L4oE5fMthgvlN+1Om2Io/o4fpjErBfWSaNQ3qdRVKXy36Cl1/8cPdAyRIIz+uhlm33T+sq0hSWCtgUs6kH4dnJXyoH/ESWFyDTy8Roi3kU6yQdme9FuhwlQT149Q398OSm/73tZaGPaqHaSp8/yV8H3djRQvPdvmMnhAAuBzIViGvVkGjBfK9MlxYN04aL/7+QEBRDqDhQZfr4F+6KO88MN7LgXH8aUjLe92PNd4P7ZqJAEhWerdSLCudFkxcDk+eR1bK5DlA19HbrZ7L7pdH66leQEPSwqY9ChDKTKfYybq063FytEEqUXZ+gGeoosBeJll2ibeZ02bFe7CAEuOKDnWpDaT8C QabBFWz+ EljxDjNgSiMjiT7e7kyeDzUcwUFW92xOf8Jh+FWefhOjKuT/45Vrev5Epm2gVWwIH85/e0HRPdnoQsp6CTQKw2Fmo1X/sISCjJJiyW9L7S1Km2ESxVI/ypHUhZgLFkZxL1oveMIm/+ZqbACRYDC6JDbU2yF1sFHla8TL3dXL0KmPMnoHXmq98aTiqF5vHPfr0IFlVEZoMroc/NFulYE3y6LRBie4qgMiFumD5gK1QWE87Wbns60w7iK1zVDPdd/libRlZKlhzoVEnHwJtqj6xgfsLXIWoryGqUkA//NMZMXrcnR/YKl1eiektTw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The main loop in calculate_order() currently tries to find an order with at most 1/4 waste. If that's impossible (for particular large object sizes), there's a fallback that will try to place one object within slab_max_order. If we expand the loop boundary to also allow up to 1/2 waste as the last resort, we can remove the fallback and simplify the code, as the loop will find an order for such sizes as well. Note we don't need to allow more than 1/2 waste as that will never happen - calc_slab_order() would calculate more objects to fit, reducing waste below 1/2. Sucessfully finding an order in the loop (compared to the fallback) will also have the benefit in trying to satisfy min_objects, because the fallback was passing 1. Thus the resulting slab orders might be larger (not because it would improve waste, but to reduce pressure on shared locks), which is one of the goals of calculate_order(). For example, with nr_cpus=1 and 4kB PAGE_SIZE, slub_max_order=3, before the patch we would get the following orders for these object sizes: 2056 to 10920 - order-3 as selected by the loop 10928 to 12280 - order-2 due to fallback, as <1/4 waste is not possible 12288 to 32768 - order-3 as <1/4 waste is again possible After the patch: 2056 to 32768 - order-3, because even in the range of 10928 to 12280 we try to satisfy the calculated min_objects. As a result the code is simpler and gives more consistent results. Signed-off-by: Vlastimil Babka --- mm/slub.c | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 5c287d96b212..f04eb029d85a 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4171,23 +4171,17 @@ static inline int calculate_order(unsigned int size) * the order can only result in same or less fractional waste, not more. * * If that fails, we increase the acceptable fraction of waste and try - * again. + * again. The last iteration with fraction of 1/2 would effectively + * accept any waste and give us the order determined by min_objects, as + * long as at least single object fits within slub_max_order. */ - for (unsigned int fraction = 16; fraction >= 4; fraction /= 2) { + for (unsigned int fraction = 16; fraction > 1; fraction /= 2) { order = calc_slab_order(size, min_objects, slub_max_order, fraction); if (order <= slub_max_order) return order; } - /* - * We were unable to place multiple objects in a slab. Now - * lets see if we can place a single object there. - */ - order = calc_slab_order(size, 1, slub_max_order, 1); - if (order <= slub_max_order) - return order; - /* * Doh this slab cannot be placed using slub_max_order. */ -- 2.42.0