From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C486CEDE983 for ; Thu, 14 Sep 2023 06:39:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 06CDC6B02AD; Thu, 14 Sep 2023 02:39:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 01CD16B02B1; Thu, 14 Sep 2023 02:39:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E098E6B02BC; Thu, 14 Sep 2023 02:39:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id CE79F6B02AD for ; Thu, 14 Sep 2023 02:39:02 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 9888281040 for ; Thu, 14 Sep 2023 06:39:02 +0000 (UTC) X-FDA: 81234250524.15.833DCE4 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf20.hostedemail.com (Postfix) with ESMTP id 6EF061C001F for ; Thu, 14 Sep 2023 06:39:00 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=aK3ONLVc; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=DVwxMGSc; dmarc=none; spf=pass (imf20.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694673540; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WL8N789tV1b/LaN3kL8qLP7fvS7C6EVprtG5NflEES4=; b=lFqgm+N5iUjdDl/tr2dIIGh2my4kpMdFO7fHutEDPA2JBf+V81a/WFTMf3/SOvG2PmTc2z oEv1DTWwUmBsqjf2UB19zZ7SVZBNw9oo7+NeUnwwqUH0sHTiMLmiYi81dDQoDA2eBhbk22 6hXplW0INs7JfwZvjPIARC5HzmLuDIo= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=aK3ONLVc; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=DVwxMGSc; dmarc=none; spf=pass (imf20.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694673540; a=rsa-sha256; cv=none; b=PaG8r5854jpX/1BEqlUnPHD632ct8CagrbSBuyF7Lxg4RvzSFbEkdThcyr/GoeANM2CTQ7 IuE4/0s2oYLzvdE0LVgUfsF1CA03ezTF4eYa8BYBoQ9KXE2evqbVjHJpZzHIAX6PReawYt 8i4DgU9cmejbAfYLO2wpXh1Wr9po3Yo= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 23EFF2185C; Thu, 14 Sep 2023 06:38:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1694673538; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WL8N789tV1b/LaN3kL8qLP7fvS7C6EVprtG5NflEES4=; b=aK3ONLVcPM42JYtEFgqTDl58EM58n7pHyG8OSxxQ2HRhBbzApbEdHI+YP3qFn3AM8tXaK6 5lWxuin3/mo25zz3YE/BEK8eE6E0UxaPeIOd2DCvJk2ZjKfYAa9UrwcRQ6kO5DRzaTAAyX 7QbWBf38XqfZ/0jqCyC/uiUoQM5Encg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1694673538; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WL8N789tV1b/LaN3kL8qLP7fvS7C6EVprtG5NflEES4=; b=DVwxMGScxeSX3Lgc20NI4N1dpMAk3b3lO8BVJg2RCj4LCf1jlvl8RKGEYEEfVjhQWM/+wc HaP4i4joHaKt+EDA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id E98CD13580; Thu, 14 Sep 2023 06:38:57 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id n/tDOIGqAmU2cAAAMHmgww (envelope-from ); Thu, 14 Sep 2023 06:38:57 +0000 Message-ID: Date: Thu, 14 Sep 2023 08:38:57 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.14.0 Subject: Re: [RFC PATCH v4] mm/slub: Optimize slub memory usage To: jaypatel@linux.ibm.com, Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: linux-mm@kvack.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, tsahu@linux.ibm.com, piyushs@linux.ibm.com References: <20230720102337.2069722-1-jaypatel@linux.ibm.com> <7fdf3f5dfd9fa1b5e210cc4176cac58a9992ecc0.camel@linux.ibm.com> <2e257eb4b3cc76f78619f5b8c9f95462421762d4.camel@linux.ibm.com> Content-Language: en-US From: Vlastimil Babka In-Reply-To: <2e257eb4b3cc76f78619f5b8c9f95462421762d4.camel@linux.ibm.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 6EF061C001F X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: hytmnbsdocafabq16euxaxmt8j3j41mt X-HE-Tag: 1694673540-945984 X-HE-Meta: U2FsdGVkX1/9VNtKETozOjp77kmRcaHRAFNlu8XTllM4RWCoglUuU3s7sauyK7ZG8v4pVywvrCtUlPbt4nD1KW0EqHkIskBkDTrNTJ3d3Sk7Fa+C3UJoEyzoQe4gBlMnVhexSKIZzRc85E/y3qDi1yoBrjuLLLW2TtsIxCA99wB5JQehTi+pRlfgZFXEqwKTdkvf0CQQr8V9JDr8c+Z/BwN/NJF5Qnmf/v4bbrwM8XsWi4ueBpYbDB4I/wRZXSZYERAnpLPIOBkgMigcz5YMNTLFHB/KV7y+/6OOgMbC3GAT9DDjfiNZeg+rkHjPAdvqT+U315cQKi+NEKCi5AJkc5SFWGqmKgkW6hf8O6skC6EiG9UWizx7zH5pNL5FM5skDl9c5309S4iAfVuLGUawhrg4Q0zE/scNbESz4RGpwb6fcKfeyAJglSXEpszlqjiov9SckMG6mO3ZvVzErsXcBnVZOdl417zAEartY+XlIHStTm7uLXaJlpVqvNPNahw/7wI5UBgBBOlEUZBo3QQobqCi0YfPbhNdITDUJ0MVtSqaWyqHscg9dU1m8Llw0shehlfyIDfyrpDfAcLWtbOViviUVh1ppoNWTAqjtknxTvECJqxpXm8cxHfhHHipEq7sYedrHAp+6BwO3wrJPM9RzRYvhTRlBk6sGYbDYeb66rZzvPentPLubeHge2ccBKEgbYmaQ1pci2WdAuDd3IbBkj3OAfNvp9pcCN8foByKpjPC4nZtzT0nmKRbKDP/N0f9H2MCJ9stlx8Vlxj5P0UT6dZqbllKRVMonCuCw/5oGLiEtZet3LI3Wg5Pnnb3ghtdnbs6IdEgtVt1MtB7DNliqhwJMJEigYqm/HrypkNjLeuYfiFS4vOcnBMxXuvj6Izx+r3SuL1py5LeORtjyc6Lq2hbadrwEMmvs9LLulqUYmMNaSGhM+qChD/Mbj1DtGnKCCMTNpFRL0Rz9BoxfLV K0lmgTii BzPhoNAlXMV4uuVqW7xLxCNWafcwSMLzwqW2iPRVvt6Ybo55OpKIy1joOM866Mlc3YGPyfG5QZnOmV5OM4C2zhxNB3wCS+BodBQaFwUImVXLzB4ao/Jq7uDfsudpyte1xBz5k61D5wRrs1trGEZu6CUhRcY0WCoOXgyglJF4Lv91gWPJch6wpX0zmx4T0gOpppt0nNR4/v26hUpekIN4EndGkFKrWUcPtyxTYsIYS6BBFbZTK0Z73ZtPts2YY8ieuBJ9YOgFV7U9NhaChglpcGzYpRMu0mxnXsvpQ29UbtCqdvVJ3tdKULaGqK+rqatrTs2utDCwq5C4GbQk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 9/14/23 07:40, Jay Patel wrote: > On Thu, 2023-09-07 at 15:42 +0200, Vlastimil Babka wrote: >> On 8/24/23 12:52, Jay Patel wrote: >> How can increased fraction_size ever result in a lower order? I think >> it can >> only result in increased order (or same order). And the simulations >> with my >> hack patch don't seem to counter example that. Note previously I did >> expect >> the order to be lower (or same) and was surprised by my results, but >> now I >> realized I misunderstood the v4 patch. > > Hi, Sorry for late reply as i was on vacation :) > > You're absolutely > right. Increasing the fraction size won't reduce the order, and I > apologize for any confusion in my previous response. No problem, glad that it's cleared :) >> >> > 2) Have also seen reduction in overall slab cache numbers as >> > because of >> > increasing page order >> >> I think your results might be just due to randomness and could turn >> out >> different with repeating the test, or converge to be the same if you >> average >> multiple runs. You posted them for "160 CPUs with 64K Page size" and >> if I >> add that combination to my hack print, I see the same result before >> and >> after your patch: >> >> Calculated slab orders for page_shift 16 nr_cpus 160: >> 8 0 >> 1824 1 >> 3648 2 >> 7288 3 >> 174768 2 >> 196608 3 >> 524296 4 >> >> Still, I might have a bug there. Can you confirm there are actual >> differences with a /proc/slabinfo before/after your patch? If there >> are >> none, any differences observed have to be due to randomness, not >> differences >> in order. > > Indeed, to eliminate randomness, I've consistently gathered data from > /proc/slabinfo, and I can confirm a decrease in the total number of > slab caches. > > Values as on 160 cpu system with 64k page size > Without > patch 24892 slab caches > with patch 23891 slab caches I would like to see why exactly they decreased, given what the patch does it has to be due to getting a higher order slab pages. So the values of " " columns should increase for some caches - which ones and what is their ? >> >> Going back to the idea behind your patch, I don't think it makes >> sense to >> try increase the fraction only for higher-orders. Yes, with 1/16 >> fraction, >> the waste with 64kB page can be 4kB, while with 1/32 it will be just >> 2kB, >> and with 4kB this is only 256 vs 128bytes. However the object sizes >> and >> counts don't differ with page size, so with 4kB pages we'll have more >> slabs >> to host the same number of objects, and the waste will accumulate >> accordingly - i.e. the fraction metric should be independent of page >> size >> wrt resulting total kilobytes of waste. >> >> So maybe the only thing we need to do is to try setting it to 32 >> initial >> value instead of 16 regardless of page size. That should hopefully >> again >> show a good tradeoff for 4kB as one of the earlier versions, while on >> 64kB >> it shouldn't cause much difference (again, none at all with 160 cpus, >> some >> difference with less than 128 cpus, if my simulations were correct). >> > Yes, We can modify the default fraction size to 32 for all page sizes. > I've noticed that on a 160 CPU system with a 64K page size, there's a > noticeable change in the total memory allocated for slabs – it > decreases. > > Alright, I'll make the necessary changes to the patch, setting the > fraction size default to 32, and I'll post v5 along with some > performance metrics. Could you please also check my cleanup series at https://lore.kernel.org/all/20230908145302.30320-6-vbabka@suse.cz/ (I did Cc you there). If it makes sense, I'd like to apply the further optimization on top of those cleanups, not the other way around. Thanks! >> >> > > Anyway my point here is that this evaluation approach might be >> > > useful, even >> > > if it's a non-upstreamable hack, and some postprocessing of the >> > > output is >> > > needed for easier comparison of before/after, so feel free to try >> > > that out. >> > >> > Thank you for this details test :) >> > > BTW I'll be away for 2 weeks from now, so further feedback will >> > > have >> > > to come >> > > from others in that time... >> > > >> > Do we have any additional feedback from others on the same matter? >> > >> > Thank >> > >> > Jay Patel >> > > > Thanks! >> > > > -- >> > > > Hyeonggon >