From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40947CA0EF1 for ; Tue, 12 Sep 2023 16:02:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C3CC66B011D; Tue, 12 Sep 2023 12:02:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BEC3E6B011E; Tue, 12 Sep 2023 12:02:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ADC786B011F; Tue, 12 Sep 2023 12:02:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 9FAF86B011D for ; Tue, 12 Sep 2023 12:02:47 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 4CA8FA0CEB for ; Tue, 12 Sep 2023 16:02:47 +0000 (UTC) X-FDA: 81228413574.14.F860D9D Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf11.hostedemail.com (Postfix) with ESMTP id 5538C40029 for ; Tue, 12 Sep 2023 16:02:20 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=BkC0Kvx3; spf=none (imf11.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694534543; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=v/ufbF5AE1kJWZUkhHfxUWS38u59OErgU7vO9+D/zZc=; b=Vwe5c09360m6SIKeS4taXDDR/6w5/t2fFKf9jE33mj1sUdVlaLN11oSx5f/0CLjB1v7rLt 7D2Ta7CQAW193CFyTMmHmhsaQ9RZDIN+gZDRDszzUcbJPXMw0TtMgJc1i1s94tWvvY0pBQ FgxMbZue6e+e3l1Emvxb+GnaofHKzzU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694534543; a=rsa-sha256; cv=none; b=Jr7DNaWKHgL856RJ9NzWsvbQEn07Ar3SxNf2WXl12/sJEmDlp0a8WT/SDfrEiYWU6inh0a /wXhENgJ//KBNHUIEkkC+kjPd38GH0edAYyrPAq95PhlPCsbVlvkRxgFrmAC+ovK7190cT leMoplYRIkBMEe3xAncI2dSpN5Jnxfs= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=BkC0Kvx3; spf=none (imf11.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=v/ufbF5AE1kJWZUkhHfxUWS38u59OErgU7vO9+D/zZc=; b=BkC0Kvx3yCVD5ZX3c+tL3s/w4j F0gMhWxTtbpUS8TtfyTsvUMqIpvBUekNvF10dTvQ6HT/Gwap/UWRoDqN0NlX9IQiZVrS13KuvCvQa CLRo1Gf1M5SYYEKkfYGgHtOfee0zp6nGabFsd8MeRk0KqqHZAqiWa93H8+PabHODYDFuhFTSvF3BI GVcr0FxKWQJjLx/yyofeMxP0aixTeqxcuT3m5wPjGmLJwDx7kUFm8nd4rewwTc+TY0lO9l4+T2lWB p2Z/TAiEwyIzUOt/eK3LqVgbgxUZD0vdFEGke8NWqFa2W++o0zHPkq0/zckdogDow2xMU0Kz06+wo V4ZMsk7w==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qg5pv-008S9t-Vc; Tue, 12 Sep 2023 16:02:00 +0000 Date: Tue, 12 Sep 2023 17:01:59 +0100 From: Matthew Wilcox To: Feng Tang Cc: Chuck Lever III , "Sang, Oliver" , "oe-lkp@lists.linux.dev" , lkp , Linux Kernel Mailing List , Christian Brauner , linux-mm , "Huang, Ying" , "Yin, Fengwei" , "Liam R. Howlett" Subject: Re: [linus:master] [shmem] a2e459555c: aim9.disk_src.ops_per_sec -19.0% regression Message-ID: References: <202309081306.3ecb3734-oliver.sang@intel.com> <84984801-F885-4739-B4B3-DE8DE4ABE378@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Stat-Signature: nksyypqa4p81ygap468a6m677bfpiguf X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 5538C40029 X-Rspam-User: X-HE-Tag: 1694534540-362158 X-HE-Meta: U2FsdGVkX18tko8gG/5mYXIQMlYgrYZ4deFELjhsFxi0y5m+c9427KCHXvxogd0hnP0g37htvbyHQ/PFim5utNNtoMrAu/kR+uc1DM9rBE5L2+vJao2T8s+2d2+ERc4BJZMGibIhg878a14m7odwpeGd7M6WD74wAQ5I5ynMqpLPWCh0yckaR77DGeFS1S5WkI6uuc9YwrEJ6RKDRuZAfJtWlocD2+MiOGYMRZBA3Sd/kkHqlq9+lt4H3Kb+1KDhVL6l39KhqWyGcGWNEPr4I8yaEimyFqDrJYMOOLElRCic5Qj+8ZM8/Yn9l0R3uCi/bLgaKapccJUBMmJO/s2kJOMSQQlxctArlyfAbI2vbrw+Op59YzWngy8MXfsVQhSW4YWzqdRA3NuFW+Qyo1rb8d5mW+bFZi8Oab166qb8cg2tC8dPJuDtgmWuyY0qdMw+YxiCRJmJczdIM6Irtz3faUsaXZPVx4YXDMiYuJ/YGoITxHc2haV8uhYHG60xJ8OLDA1mEKC2D0+4rRMXK7pT/UrorrH058BOKQ9YnOBjp2p0B34xg4DNs/2MVd1U9pt6xd0DPqf9muVZxkWZl8leolAxzKx8KZNyFF5/ZC8b5u04lHOgLznTzvSYIqgupHlxP72X1o4+UiarWXkicLvsWPhZYtJAiJuTlWS1bOgh8hOIGqbm3J4LY7vdZ7GiV88ZpX6A18kS31RsXGmD7zswZko62K2k0oazFjkhgIX08+e7GcNZohNZ+xCpfyAFwf6uEmfAjDoULCxaX3zV3eBSHie5qYcvDYvDc11vCvKTpvKb4Mq92usSKt8CrXcDz0UK5UAlRbLa1LSjOaDMak2iApcP9vuvLPNp6qLz/r/SEho9Pq6hRypPAtQ0663nu4VuMgQtnuVVOKb5WoEbt6u5dhzt//hORm0D0Q5Sa/FHRXtXorgxB39/rdNkYlLhTfK4XCl+D8zZnApcAWtxwCw 1P9VrhLV PvkVHAF6KanRs/pJUYnQvSTsaxAcinE6gsvljlRaQ5GsuXakTXZscNsHb59J+9iFG/7EHbQ44Mu6SjvjXi5f0YiKcwCGXfIRrnk+WfDgX5Se7/3Bmsc36qUxhZuSnVT8bmoPFGduH+zMGxB3+6RSL5sG9g3F55G7s/afMYz9SpA0OyNtIETCGOICFuO5ZuS8q3gQs5rVvl6IIzqpOlCG2e+Pp5RFDmKm5tsFA X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Sep 12, 2023 at 11:14:42PM +0800, Feng Tang wrote: > > Well that's the problem. Since I can't run the reproducer, there's > > nothing I can do to troubleshoot the problem myself. > > We dug more into the perf and other profiling data from 0Day server > running this case, and it seems that the new simple_offset_add() > called by shmem_mknod() brings extra cost related with slab, > specifically the 'radix_tree_node', which cause the regression. > > Here is some slabinfo diff for commit a2e459555c5f and its parent: > > 23a31d87645c6527 a2e459555c5f9da3e619b7e47a6 > ---------------- --------------------------- > > 26363 +40.2% 36956 slabinfo.radix_tree_node.active_objs > 941.00 +40.4% 1321 slabinfo.radix_tree_node.active_slabs > 26363 +40.3% 37001 slabinfo.radix_tree_node.num_objs > 941.00 +40.4% 1321 slabinfo.radix_tree_node.num_slabs I can't find the benchmark source, but my suspicion is that this creates and deletes a lot of files in a directory. The 'stable directory offsets' series uses xa_alloc_cyclic(), so we'll end up with a very sparse radix tree. ie it'll look something like this: 0 - "." 1 - ".." 6 - "d" 27 - "y" 4000 - "fzz" 65537 - "czzz" 643289767 - "bzzzzzz" (i didn't work out the names precisely here, but this is approximately what you'd get if you create files a-z, aa-zz, aaa-zzz, etc and delete almost all of them) The radix tree does not handle this well. It'll allocate one node for: entries 0-63 (covers the first 4 entries) entries 0-4095 entries 3968-4031 (the first 5) entries 0-262143 entries 65536-69631 entries 65536-65599 (the first 6) entries 0-16777215 entries 0-1073741823 entries 637534208-654311423 entries 643039232-643301375 entries 643289088-643293183 entries 643289728-643289791 (all 7) That ends up being 12 nodes (you get 7 nodes per page) to store 7 pointers. Admittedly to get here, you have to do 643289765 creations and nearly as many deletions, so are we going to see it in a non-benchmark situation? The maple tree is more resilient against this kind of shenanigan, but we're not there in terms of supporting the kind of allocation you want. For this kind of allocation pattern, you'd get all 7 pointers in a single 256-byte node.