From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06F66C77B72 for ; Sun, 16 Apr 2023 06:41:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2061C6B0074; Sun, 16 Apr 2023 02:41:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1B5986B0075; Sun, 16 Apr 2023 02:41:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 07E336B0078; Sun, 16 Apr 2023 02:41:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id E996D6B0074 for ; Sun, 16 Apr 2023 02:41:41 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A619CAAFE8 for ; Sun, 16 Apr 2023 06:41:41 +0000 (UTC) X-FDA: 80686308402.29.BE17831 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf15.hostedemail.com (Postfix) with ESMTP id 79FB2A000F for ; Sun, 16 Apr 2023 06:41:39 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=uI2vWvZi; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none); spf=none (imf15.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681627300; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=feNbonEkJBZpWgMwe8jt6vuLvISXrUj/SEXS0kUAnlc=; b=cWjV3F9E6HJXg9lGVGUXMFiWMr6tTaJ0HBWm/EHNXptau9LC8orXyp+NxRC/PKIQAl6wWg /VBdUtaVCl0DHChkcYHXQ1bMzdXOwzpJ20dA8mxLyxXXT50qQgp2thFnerI+QTwq82OBH6 mH1tsg3j9qv6hSCI9AkanVWRvIuXkS0= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=uI2vWvZi; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none); spf=none (imf15.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681627300; a=rsa-sha256; cv=none; b=2oXsd9sJWE8o/LUegrKNKnM/OSLMW5meD3Yko09i0HYtRhFCUU70OfuKELdWqQnV9FEPyM NGFQ5y4OlKcJnPmLO5EgpmSNih+F3J/H/s5kfSqXDgAWWLHZT34sscyYRnSjiRvHHirTIg B+oP+hRLe3RvLEHKhTOBAbJmnxMfxaQ= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=feNbonEkJBZpWgMwe8jt6vuLvISXrUj/SEXS0kUAnlc=; b=uI2vWvZiRzbowyrTuwWuv3DGFt wJPQJ+LEhJc1KTs+6wAShf2paCygyaRrvABmu2+l1jqQNg3PhICDLlhr50BtqIoL9D5mnXRCDCmVh dUzkbSV0gkiuxdN0MWKYXzIiqiJXk6at5GkEiw/0CeJ8IFzH9R92b9QO8L3YR6OJAU0sTwO1mMjpP KoF2uPLBtgHoqzG/vDxlgkUqzer3SIp30WRSrnF3AgP2b05g7mzOZrm3GEtkFGHhDLQ2DTi5GmUXP amW2thIB4qoG7Rr/csp8axVft5ZI+cm1c2Ix6nBhjiM2fdNh3Sfwk+izpgnFjejiMctGJRn/d6+TV wKqo2Hsg==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.96 #2 (Red Hat Linux)) id 1pnw4m-00DFIg-2j; Sun, 16 Apr 2023 06:41:28 +0000 Date: Sat, 15 Apr 2023 23:41:28 -0700 From: Luis Chamberlain To: Christoph Hellwig , Kees Cook Cc: david@redhat.com, patches@lists.linux.dev, linux-modules@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, pmladek@suse.com, petr.pavlu@suse.com, prarit@redhat.com, torvalds@linux-foundation.org, gregkh@linuxfoundation.org, rafael@kernel.org, christophe.leroy@csgroup.eu, tglx@linutronix.de, peterz@infradead.org, song@kernel.org, rppt@kernel.org, dave@stgolabs.net, willy@infradead.org, vbabka@suse.cz, mhocko@suse.com, dave.hansen@linux.intel.com, colin.i.king@gmail.com, jim.cromie@gmail.com, catalin.marinas@arm.com, jbaron@akamai.com, rick.p.edgecombe@intel.com Subject: Re: [RFC 2/2] kread: avoid duplicates Message-ID: References: <20230414052840.1994456-1-mcgrof@kernel.org> <20230414052840.1994456-3-mcgrof@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 79FB2A000F X-Stat-Signature: 89e78waefwgg3xi58wpyt37si4gzq9fh X-HE-Tag: 1681627299-137613 X-HE-Meta: U2FsdGVkX19qJMvHShojt3D+dU4eivbQ7+1BWYIZUjhMbVj48XGzNW3nCrAj9DAB2Wz0qU3x1VGAzAu3w7dnPkx2mCzkB+P0p36b84GphNMTolXfuT/ij1pw+o7YaOgFQbmUZyzHFi+f7aWGyErhIoZDsvMKfMJrKxXsGAUfBJPwrIBvkfjYVkVWNPWRb58iHXouXx+WKCtThjuOwu94yaUfDOWdwaDL61qZ4XYp3Rf/VKLdGjyJmcxlaltnOAUy1obB0oYwIY2ffxwhfbQKGAZd0H/cKt4eViiN9Y8zxzk/aQq2GsrPAb78wc9XrfLo7NvzacujecgNMHo6kh+3LQZZJqD9qM82NN0y6OGVjhOliaTLp+HTVU014FBvM9S+aWSHEpH9CjxHib9FtYqfJfn41UjQeUYYrXfiMFT68D4o2d35t3NcXNaCAI3uHi95hLddYTnjdsfo17BgNU90W6LGCZBmwbd2/mppmyFlMk8HRyoODzQpPOwqpw+OqF/L7wE22EZDxiD14vrn4byBw4GQ3DMzVfI8ZXWTRLB3GieFnCEys/HKmJt8z9FWhXDChb8lzt/tkks8SwXghzOPKUKgTb3yVx/Bt5z1+ltJAusOJrLePy4UjOMDxIgtGFQ6ytfS0ROjNeR1GV6T3DaCbm7SA25+kzZgrQdhxUC84v9fODnRnR1cLVo4BtYUPFe8BBgS+scoaqUcSegyfvVf9KEZXrDtapsBNQEmkP9g1/FR+R+54mPttSUnRwCgfj6oxqR1pKGCa18BtSOTrRx62m1KuFBNoFFYmD+Lngn81PSnboiCLVXVAs4y3KDiUcBZwtVPEF8HHWuD7EWcM9YaaaLeqYJ1AuYzX0sN7ziBkZ/5TkITZ9ttAnzsKKrDhacHVNJgH/xtipNCtStExcEE8sIsH4Nv9/pFThI2zaABiAkiGXEY6GYV7LVgpvr+Sj3ztdaCVebv8XBlSKLkzXc RblF2Rhk RW6fcYJQ7ARqEMfWXHRCq6OxgeCqr2Z0vHbs+9DbKDrtdl0Evtb33Y9ibK8pdbTAGMhBeWH0WIqb6ljOisSAGUOf3u2j71Qw7TPGlwWA4UskeTWFE75dDjh3xtOs8cDQKNTKU2vYd1fgmCWzeCQC8GsskMUXb24SlNiVP3OJC6KvBWdYVBWS+EBUdnqejP8phB6slEruGlwwwP19dLD+l3F/i/8Z1qG6aaKFHu7ex4MBKpmYD27jZECELFb12jYoO7gC/Ja2gjhz5mteSNAZs34jjBB4LsHfwFNVqcT+KyZJT9XDYMGOVPmeCwDkQfD4VHqxqqGjzUJAPca11OIeZLVnTQJNib9rB1RiyD2p1AUXUgCOACrh8WP92yIWSpRROr5MYhSmi2N1pA0Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sat, Apr 15, 2023 at 11:04:12PM -0700, Christoph Hellwig wrote: > On Thu, Apr 13, 2023 at 10:28:40PM -0700, Luis Chamberlain wrote: > > With this we run into 0 wasted virtual memory bytes. > > Avoid what duplicates? David Hildenbrand had reported that with over 400 CPUs vmap space runs out and it seems it was related to module loading. I took a look and confirmed it. Module loading ends up requiring in the worst case 3 vmalloc allocations, so typically at least twice the size of the module size and in the worst case just add the decompressed module size: a) initial kernel_read*() call b) optional module decompression c) the actual module data copy we will keep Duplicate module requests that come from userspace end up being thrown in the trash bin, as only one module will be allocated. Although there are checks for a module prior to requesting a module udev still doesn't do the best of a job to avoid that and so we end up with tons of duplicate module requests. We're talking about gigabytes of vmalloc bytes just lost because of this for large systems and megabytes for average systems. So for example with just 255 CPUs we can loose about 13.58 GiB, and for 8 CPUs about 226.53 MiB. I have patches to curtail 1/2 of that space by doing a check in kernel before we do the allocation in c) if the module is already present. For a) it is harder because userspace just passes a file descriptor. But since we can get the file path without the vmalloc this RFC suggest maybe we can add a new kernel_read*() for module loading where it makes sense to have only one read happen at a time. Luis