From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7A10C77B7A for ; Mon, 17 Apr 2023 22:08:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F06C08E0002; Mon, 17 Apr 2023 18:08:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EB61B8E0001; Mon, 17 Apr 2023 18:08:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D7E5F8E0002; Mon, 17 Apr 2023 18:08:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id CB6A98E0001 for ; Mon, 17 Apr 2023 18:08:50 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 75FFD405C5 for ; Mon, 17 Apr 2023 22:08:50 +0000 (UTC) X-FDA: 80692273620.06.242706F Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf20.hostedemail.com (Postfix) with ESMTP id B5E7A1C0003 for ; Mon, 17 Apr 2023 22:08:48 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=oeViYi4R; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none); spf=none (imf20.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681769328; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Cl/KOggIUYwh125y+9YwbpbwhMMF2Br/gvzyehMp5l4=; b=xBu2BPw+DbJfzGYVy40ox7q5QZ6mZHxPA0eE14wI3wjfBdDtu2gs3KVOpNh7Jer6vjrWrq lg6lCWK+3NCJbg4a9ZrvzaVgg4KROJBgMVdCUZcId6GT08ncO3ycIUSDghZG0pwmk1dy5R L3NYe3hQQvr4zXsFSJuLhojD3HjoTYY= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=oeViYi4R; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none); spf=none (imf20.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681769328; a=rsa-sha256; cv=none; b=0yUU8XpI2GZi4YOX/Q2ybKDMom/vBk2C+XBSAJH17wmSlEPKL98Up3oqSz+IXBqR3oilnZ myDn0toJ66Q+lqcRduERuodoUtn8Zi+dqyH/kCo5eUDzGdN3uthTS9qlM29o01mxwXbevW 4O5ZJrMj+sug7S9qPz05zGekQcSxnvg= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:In-Reply-To: Content-Transfer-Encoding:Content-Type:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description; bh=Cl/KOggIUYwh125y+9YwbpbwhMMF2Br/gvzyehMp5l4=; b=oeViYi4RB8m0SpCud7goSGJLhY p9mePiQHwRTyX2C9LAmhxB6gnq7TJJIDRzwaAKAeDCt8WsuoaEVbjrwb1G3OIFH9HsUtWwnTCEVi8 3xHbCv2+vpPhAahrADkj0pPHkhPXmTvr/uf2Cof/fVZAMBSe/Fe8/FyHnxKQGuZzy081Yan/kdI2W wAN2iF6yC2b9JekfZ8nftG5JzEJisniUBvCNvaD7cKVxg8ctSATi682rFhbTjAdRWelfJYesO1AaN HEz9Gwcrgw1WyOaMMIDblI+04xI67UexrokU28LEWBtVKVzn3yLnErnYVGk8BXNI6dXZGBqxNmNr9 a3K6ijWA==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.96 #2 (Red Hat Linux)) id 1poX1W-000Emt-2k; Mon, 17 Apr 2023 22:08:34 +0000 Date: Mon, 17 Apr 2023 15:08:34 -0700 From: Luis Chamberlain To: "Edgecombe, Rick P" Cc: "keescook@chromium.org" , "hch@infradead.org" , "prarit@redhat.com" , "rppt@kernel.org" , "catalin.marinas@arm.com" , "Torvalds, Linus" , "willy@infradead.org" , "song@kernel.org" , "patches@lists.linux.dev" , "pmladek@suse.com" , "david@redhat.com" , "colin.i.king@gmail.com" , "linux-kernel@vger.kernel.org" , "dave.hansen@linux.intel.com" , "jim.cromie@gmail.com" , "vbabka@suse.cz" , "christophe.leroy@csgroup.eu" , "linux-mm@kvack.org" , "tglx@linutronix.de" , "jbaron@akamai.com" , "peterz@infradead.org" , "linux-modules@vger.kernel.org" , "gregkh@linuxfoundation.org" , "petr.pavlu@suse.com" , "rafael@kernel.org" , "Hocko, Michal" , "dave@stgolabs.net" Subject: Re: [RFC 2/2] kread: avoid duplicates Message-ID: References: <20230414052840.1994456-1-mcgrof@kernel.org> <20230414052840.1994456-3-mcgrof@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: B5E7A1C0003 X-Stat-Signature: suomor1mez4kpkzad9dpemuayq5gqrry X-HE-Tag: 1681769328-356496 X-HE-Meta: U2FsdGVkX19QAQ65eTpKkMHE0x+V+ANHX1hWVInNvusv4gbGZok7yypJeV//tiRs1YB0ULyeTWzjU8bu/OGuYFgmgUg0KfjaWM5jxHv5dbCo7+JiQdNfiMMwLgbBJTY1bwpSFZyv7EAi22QJB/TTN2zGVKhGuhk5kBWqB9xzzMoY0wZxFgYvo4EdJYaieXIqgRm/7KgCemUFrdPzijUqEGXQjueA4QMmaw74UXjoEDSdJAj82Hjj3erdSSfGc24ao2boLbEPQt2aKTQ7kDfiCxvxlcM7wXrz9o8sdfrP8wfi63XTSPhZ6Nf+iG9OwurNbEfT3rhMXznvlnNdbOwFQy7yj8nV+9p+YywA+cXw0LyP5RfgjaqlQYsBUmtphTHCn70HXhQ+lG3flzTloseXf2erC2TU+GwB0hET4VSZwjGI8Qin522yvUEif5oda/EXrDSkpDxji4rhaW/QDmru0etYCNOvw7KIxxrs8nAZ2sFTLxFVouaHp5QZQGsaKCL2+dhN0YZqm4dNrWtZ9OB5XmwSZ1mxWMl9w4n5Zb/bHAm6OmD85rU8+ICTV7VQFwSuve0u6PWvPf9/GZxzmd8wmj1CIGLxoJnrvxE2zFHvp39y+PhhjtRA7EK3z/k9NRas2YTuQUB2iVjcGy7hB+0Almfgdo2h9zUS4rcWROJlRqL439ho6xNzY1YYHGzGePZHeQUirQQWbehxKPMGmE3M+6g3/sKnJbyltMdbyCeeakA/W+BqBKEgwAfARrW/r/HAObmwV2etf1mFLwhQEXnzIA2CCf0gpwPSPYHZstkJ6VEnQNXcB7u3jARkvUTfXFi1Xz90sQa+81Sww70XDXH32yjfv6fJAGRKf4P+Y3VZM2rW60hq6s6LBYYXFC1hFSJVJfTRy+azBrmLwFgwTNMX6AvM4mPtqId7MW8gqaxBRQptWfyY1tyZEYWibx7M/HQKpbxry26rQV429AkyugO Y/JDkICk dXbsg8g08ttRU8LULZGxDILrXhowCyC92KurNJqS4U4IBJcWd7IGZ8V3lF//DaJxSaEPNfRdJASP5o8Rvbsv3tlg3MmPnVBU76YY3Q4SxM+rCeS50y8N0tzyUNBeTGuGALv//p/m74w0S1ZyoipzWx03gSU3X1JHhdlIoayyyzRNxPiltFqKU+51skpFAsULOiSzMzcJM7dGFm8jqSE2NAkPd6CuN1ZAfX9hqu+RK89eLke7rqhvA95K9Nh3HhCfrO1ol+4FBYRw5DvqKixH7em2NaOmahySpr/bHV0P2iOrVFBbWj9gJHHmc4Q0Th32KB3IvKnNyNqAPCl/t9sE/7OSSc3DHe2icr2LxkorSuS7wcrOmaU6+nRH2wWAP094Ay9WmGiXDjOp6yQVxAEIsMUc1vN3wwvUjZ03o6MfUjoIA5a0iflBBtMsNTQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Apr 17, 2023 at 05:33:49PM +0000, Edgecombe, Rick P wrote: > On Sat, 2023-04-15 at 23:41 -0700, Luis Chamberlain wrote: > > On Sat, Apr 15, 2023 at 11:04:12PM -0700, Christoph Hellwig wrote: > > > On Thu, Apr 13, 2023 at 10:28:40PM -0700, Luis Chamberlain wrote: > > > > With this we run into 0 wasted virtual memory bytes. > > > > > > Avoid what duplicates? > > > > David Hildenbrand had reported that with over 400 CPUs vmap space > > runs out and it seems it was related to module loading. I took a > > look and confirmed it. Module loading ends up requiring in the > > worst case 3 vmalloc allocations, so typically at least twice > > the size of the module size and in the worst case just add > > the decompressed module size: > > > > a) initial kernel_read*() call > > b) optional module decompression > > c) the actual module data copy we will keep > > > > Duplicate module requests that come from userspace end up being > > thrown > > in the trash bin, as only one module will be allocated.  Although > > there > > are checks for a module prior to requesting a module udev still > > doesn't > > do the best of a job to avoid that and so we end up with tons of > > duplicate module requests. We're talking about gigabytes of vmalloc > > bytes just lost because of this for large systems and megabytes for > > average systems. So for example with just 255 CPUs we can loose about > > 13.58 GiB, and for 8 CPUs about 226.53 MiB. > > > > I have patches to curtail 1/2 of that space by doing a check in > > kernel > > before we do the allocation in c) if the module is already present. > > For > > a) it is harder because userspace just passes a file descriptor. But > > since we can get the file path without the vmalloc this RFC suggest > > maybe we can add a new kernel_read*() for module loading where it > > makes > > sense to have only one read happen at a time. > > I'm wondering how difficult it would be to just try to remove the > vmallocs in (a) and (b) and operate on a list of pages. Yes I think it's worth long term to do that, if possible with seq reads. Luis