From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB8FCC77B7E for ; Thu, 25 May 2023 18:50:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5A71B6B0078; Thu, 25 May 2023 14:50:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 557C0900004; Thu, 25 May 2023 14:50:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 44590900002; Thu, 25 May 2023 14:50:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 33D686B0078 for ; Thu, 25 May 2023 14:50:44 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 14BD3160C80 for ; Thu, 25 May 2023 18:50:44 +0000 (UTC) X-FDA: 80829668808.08.A503294 Received: from mail-ed1-f54.google.com (mail-ed1-f54.google.com [209.85.208.54]) by imf08.hostedemail.com (Postfix) with ESMTP id D0246160021 for ; Thu, 25 May 2023 18:50:41 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=google header.b=baNIGmCo; dmarc=none; spf=pass (imf08.hostedemail.com: domain of torvalds@linuxfoundation.org designates 209.85.208.54 as permitted sender) smtp.mailfrom=torvalds@linuxfoundation.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685040642; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aIsIFxmLlj97Nfx6D9vKSgsX1a6jDH3TwyPUkCdOSPQ=; b=AyONAIHCOHAO58N9fN8IXN4S98fipKWJ9d0+aLZ8UDZ1YarVib57nuU0/DzfdcbfOU/ufm zx0CfbnR/7JVhBaHcF4BSXofwtPpVOiqdvJsHYkjdTbduZl+F+KVboFJsWTRDAa2wY96/R sX6mJA0F+l5MXGEYTd3o/0tKW+l8gdI= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=google header.b=baNIGmCo; dmarc=none; spf=pass (imf08.hostedemail.com: domain of torvalds@linuxfoundation.org designates 209.85.208.54 as permitted sender) smtp.mailfrom=torvalds@linuxfoundation.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685040642; a=rsa-sha256; cv=none; b=NMUn7iiMtFHcMeM1u3go5UjlMSC1fkAc46hf14yXqGL5gRltNjNNjZhNqBcIwmQonTgDbZ mEBDKLB4PTf5W/fzqF0VlNCSBw2xEcqSVcXLX3prZnByjXd4vBC51BYdaVVfV11GLC3rQu rccroyfQxnrvafcwqKuDsuq96uSo+/o= Received: by mail-ed1-f54.google.com with SMTP id 4fb4d7f45d1cf-50bcb00a4c2so4406884a12.1 for ; Thu, 25 May 2023 11:50:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=google; t=1685040640; x=1687632640; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=aIsIFxmLlj97Nfx6D9vKSgsX1a6jDH3TwyPUkCdOSPQ=; b=baNIGmCoH4FutF1V6uTprsJt6A5it1rxKqCQZYOX/MldZ8tPo1WiL+FHKf7x1y6KD5 Vs1Dfz//YqrNgktlcqNunhd6YiVE8S7JiOIo6o+BMgmkuRpQMVObzRwAluvanaA9uJSX XkLdX9dR+WDIJrC5DqQ+t7zhQT7bx9TFjtATA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685040640; x=1687632640; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aIsIFxmLlj97Nfx6D9vKSgsX1a6jDH3TwyPUkCdOSPQ=; b=beqSIKmrzuWHAIH88YHHNS9g2Be8qxsbgqsRfsYMji1GbeafW7NcLjuq/Zs9uTs6ve OVeBrKHNBRNNyJsdEzebTbhzvDDDI4PzqGAPysTnmbLcPjr0xtaEXQUXe3JYmWhs8i7E oLfxwCA1UERz2hMxOjIWzENb7EMgPAMjj/nTlZaB/UHtPdIsr8lJ8kndLcyDw24SwU16 +ZGBTUctAs2nvR2WPKFhyWTotBRbeBP/up/b2nqzWAHMPae4TKeLC63JOmfn70VXurIA rhzISO2teT3NBJId5Co/5VykSt/itXINklTyUVfDyktQm0A/1ZXxu+7FT7zqZUloUXau NNlw== X-Gm-Message-State: AC+VfDycjSSiIQ5Sefww5wLnObmPfJ9ijZ0Ht3F1PxeFJBBTw6S9BON5 C10G2d0gDn49PAO0jgIon7Ut7x5fjihgS6d64f6vi32w X-Google-Smtp-Source: ACHHUZ6yzMnakHQceA+0sYATImmjWzmSsOsPYEitzpyrS3shFtlUAhYsBeYzhXg/5XTGYSIayk/PsQ== X-Received: by 2002:a17:907:785:b0:966:265d:edc9 with SMTP id xd5-20020a170907078500b00966265dedc9mr2331739ejb.74.1685040640089; Thu, 25 May 2023 11:50:40 -0700 (PDT) Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com. [209.85.208.49]) by smtp.gmail.com with ESMTPSA id gj23-20020a170906e11700b009600ce4fb53sm1168515ejb.37.2023.05.25.11.50.39 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 25 May 2023 11:50:39 -0700 (PDT) Received: by mail-ed1-f49.google.com with SMTP id 4fb4d7f45d1cf-50db91640d3so4443317a12.0 for ; Thu, 25 May 2023 11:50:39 -0700 (PDT) X-Received: by 2002:a17:907:6d9e:b0:96f:4ee4:10d4 with SMTP id sb30-20020a1709076d9e00b0096f4ee410d4mr2465428ejc.43.1685040638863; Thu, 25 May 2023 11:50:38 -0700 (PDT) MIME-Version: 1.0 References: <20230524213620.3509138-1-mcgrof@kernel.org> <20230524213620.3509138-2-mcgrof@kernel.org> In-Reply-To: From: Linus Torvalds Date: Thu, 25 May 2023 11:50:21 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH 1/2] fs/kernel_read_file: add support for duplicate detection To: Luis Chamberlain Cc: Linux FS Devel , hch@lst.de, brauner@kernel.org, david@redhat.com, tglx@linutronix.de, patches@lists.linux.dev, linux-modules@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, pmladek@suse.com, petr.pavlu@suse.com, prarit@redhat.com, lennart@poettering.net, gregkh@linuxfoundation.org, rafael@kernel.org, song@kernel.org, lucas.de.marchi@gmail.com, lucas.demarchi@intel.com, christophe.leroy@csgroup.eu, peterz@infradead.org, rppt@kernel.org, dave@stgolabs.net, willy@infradead.org, vbabka@suse.cz, mhocko@suse.com, dave.hansen@linux.intel.com, colin.i.king@gmail.com, jim.cromie@gmail.com, catalin.marinas@arm.com, jbaron@akamai.com, rick.p.edgecombe@intel.com, yujie.liu@intel.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: D0246160021 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: t6gy5ujfubuxup5hezbcfyrquoyxu31o X-HE-Tag: 1685040641-74359 X-HE-Meta: U2FsdGVkX18inmOMARZ8cGXrQ43z2GXpWDUvBPRRh03fSsdaszFw2i2fgI0E7fyYOq9rO9L7MW2Q/KUqlfDmnAmrG7daH5O5YKlxQnfSEs7wMfkQ1yY83DPrl2HjXLaDMKdYRaDNspNpgZnW6khVT3HA5rKf358QU02znpm8pZ7CzF9nxBFsbeM8hNmzqitzjL0KdQhW4thR08KSB4TGOj1XGXQG/uR/kczcsHg8NO6rKDjgRIppq66n/QcbUNUrkTY4rqXDcIwI5rqMr9BKHfCxrG16RTNoUkhaAMMSaLSQLn+un9tMcITRrekky0IYx44S9484g2IIuCXxJYnGSCzwkpTOSw4aTSWIbApYTk1u+62Ta3vQS/20M96z8GfkSbU3oOB+zeuSMIioYGCQrYPd3y1T+ibuWY6+2uDtE/Z/7Zjhr0nWI2CUWZn+zz2ZjyE1tPA8pVxdOTK+i3UsWsiP04kh4dsM8BkB+NZ1jgvJ+FdXaK4n4ZVLuoCkwUuEnhvZi9Cymh9j+Jjdi/8GukJXG7h9L74WvYjIB2VA7u/u1EvNwPqvvjxBBEG3d5RGUJHQjQS2+dk/qzZOslPT0xn8QsYZ+rRlfltggMvn7p51Ci8G6jB5VepG1xpBJ93V0bHebkYxhLk8mG8lULROMjCZwgvg1xyLISVeDgsRFH2ZNy1jr+CjxiqJXa8qoq/7GUkF6Mp/Fx5OFj0VM04MfLk1S4AcI/AokVNwZ75UpNwk/7vmDI3eaNXteozooWbup3F5sObmxd/pHSMZ9kxbrMSbsmaUKkFCjHgVWk+IA+J2AK0euHESbOZGRv9KE9h9oiJshvouGoB+aeRHRGYvWaTg53CBeWJ5KJxZR1DXbAbAN40Znio6LLQVXUjYAgfCCbMisb5QiXi1cxV5Bqa57LFIuUgICmMtUKefTZhD6hmDxwDoXCNPa+ye9g+mW06leMTl2npoJNe8NnS2yVx NwMe1TJU Qnw0UXmNJsW7jpuSysZI1tnysLuqfa80wNOrhnBteP9Tng6Q2dSvMgdkkJ38D6QoRrSON0Pg4urM8/sLZrKTufaXGO3oIZd8KEqpnIpEgn9zil1ZIIEBbtF/FRhAeJGnz6pSi/3RuZ/by5dyr2j1XlVxNtC8NzY4sO+27bo6UVGTKkm8r595y/LyyYo+grvI4ylWZyxPIfsIQD8vbMPxgY1PyalzrCUkH3o5VH6NkFHbfdoVVTYQmoUC1d2KMWgmCI6cDjaG4600qOXhtq0u9GAAtZwwlVVGaV/CbxDBcemT/TjEyOMMTHhqvnUJtIYYS5AU1u8pidKRtIiQcc4DRWR8KKXsxVj8B7Vl+TLMuJP+BdyXqfvI1QK/UCQLgd0joFIX5+MyxMlpGaURvkAAqLyqiiqeLtBFFeumSt2Qb4rcqOI5qOYmGFv9jSAByzWU8I9ED6/zVPt/xhhQpJs9qu5gEhPvlgFtOlT2e8qgmN9rKzDotHkA7nQR9CgbbPBhJD74+xPNViVLbPtrD6w91agTsQkWqpqsbEtFnI1B9Gs1BBP97tS1R8NVac4J2YVyKn5Ux3KECQ99d3pBqwBfZPCo4pkCqk++xmadV8ttBwVTIHWi214Q6joCZ+1rsJGjXlBKAVNg1tPyHoY4QbAW9KJfRfC9RrsB5dOoQy9Tr0lFfhUs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, May 25, 2023 at 11:08=E2=80=AFAM Luis Chamberlain wrote: > > Certainly on the track where I wish we could go. Now this goes tested. > On 255 cores: > > Before: > > vagrant@kmod ~ $ sudo systemd-analyze > Startup finished in 41.653s (kernel) + 44.305s (userspace) =3D 1min 25.95= 8s > graphical.target reached after 44.178s in userspace. > > root@kmod ~ # grep "Virtual mem wasted bytes" /sys/kernel/debug/modules/s= tats > Virtual mem wasted bytes 1949006968 > > > ; 1949006968/1024/1024/1024 > ~1.81515418738126754761 > > So ~1.8 GiB... of vmalloc space wasted during boot. > > After: > > systemd-analyze > Startup finished in 24.438s (kernel) + 41.278s (userspace) =3D 1min 5.717= s > graphical.target reached after 41.154s in userspace. > > root@kmod ~ # grep "Virtual mem wasted bytes" /sys/kernel/debug/modules/s= tats > Virtual mem wasted bytes 354413398 > > So still 337.99 MiB of vmalloc space wasted during boot due to > duplicates. Ok. I think this will count as 'good enough for mitigation purposes' > The reason is the exclusive_deny_write_access() must be > kept during the life of the module otherwise as soon as it is done > others can still race to load Yes. The exclusion only applies while the file is actively being read. > So with two other hunks added (2nd and 4th), this now matches parity with > my patch, not suggesting this is right, Yeah, we can't do that, because user space may quite validly want to write the file afterwards. Or, in fact, unload the module and re-load it. So the "exclusion" really needs to be purely temporary. That said, I considered moving the exclusion to module/main.c itself, rather than the reading part. That wouild get rid of the hacky "id =3D=3D READING_MODULE", and put the exclusion in the place that actually wants it. And that would allow us to at least extend that temporary exlusion a bit - we could keep it until the module has actually been loaded and inited. So it would probably improve on those numbers a bit more, but you'd still have the fundamental race where *serial* duplicates end up always wasting CPU effort and temporary vmalloc space. Linus