From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAFCBC48BC1 for ; Wed, 14 Feb 2024 16:38:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 70EB16B009C; Wed, 14 Feb 2024 11:38:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6973E6B009D; Wed, 14 Feb 2024 11:38:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 510A36B009E; Wed, 14 Feb 2024 11:38:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 3DEC56B009C for ; Wed, 14 Feb 2024 11:38:30 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 16621160DEE for ; Wed, 14 Feb 2024 16:38:30 +0000 (UTC) X-FDA: 81790967580.04.8F095C2 Received: from out-178.mta0.migadu.com (out-178.mta0.migadu.com [91.218.175.178]) by imf28.hostedemail.com (Postfix) with ESMTP id 05EF1C0011 for ; Wed, 14 Feb 2024 16:38:27 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=Xm+zNKCM; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf28.hostedemail.com: domain of kent.overstreet@linux.dev designates 91.218.175.178 as permitted sender) smtp.mailfrom=kent.overstreet@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707928708; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GhC8U54i48aAhQsKXgUpTdpqc3Q9oBEWvXQFXanbxaU=; b=uQW8BpF/6SC2EFtSX0oLVPApwgt6kayureLLH/lCL18SnKis5TapXL6QxndH2HwScDwVMR pj11DTkc4+c6LMQ5ickvXsszcXRBJPFKBcSGqobj4aU2+Zs3ujxlN9F8CE3RKtXxA6NWpE KWNqujLZaEVox6GEWOqcFv+8SX1x90E= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=Xm+zNKCM; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf28.hostedemail.com: domain of kent.overstreet@linux.dev designates 91.218.175.178 as permitted sender) smtp.mailfrom=kent.overstreet@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707928708; a=rsa-sha256; cv=none; b=GPonFveb4S1W+D6b9Ub8HvPMNWd/ubLLOqgCHu8ZhViOuFDOnFbLxko6n/9ZVlSJp5yvij BiTVvnQiLnb5iXUxkD+1pESmt6Z3tHZ0B82CFjkbEVAFXOLP/X2P5G2qWBBhc/C+mIrwI2 OtWRVsoJYtjniM5fmc8CL8/setrJFUk= Date: Wed, 14 Feb 2024 11:38:14 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707928705; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=GhC8U54i48aAhQsKXgUpTdpqc3Q9oBEWvXQFXanbxaU=; b=Xm+zNKCMeqV79v7rJ5E5an4MzgIP4PD8/8ugFWMqKQm3NKxkYrYq0OYQxclkHYIL8RZCoP TC2LBaoxPghcPUQzNf8N/tkj2r5IU55ThbSV14NQ/Bs/1muwGdvLMDBwjC0Nd8EVp21WU+ XnEe90fkimXt0ALl4VLMlcoyT6d4gss= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Kent Overstreet To: Vlastimil Babka Cc: Suren Baghdasaryan , David Hildenbrand , Michal Hocko , akpm@linux-foundation.org, hannes@cmpxchg.org, roman.gushchin@linux.dev, mgorman@suse.de, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, corbet@lwn.net, void@manifault.com, peterz@infradead.org, juri.lelli@redhat.com, catalin.marinas@arm.com, will@kernel.org, arnd@arndb.de, tglx@linutronix.de, mingo@redhat.com, dave.hansen@linux.intel.com, x86@kernel.org, peterx@redhat.com, axboe@kernel.dk, mcgrof@kernel.org, masahiroy@kernel.org, nathan@kernel.org, dennis@kernel.org, tj@kernel.org, muchun.song@linux.dev, rppt@kernel.org, paulmck@kernel.org, pasha.tatashin@soleen.com, yosryahmed@google.com, yuzhao@google.com, dhowells@redhat.com, hughd@google.com, andreyknvl@gmail.com, keescook@chromium.org, ndesaulniers@google.com, vvvvvv@google.com, gregkh@linuxfoundation.org, ebiggers@google.com, ytcoode@gmail.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, bristot@redhat.com, vschneid@redhat.com, cl@linux.com, penberg@kernel.org, iamjoonsoo.kim@lge.com, 42.hyeyoo@gmail.com, glider@google.com, elver@google.com, dvyukov@google.com, shakeelb@google.com, songmuchun@bytedance.com, jbaron@akamai.com, rientjes@google.com, minchan@google.com, kaleshsingh@google.com, kernel-team@android.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, kasan-dev@googlegroups.com, cgroups@vger.kernel.org Subject: Re: [PATCH v3 00/35] Memory allocation profiling Message-ID: References: <9e14adec-2842-458d-8a58-af6a2d18d823@redhat.com> <2hphuyx2dnqsj3hnzyifp5yqn2hpgfjuhfu635dzgofr5mst27@4a5dixtcuxyi> <6a0f5d8b-9c67-43f6-b25e-2240171265be@redhat.com> <4bb7b1e4-d107-4708-bb65-ac44d4af9959@suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4bb7b1e4-d107-4708-bb65-ac44d4af9959@suse.cz> X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 05EF1C0011 X-Stat-Signature: fd4mbydiz6paozkh8i4c1gcwd9zka6nz X-HE-Tag: 1707928707-905605 X-HE-Meta: U2FsdGVkX19QLmk1t6IgVwf6u4pSF7O0JX8Rv0EAzvj8hRrJMJ6J511J3n8gh255I0XQtXOWanXvSWvL84ABAC+9aCF5vYCnqT4XnxHjVr4chio1mfLTkicn4Zz5pIs9nke5ume8qlbPKCHosse2D9XKSsRIzAmd/kJksrkkmt6deyoCHMULvtNGeAO6xvQlkdE2m3oaHMA5okYtEGUO8CM0j2UOUbYKhpqScI7EHgiKSNgpj5siB/pJrqrV1AIL/JLKVy6WRjuyg0PSqi84gXj8/SAhJHMrGTVCvhPJ3bToDZAchSYONwgGNZzEis9kf/3HfMttOworxnxHUkz9a0mWApiqtAPf3sTU4OOTwX6d7h1n3QdXTFgpqk7cTS7fySQi6f91p2Ps1RRcrrB9jHR2v4z1ZaOqtHX3sHwryMwZBWeqWL4vJNT+rOu8Wmu555cCNKU93y8hd2fWINB2VEWKxNaCMC4tLO+YkL69ORIgoN0vkRYH+7ASnJVJ/KJyhkR6dbR6661BWzzNaFn9QtqV4i354Sl200GOzZeUBalNX5hToYqeabqWvVyUQhIkV0IoJ8F1LC6Eq728JFTwkeTI3g9m5e9CsjYuIIfvpenawbIvlylMGG8030JrOqGDj18L91cjiFCJeA/zzfiFzrZSTJjsbTToPSrF6xH5SePwbIMTsiX+toqQ5QcahtvJz+dQeMY2meG3ONIgTkLXw7WqFEFahTLJ6nLjlSHqq5H7jyu6zApqV651F2qIpbZvgNLTbnpR6Tcju1zy8xg/cCnvECPOmQkvWdVt0tTKjyn4nY/KyfLujVAZNnQGSDlVr/e5fwnYXANedeMIRoRI5LC02I84gRU+zRMriTf2feAOUmC/+rkKV6tr5wCw1aMepDm7T5bPiu9x+MwqP5/UR/OcJzTSqJnTEnGPByGO91XJ14MIYDeaIDuK13H2zpR7oEdvwYOjk4Skh6ctWaf ADGb0qMx YYEaJL9+ZqaGBBP2e+c4wtQ6sBqa5sTpiMfxM9fSidfo3QeRCyjGAQgJZx0UHPRoOlp1Y2MgTf+62c2yF1Zx5WGkxziPcPv202o2KydRhS8+0+yUvCiCFphzIcw8bDGLRtPGDow6bv/CNDZilfziunmN0yQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Feb 14, 2024 at 11:20:26AM +0100, Vlastimil Babka wrote: > On 2/14/24 00:08, Kent Overstreet wrote: > > And, as I keep saying: that alloc_hooks() macro will also get us _per > > callsite fault injection points_, and we really need that because - if > > you guys have been paying attention to other threads - whenever moving > > more stuff to PF_MEMALLOC_* flags comes up (including adding > > PF_MEMALLOC_NORECLAIM), the issue of small allocations not failing and > > not being testable keeps coming up. > > How exactly do you envision the fault injection to help here? The proposals > are about scoping via a process flag, and the process may then call just > about anything under that scope. So if our tool is per callsite fault > injection points, how do we know which callsites to enable to focus the > fault injection on the particular scope? So the question with fault injection is - how do we integrate it into our existing tests? We need fault injection that we can integrate into our existing tests because that's the only way to get the code coverage we need - writing new tests that cover all the error paths isn't going to happen, and wouldn't work as well anyways. But the trouble with injecting memory allocation failures is that they'll result in errors bubbling up to userspace, and in unpredictable ways. We _definitely_ cannot enable random memory allocation faults for the entire kernel at runttme - or rather we _could_, and that would actually be great to do as a side project; but that's not something we can do in our existing automated tests because the results will be completely unpredictable. If we did that the goal would be to just make sure the kernel doesn't explode - but what we actually want is for our automated pass/fail tests to still pass; we need to constrain what will fail. So we need at a minumum to be able to only enable memory allocation failures for the code we're interested in testing (file/module) - enabling memory allocation failures in some other random subsystem we're not developing or looking at isn't what we want. Beyond that, it's very much subsystem dependent. For bcachefs, my main strategy has been to flip on random (1%) memory allocation failures after the filesystem has mounted. During startup, we do a ton of allocations (I cover those with separate tests), but after startup we should be able to run normally in the precence of allocation failures without ever returning an error to userspace - so that's what I'm trying to test.