From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 928CEC3064D for ; Thu, 27 Jun 2024 19:46:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2AD7D6B00A5; Thu, 27 Jun 2024 15:46:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 25CC16B00A6; Thu, 27 Jun 2024 15:46:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 125236B00A7; Thu, 27 Jun 2024 15:46:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id E88FD6B00A5 for ; Thu, 27 Jun 2024 15:46:47 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A7F2AA41F0 for ; Thu, 27 Jun 2024 19:46:47 +0000 (UTC) X-FDA: 82277701254.10.4DC139B Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf28.hostedemail.com (Postfix) with ESMTP id D9315C0005 for ; Thu, 27 Jun 2024 19:46:45 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=KHE8SNVF; spf=pass (imf28.hostedemail.com: domain of kees@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=kees@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719517588; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1EwtUbctZX1+t1W46CJFwWp+QFZRUMZq2IX+j15j3Ik=; b=vyLLEv14HHg4HREjNaXsSI3nvR14CAncwv4jIGs8v+K47VxjtBU3juw3KaFNrxSYtELm4y GeXS9wlvR/zi7EilzjTp2sDylu4n5xSyY9tVo7pKEchc89y3OtNEeMtQY/7uOCBQeZhSKS dywG8BDt1ycBC+//y4hfqK+h8ktWZI8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719517588; a=rsa-sha256; cv=none; b=GNH6ZL2GykAIVjLgtusflfM4fMJxS/Tjf7L/xmupdm+ZWqap/YztnhSNiL2IbRU4EKIsjU 2zTHr50s0KdvS+eL445Y4nQbEPO88JGfC1ulVsuaRehfsBrL4KKqmbRCdF5Sve7a9NeztI UB4xoNagWij5RN9eSxJt//CzIzDom2w= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=KHE8SNVF; spf=pass (imf28.hostedemail.com: domain of kees@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=kees@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id AA71A61F69; Thu, 27 Jun 2024 19:46:44 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 46939C2BBFC; Thu, 27 Jun 2024 19:46:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719517604; bh=IfSpmibmopaiEUnoS5Sj1W0By6K3R/JpHzodLmPYEk4=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=KHE8SNVFwNlZ407ZHXT9e3Im2lLVpRzsiEnU3MOqwdtEA+DtbS832M2iVxCgWtqEG bTE4nRyJjyaIz+mD2pRjWPnjYrITHDbeZGsJumRxEtAhczW3atejlwUPLacmMy2rhl BGeqGJ7c9GiNkx606wRbnGOTZL0cH42vZ3A0GVWsDg/kdieefLBdZudd5xYUuCF3VZ d2ktNgCStqYUzBiWFRpVYq73jof7qZz4/OaTvHdFF7ZbJAumg4nKUNv7HJIoyrdu4X 2/Qr9sI4MOvljIuvmHJKfFb0WFmkDTW0eMdBcd3gYe9reXBN4von9LmKACSwlZLKjK Vfs9OA/4Ya8nQ== Date: Thu, 27 Jun 2024 12:46:43 -0700 From: Kees Cook To: Lorenzo Stoakes Cc: "Liam R. Howlett" , Andrew Morton , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Vlastimil Babka , Matthew Wilcox , Alexander Viro , Christian Brauner , Jan Kara , Eric Biederman , Suren Baghdasaryan Subject: Re: [RFC PATCH 7/7] tools: add skeleton code for userland testing of VMA logic Message-ID: <202406271240.80ED0EF358@keescook> References: <22777632a0ed9d2dadbc8d7f0689d65281af0f50.1719481836.git.lstoakes@gmail.com> <202406270957.C0E5E8057@keescook> <5zuowniex4sxy6l7erbsg5fiirf4d4f5fbpz2upay2igiwa2xk@vuezoh2wbqf4> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Stat-Signature: tw9jw7c8mnxokpeqwnrjogwp8xzayemj X-Rspamd-Queue-Id: D9315C0005 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1719517605-161873 X-HE-Meta: U2FsdGVkX187O4FkLKKSr/glyJUMxbashOj7WmOdVQqWm6UBxLb71JWd8he7Y+aoJdKH2uY90WaHxvj7IfbKznqO/dU3rQmfrtwTFK3t66tiu/UhysIrevyN1kOjltt7Tg8VakyYpH9cHFxMcyvtD3h+0VKG5yrMZH6ZAdnVD2PJ5zMz9CAaXIQspw3kB6geyHbxQGdhIjpCjHwMokVzF2471rcw35Xe1KxWg1oAiaFwyXCvemvCFjSSP/lTscPj7LepFWvD19WQNakCMF+TpXnL1BR1Q3xAtcPKUL5ixrI1xk3eiwQBY3zjRLByRXinfEw5+KIDxkp7aRaBs9uIBzgwJ46Gczg/1TKF8Eld09aHcJZ8AjbOvh0QfHbC8eWsB8KQYbIEtAIqLgtK9MEZ2llRq7XG9Dc210vQka/ncpL24i+Ac3pFGuc6zLIYN4tk55rpf84qwXbvhOYhU+EsthXnnanws64OPno4YWr0/4fDUdiQpnQLz4AbzeLeCmDNIRJ1y86vqauiO8kf/rYj1iaNAJNDws/zxHYYMInc393RQTF/y0O1FKJUBICGRkfSSbrbKsY7ZUd113vn++qr+ZZQidYjatPwqzHUwLLJP9pNk8ONZ4H9Vkku8FpNjzqLxGSue1s6nws7qqG77Fdwh8FjvtNIjiNxuajr/dr+TDeDd+xVMnnik0LNPNF2iqT2r2iE6tv4ZmqgMg2LrwNX/4rDEeVd5ZRHuTx6gxQnOFZnEW5GJ/WyYx20DYgd1yXq1txH3vV13lxlARB71xWGu+04920MWxjDferz2wO6/t4oc7u4uIIs7rf8g2hID2vjJspABQKzxng/qR5uGrZojL+NnF9BIhSJ9Az6qhQbNTkaRxBD1hUXkBWcQ9KSXSXzqrfQd+SFt280zIOm0CnEmMOQgJX44yZTgNh/+E1RhkGHyX2dwkZMoakWygeRiXXH3LvY/pAqzGZYRA8VSrj r9fIm3Pd wS2PFcDa+WA5IP1wcBKIJM1n0zym5Joc9yXT8rF/sXeB+buhUzJ/IbFUl6pDFn5fJQQSv4v8VgDLcDIza0Q8P1173Oyc3uZG+sPphfxXfGm3YV3C4u5voxWJwoKGoYLqD4FyyQT22F8Vm4dsRjgFTw+bqaoJlQTBc4kWf2R8n79ehPF39qmraP4oFPpc1Uz6DNxO7zKo1I6d5QtXGabANZv/EvgpU+ozk7TYeFYKyDNmE7dr1CCWSWIur1q17CpAR8JnMiaUqr3lj0rZ0SI/yzZ/PPg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jun 27, 2024 at 08:31:07PM +0100, Lorenzo Stoakes wrote: > On Thu, Jun 27, 2024 at 02:25:36PM -0400, Liam R. Howlett wrote: > > * Kees Cook [240627 12:58]: > > > On Thu, Jun 27, 2024 at 11:39:32AM +0100, Lorenzo Stoakes wrote: > > > > Establish a new userland VMA unit testing implementation under > > > > tools/testing which utilises existing logic providing maple tree support in > > > > userland utilising the now-shared code previously exclusive to radix tree > > > > testing. > > > > > > > > This provides fundamental VMA operations whose API is defined in mm/vma.h, > > > > while stubbing out superfluous functionality. > > > > > > > > This exists as a proof-of-concept, with the test implementation functional > > > > and sufficient to allow userland compilation of vma.c, but containing only > > > > cursory tests to demonstrate basic functionality. > > > > > > Interesting! Why do you want to have this in userspace instead of just > > > wiring up what you have here to KUnit so testing can be performed by > > > existing CI systems that are running all the KUnit tests? > > > > The primary reason we did the maple tree testing in userspace was for > > speed of testing. We don't need to build the kernel, but a subset of > > APIs. Debugging problems is also much quicker since we can instrument > > and rebuild, iterate down faster. Tracing every call to the maple tree > > on boot alone is massive. > > > > It is also difficult to verify the vma correctness without exposing APIs > > we don't want exported (or, I guess, parse proc files..). On my side, I > > have a module for testing the overall interface while I have more tests > > on the userspace side that poke and prod on internal states, and > > userspace rcu testing is possible. I expect the same issues on the vma > > side. > > > > Adding tests can also be made very efficient with tracepoints dumping > > something to add to an array, for example. > > > > Finally, you have ultimate control on what other functions return (or > > do) - so you can fail allocations to test error paths, for example. Or > > set the external function to fail after N allocations. This comes in > > handy when a syzbot reports a failed allocation at line X caused a > > crash. > > > > This has worked out phenomenally on the maple tree side. I've been able > > to record boot failures and import them, syzbot tests, and fuzzer tests. > > The result is a huge list of tests that allowed me to rewrite my node > > replacement algorithm and have it just work, once it passed the > > collected tests. > > > > I haven't used kunit as much as I have userspace testing, so I cannot > > say if all of these points are not possible, but I didn't see a way to > > test races like I do with rcu in userspace. > > > > Thanks, > > Liam > > Liam's response is excellent, and obviously I agree > wholeheartedly. Additionally, I'm not really experienced with kunit, but > surely it's implemented as a kernel module somehow? If so, these interfaces > are largely not exported so it wouldn't be functional as a unit test. (KUnit has a method for doing optional exports.) > And also as Liam says, it'd be very difficult to test this stuff _in_ the > kernel without unwanted side-effects triggering and it'd be very difficult > to isolate or mock components we don't want to play a role (for instance - > rlimits that we might not be able to control). Yeah, this all makes good sense. Thanks for the details! I've been looking at ways to test signals and task tree helpers[1] and avoiding side-effects in the kernel has been quite difficult. I might have to study your approach here and see if the same could be done for my cases. :) -Kees [1] https://lore.kernel.org/lkml/20230814210508.never.871-kees@kernel.org/ -- Kees Cook