From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0CEAC83F22 for ; Wed, 16 Jul 2025 17:52:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4F27D8D0002; Wed, 16 Jul 2025 13:52:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4C9B98D0001; Wed, 16 Jul 2025 13:52:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3DFAA8D0002; Wed, 16 Jul 2025 13:52:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 2EBA78D0001 for ; Wed, 16 Jul 2025 13:52:29 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id AC5F1B9C5C for ; Wed, 16 Jul 2025 17:52:28 +0000 (UTC) X-FDA: 83670872376.27.528CA26 Received: from mail-qv1-f41.google.com (mail-qv1-f41.google.com [209.85.219.41]) by imf06.hostedemail.com (Postfix) with ESMTP id 607B718000D for ; Wed, 16 Jul 2025 17:52:26 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=EhTUxvhm; spf=pass (imf06.hostedemail.com: domain of boqun.feng@gmail.com designates 209.85.219.41 as permitted sender) smtp.mailfrom=boqun.feng@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752688346; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NhZYBS1TYlYTN8bgdtKESmbKm8B642FoPKPSdfLXodE=; b=xG0P5/Occ5kTpibMBd63WqCNLowFnbaddpKCPcG8t5mCaw9YaZZ9q0XmgJrd6XGKzJUarU GfOS/6ZDvmE8efLoHhkqLWCWGDO9iKDKcAfMqZRFtv+Ukk/pXOMSMFx8IxtqC47vyigLc7 NFZv0l6/GqZV4092K3G9AWu4gmrdYIA= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=EhTUxvhm; spf=pass (imf06.hostedemail.com: domain of boqun.feng@gmail.com designates 209.85.219.41 as permitted sender) smtp.mailfrom=boqun.feng@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752688346; a=rsa-sha256; cv=none; b=5ii254dpMZchoOHx5VcgOgnu+2Cbxla4Rj5EA9SltfgvJ7WC2rquE6no0TytlpTi5IN+vb GtU0O3vgXo5PsuNRxjGRV1NqW+FhY8EWDVid77QT6xf55RiJO5whTTb1igDrGV8TOpc+VS vP8hfXVOlVzKpRhAvih3vTpvVKC8b4A= Received: by mail-qv1-f41.google.com with SMTP id 6a1803df08f44-6fd0a91ae98so1476156d6.1 for ; Wed, 16 Jul 2025 10:52:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1752688345; x=1753293145; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:feedback-id:from:to:cc:subject:date :message-id:reply-to; bh=NhZYBS1TYlYTN8bgdtKESmbKm8B642FoPKPSdfLXodE=; b=EhTUxvhm6ROWUsHVxVW1YtXY47gQfTCrFYH4XPvrf17AcmXetrjNS4uDg44L7FIve3 kGiEiAuLAKb/8VquY3J7pbyKJ9FR8eNHIwdKAhqBv/xCTQploRYZTMczyb4SuUugow8U QyKj+B9Ke3ctJScm+zjnBzq11Qmq+fTb4Aagz6OdrYYmdrBfhkB0fcLdhmABr90CUhTZ Cd8UcQXpnUffFW5PD45jpjuC9B1m6k8lxXNLiVZJQeA37EFEuGZ2RaKiAiBnTP/eaRgQ NCWwoLMMrLiB4YrRQOHh/S6zfeNGc7u+ua6JORJOADGqOKs/p2m8DInO95ApyWmz604d bJYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752688345; x=1753293145; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:feedback-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NhZYBS1TYlYTN8bgdtKESmbKm8B642FoPKPSdfLXodE=; b=wUp4RPagFKeSwg2OrXq2ss5BvFES2Arxiuhf9wVTF2p0IijnIw7cdmDFlYg3MIbMIJ VZgirMDcUv4cwSteigDf+TJPfZbO5GLX4SQ7hSmm6nfVtfgTlHHdXjLjrZDp6hlqkZVz aOND7T8dZfpsYRcgbe0Ajy2lVbaGgGegxWFTgpTwd/hfWf0+e2FpVmjzmfVbgHzPd94x T7UpMEsDO+6CLWq9t7FU6RBwrN00hF2roQ/0j7/ViyRhrGTjsoBIS3DnqonrqIksck+/ 7dXYvRinjWyapxaRdJx9x3UvUJeTGcRKJK/KypayncfqCMuWHYjABxkEeF39KNqWcHjF kDUA== X-Forwarded-Encrypted: i=1; AJvYcCU7d2Fip2uNWGAUcbs6+2DqdK+kkHuspzTeFh/LxNMGpqeYt30xxVh4GDTBo86yXv3oGg5EiD5rGQ==@kvack.org X-Gm-Message-State: AOJu0Yw8DFj8dtY0rNtUtlXu3tPeKbfm9EoSMwvDPZyWM3u5I1WMsZDZ hfPLI60lNumMLYtF/MPqsemcVICDO8Oqi3M/7Y69XUmbDy5N0HrEiqNo X-Gm-Gg: ASbGncs+HNjYdF7S/q/BOZYeFKOXm0XWAHc4C40vfo7oWrZZeI+YbD/CFdXC+p6KR39 x0ivp2PCz04M774OwOjaxl/5M7PtssHFcs4E0iPMxTIAAwauJ7UGm4/ezd+AoZMJ8LlIFuHpC8V u1BkjYtVNZhdMll/g95IIAVDX+BLgFsPNtAqPepQdjL8bV9X85L8e8HcEnNq7Fxvm4v1HEI+90i PPV+s+RV4p0mXIuH1vyFzNwOy/4nQVNDXN5sSAJfzu8HX0LruM4SGOygn5QkqPSM+WsEQ2XcCZB A+ryJJbRyTphQr3XvAAOf/8ySk2kVkQxzQZcv9e/1K38JtczjD63lwD8wT93JJ5BYOHE84EBQTq qLtkJfQ4XfXAUOiI9sCDVi57yyiqKBjDw4TVPmHnMtHOWuybYa+23SoC1/7ggzF3jAQJFckL+bG 4pQU0mmcMlKY4j X-Google-Smtp-Source: AGHT+IHS5VRv6VkyUc17Gj+CedIfSsDAPemumVPyvM5IQ2aSLIbA2ACUNS1G1fJCAa9iWMY5tvez7g== X-Received: by 2002:a05:6214:5bc6:b0:704:a6dc:525d with SMTP id 6a1803df08f44-704f6aa03d3mr49677336d6.25.1752688345220; Wed, 16 Jul 2025 10:52:25 -0700 (PDT) Received: from fauth-a1-smtp.messagingengine.com (fauth-a1-smtp.messagingengine.com. [103.168.172.200]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-704979f498esm73114646d6.53.2025.07.16.10.52.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Jul 2025 10:52:24 -0700 (PDT) Received: from phl-compute-07.internal (phl-compute-07.phl.internal [10.202.2.47]) by mailfauth.phl.internal (Postfix) with ESMTP id 420BDF40066; Wed, 16 Jul 2025 13:52:24 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-07.internal (MEProxy); Wed, 16 Jul 2025 13:52:24 -0400 X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdefgdehkeefiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjug hrpeffhffvvefukfhfgggtuggjsehttdertddttddvnecuhfhrohhmpeeuohhquhhnucfh vghnghcuoegsohhquhhnrdhfvghnghesghhmrghilhdrtghomheqnecuggftrfgrthhtvg hrnhephfegtdefveeljeeuueeltdevleehfeeludegteekhfehveeuleegkeelkedtjedt necuffhomhgrihhnpehophgvnhhgrhhouhhprdhorhhgpdgsohhothhlihhnrdgtohhmne cuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepsghoqhhu nhdomhgvshhmthhprghuthhhphgvrhhsohhnrghlihhthidqieelvdeghedtieegqdduje ejkeehheehvddqsghoqhhunhdrfhgvnhhgpeepghhmrghilhdrtghomhesfhhigihmvgdr nhgrmhgvpdhnsggprhgtphhtthhopedukedpmhhouggvpehsmhhtphhouhhtpdhrtghpth htoheplhhoshhsihhnsehkvghrnhgvlhdrohhrghdprhgtphhtthhopehlvghvhihmihht tghhvghllhdtsehgmhgrihhlrdgtohhmpdhrtghpthhtohepohhjvggurgeskhgvrhhnvg hlrdhorhhgpdhrtghpthhtoheprghlvgigrdhgrgihnhhorhesghhmrghilhdrtghomhdp rhgtphhtthhopehgrghrhiesghgrrhihghhuohdrnhgvthdprhgtphhtthhopegsjhhorh hnfegpghhhsehprhhothhonhhmrghilhdrtghomhdprhgtphhtthhopegrrdhhihhnuggs ohhrgheskhgvrhhnvghlrdhorhhgpdhrtghpthhtoheprghlihgtvghrhihhlhesghhooh hglhgvrdgtohhmpdhrtghpthhtohepthhmghhrohhsshesuhhmihgthhdrvgguuh X-ME-Proxy: Feedback-ID: iad51458e:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 16 Jul 2025 13:52:23 -0400 (EDT) Date: Wed, 16 Jul 2025 10:52:22 -0700 From: Boqun Feng To: Benno Lossin Cc: Mitchell Levy , Miguel Ojeda , Alex Gaynor , Gary Guo , =?iso-8859-1?Q?Bj=F6rn?= Roy Baron , Andreas Hindborg , Alice Ryhl , Trevor Gross , Andrew Morton , Dennis Zhou , Tejun Heo , Christoph Lameter , Danilo Krummrich , linux-kernel@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v2 3/5] rust: percpu: add a rust per-CPU variable test Message-ID: References: <68762e19.170a0220.33e203.a0b7@mx.google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Stat-Signature: 7mgkftyw9e76s7cemct7j5rpwduushxy X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 607B718000D X-Rspam-User: X-HE-Tag: 1752688346-905983 X-HE-Meta: U2FsdGVkX18LfTXg5HgUn0tgKncWgA/ScpKc6dudhjR4zBaoPGVT6fs69Ls0Zni0FC/ALSwfigAvzIr3tXin7x5J0ARrI7bnKqnApp+9yP/cbu1+kf0QOHZsJdND3zIlKblD1UZBvZQ1tSHCjhmFAhFONPnYM/6Y+TAe1iJuf7gAqSgJBQR3opkCwU88aYCqb9rF4iDNGQI8cX+hSrIxbw38aHn+h0tr/UjHfnB1prSJN9Lx5OWIVDKQYKITe56rtmG+CMFFpxmoSeAPCdsTKH08JDYOQ2mRZKe17u3mRfctDaH7W2D2QEYeg8mzdabb7c/eh4Pb0PrH49dzBc4ZJ0aTYAj5Gjm4UinNKYNnhGKdc8A4CC4tlXtGfoUS5WqvdQvj3K8SZYWNmkU6tM1aSIRg1ezs/pBc1srXqRbnIPKJoQQ4YAqKL2C8jkh6Gdhm0tWytYd19cQHV+VmqD9Z5QdJmIz7rNWFsyiRvmKx+D5HHco1+wBE1/IpLIJeUMQRzipepdNvPqbxeYHFSe23a4l85swtwKsh5zFcyr/e5cR5gMSCt3Ie023AwuNqAnJR3KQilHCmpHixk7F2AFWxM77/eX+fH1Cspw5L86PI5pxZczcWZdaSWF3hiW9OahUwl/WUN1ODOHZegGjwaxCWUVVEccmA68NjNJ+AsUBQLhULu9TQzVIIz+jfcbZS57aoziF2Hargy5tSBwGnsL8KckWANOapFtIEliXUSBTlCxLhmOuiZA9LXMvKcUU1VCMV2Gtcs2tKLV4fxrPrQ7QrPtIwl0u7cBtVCS4N/6iwoO8oHokY4QK1bC7Fm1SD99YAuTXFTQajjmyJ6/T6BmrCaM9YpKlr/dNgpNw5vf4xS5fFlIz188+ts1ckK/gMLIPRN901nb3rnmOaCKIhXrXOYy+wwQAprjCuM9IE0vHchVPMKNQFExM0MCSOPqN7btDXZVWDVjDkU84SBEWYeSV 0SGcMCd1 rcnx3nUdjAcImyFRshslQJHmHeMQ4p7FZmeupXclnT9Xewb6nAf4kB4q96pi5DTGzthf5zV51UEVup6rh0oUHKyRkcIKj5Cl31za0HpNbaC36sg2QzQ88xd33xMM7gRwKxMjnVHk7581lJyafG+A1qAtRkYM1vgZLPgB5T7jTUh0K9iMxZHsPZYA1rt6fOEA3zujnLCMJuVX8f35JkamwkE2OyeLrRxjQViV4ZD9FbvafkLh/sbRPtIrmYuGOwa3T8dgHP6IOeEeMDbclwjbC9EpVToIV2qyLy3MAH0aplkjXb3zOuFcd4LMsMEOni8uqzCcvs7Ed7SWi5xwC4p01JIj6eaQn17EVjOl8/dDH/PWHeTjx72RUdnZfzC7uJxOgiBmWOznjSl0T+70yFP4TO8py0EHuIuzez/isBT861AgOMTGT2foBHl2riHmpS3CWzXw7ApzaHr7FTOhngl4smBCfxh96fCZRB4KtaennDnKLnZZuNP/A9g/dhd/QYbNHz9HSYSPGrTx/fOM8+QWKPsIgxFa31MkYaD6CjbUr3jVnJZehrNhXxYwLYk5dm4AaLysrpvamyFj3hYc1M7rCAOgh2CbL3F5XL1W2jh76ScQKEI67p63emcdG2gjp3X/TPaq2 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jul 16, 2025 at 07:21:32PM +0200, Benno Lossin wrote: > On Wed Jul 16, 2025 at 5:33 PM CEST, Boqun Feng wrote: > > On Wed, Jul 16, 2025 at 12:32:04PM +0200, Benno Lossin wrote: > >> On Tue Jul 15, 2025 at 11:34 PM CEST, Boqun Feng wrote: > >> > On Tue, Jul 15, 2025 at 07:44:01PM +0200, Benno Lossin wrote: > >> > [...] > >> >> >> > > >> >> >> > First of all, `thread_local!` has to be implemented by some sys-specific > >> >> >> > unsafe mechanism, right? For example on unix, I think it's using > >> >> >> > pthread_key_t: > >> >> >> > > >> >> >> > https://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_key_create.html > >> >> >> > > >> >> >> > what we are implementing (or wrapping) is the very basic unsafe > >> >> >> > mechanism for percpu here. Surely we can explore the design for a safe > >> >> >> > API, but the unsafe mechanism is probably necessary to look into at > >> >> >> > first. > >> >> >> > >> >> >> But this is intended to be used by drivers, right? If so, then we should > >> >> > > >> >> > Not necessarily only for drivers, we can also use it for implementing > >> >> > other safe abstraction (e.g. hazard pointers, percpu counters etc) > >> >> > >> >> That's fair, but then it should be `pub(crate)`. > >> >> > >> > > >> > Fine by me, but please see below. > >> > > >> >> >> do our usual due diligence and work out a safe abstraction. Only fall > >> >> >> back to unsafe if it isn't possible. > >> >> >> > >> >> > > >> >> > All I'm saying is instead of figuring out a safe abstraction at first, > >> >> > we should probably focus on identifying how to implement it and which > >> >> > part is really unsafe and the safety requirement for that. > >> >> > >> >> Yeah. But then we should do that before merging :) > >> >> > >> > > >> > Well, who's talknig about merging? ;-) I thought we just began reviewing > >> > here ;-) > >> > >> I understand [PATCH] emails as "I want to merge this" and [RFC PATCH] as > > > > But it doesn't mean "merge as it is", right? I don't think either I or > > Mitchell implied that, I'm surprised that you had to mention that, > > Yeah that is true, but it at least shows the intention :) > > > also based on "I often mute those" below, making it "[PATCH]" seems to > > be a practical way to get more attention if one wants to get some > > reviews. > > That is true, I do usually read the titles of RFC patches though and > sometimes take a look eg your atomics series. > > >> "I want to talk about merging this". It might be that I haven't seen the > >> RFC patch series, because I often mute those. > >> > > > > Well, then you cannot blame people to move from "RFC PATCH" to "PATCH" > > stage for more reviews, right? And you cannot make rules about what the > > difference between [PATCH] and [RFC PATCH] if you ignore one of them ;-) > > I'm not trying to blame anyone. I saw a lot of unsafe in the example and > thought "we can do better" and since I haven't heard any sufficient > arguments showing that it's impossible to improve, we should do some > design work. > I agree with you, and I like what you're proposing, but I think design work can be done at "PATCH" stage, right? And sometimes, it's also OK to do some design work even at some version like "v12" ;-) Also I want to see more forward-progress actions about the design work improvement. For example, we can examine every case that makes unsafe_get_per_cpu!() unsafe, and see if we can improve that by typing or something else. We always can "do better", but the important part is how to get there ;-) > >> >> >> I'm not familiar with percpu, but from the name I assumed that it's > >> >> >> "just a variable for each cpu" so similar to `thread_local!`, but it's > >> >> >> bound to the specific cpu instead of the thread. > >> >> >> > >> >> >> That in my mind should be rather easy to support in Rust at least with > >> >> >> the thread_local-style API. You just need to ensure that no reference > >> >> >> can escape the cpu, so we can make it `!Send` & `!Sync` + rely on klint > >> >> > > >> >> > Not really, in kernel, we have plenty of use cases that we read the > >> >> > other CPU's percpu variables. For example, each CPU keeps it's own > >> >> > counter and we sum them other in another CPU. > >> >> > >> >> But then you need some sort of synchronization? > >> >> > >> > > >> > Right, but the synchronization can exist either in the percpu operations > >> > themselves or outside the percpu operations. Some cases, the data types > >> > are small enough to fit in atomic data types, and operations are just > >> > load/store/cmpxchg etc, then operations on the current cpu and remote > >> > read will be naturally synchronized. Sometimes extra synchronization is > >> > needed. > >> > >> Sure, so we probably want direct atomics support. What about "extra > >> synchronization"? Is that using locks or RCU or what else? > >> > > > > It's up to the users obviously, It could be some sort of locking or RCU, > > it's case by case. > > Makes sense, what do you need in the VMS driver? > In VMBus driver, it's actually isolate, i.e. each CPU only access it's own work_struct, so synchronization between CPUs is not needed. Regards, Boqun > >> > Keyword find all these cases are `per_cpu_ptr()`: > >> > > >> > https://elixir.bootlin.com/linux/v6.15.6/A/ident/per_cpu_ptr > >> > >> Could you explain to me how to find them? I can either click on one of > >> the files with horrible C preprocessor macros or the auto-completion in > >> the search bar. But that one only shows 3 suggestions `_hyp_sym`, > >> `_nvhe_sym` and `_to_phys` which doesn't really mean much to me. > >> > > > > You need to find the usage of `per_cpu_ptr()`, which is a function that > > gives you a pointer to a percpu variable on the other CPU, and then > > that's usually the case where a "remote" read of percpu variable > > happens. > > Ahh gotcha, I thought you pointed me to some definitions of operations > on percpu pointers. > > --- > Cheers, > Benno