From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6ACE0C5479D for ; Tue, 10 Jan 2023 00:04:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 030C48E0002; Mon, 9 Jan 2023 19:04:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F23508E0001; Mon, 9 Jan 2023 19:04:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DEB578E0002; Mon, 9 Jan 2023 19:04:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id CCE528E0001 for ; Mon, 9 Jan 2023 19:04:23 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 871A1140F5E for ; Tue, 10 Jan 2023 00:04:23 +0000 (UTC) X-FDA: 80336942406.03.6882A56 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf29.hostedemail.com (Postfix) with ESMTP id E1099120002 for ; Tue, 10 Jan 2023 00:04:20 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=td0i11Px; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf29.hostedemail.com: domain of "SRS0=fUii=5H=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=fUii=5H=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1673309061; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=w3kuwFMLXNNj7Hanbp9yezr84nGU9RFRKoTPP48Rk04=; b=KAzwlKb24R+oKLbuCW4xqTABhgDyi5eozepdXOiObUGXrIHoUvHZ/m18ubJO6NwAgQo8IJ UXAsFjLSCmzZ3K6JTacB0yRtY89BgHTYTFuIB7MgVGPANH20EoT/+oCYXdUfIRTAh5ON8k z5E37yJsA7k5Hwi12Cb8c35SYBv/PNw= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=td0i11Px; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf29.hostedemail.com: domain of "SRS0=fUii=5H=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=fUii=5H=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1673309061; a=rsa-sha256; cv=none; b=rFwdJkfQ1aOeqSF6b06FxirsbbxIQbWd5uxaBAGqW8oGx66DOPSfBeQfugfQRBXJFoIpsu iaXKZBnSNfS9JSibkgB+lUzekCW7k2JYAGCO6Vc2M10QFEV813U2SrR2ef6fK26sTcuuZM a1quJ2ggCGFiA/+euK08eEPYGY+vCFc= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id EF56061480; Tue, 10 Jan 2023 00:04:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 385FCC433EF; Tue, 10 Jan 2023 00:04:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1673309059; bh=3Qr/J61teHadnpvjC8KtkGO+RRh19zydnqOQ2oGU/ec=; h=Date:From:To:Cc:Subject:Reply-To:References:In-Reply-To:From; b=td0i11Pxrn8S/hvNZ4jev4LBcAFQdcalVQih3l3KfnWGHR/9P4BrV8volHDOAW5Wk RFIarAUP7G0eklDoK0nxN19VyEvm8V/b31JnX29Ky9QW5qAKqrs5dEJeHfTWc7SqlG DXyi1nZz+X2dt11mYflLUsD65NJBQNzgh7AajUrIa08AsYz6u9WD+x+bSL3E0l0Hnm JjX6d+UWzG7C18APH688iU+HPme2eCY+ksNZSe6rb+k4+LN4pVWh50NHcUqtlWEWCQ z271/FKvmGXRV0c1uH2BEsHIkQOMyK03whM4m5DemQLx6qzmS/4/3sds09Dg1uoBCm 1t9p23+d44tQA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id D78105C0623; Mon, 9 Jan 2023 16:04:18 -0800 (PST) Date: Mon, 9 Jan 2023 16:04:18 -0800 From: "Paul E. McKenney" To: Yang Shi Cc: linux-mm@kvack.org Subject: Re: [QUESTION] Linux memory model: control dependency with bitfield Message-ID: <20230110000418.GC4028633@paulmck-ThinkPad-P17-Gen-1> Reply-To: paulmck@kernel.org References: <20230109173155.GS4028633@paulmck-ThinkPad-P17-Gen-1> <20230109231106.GZ4028633@paulmck-ThinkPad-P17-Gen-1> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: E1099120002 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: 6b5snq6hbzoo9899rfxdm8x63xh4ek9w X-HE-Tag: 1673309060-714421 X-HE-Meta: U2FsdGVkX1+ntxN+8AnuXHRSFvtRGJORbJNsYdYGDjfEYHzHkHsYYoeCcvKQ2bRYmWiB6TNgoO8/X6+ovRFoEg6HD+EmbstWJ5o1LRzW4Gklj708nTl3NX1Dl6JNCexlW8T4XC0f1zkEZ+gkEQ4Lv4MNuUxBzoM5mOnIPu4b67oYm6Ez+dZQ2kK7HU/NDn3PPU7Ibil7rWIF/RNx9d9l/5Cz55B4/Or0BGVWthTqbXkzUnZGYXJWJQcHddYXEIu13m4tZZHx/Dz1mOdoBwn3A3nCskRNxmhbL09QDoFr4V3VFJsxJ2vgwahhI5IqQ+NdxFAYjjJ+U7djDsk1CU6sxbCSV96la5Tsb+HXRq54MD6QZrN+21eE2KcM+Rpq/ocBLOYZ8+Obmqowp5TOQqBTfMZW/BgmyainxnCyGzfcQsGCH6VthvbNtR1J3ahT3PWD4pkc0fK323DrcazODwckvnA3uSnngPXUjabyRkMZeRqmsMUFzmp7uZY03Ul7ZJMewFYoasWSwg2Xe09+bwExKqLCeruDr7dLUzD2Xf+JJlaW8A2OTkwSga5H6t5fbOcYHX0BbgkzSpVFE3/ReeuchrWnXh5svkz0dlZAGHs4u4tEL+At//YSgek1sZEd1MwOkMn1Y2BZIwal8YcEoMFJWSqG1G2spbAVbbk8vQHz7MBK4qEuuEGcw3USdu/xlWIiMYwhwDUJiQlYM1/wxDLWuQ7nJ+xk5dSZLce6iOKUQqf2eKVKzkBxfKX8MrEy57HbzXG5dpUByxGDIZDfvyoiCqC39oRfbx21D2iHS53kVppR2zB3qqPHDabx93w/argErY3HOk/YYpHov9F7YFns3lws8a0Mc4QrISFoxL4D2xWpojhtNSXu/9hpUFqMxme+P99/GKRIKlhZL77VmSQtxjcvj4E5axOlcbEmwkdhm8OySAVCPWRhkeEOF89bAyik+A85sRAG2y8ZYS+w0Up o1ob6NZ/ 5enB3CzuQTygwnueyK+sgt/nIcRrg6dpwxeQdRTW9A8ycXx+KMIpwfLNPKdFfV+kdjzZwQuJx/uqRHagfrPMVfIH5zlXbXuT5CyU4Ynyq/bBclA1K8FKjvwL+kix163ol7K5Oqd1vcm7jaPdlbbYv/4obck0281XshlL5Y7E4PtpT5RlLZFjm8aSsScDBagahTEwg1nwgGmbutCqzmzQkRf0mes9D9h8A5Gcw4KdAxe351Ed2WO+Tf1zlU7/WMqHZUD9Zoiqx6ynasSaR0kMmNie5zAMH5jiIQJ3/faJhVgzRKZIOA7wYwcOo+89SQgdENlf8CJ4sWSjAXNNs6JudhBDHLcmnVntAezB8SPQHEKi6nVJGJisYQ5d5yp1BNnt6Z4HdKCe9XlUq5hCM/J4Pvc7ua7ec5OcGfasEFIPaX3UJ4U8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Jan 09, 2023 at 03:44:54PM -0800, Yang Shi wrote: > On Mon, Jan 9, 2023 at 3:11 PM Paul E. McKenney wrote: > > On Mon, Jan 09, 2023 at 02:08:35PM -0800, Yang Shi wrote: > > > On Mon, Jan 9, 2023 at 9:31 AM Paul E. McKenney wrote: > > > > On Mon, Jan 09, 2023 at 09:14:19AM -0800, Yang Shi wrote: > > > > > Hi Paul, > > > > > > > > > > Hope this email finds you are doing well. I recently ran into a > > > > > problem which might be related to control dependency of the memory > > > > > model. Conceptually, the code does (from copy_present_pte()): > > > > > > > > > > acquire mmap_lock > > > > > spin_lock > > > > > ... > > > > > clear bit (a bit in page flags) > > > > > ... > > > > > VM_BUG_ON(test bit) > > > > > ... > > > > > spin_unlock > > > > > release mmap_lock > > > > > > > > > > > > > > > IIUC there is control dependency between the "clear bit" and > > > > > "VM_BUG_ON" since VM_BUG_ON simply tests the bit then raises the BUG. > > > > > They do touch the overlapping address (the page flags from the same > > > > > struct page), but they are bit field operations. Per the memory model > > > > > documentation, the order is not guaranteed for bit field operations > > > > > IIRC. > > > > > > > > > > And there are not any implicit barriers between clear bit and test > > > > > bit, so the question is whether an explicit barrier, for example, > > > > > smp_mb__after_atomic() is required after clear bit to guarantee it > > > > > works as expected? > > > > > > > > I am not familiar with this code, so I will stick with LKMM > > > > clarifications. > > > > > > Yeah, sure. This is why I tried to generalize the code. > > > > > > > First, please don't forget any protection and ordering that might be > > > > provided by the two locks held across this code. > > > > > > Yes, but for this case I just care about the code between clear bit > > > and VM_BUG_ON. > > > > Fair enough! > > > > > > Second, a control dependency extends from a READ_ONCE() or stronger > > > > (clear_bit() included) to a later store. Please note "store", not > > > > "load". If you need to order an earlier READ_ONCE() or clear_bit() > > > > > > So you mean: > > > > > > clear bit > > > ... > > > if (test bit) { > > > load_1 > > > store_1 > > > load_2 > > > store_2 > > > } > > > > > > The dependency reaches to the first store? > > > > It reaches both stores, but neither load. > > > > That means that your example might well execute as if it had instead > > been written as follows: > > > > load_1 > > load_2 > > if (test bit) { > > store_1 > > store_2 > > } > > > > Assuming that you mean the test_bit() function. If you instead mean > > a C-language statement that tests a bit, then the compiler can do all > > sorts of things to you. The compiler can also do interesting things > > to you if the stores are plain C-language stores instead of something > > like WRITE_ONCE(). > > It is a test_bit() function. Is it possible clear_bit() is reordered > with test_bit(), or test_bit() doesn't see the result from > clear_bit()? If the various calls to test_bit() and clear_bit() are to the same location, then they will not be reordered with each other. If they are to different locations, they can be reordered. But in that case, they would not see each others' results anyway. > > > > with a later load, you will need acquire semantics (smp_load_acquire(), > > > > for example) or an explicit barrier such as smp_rmb(). Use of acquire > > > > semantics almost always gets you code that is more readable. > > > > > > Does the load acquire have to pair with a smp_store_release()? > > > smp_mb__after_stomic() is not needed because it is too strong and the > > > weaker barrier is good enough, right? > > > > It needs to pair with some type of applicable ordering, but yes, > > smp_store_release() is a good one. > > So, it should look like IIUC: > > clear_bit() > smp_load_acquire() > ... > if (test_bit()) { > smp_store_release() > load_1 > store_1 > load_2 > store_2 > } Just so you know, smp_load_acquire() does a load and smp_store_release() does a store. Also, is this code executed by a single CPU/task? If so, you need to also consider the corresponding code executed by some other CPU/task. Thanx, Paul > > > > Does that help? > > > > > > Yeah, sure. Thanks. > > > > > > > > > > > Also CCing linux-mm@kvack.org in case someone with better understanding > > > > of that code has advice. > > > > > > > > Thanx, Paul