From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5AA6BC54EBD for ; Mon, 9 Jan 2023 23:45:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 940518E0003; Mon, 9 Jan 2023 18:45:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8C94B8E0001; Mon, 9 Jan 2023 18:45:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7431E8E0003; Mon, 9 Jan 2023 18:45:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 5E2EB8E0001 for ; Mon, 9 Jan 2023 18:45:09 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 18BBB1215E5 for ; Mon, 9 Jan 2023 23:45:09 +0000 (UTC) X-FDA: 80336893938.30.023D830 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) by imf22.hostedemail.com (Postfix) with ESMTP id 8B0F8C000D for ; Mon, 9 Jan 2023 23:45:07 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=VTzHdFa0; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.173 as permitted sender) smtp.mailfrom=shy828301@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1673307907; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lVhYNhEOnYUipOFW8SAAdR2+FSu/I64U2KyoO+MVR00=; b=KlpDdj6Uiko6ud87OzbDIn7CkiQTELM4mvTU5eBKsta07aO9u91lEnteS1gPMDfUKKBkgK CD2ihnDuY3gxOUNDpib/7iGRYq/tI7pCYVEGSydUS6Aa6s77euULQz3Qk0bzApCkqPRF2k hxznmxIkig5D2AMtxf/YNaeqAyJVMxY= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=VTzHdFa0; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.173 as permitted sender) smtp.mailfrom=shy828301@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1673307907; a=rsa-sha256; cv=none; b=vZLk7CqFHU+gDqutpmxKIEOsSNV27JeIHrtuTTW93N7aN5bGcNcsl0XE8+kwGGmLfyQIDM 5DDxcOR4Z8vPTHqlDz8A+l5nxgzTbFzCb94DR1Ihj/tBOb0wfeat1JS6zB7lIO4t/1D9dY OZe3VQgXCU63BOqhZzPocifgbmLjIKI= Received: by mail-pf1-f173.google.com with SMTP id 200so1286707pfx.7 for ; Mon, 09 Jan 2023 15:45:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=lVhYNhEOnYUipOFW8SAAdR2+FSu/I64U2KyoO+MVR00=; b=VTzHdFa0mKH5a6/u0G0cmcLZal0Y7DjsghlGzlDTmlwM2TEg/V3xcjOGLnQZovgtF9 cg6eD2RrBzvIh5wrlMtjDP71Ew850P+2H0OZh58yYP7sbrbJbvVTu9gcj0EeOLVzJKTt E/H4rDdCgVwb9XRaDKjteNqaWfoDMVldjCtT2ovwcwo4T+xmHbziPAcDdf6CMcJNpXUD ksBkl5T7gu68u5GG+8hoy/VcDJU/2EsKE1hZH4gZRQUZtepIpNxICgs7XkkhFY++Glat qGLCP0cHyD6Mlgz6+AH87sAB7UT6soPLE5euPBCxMV1QL4Jq8UaQhcnLsBkVZJPByPui h0ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=lVhYNhEOnYUipOFW8SAAdR2+FSu/I64U2KyoO+MVR00=; b=ZKwwFpj6UGHlAWWvX/XtUCqRHUn9u6krfBvdwCe272NC+HgGco2imkAR06gJnudnDX yJ/mK3FmcyE04/kjL5vU+LClk/1pOJPufzo8Cn7n2rtZqL7Z41F3qSNpQv6tckqIweOO tBjGaKSR4w6JGjUHAJ3WqnhbtAtcjX1qFLUXqfM2aV1tM7rzLWgFFfxay8BU95aQUvLs Ggsci83SD+ZPzVxbbHQul3AiqP9Ho0a+QrTWFMHKEiRWws3gRuN6HBF41TOiTG0lRmq9 tfl2W0ejEwaapMgvvVK/sY1NEqlN4GuJUP60x0YGnZs6rvjkyzGdJ6pZK7kvxSV2H2Tm fnJA== X-Gm-Message-State: AFqh2kq07lnEojrLw32mfKmi2xXep786xXF1pgLROlNsOfIZvNIgA/OD +raOKCeOPi8NkFHf8b9ENzW7dZlsAFRRKUtwD2Q= X-Google-Smtp-Source: AMrXdXt668Eq+G/JU9As2WY41Aajz5CgulakY0fiPrjV3vb9M01FMwS4+ASljYtL4q9MqDgjkpuC8EuwbG7Sr67H5/g= X-Received: by 2002:a63:1944:0:b0:492:50ad:d177 with SMTP id 4-20020a631944000000b0049250add177mr5330457pgz.310.1673307906304; Mon, 09 Jan 2023 15:45:06 -0800 (PST) MIME-Version: 1.0 References: <20230109173155.GS4028633@paulmck-ThinkPad-P17-Gen-1> <20230109231106.GZ4028633@paulmck-ThinkPad-P17-Gen-1> In-Reply-To: <20230109231106.GZ4028633@paulmck-ThinkPad-P17-Gen-1> From: Yang Shi Date: Mon, 9 Jan 2023 15:44:54 -0800 Message-ID: Subject: Re: [QUESTION] Linux memory model: control dependency with bitfield To: paulmck@kernel.org Cc: linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 8B0F8C000D X-Stat-Signature: c8feb4qnehkkbsqnhzkz9z3ho6xr9jau X-HE-Tag: 1673307907-555072 X-HE-Meta: U2FsdGVkX19cYbj2PjsLB6M/gufYfbhW2/IpJWXGbFvwe6MrF25rjCvfQtxGq3e1M6PVpXqpQcJVHJUbREFhCZlMZp8B5ZAsOWuiXsXIwfn+btspdfELm+pWErbvTHYGU2BSn96b9hp5lzmEM00VS3vCz8zO5fX11+VC60bRcl7RQxo1osxCg3Hjl7pF/WefY1yxHXcuCu630NLhGZBVL4CJEe1/vy2k0FjK23MbdDt39yVnKdmg4KPlrbmZozKx/BkU9A/0ZXZeRHXORqC8tDZ6qv0m6DcNCg9D/tO6HPjpMPUsuXdt0eFtDHUe0rkoekOj+LzY+a+fLnQv3guCc+NrlWY22Z4tPwWGW5XFpr9g/tyhIsNLOKFuFVrzpxWB45FENG8dds+haxBF0eEPFvfSTphAXjdtQLeu0NDw3yZMsHHRtTJpQtOTJWwf+VCnIMfneyNyFI6xLyGUrOyC4DS2ues8J6cjEIH6Rl2W3gjZ2KpRhYf7c6p2NmpXAPvvDnlGkyVwPEKElwL8jU0oJv7wwyQhOH/nu/YJHYwLhKsxE1loL7ZNieMFKKiMq3HDp1exf1Sp8/vpS4QTwTsRBd18nH2HAwxXpTvvqhJ7CJE4jdrSStz4W7hMURXmtGsQ9hV/VEuoiilseqAjaN6aHX2vGCQxb5+cSJKqlD886K3BZ4bGPzZFBlSqvdtEpdSI0Oj4Eds11KcwjWxCMX99kJRyNUsM/IDkFNpf6gdJUwsbM8uuU7w9TW6Yae29ZjjHP53QFpDpPfmMGtWbnibkvhoEjZvG2WylmARhX7kJB4HMeeTXCEjMLGspkO8QpM1nZ9Cc6WVr1+jObkXGhIx13hc2zi5uxlCvsxTaSh9LcQQlPLvID4q4IV8UOEdHEreaqF17avDiQneDsaYovqeCTeQQUUNw2g0QAP3zEkMWa+Nnfslebk8jzWepNhgiOfO4uVJ7oBk8JS91DfF8aho n+bXBMH/ gbzlPQzW424d7nhAlLuvg63ZpSAE0QgZg1ULArFSLI9/fXO5KD6djuXRh8Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Jan 9, 2023 at 3:11 PM Paul E. McKenney wrote: > > On Mon, Jan 09, 2023 at 02:08:35PM -0800, Yang Shi wrote: > > On Mon, Jan 9, 2023 at 9:31 AM Paul E. McKenney wrote: > > > > > > On Mon, Jan 09, 2023 at 09:14:19AM -0800, Yang Shi wrote: > > > > Hi Paul, > > > > > > > > Hope this email finds you are doing well. I recently ran into a > > > > problem which might be related to control dependency of the memory > > > > model. Conceptually, the code does (from copy_present_pte()): > > > > > > > > acquire mmap_lock > > > > spin_lock > > > > ... > > > > clear bit (a bit in page flags) > > > > ... > > > > VM_BUG_ON(test bit) > > > > ... > > > > spin_unlock > > > > release mmap_lock > > > > > > > > > > > > IIUC there is control dependency between the "clear bit" and > > > > "VM_BUG_ON" since VM_BUG_ON simply tests the bit then raises the BUG. > > > > They do touch the overlapping address (the page flags from the same > > > > struct page), but they are bit field operations. Per the memory model > > > > documentation, the order is not guaranteed for bit field operations > > > > IIRC. > > > > > > > > And there are not any implicit barriers between clear bit and test > > > > bit, so the question is whether an explicit barrier, for example, > > > > smp_mb__after_atomic() is required after clear bit to guarantee it > > > > works as expected? > > > > > > I am not familiar with this code, so I will stick with LKMM > > > clarifications. > > > > Yeah, sure. This is why I tried to generalize the code. > > > > > First, please don't forget any protection and ordering that might be > > > provided by the two locks held across this code. > > > > Yes, but for this case I just care about the code between clear bit > > and VM_BUG_ON. > > Fair enough! > > > > Second, a control dependency extends from a READ_ONCE() or stronger > > > (clear_bit() included) to a later store. Please note "store", not > > > "load". If you need to order an earlier READ_ONCE() or clear_bit() > > > > So you mean: > > > > clear bit > > ... > > if (test bit) { > > load_1 > > store_1 > > load_2 > > store_2 > > } > > > > The dependency reaches to the first store? > > It reaches both stores, but neither load. > > That means that your example might well execute as if it had instead > been written as follows: > > load_1 > load_2 > if (test bit) { > store_1 > store_2 > } > > Assuming that you mean the test_bit() function. If you instead mean > a C-language statement that tests a bit, then the compiler can do all > sorts of things to you. The compiler can also do interesting things > to you if the stores are plain C-language stores instead of something > like WRITE_ONCE(). It is a test_bit() function. Is it possible clear_bit() is reordered with test_bit(), or test_bit() doesn't see the result from clear_bit()? > > > > with a later load, you will need acquire semantics (smp_load_acquire(), > > > for example) or an explicit barrier such as smp_rmb(). Use of acquire > > > semantics almost always gets you code that is more readable. > > > > Does the load acquire have to pair with a smp_store_release()? > > smp_mb__after_stomic() is not needed because it is too strong and the > > weaker barrier is good enough, right? > > It needs to pair with some type of applicable ordering, but yes, > smp_store_release() is a good one. So, it should look like IIUC: clear_bit() smp_load_acquire() ... if (test_bit()) { smp_store_release() load_1 store_1 load_2 store_2 } > > Thanx, Paul > > > > Does that help? > > > > Yeah, sure. Thanks. > > > > > > > > Also CCing linux-mm@kvack.org in case someone with better understanding > > > of that code has advice. > > > > > > Thanx, Paul