From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FFDBC46467 for ; Thu, 12 Jan 2023 00:01:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BE3098E0003; Wed, 11 Jan 2023 19:01:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B93658E0001; Wed, 11 Jan 2023 19:01:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A81F98E0003; Wed, 11 Jan 2023 19:01:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 9A0938E0001 for ; Wed, 11 Jan 2023 19:01:49 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 6C75512084F for ; Thu, 12 Jan 2023 00:01:49 +0000 (UTC) X-FDA: 80344193538.30.B243F5C Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) by imf03.hostedemail.com (Postfix) with ESMTP id B36682001B for ; Thu, 12 Jan 2023 00:01:47 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=nspgO5Qi; spf=pass (imf03.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.182 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1673481707; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8Dl3nn/Wn2tOf4PrdMxduCS7/HWymWvhZ9rnPEXGjpw=; b=pX89mz2MNHGIuGvGGtFWb/+yNVHJPIPIw6+t0Zw0egdPkYhGREAynXZD1K43aZpeamAtUI BIzv4tLQT3m7VFZF9eaJ2BzjFCjrQbbxXHeKqz9yfIX5rFMO8DHWJCRp1xJXfGiFEVHrdg 8/NaXBo8R2thh1ttTuHp7/BiaMA0D3Y= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=nspgO5Qi; spf=pass (imf03.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.182 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1673481707; a=rsa-sha256; cv=none; b=YvhDg/lNIcfUQghf7oLUi6oIiYvdsSJPzjtZxC54abYgXR0iGDzcB4XSkQq+VR0wqMfZks 6qklpDn8knBaKK50CE8Pjf3LlO3tA1xeyeaoO31Uzp91qVb24MjF5lG/RR4HAn15RiDclG MT0Ba9F+XbpjMJd6GGtToTV3OWnUTV8= Received: by mail-pf1-f182.google.com with SMTP id c85so9345672pfc.8 for ; Wed, 11 Jan 2023 16:01:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=8Dl3nn/Wn2tOf4PrdMxduCS7/HWymWvhZ9rnPEXGjpw=; b=nspgO5Qi+pBHYKEe1I4IfJsxUIFBh8zRkAFIHSUrLtpc5KbBYhCP02IaxQrM/CRUHy Y4TY/tZCPmkXE8kwufBKlnQ7XiOmedKZIa3AfDIwTVbucz/U1MhYZ+wKP7ufN7B29UQl 3olgHVPyfUF6HaVjRF/bhs7XOrj20fq/TtlYs96s3eU7m1/HZBafHK3lRPGLsCqqsUd3 BGTbmzO6yl3eGCG/+ecj9X/9HlPw3fZZ2tGjVsKY68E70XToUGTfrgnAE0KD2PIPxxN5 OkSS0k7m2X7Ak4M3EXq/C1cvwt28rwCgLVrZBCeVF8lYJiUXeB5nZoU/b5tOWtf18rBE 6pqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=8Dl3nn/Wn2tOf4PrdMxduCS7/HWymWvhZ9rnPEXGjpw=; b=TW2TytsxCd1i6oayjP51PVGEHOCone6rHy7s+9pHHJ4iSVX8eUZcD0F7i+bNfijT1n 25rhWp4+djTRA27HBnQHEOh9ML4bTp7DMq5kS/2C+PRkLNMIG3gEnEoxhmJosZhPR3r0 HRT8QJ+xa3hfSfSXxdPmr+vLZjdLUkZ9MXZTWV/PtZxmK6VBZFLrjruMiNGHzE/2EtnK ewsahHFmHupo2ZrtkwzBzh42s993vcRYrYst8pEffAIgci2ELC+vvQDszSapfaxK/lEg tfYLZLbrco9RVnsb8AvZotTwv4P92rmDdiLauhiHr+ddgKSBb7pLmfoOiesfZb5oJ40k hMNw== X-Gm-Message-State: AFqh2krvzHjYti60l8w+ibKU4pa45ugXDUUt6uvCxbCVN05Ly/jAc8KI fpm9d5kz6KfPJpwpkOxmjT3kLlBw31RbZkVZgj4= X-Google-Smtp-Source: AMrXdXtQQPaT80U5kZr8QPKLEnrW2ye44T3a40dof+nmE8neZFw0XWVNRxFjcXxHiI0txA2sYTJQWr8AV1sGTZWZQlU= X-Received: by 2002:aa7:8051:0:b0:582:e939:183d with SMTP id y17-20020aa78051000000b00582e939183dmr2974824pfm.63.1673481706391; Wed, 11 Jan 2023 16:01:46 -0800 (PST) MIME-Version: 1.0 References: <20230109173155.GS4028633@paulmck-ThinkPad-P17-Gen-1> <20230109231106.GZ4028633@paulmck-ThinkPad-P17-Gen-1> <20230110000418.GC4028633@paulmck-ThinkPad-P17-Gen-1> In-Reply-To: <20230110000418.GC4028633@paulmck-ThinkPad-P17-Gen-1> From: Yang Shi Date: Wed, 11 Jan 2023 16:01:34 -0800 Message-ID: Subject: Re: [QUESTION] Linux memory model: control dependency with bitfield To: paulmck@kernel.org Cc: linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: B36682001B X-Rspam-User: X-Stat-Signature: aq7nmrs6e56ejcaebypeiy68mkaykpri X-HE-Tag: 1673481707-545189 X-HE-Meta: U2FsdGVkX1+66AqA5Lt2sS3m41eoq1GolYOjI44SonCnw3kBzoEiIkDFDepRMAn9MZ0gY6+v4i7j8pXVHBrg9ncvuiJ045JHZdYZjVkr6pEx+9CN7MXLNxpsp0hnZoJsXHiihjxpvraqs5C9fvE9OUR4EWHMPiOy3c2bbIjcKGqIb4tmNg+XuoYTch0vJ/tmZMl/WItfue/u7CBIVGLoDFxKxTJGS4b45qVbgWH1zZj16G0I3UKc+ZHPUCUtfB6pzVPfrzyUXfpGB3/GsXaYU02e7zeORo6zz6y2ZcHeXXGoC4Djt2omsDbD0kg7jMUycfQ7xsjBk2lE8rTiuEh5avcUZ0SLrgu0z2JCq/0R9oBHE29rv9zEI+RtD7tLEwoi0Y8wKiCbBbM4tvN1CZn2YLQJWDMtFs+1qu0XmQjVmPAdnp2zRAgcJyWbYfWhue/co4Gu9LnTsp6PXama43do3Kq9tMlEjbGvzXKRX8MVdZcVYVOba4JoR/4eARdZ4BkjKdzlMqIWBGL3eMe6vvCBv39ITmM1vv/hOXM4DL8hJgdqY+VwH2UCiNPH9mTQy9hbhlOKK0YirYYmqxDs7axRTy1eNEYLiqwAM/eYngO0K8JjdpYku0vLXYrwHBVN1IdBnvS3mieodJ49/1iCSP2fUYX6R4PIlkoKhWlylsIoPHC8m4bYJTWtzjltkUSi8n0zrKiFuerhWXCpGr8STy5VFjfu9/BwMznqCMgJA6XJnQ61kqV9RQjrjopXG628xLKeq6xsRBiyNSJ52p5tR3T8XsjbtjYj2MkNO6TMVqcCAJrxHUSaOskkwskbZaDzd6rUd+BTPNGFVYjhgwSs8RKlgN1G37xjIv4trZk8olmsS6ILAF4Yy0j/qYn43vDACI3qtpSEsXFjyfs6ZAO8Xf1umF/tKN/vpxhsxHo1o18pgY0mJdP+5S8poEkJJ07rofZujZyBHfP4frSOgOVcbLD Y5DeF7OF glHq5okBK9molDy4YL1qpqYr2pqUAvkRf0YRDcJVUYCkFY6VXdvBPdP7ygQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Jan 9, 2023 at 4:04 PM Paul E. McKenney wrote: > > On Mon, Jan 09, 2023 at 03:44:54PM -0800, Yang Shi wrote: > > On Mon, Jan 9, 2023 at 3:11 PM Paul E. McKenney wrote: > > > On Mon, Jan 09, 2023 at 02:08:35PM -0800, Yang Shi wrote: > > > > On Mon, Jan 9, 2023 at 9:31 AM Paul E. McKenney wrote: > > > > > On Mon, Jan 09, 2023 at 09:14:19AM -0800, Yang Shi wrote: > > > > > > Hi Paul, > > > > > > > > > > > > Hope this email finds you are doing well. I recently ran into a > > > > > > problem which might be related to control dependency of the memory > > > > > > model. Conceptually, the code does (from copy_present_pte()): > > > > > > > > > > > > acquire mmap_lock > > > > > > spin_lock > > > > > > ... > > > > > > clear bit (a bit in page flags) > > > > > > ... > > > > > > VM_BUG_ON(test bit) > > > > > > ... > > > > > > spin_unlock > > > > > > release mmap_lock > > > > > > > > > > > > > > > > > > IIUC there is control dependency between the "clear bit" and > > > > > > "VM_BUG_ON" since VM_BUG_ON simply tests the bit then raises the BUG. > > > > > > They do touch the overlapping address (the page flags from the same > > > > > > struct page), but they are bit field operations. Per the memory model > > > > > > documentation, the order is not guaranteed for bit field operations > > > > > > IIRC. > > > > > > > > > > > > And there are not any implicit barriers between clear bit and test > > > > > > bit, so the question is whether an explicit barrier, for example, > > > > > > smp_mb__after_atomic() is required after clear bit to guarantee it > > > > > > works as expected? > > > > > > > > > > I am not familiar with this code, so I will stick with LKMM > > > > > clarifications. > > > > > > > > Yeah, sure. This is why I tried to generalize the code. > > > > > > > > > First, please don't forget any protection and ordering that might be > > > > > provided by the two locks held across this code. > > > > > > > > Yes, but for this case I just care about the code between clear bit > > > > and VM_BUG_ON. > > > > > > Fair enough! > > > > > > > > Second, a control dependency extends from a READ_ONCE() or stronger > > > > > (clear_bit() included) to a later store. Please note "store", not > > > > > "load". If you need to order an earlier READ_ONCE() or clear_bit() > > > > > > > > So you mean: > > > > > > > > clear bit > > > > ... > > > > if (test bit) { > > > > load_1 > > > > store_1 > > > > load_2 > > > > store_2 > > > > } > > > > > > > > The dependency reaches to the first store? > > > > > > It reaches both stores, but neither load. > > > > > > That means that your example might well execute as if it had instead > > > been written as follows: > > > > > > load_1 > > > load_2 > > > if (test bit) { > > > store_1 > > > store_2 > > > } > > > > > > Assuming that you mean the test_bit() function. If you instead mean > > > a C-language statement that tests a bit, then the compiler can do all > > > sorts of things to you. The compiler can also do interesting things > > > to you if the stores are plain C-language stores instead of something > > > like WRITE_ONCE(). > > > > It is a test_bit() function. Is it possible clear_bit() is reordered > > with test_bit(), or test_bit() doesn't see the result from > > clear_bit()? > > If the various calls to test_bit() and clear_bit() are to the same > location, then they will not be reordered with each other. > > If they are to different locations, they can be reordered. But in that > case, they would not see each others' results anyway. Yeah, make sense. > > > > > > with a later load, you will need acquire semantics (smp_load_acquire(), > > > > > for example) or an explicit barrier such as smp_rmb(). Use of acquire > > > > > semantics almost always gets you code that is more readable. > > > > > > > > Does the load acquire have to pair with a smp_store_release()? > > > > smp_mb__after_stomic() is not needed because it is too strong and the > > > > weaker barrier is good enough, right? > > > > > > It needs to pair with some type of applicable ordering, but yes, > > > smp_store_release() is a good one. > > > > So, it should look like IIUC: > > > > clear_bit() > > smp_load_acquire() > > ... > > if (test_bit()) { > > smp_store_release() > > load_1 > > store_1 > > load_2 > > store_2 > > } > > Just so you know, smp_load_acquire() does a load and smp_store_release() > does a store. > > Also, is this code executed by a single CPU/task? If so, you need to > also consider the corresponding code executed by some other CPU/task. The code could be executed by multiple tasks in parallel. > > > Thanx, Paul > > > > > > Does that help? > > > > > > > > Yeah, sure. Thanks. > > > > > > > > > > > > > > Also CCing linux-mm@kvack.org in case someone with better understanding > > > > > of that code has advice. > > > > > > > > > > Thanx, Paul