From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1568C77B73 for ; Fri, 26 May 2023 16:48:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5192D900004; Fri, 26 May 2023 12:48:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4A1A9900002; Fri, 26 May 2023 12:48:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 31BC7900004; Fri, 26 May 2023 12:48:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 1B07D900002 for ; Fri, 26 May 2023 12:48:59 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id C3786AE2AC for ; Fri, 26 May 2023 16:48:58 +0000 (UTC) X-FDA: 80832990756.11.D35FE84 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf11.hostedemail.com (Postfix) with ESMTP id 5464140019 for ; Fri, 26 May 2023 16:48:56 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=PwRJv7iW; spf=pass (imf11.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685119736; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Sxic9C46g7G0xFpFyBBflZoBYb0LNNTJdlt5DOHiHJY=; b=zn9nb8hnmg39hNU7Eum+lG63THicljMUjMl3qTjr0/QIZn0hOSCyKbUCvvnNsBLYD9Fq5p 5vBi+TRuuhKkvYqt7zDJU1kGZYOqCB+UQ0VSN4pHsanNrob/QcOJMBuTh1CIJMwp2gyAte BYXRj0lmyyDdVwhutwCOEAqIJo3BuQ0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685119736; a=rsa-sha256; cv=none; b=AKP6+VGre6wmqfFHelv+BJBxlekKOBn4SCTUSL+epaQ2XTi8M4+e8DtbyRtFgJygKTyBYm oKAmC47p+FP2ISafwZ7nA9qMqI7K4ZCw3gwbkFLx48dmC3Au7xyVFgZkIk59itLsPtZBpz tKlH971bGjadZTCy8moqpR6PH2Rlf9s= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=PwRJv7iW; spf=pass (imf11.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685119735; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Sxic9C46g7G0xFpFyBBflZoBYb0LNNTJdlt5DOHiHJY=; b=PwRJv7iWrrEkMrJ6G8PogvVbaIvWzdoZVZOOStqs6+IWmDKKFzBVBfii7XofLFe95DTFE6 QNgTQABYLYe7r7KOfh2kTb3+LvcG+hDi8vObzzsfQqvv7QmnROHNToadWj7Wfv2VzVrjRT fjhAh8khidUN8j0bJY7sxBgD99QgSxU= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-264-6ntUOz45OX2Ul7FA-FqFjg-1; Fri, 26 May 2023 12:48:54 -0400 X-MC-Unique: 6ntUOz45OX2Ul7FA-FqFjg-1 Received: by mail-qk1-f200.google.com with SMTP id af79cd13be357-75cb47e5507so4137185a.1 for ; Fri, 26 May 2023 09:48:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685119734; x=1687711734; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Sxic9C46g7G0xFpFyBBflZoBYb0LNNTJdlt5DOHiHJY=; b=BLhtQK78n5llFQ7cLO9jvf1jCOfEGSm6GH0LNCK7U6s05yifiImjFmeh7sGRtFefzI rOeDJ4Cv+KxQy+VTVa9Dj1Q4A1PCL5Z7nBThw0NTGDxkMHD8YLpW/6pbnvsLYZj8gysg TQjNiFeeeo3hfenDal899xXS5t2ulWSZDxIyUxfPxEsxD7vMEEYV8p33HB6Yho/16wyX 7vYi9m7QDIQt8Ej12yFrHmIagaMAnyQpPOXtpcBJ9OLe5NqfhZr6Lnf0K1Evfc1+qtzG ByveP2K/bAayswyn8YXWNecndzekC52qiBKiQjSfADLwlt9n/5tq8ube0nVW2yGNc4L2 S9dg== X-Gm-Message-State: AC+VfDwi6/wpm1RR+JF1KAjNThsPBQHrY03BSFWfZwhcoaQuKaBacleq 2diGt57neWn/alTIPD8aJPzJpyhSFyCGM60aBRV6T1CiI4jFH/BgdywJcHcR9FitFSYPrYdRKQs 8xzIaQ4k30SE= X-Received: by 2002:a05:620a:8f06:b0:75b:3a99:241c with SMTP id rh6-20020a05620a8f0600b0075b3a99241cmr1986884qkn.7.1685119733667; Fri, 26 May 2023 09:48:53 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7Zj6MOk0fgSC2Gs+lb8bZgbtw6HOuPPCYdHF93Hvwa10uJ6LUEwh6R8dNOTYn7x41D4Dwn9g== X-Received: by 2002:a05:620a:8f06:b0:75b:3a99:241c with SMTP id rh6-20020a05620a8f0600b0075b3a99241cmr1986862qkn.7.1685119733280; Fri, 26 May 2023 09:48:53 -0700 (PDT) Received: from x1n (bras-base-aurron9127w-grc-62-70-24-86-62.dsl.bell.ca. [70.24.86.62]) by smtp.gmail.com with ESMTPSA id c22-20020ae9e216000000b0075b18a40a85sm1289471qkc.46.2023.05.26.09.48.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 May 2023 09:48:52 -0700 (PDT) Date: Fri, 26 May 2023 12:48:50 -0400 From: Peter Xu To: Hugh Dickins Cc: Andrew Morton , Mike Kravetz , Mike Rapoport , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Suren Baghdasaryan , Qi Zheng , Yang Shi , Mel Gorman , Peter Zijlstra , Will Deacon , Yu Zhao , Alistair Popple , Ralph Campbell , Ira Weiny , Steven Price , SeongJae Park , Naoya Horiguchi , Christophe Leroy , Zack Rusin , Jason Gunthorpe , Axel Rasmussen , Anshuman Khandual , Pasha Tatashin , Miaohe Lin , Minchan Kim , Christoph Hellwig , Song Liu , Thomas Hellstrom , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH 01/31] mm: use pmdp_get_lockless() without surplus barrier() Message-ID: References: <68a97fbe-5c1e-7ac6-72c-7b9c6290b370@google.com> <34467cca-58b6-3e64-1ee7-e3dc43257a@google.com> <427ea01f-345a-6086-d145-fe573894dbe@google.com> MIME-Version: 1.0 In-Reply-To: <427ea01f-345a-6086-d145-fe573894dbe@google.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-Rspamd-Queue-Id: 5464140019 X-Rspam-User: X-Stat-Signature: z8k1eu7ru7bwqjahgut1rcsaqh6a97cd X-Rspamd-Server: rspam03 X-HE-Tag: 1685119736-221083 X-HE-Meta: U2FsdGVkX1/MYlNDU/KiJ/X/PT8MHL0bHEmiIbY4c1/+631PsKc9gcfoKl0/qN9dulM0mksDl2Jv9r5OHZ+mKqHVHLH5/Chmv7Ebiq1LMKYKQUKmtFN46EJblxRPGE0VvvDe9dZSUSj4iVmZVpzkCAMEozODvY0KDAkJX0ZOBoERSb5VKANjlp8ilVKmCuo31sEUC68LMJcJ/BeVSpprIWCCZtXRyDtavnBHvX9gMkKQuDmnaG7HkuLEjpe2+Y+/fE2V5Q7j3Q3Dgut3gl7IptEgYk0CFuuQddM299jtalyjAUPmPi3wM06N/u65IJF+rs0gshm/CmMDqmru4hTNwj9YcpXmITolWMKk/IQM/RLMoqxKo8vu22FxNtQu/wdQTQDGALQHB4teE6QoB9HEszGq0BKHRPfDhA2/Mmw3YHFKl/lz5RybEJeNrbm/0EhIv8WKkbIz2IZ58stjYxZPZIVuN2/UrHE8ErNF1LVI7/w9Ahw008wpFZylrMrGCnEKXNa9RbiqQQUiZo7RsDblcigFayzR4lclLVBKEr6mhWcM9PRq6B0qaEjgAeGECw3/hQ+loymrHf4JjmaB0cdolDKAXOuazhmWWgK2lILajsXpzO0m4Ll7/7QMy2sRsXDweChNuTnPA5VlnlIhI4mGEuqDwWDCdZlqikk8sECs4GiftTLw5HDHxf17iQo/8qULRZCeKdymHJM9MePXWalBu4P60JpGRe0+xgQeJ4VHUJE0XmZ3/BXSsWI+5DfDMbTHletE5SCb7rAMWo2g55SOxNuRczqj3LPfHmlxVLFcNbXqhLVswUFII/CmomfRP2SNIpsH/a7K3rn9fdDSGXwLQNomKC3EmVF4zej+lO5vC9BpB10Uzk5DnmIoP2htFeLYD7qL1WDZAaKXC6aoeHPx7mbm1r3tj8Hcw4w5VwtWVTv9ruyGI7ADBywJ6b4FK6r2C6pa8nIQbWYTdgFFzoe 3bc6cxKj CyIaHjovvGrASsovNufJJwFIW84SF7jw8TXgK5v20vx6D6Z1WwPU3ut9vLu/8aIJDPVTxM3FQhFCBKJd/coKoXsnPquM/nciMpXok+q22sEUl+WQEEEmn+ZXSPiPMJLdwGCzXfbNlXcQuVun2drNZxacgU5Rv0iT9VlPZaKbB7KbE/q1XIIbySggohN5NoAAqxAIcMfoSOA+i/TXi7Tk6cOWobmGC/K6ZUcMNCXvzNAkGRIDO4Ig7/un7JMAcQrqHANKTsVXXo9g73/R/Lpf0KF1lnAf1vhyiuDTtaip/OXj2/wGrmAd7gl7dtHzSKHFdC9jKbwbi1JWsE3EEwAStpHq7DD6Obb5/kq/U4FBr3s3Cxcbl2aaOPCrmzEnPjHtEHSzVYlhCHDYmP65BHRcQcv2aLUWYnjO6IIqc4eAntg8CKtXN8kZOTGJyHQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, May 25, 2023 at 03:35:01PM -0700, Hugh Dickins wrote: > On Wed, 24 May 2023, Peter Xu wrote: > > On Sun, May 21, 2023 at 09:49:45PM -0700, Hugh Dickins wrote: > > > Use pmdp_get_lockless() in preference to READ_ONCE(*pmdp), to get a more > > > reliable result with PAE (or READ_ONCE as before without PAE); and remove > > > the unnecessary extra barrier()s which got left behind in its callers. > > > > Pure question: does it mean that some of below path (missing barrier() > > ones) could have problem when CONFIG_PAE, hence this can be seen as a > > (potential) bug fix? > > I don't think so; or at least, I am not claiming that this fixes any. > > It really depends on what use is made of the pmdval afterwards, and > I've not checked through them. The READ_ONCE()s which were there, > were good enough to make sure that the compiler did not reevaluate > the pmdval later on, with perhaps a confusingly different result. > > But, at least in the x86 PAE case, they were not good enough to ensure > that the two halves of the entry match up; and, sad to say, nor is that > absolutely guaranteed by these conversions to pmdp_get_lockless() - > because of the "HOWEVER" below. PeterZ's comments in linux/pgtable.h > are well worth reading through. Yes exactly - that's one major thing of my confusion on using {ptep|pmdp}_get_lockless(). In irqoff ctx, AFAICT we can see a totally messed up pte/pmd with present bit set if extremely unlucky. E.g. it can race with something like "DONTNEED (contains tlbflush) then a POPULATE_WRITE" so we can have "present -> present" conversion of pte when reading, so we can read half pfn1 and then the other half pfn2. The other confusing thing on this _lockless trick on PAE is, I think it _might_ go wrong with devmap.. The problem is here we assumed even if high & low may not match, we still can rely on most pte/pmd checks are done only on low bits (except _none() check) to guarantee at least the checks are still atomic on low bits. But it seems to me it's not true anymore if with pmd_trans_huge() after devmap introduced, e.g.: static inline int pmd_trans_huge(pmd_t pmd) { return (pmd_val(pmd) & (_PAGE_PSE|_PAGE_DEVMAP)) == _PAGE_PSE; } #define _PAGE_PSE (_AT(pteval_t, 1) << _PAGE_BIT_PSE) #define _PAGE_BIT_PSE 7 /* 4 MB (or 2MB) page */ #define _PAGE_DEVMAP (_AT(u64, 1) << _PAGE_BIT_DEVMAP) #define _PAGE_BIT_DEVMAP _PAGE_BIT_SOFTW4 #define _PAGE_BIT_SOFTW4 58 /* available for programmer */ So after devmap with CONFIG_PAE, pmd_trans_huge() checks more than low bits but also high bits. I didn't go further to check whether there can be any real issue but IIUC that's not expected when the low/high trick introduced (originally introduced in commit e585513b76f7b05d sololy for x86 PAE fast-gup only). > > You might question why I made these changes at all: some days > I question them too. Better though imperfect? Or deceptive? I think it's probably a separate topic to address in all cases, so I think this patch still make it slightly better on barrier() which I agree: Acked-by: Peter Xu Thanks, -- Peter Xu