From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB1CEC3DA61 for ; Wed, 24 Jul 2024 12:11:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 407E16B008A; Wed, 24 Jul 2024 08:11:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3B7576B008C; Wed, 24 Jul 2024 08:11:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 27F376B0093; Wed, 24 Jul 2024 08:11:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 07A1E6B008A for ; Wed, 24 Jul 2024 08:11:06 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 7B4E91408DF for ; Wed, 24 Jul 2024 12:11:05 +0000 (UTC) X-FDA: 82374530490.17.F82DF70 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf14.hostedemail.com (Postfix) with ESMTP id BB72C100018 for ; Wed, 24 Jul 2024 12:11:02 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=k+DNDFzd; spf=pass (imf14.hostedemail.com: domain of ardb@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=ardb@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721823015; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1wuR3K464iGX1BnZY20sCoE+M156R+Mh/IVZgJpVHtY=; b=f3sSonaY8AfHxIwAfazsTwT5fgGkELy5wud78H637AIwM+pGeeeyto9G5nOpYb0J11NMZA CIO556rcrfYCCWfNhpMYc+IY58TrveER/dgYdBKU1L1ULRpoa06NmtFqDEHLuPwKffJSkH jSsqSIlDoEckouS2qp6vCshSZSkLl/0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721823015; a=rsa-sha256; cv=none; b=DqJaV0x6E1hsSLMuKDguZrYVQAU1b6S/IKibloC33b/bpjWfRU/5d+A9Re5kwkhUkQfF+k 6zuEnRxEnaM8Vbxi1qRutIAJ1xgHbEGSwOGMgbmwFUxL0VPMnwSdDKBuB8A7R2BHJeRkuP qRyqd2STLr1mugs2nhR3jDZcFMvwT4s= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=k+DNDFzd; spf=pass (imf14.hostedemail.com: domain of ardb@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=ardb@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id C3517CE0FA0 for ; Wed, 24 Jul 2024 12:10:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1F189C4AF0E for ; Wed, 24 Jul 2024 12:10:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1721823058; bh=csg4Z3+s45IbHVOgFZkpPLJvQyuq8WsMMCvwBc1Qr20=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=k+DNDFzdWGuYzsis1lCTinqYPExVgiTnNLPkL1Es88n0DD8LHNGjcisI6jPhBWbT9 TBpQlbt03WDIwlW29BuIiQ3V04c5EZCWeilT2zD20//NjtF9cW5Hgc5T0dRhfQ6MIi HzZF8K/15FbRAW8Me0kG5lh6jOwy519JMEx5OsyNoBjtVffTlSDaBySZOXEvcVUcWM 54SSsDOwmIdjQDOjB5UQBL6dQnIb/GvKyaJ14FVwD2wa2Q48khHpBlyDboaC4zKkNH XSiZ8xtpfnJCDDMMbHZqk7BWhwn3dYaOJWmXtwBlIwHf5XQXUaiwudVi1W1DEQgLC/ loU6Ryuv0cMkQ== Received: by mail-lf1-f53.google.com with SMTP id 2adb3069b0e04-52f04b4abdcso3406927e87.2 for ; Wed, 24 Jul 2024 05:10:58 -0700 (PDT) X-Forwarded-Encrypted: i=1; AJvYcCWPzJIFd7+sJOyNhn8aby7GMQbGvTQIcfaJkEUpCUgyrEbadb5CrzIF5pq5WdUm24fRNFq79TPQnbJlJHdffeLlaN0= X-Gm-Message-State: AOJu0YxEMWiqu9u4/L8dsrgGJqJItyBnW/qoKEYeWgpcjs5X6hxKgTZn 6mff59FUI8PjDtWj0bhaMtZZdbJ/8nMPRcrRfML9Yf1XYDRb++unECDBaLM0dSuzCGMtTF57zTN /SpAvwWzl5XRosDhQq7uJS5FBBVU= X-Google-Smtp-Source: AGHT+IEv0BF/mQwbqGY48LUWq4SUUbjc31/O/yu0siZ+PJnKmYA/IJfOGhnHtWWy3JVNKSHy8ZslU4klyRnXYs1GV0E= X-Received: by 2002:a05:6512:ad4:b0:52e:9a91:bba3 with SMTP id 2adb3069b0e04-52fcf007c36mr1533119e87.15.1721823056391; Wed, 24 Jul 2024 05:10:56 -0700 (PDT) MIME-Version: 1.0 References: <50360968-13fb-4e6f-8f52-1725b3177215@asahilina.net> <20240718131428.GA21243@willie-the-truck> <20240723145214.GA26403@willie-the-truck> <20240723160543.GA26546@willie-the-truck> <20240724113340.GA27474@willie-the-truck> In-Reply-To: <20240724113340.GA27474@willie-the-truck> From: Ard Biesheuvel Date: Wed, 24 Jul 2024 14:10:45 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: LPA2 on non-LPA2 hardware broken with 16K pages To: Will Deacon Cc: Asahi Lina , linux-mm@kvack.org, linux-kernel@vger.kernel.org, asahi@lists.linux.dev, linux-arm-kernel@lists.infradead.org, Catalin Marinas , ryan.roberts@arm.com, mark.rutland@arm.com Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: ufks4d9bpzbkkb1k3aid8rais6g63djr X-Rspamd-Queue-Id: BB72C100018 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1721823062-427728 X-HE-Meta: U2FsdGVkX1+2ueqnMbUzz9WANHZ8WyhOdgtdikojJsz4siFtbs9f4/wIQCUmuiASwkMrC1ev99VSNfUKdqK8qlc1qs8RTGaOc2zNITedak8VILhSQWeSQwx0zJwtfFMW3RRGnYaqfu1mUJgin0//zDswNJqKvK7bQah4MdyPp9oZq4VOn7A2DzIpGEowhstFTgQgc8ENQmKZ81KV9JvIl1FPn6f3jfl6JkMBwJXgLMi+JbKJjpUBgMfVAFoL34IQ+WsD0iEA9Qw8nqriTOy5uDKOhKEYfuVGJirBRzLvaqPg4iI9nAcn1/vx8IwNhpETOuEmkO8sbzhlkkLuIUQN8cOnkg/Yl+kZxiCfIlo3iwgJ9R2Wvdna0TaB5fx0OOlmXiEYy+edt5bbTNK2xtokSCyOja6G6OasesUP1+5gozWT5V/ZwTK9H6ekXLZOv5OJpwmlbNieWCdy3pEj7xsms+xqekPAbBAsFc24QJfhJqdKap6BQft5TdllSLk+xUay0LhkebsJ3TjHRhC8xGB2yh/hX1XaqZ5wpeocLheUrRX09BQd17d7tMH9lJj3PFS2zZoRIV76RCRMfzjYZAeUqKk4GC+svp6WslVg5/bExCvWAzDqVs3Pu0cqfFarQjNUytvYZQb2JKojOHxlXOn/TJr/MLR+DeVu3W5j9zhU2VmfglLPNi6pPzrDnwvF5OImus+dCpq+7IAjgBSRmmaW3sh/e2TSZeNs063Rg0nZ7LotLsgBFNXhDQ94mYGg17MOsiLBqffAbJEUhYP3Hx/bWqQuxXnAgTATmEmX5vdqPY7I9m/LIjZd2akqq1XeSy+rlfcEjW9jo17o+EzAKlmiiBWrpgntbaDO4wTUPk15d7zI9pN9ohMLu6+sX7xn8oJ9p+SgpJ/P6owTPFmQ02IP1xfQIBQ/TIF9yDi6IB7IQh7tOkExKpHw4rOaSnKbJ62RUlcZgPIHTTIK8bedY9Q BKsB0QOd 1xV+w28nGp5vm6XS6RZAX0R53kjsNwf13iKHEdjmGc9oMjOOcMBf+3mpGx6v8EpV1dMUd55q0XVhGCxkqfXN1Qlztq9rgQdNo7LjhS11ktHR67kqC2Jhlc62hfdgxiTbmraMpzu4OPb6Z7pHfif2sCOdLxYcWwgV9Z/ugMsNv/RKAPfcGfAZyRaZSwqmUFCehLV5bvEn31P9tZc6LCsdL+zf41uhsXEpT5vVbJ9kGRsOx6PTYcKex4ct2uOm8BcNoHgFZTD5iNB/4QWuYy9fUeGA9Ehc3EPj19Vk36MW48c97RBPLNdMNhYLDMQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, 24 Jul 2024 at 13:33, Will Deacon wrote: > > On Tue, Jul 23, 2024 at 06:28:16PM +0200, Ard Biesheuvel wrote: > > On Tue, 23 Jul 2024 at 18:05, Will Deacon wrote: > > > > > > On Tue, Jul 23, 2024 at 05:02:15PM +0200, Ard Biesheuvel wrote: > > > > On Tue, 23 Jul 2024 at 16:52, Will Deacon wrote: > > > > > On Fri, Jul 19, 2024 at 11:02:29AM -0700, Ard Biesheuvel wrote: > > ... > > > > > > > > > > > > We might add > > > > > > > > > > > > if (pgtable_l4_enabled()) > > > > > > pgdp = &pgd; > > > > > > > > > > > > here to preserve the existing 'lockless' behavior when PUDs are not > > > > > > folded. > > > > > > > > > > The code still needs to be 'lockless' for the 5-level case, so I don't > > > > > think this is necessary. > > > > > > > > The 5-level case is never handled here. > > > > > > Urgh, yes, sorry. I've done a fantasticly bad job of explaining myself. > > > > > > > There is the 3-level case, where the runtime PUD folding needs the > > > > actual address in order to recalculate the descriptor address using > > > > the correct shift. In this case, we don't dereference the pointer > > > > anyway so the 'lockless' thing doesn't matter (afaict) > > > > > > > > In the 4-level case, we want to preserve the original behavior, where > > > > pgd is not reloaded from pgdp. Setting pgdp to &pgd achieves that. > > > > > > Right. What I'm trying to get at is the case where we have folding. For > > > example, with my patch applied, if we have 3 levels then the lockless > > > GUP walk looks like: > > > > > > > > > pgd_t pgd = READ_ONCE(*pgdp); > > > > > > p4dp = p4d_offset_lockless(pgdp, pgd, addr); > > > => Returns pgdp > > > p4d_t p4d = READ_ONCE(*p4dp); > > > > > > pudp = pud_offset_lockless(p4dp, p4d, addr); > > > => Returns &p4d, which is again the pgdp > > > pud_t pud = READ_ONCE(*pudp); > > > > > > > > > So here we're reloading the same pointer multiple times and my argument > > > is that if we need to add logic to avoid this for the > > > pgtable_l4_enabled() case, then we have bigger problems. > > > > > > > The 3-level case is not relevant here. My suggestion only affects the > > 4-level case: > > That's exactly what I'm uneasy about :/ > Right. > > if (pgtable_l4_enabled()) > > pgdp = &pgd; > > > > which prevents us from evaluating *pgdp twice, which seems to me to be > > the reason these routines exist in the first place. Given that the > > 3-level runtime-folded case is the one we are trying to fix here, I'd > > argue that keeping the 4-level case the same as before is important. > > I think consistency between 4-level and 3-level is far more important. > Adding code to avoid reloading the entry for one specific case (the > pgtable_l4_enabled() case), whilst requiring other cases (e.g. the > 3-level compile-time folded case) to reload from the pointer is > inconsistent. Either they both need it or neither of them need it, no? > The thing to keep in mind here is that the path via p4d_to_folded_pud() does not dereference the same pointer either. It just converts a p4d_t* to a pud_t* by deriving the page tables address from the p4d_t*, and applying the PUD_SHIFT rather than the P4D_shift which was applied one level up. So a) it does not dereference, and b) it refers to a different entry so the prior dereference loaded the wrong entry. OTOH, the 4-level path is not only used by 16k+lpa2 (or 16k/48bits), it is also used by 4k/48bits where pgtable_l4_enabled() is a compile-time constant TRUE, and this case will no longer be 'lockless' as before. However, if I am understanding you correctly, you are saying that a) p4d_offset_lockless() for <5 levels cannot race in the way that these macros are intended to address, as the folding implies that the next-level load is in reality a reload of the same entry, and so we will be using the latest value b) reloading the same value is not an issue because this is not a performance optimization but a concurrency/correctness thing. I suppose it would be good to clarify this in a comment, as doing the reload in the implementation of a helper that exists to omit it looks rather dodgy. But I agree with your analysis, and the additional bits I suggested are not needed.