From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3777C0218F for ; Fri, 31 Jan 2025 17:06:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3FA8E6B0093; Fri, 31 Jan 2025 12:06:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3AA316B0095; Fri, 31 Jan 2025 12:06:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 224F26B0096; Fri, 31 Jan 2025 12:06:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id ABB5A6B0093 for ; Fri, 31 Jan 2025 12:06:06 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id AE744160E9F for ; Fri, 31 Jan 2025 17:06:05 +0000 (UTC) X-FDA: 83068374690.17.4F10C46 Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) by imf09.hostedemail.com (Postfix) with ESMTP id 26C0214000D for ; Fri, 31 Jan 2025 17:06:02 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=ffwll.ch header.s=google header.b=Lfcf4VBI; spf=none (imf09.hostedemail.com: domain of simona.vetter@ffwll.ch has no SPF policy when checking 209.85.128.45) smtp.mailfrom=simona.vetter@ffwll.ch; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738343163; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=G+G4waO4iWi5ppsP27MVzR5EHmBlY2mxPNF1CuedbFM=; b=oYtmoHJpbTklLoqMyGubQ+YkXQbYlJ4eZ7+l0SYgOaBPp36pAEawO0s6tGrvWX9mf/o9ct WcccqwNByek/CCjblpz9O7o0ecep9/yqgc0EoulqOTTAisJA0BppnxK8U1+7RxyL46isbz aFFfd/tsvnHIT00gSFCmRbJf4LHQp6I= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=ffwll.ch header.s=google header.b=Lfcf4VBI; spf=none (imf09.hostedemail.com: domain of simona.vetter@ffwll.ch has no SPF policy when checking 209.85.128.45) smtp.mailfrom=simona.vetter@ffwll.ch; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738343163; a=rsa-sha256; cv=none; b=rU/g19A0+nG4muFJP6GYa/etvhOGO16N4eQN3RB7Gx8HGAWRD3FgXvaGwdDZxRSpVUJpgJ 9EgAeuqdyXmytLnIqneSRm9j6v3TcN1svVn3NZm8AP8fHegmde7R6Vqws21kJNfbdCF9pd lYitMrCl6jgoXCr5A2T5Zclq2iyx1f4= Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-43626213fffso21709815e9.1 for ; Fri, 31 Jan 2025 09:06:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; t=1738343161; x=1738947961; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date:from:to:cc :subject:date:message-id:reply-to; bh=G+G4waO4iWi5ppsP27MVzR5EHmBlY2mxPNF1CuedbFM=; b=Lfcf4VBIAT6KSc8kYs8cvXp1biuNG6fSzZAxQ1P5fA3Z1ES2PNCGHy7AcyugrlZhiI TUIYZp55zC8za6XeLJoWrwZ5M40z7H4lo+/rDVTBopd9YDTRgLDspKTJ4Mwuy41+zEeP 66QmMfEVWznRADp3sa1eoLxIz+9T2dQLbR/O4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738343161; x=1738947961; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=G+G4waO4iWi5ppsP27MVzR5EHmBlY2mxPNF1CuedbFM=; b=VQq7wjzivDR+b61EPTvGQXbE8Sd8B9zP8/2RyIcV6mcl4c9M0qNSH256yCi/beBHHp rfG9LzKX88JNFHLW99HwgPymgW3IyxUFF2VJZCpmjCkY9FWnfHxsWTXKD3U4qn74wbpJ RJiTd9BTSP2bW84eCPrxM+KqWwp8HpbtRFnrE4c0R1CMAGPzOnGpiFXmQsvHNZYXmtIg CBcYpOoHsMoRvKK1iMcDLii6z3g0l5sjXUPK5XFKLGvSmhO3LK00PP2uU7kPd23pht1i 5FyJblB5+ittnaaMh7ft7D94VeGyydNgNijqLHntqhb7YE6qDz5aAjUNp6pQPXMeej94 n+UQ== X-Forwarded-Encrypted: i=1; AJvYcCWchXbvd+0pc4y+H0TWggZ18aTdFs3meJVHC/e22Kogl7sVXHpxZTcZRMCaawo8h9mqsYUeOplJOw==@kvack.org X-Gm-Message-State: AOJu0Yxyqg09Um5v4R1Mzggr0OnBp9ZZ/C/y5AcS31R7h7MXpKFtHpKk EpLd7p9Q351DQEb9KGfcYz91kOm0ntvEzB3CB4ULjus8gBzy2M635glaKJpa+q4= X-Gm-Gg: ASbGncugLaIfLNJFH8E71Es1SFJkyyJVoR+C6eHhH/n8MRUG+TsC3aNypMF34Kj1EvW o462dRNLv6f/YoUm3K1xQLAAofqTLLcT/yQW4b1xV99WwGpQTOtHyFR21OixDZFOuoXT7ziOQB+ kf2N+vJzgGIOdaFzxnEYQc5V1Oac7eOt2MYKIND1pRzyKmkCi0/guqJNeOp0JqT+hTNgDdrRlRp jXoR7CjHAnux5t9GxfDjKuQZy2H7Dcn7o1QVfTKy3rFt72ALSwnHiBPtKchcbHjFnW6j4qxfg4M iJDtEY1ARpakkdJu3LWPgvx2zI4= X-Google-Smtp-Source: AGHT+IHqJ7SXWObwA7TE2hNt8h99Mz+Rh4lA4VBEyCv4JI/V7wPQqHQuFCcfla/1mLYWhuuiSze4/A== X-Received: by 2002:a05:600c:83c6:b0:434:f2af:6e74 with SMTP id 5b1f17b1804b1-438e170e7b5mr73728995e9.15.1738343161267; Fri, 31 Jan 2025 09:06:01 -0800 (PST) Received: from phenom.ffwll.local ([2a02:168:57f4:0:5485:d4b2:c087:b497]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-438e23e5e53sm60120095e9.17.2025.01.31.09.05.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Jan 2025 09:06:00 -0800 (PST) Date: Fri, 31 Jan 2025 18:05:58 +0100 From: Simona Vetter To: David Hildenbrand Cc: Alistair Popple , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-mm@kvack.org, nouveau@lists.freedesktop.org, Andrew Morton , =?iso-8859-1?B?Suly9G1l?= Glisse , Jonathan Corbet , Alex Shi , Yanteng Si , Karol Herbst , Lyude Paul , Danilo Krummrich , David Airlie , Simona Vetter , "Liam R. Howlett" , Lorenzo Stoakes , Vlastimil Babka , Jann Horn , Pasha Tatashin , Peter Xu , Jason Gunthorpe Subject: Re: [PATCH v1 05/12] mm/memory: detect writability in restore_exclusive_pte() through can_change_pte_writable() Message-ID: Mail-Followup-To: David Hildenbrand , Alistair Popple , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, dri-devel@lists.freedesktop.org, linux-mm@kvack.org, nouveau@lists.freedesktop.org, Andrew Morton , =?iso-8859-1?B?Suly9G1l?= Glisse , Jonathan Corbet , Alex Shi , Yanteng Si , Karol Herbst , Lyude Paul , Danilo Krummrich , David Airlie , Simona Vetter , "Liam R. Howlett" , Lorenzo Stoakes , Vlastimil Babka , Jann Horn , Pasha Tatashin , Peter Xu , Jason Gunthorpe References: <20250129115411.2077152-1-david@redhat.com> <20250129115411.2077152-6-david@redhat.com> <2670f65f-e973-483e-aed6-526d00125ad7@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: Linux phenom 6.12.11-amd64 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 26C0214000D X-Stat-Signature: hw8gx7ybustyfpkoj9qwtgam5n3hugbc X-Rspam-User: X-HE-Tag: 1738343162-579896 X-HE-Meta: U2FsdGVkX1/egXDzqUt2wrVwGdsuS6oEXOCWekQ8zfljNGLcbaQk8So16K6mQVjjwWxqjQVKSEZtBOjbT61wS/so+Z/cm4Yr48eu/tdLWEzUmeMMcNEJm+n9a5gJw1UkV659OaLx0VQy/ukEFJRP7Wck8XIuHVxK6gtqsNKRU/74JygsEF3JAsmyNDZflI0JvXAmSiS/jREk5kasFs0Nn0HSGBp82P2d/APETTJEUrFcQnfd0837HkysNzIghCmoU0enKjyc/aenDDbKnX0OA4Kc4vt/6sDJMv8f8+qMm8Tog6TYPhKxh9pqQsQ/RJk4n7R5lJjBKBxm7dhMhOnQaPcN9VZr1evW+OlIZSMC5w2xA8ZAyU3s3IQTr6PCvH+28Z7Xrd3LGGci7ZA1wjNhY7S6q3zr6Z6wQrpR8SxGJRNccy6dyjFEWEIuHLyWGM/c5yoXHfNaBCadliz8djzq7cqnlZTn2kWXWuEKyXgkhbe2UDfT2QiXfrJnVulgix1i56GPuuatAcRpKZcnlnWFTo1xfrgZJI6ZQMha+XqZ56XMM9f3xqnRj1lvV6s31kt66J7ysX0Iflw931zCBBOcak0t9Mot/GVNQhOLEYw2XwH3+qzSG4N5yBLEvjXZ1pZO6pBFsaQGkNwj9Rua1ztW0PrjK0xxXK3rlNjzID5BucAwJpaMu6jJBbjh6ZdzmUIQDK+TZQHNpRyAG4BaeOzMlln68AVQT53/6GsvrxFStTMKExWaL6jrSVlNvsi3Wqs3cOVOW3t+jpVUdnl2XlLRcap4TAK3PsMWBzwr65vuc+g2q3K4zu3FeAB6KCD2FgxtVS0Xi6GsbQGI2HfHGuVtr8a+ZT/CvQ6WmSxpVKlsyc8emkCOag5XoIHsyC2BdlwYMEHj0/WtdvJiO+55pP8htJ31nVxyeJjvEq+z0zkkMZuwhWWO8Qwd/iMvw79zteue9/4BqoN75pi18dpa9Jj l/Snq2UN tCxAd8mf6iwQ/KtyAx3jVcw+VtYO7wkF815D484HROCqFK718wBBkpldG9S/ZTT88LWUR3/ab4a6dsxxrGpg0E7jqLwf33TIuVavft7+dsN3bNC6bOkIlhnk4gucovn5Czzpt0q6oOV+wIXboHspkdyMMX1uDP5bTtwrKccluWnkICUSRfepqdyzzx3ukCVgB5YgJgcATYufI8xsBxp3ylb1bFQXVAa61xD+GgWXE/Ru7lwFu1XY5fFeaF9DJb3cfCB+OMLJjxVVHjJySyf1xP1dFBZOTxGY36VdGfx/wJLA0PzrRHaTVI3kc3Oq87Vt0hqRg7Rs4ehFgFDg8YZ73QZNi5Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000077, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Jan 31, 2025 at 11:55:55AM +0100, David Hildenbrand wrote: > On 31.01.25 00:06, Alistair Popple wrote: > > On Thu, Jan 30, 2025 at 02:03:42PM +0100, Simona Vetter wrote: > > > On Thu, Jan 30, 2025 at 10:58:51AM +0100, David Hildenbrand wrote: > > > > On 30.01.25 10:51, Simona Vetter wrote: > > > > > On Wed, Jan 29, 2025 at 12:54:03PM +0100, David Hildenbrand wrote: > > > > > > Let's do it just like mprotect write-upgrade or during NUMA-hinting > > > > > > faults on PROT_NONE PTEs: detect if the PTE can be writable by using > > > > > > can_change_pte_writable(). > > > > > > > > > > > > Set the PTE only dirty if the folio is dirty: we might not > > > > > > necessarily have a write access, and setting the PTE writable doesn't > > > > > > require setting the PTE dirty. > > > > > > > > > > Not sure whether there's much difference in practice, since a device > > > > > exclusive access means a write, so the folio better be dirty (unless we > > > > > aborted halfway through). But then I couldn't find the code in nouveau to > > > > > do that, so now I'm confused. > > > > > > > > That confused me as well. Requiring the PTE to be writable does not imply > > > > that it is dirty. > > > > > > > > So something must either set the PTE or the folio dirty. > > > > > > Yeah I'm not finding that something. > > > > > > > ( In practice, most anonymous folios are dirty most of the time ... ) > > > > > > And yup that's why I think it hasn't blown up yet. > > > > > > > If we assume that "device-exclusive entries" are always dirty, then it > > > > doesn't make sense to set the folio dirty when creating device-exclusive > > > > entries. We'd always have to set the PTE dirty when restoring the exclusive > > > > pte. > > > > > > I do agree with your change, I think it's correct to put this > > > responsibility onto drivers. It's just that nouveau seems to not be > > > entirely correct. > > > > Yeah, agree it should be a driver responsibility but also can't see how nouveau > > is correct there either. I might see if I can get it to blow up... > > (in context of the rmap walkers) The question is, how do we consider > device-exclusive entries: > > (1) dirty? Not from a CPU perspective. > (2) referenced? Not from a CPU perspective. > > If the answer is always "no" to all questions, then memory notifiers must > handle it, because we'd be answering the question from the CPU point of > view. > > If the answer is always "yes", there is a problem: we can only make it > clean/young by converting it to an ordinary PTE first (requiring MMU > notifiers etc.), which makes it quite nasty. > > Mixed answers are not possible, because we don't know just from staring at > the entry. I think it's the gpu's (or whatever is using it) responsibility to update folio state while it has ptes pointing at memory. Whether that's device-exclusive system memory or device-private migrated memory. Anything else doesn't make sense to me conceptually. And I don't think we can just blindly assume even for device-exclusive mappings that they will be dirty when we convert them back to a real pte, because we might have raced trying to set up the gpu mapping and restarted before we even put the pte into place. Or maybe someone was real quick at writing it back after the gpu already dropped it's pte. I guess maybe some clear documentation in all these functions (make_device_exclusive, hmm_range_fault, migration helpers) that it's the drivers job to dirty pages correctly would help? But nouveau definitely does not look very correct here, pretty sure on that. Cheers, Sima -- Simona Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch