From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05FBFC02182 for ; Thu, 23 Jan 2025 15:08:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 572C56B007B; Thu, 23 Jan 2025 10:08:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4FA386B0083; Thu, 23 Jan 2025 10:08:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3E9B76B0085; Thu, 23 Jan 2025 10:08:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 239EC6B007B for ; Thu, 23 Jan 2025 10:08:23 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id B92B21410E5 for ; Thu, 23 Jan 2025 15:08:22 +0000 (UTC) X-FDA: 83039047644.03.B551589 Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) by imf19.hostedemail.com (Postfix) with ESMTP id 793581A0021 for ; Thu, 23 Jan 2025 15:08:20 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=ffwll.ch header.s=google header.b=BW8FOv6R; spf=none (imf19.hostedemail.com: domain of simona.vetter@ffwll.ch has no SPF policy when checking 209.85.128.54) smtp.mailfrom=simona.vetter@ffwll.ch; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737644900; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vnvF918MExGqpW1JhPiFqDMhjdc3YrQOLeaZRhdmR1k=; b=j37wzuWyaEKOTAylaAuUPTgAQX67ReTm+tPg8gPvFR6bb4IBdC7DZKE3AFnnWQTkGVmjRC Mg3NT7Y82W0k3EpmhImUGJW1HYrWN7/MJvaA8kaKaX0oYmWkeaynU4yaWIMl0plnuRLgVM ipe4+vV0WP11eQnhh1i8qnYlBbJqrwI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737644900; a=rsa-sha256; cv=none; b=8pB5mR+FbbCVXjB5CDIGsSHdzxK6vnedLu2vkR6EGw6DxyTty9Wvnb/2U8D05itYfnAEu7 osoJxV1pEqhalVhS22t4YQtuULM/kIkjQ180QnbFvO32HfTCr94uxVez5utHC09MMG+7LF j1nTRv5dXvMWCvFhdZg9szWvhJ87068= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=ffwll.ch header.s=google header.b=BW8FOv6R; spf=none (imf19.hostedemail.com: domain of simona.vetter@ffwll.ch has no SPF policy when checking 209.85.128.54) smtp.mailfrom=simona.vetter@ffwll.ch; dmarc=none Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-4361f796586so10980975e9.3 for ; Thu, 23 Jan 2025 07:08:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; t=1737644899; x=1738249699; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date:from:to:cc :subject:date:message-id:reply-to; bh=vnvF918MExGqpW1JhPiFqDMhjdc3YrQOLeaZRhdmR1k=; b=BW8FOv6Ra90iRWC6Qd7Z2Zm+ADfG7PaIbnGqZdoD0fkdmmxsSPuXchWAzXD1KqyZKH itzpzYmv50uJHHX+0+Kkvdn/6hMof61YnQpA+Vc9GwSr/HcgFgeY4oLf/e1Pl0ykYK7M FNPb0Vgnpc01jrLKoW9P/KhU0hN50Qnz3p3Mk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737644899; x=1738249699; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=vnvF918MExGqpW1JhPiFqDMhjdc3YrQOLeaZRhdmR1k=; b=EGNuXOXEbSLVKMQ0fq6w4r1GolYeo9kl2Zc0+mAVTkWkSMEkgRylKHWpDty0ZZ2z1q +eawxg3MnU3tCOuGRRQreWMHV00IFcsslEViefqcmzOfWNPdVCOWUjWl3JesL6YFRD+t YjRgyOkLzPu9xwNYubhDV9vXe1KAglgcOKqs+8pM/MZvCG+OOV+i+cpRyZazmZ0kysKW qR0zvl8Hg04rDaPXgVXOlO9e37bbnevSsFWR9rvq1du36oi1wFe2ue05o62ifcQq26cn jhEHLH3v2Hb3WEAR7mu4x2oM2rJY0b4WZwoBdHCA8Usjt1+3RHe/ae+0UANmMxvEV1nM /w7A== X-Gm-Message-State: AOJu0Yyt13dMjkIiA5WBxYttTmuVEPacdDa8im94lMoGgE5wftaRshsk JCwWqJOCzFpAFwc6/dcKN+AUMpLuj+dO40PYRpfssbZVJgzamAsEXb8n5BIMZww= X-Gm-Gg: ASbGncuwaBD68p48hjHMaJ70QBUGwKRU3srv08wEXv0sF+nb/liVpcY7UKQD+l72AYC q+bmTktacTLj7LkacxSr44+pT5enLU3xMamiKYippJvbnr99V2//m+Cd8SDb/anvGv5iT4tKILf YlxaXPWs6/YivREQ1zhgRrXZniOkv8F9pOtPAnZEiBYLQ2zXfWlqTwF4gjdE2e6O6EYKSeVNFvL 6l6CNOCMgflMly6rBonKpOBWQ8MlBikUUoTMPUH4CWxYMo32dPceomPzTPG4HL/XYLavWbdGVrP THId/EchIn45E8Uq X-Google-Smtp-Source: AGHT+IFe5bvrufD5E2WCU0xgHozwkEfe52bL8rsuxcBOdIdAdnpMPzPUDgbaIWl03oauCdgHt4m7yA== X-Received: by 2002:a05:600c:4e06:b0:434:fddf:5c0c with SMTP id 5b1f17b1804b1-438913c60e1mr250217565e9.4.1737644898467; Thu, 23 Jan 2025 07:08:18 -0800 (PST) Received: from phenom.ffwll.local ([2a02:168:57f4:0:5485:d4b2:c087:b497]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-438b31ae155sm64915825e9.20.2025.01.23.07.08.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jan 2025 07:08:18 -0800 (PST) Date: Thu, 23 Jan 2025 16:08:16 +0100 From: Simona Vetter To: David Hildenbrand Cc: "linux-mm@kvack.org" , John Hubbard , nouveau@lists.freedesktop.org, Jason Gunthorpe , Alistair Popple , DRI Development , Karol Herbst , Lyude Paul , Danilo Krummrich Subject: Re: [Question] Are "device exclusive non-swap entries" / "SVM atomics in Nouveau" still getting used in practice? Message-ID: Mail-Followup-To: David Hildenbrand , "linux-mm@kvack.org" , John Hubbard , nouveau@lists.freedesktop.org, Jason Gunthorpe , Alistair Popple , DRI Development , Karol Herbst , Lyude Paul , Danilo Krummrich References: <346518a4-a090-4eaa-bc04-634388fd4ca3@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <346518a4-a090-4eaa-bc04-634388fd4ca3@redhat.com> X-Operating-System: Linux phenom 6.12.3-amd64 X-Stat-Signature: 3dcf73obtosfeyxwpwqnd9q5jynnboeq X-Rspamd-Queue-Id: 793581A0021 X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1737644900-336404 X-HE-Meta: U2FsdGVkX1/QXIhCRwmvA6dh37nodF+TRJvkn1MmSK9oC99wci62nhTLlVnmOgTWzm96qQHhr6ZsmqKXLY9O/688LUkkkb3DvBgYqZQqlZ/spEvg4mAXR2abA0b05CTLm5O/7KPQMiQi926UPL7CmkKrYzuDXvDCSaxebdOwx/yca0mg4fhSlYyb7/TIzym3jTmgGEMCzcGWYExL0OBLMY3mLUP4sPuqWEiKPDfCzpQvxp7mou792SDst4xhbuvqOi7DZxvJuI0DHolXoCOrWtzKWIqhS+lthlMlvRIluXcw6Xg6L2bpHYS6eTYOVNkaeYeH1oxjIS4EE17BJDy6/wZ5n7L6AU+/+Ua01E6ByRQFEF10vI9bveKchSrq0YQZ70024djIW+Wng+J6kbyMsLfK/sV+fY46T4kdvW3cicr5eu9yVtIwIg7Uf5i7eZqYEABj8bibOZLqJp5/S/cnhc+1OIthqOvkinM5+izre/9ya/4F39l7JLg0iP0RHW2fybsuSm9qvs57+7kyX+kacC6e2J+APY+4WS0/2/8LPh9hlc+I0q5jpArLz46/Ywe2WrLEVnUpGnAqbt3DkZbW+G7NDuwoF3B7a0Abrrw9MJ2Nh+IyJE302PQYs8/gnLTnkFIT7/C/DcrD+BReFew+rMD9lTPnZMir/m6tc5ygRas+/VH2R1S9bqO1P1UKlEfs7qfGxERwZT1LViYhU9oNIh+jW+i7PAL0WhjmskZATzwLkNBMgID1fRH28x+PW9EpLw5bK4sy/xzUCLhL0tZxy3URiXcq3MC2DvctBuPU9AUVK7DkwVNp9rAKngw29F1AqhgjicTSLGF4PfCAyW1xqoZdnhaxcWohON2O/2qrcjMrZnt6cqQBs+R0toM9GzTzuFuq6x7+yUM5f5TdGqwaMCRmu0+uj20TIp8USc0zAfHqBqxWe9JCWnGuB1J7qbPHFz1O0Zk9BkzHCrPongF c4mWnB0G 8I/gVEgY4afaPiYJ5K92QLhUf/O0whd7TPaIbJhnF73PQ/POxkm/eYfsGsVRB+HKuDXmpfkRzTtYiFZBhj/TaKpPZwLo+HoXyzvbq89EtiYF3UavRuHLCj6XBH2mnNk83uuAjAZMiOlLqUBTfiXU53Eyg1jj7rUj41Y9GPv5Lnjwvt454cYxHY8A3dpNQKHStakJLpPtf91rN+HAQOsSng43etSodgQD78hB669tL1BTTBJL0vswkqDenbLowU1iBuqqW/tCV9ITCLyLyi4jws3IA/u6Y2d8Iw3E5V8srdnvnd7GBR7I9hRpR3g/1nTAznAzPsnxndbDjtm4Hr+DLJ9/x9c1YPP6dwD78N5wHgX/zPV75MZGhrplBcJaUiss+4U4IZpYD0QkdKhREmxX4J+ejbg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.111169, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jan 23, 2025 at 11:20:37AM +0100, David Hildenbrand wrote: > Hi, > > I keep finding issues in our implementation of "device exclusive non-swap > entries", and the way it messes with mapcounts is disgusting. > > As a reminder, what we do here is to replace a PTE pointing to an anonymous > page by a "device exclusive non-swap entry". > > As long as the original PTE is in place, the only CPU can access it, as soon > as the "device exclusive non-swap entry" is in place, only the device can > access it. Conversion back and forth is triggered by CPU / device faults. > > I have fixes/reworks/simplifications for most things, but as there is only a > "real" single user in-tree of make_device_exclusive(): > > drivers/gpu/drm/nouveau/nouveau_svm.c > > to "support SVM atomics in Nouveau [1]" > > naturally I am wondering: is this still a thing on actual hardware, or is it > already stale on recent hardware and not really required anymore? > > > [1] https://lore.kernel.org/linux-kernel//6621654.gmDyfcmpjF@nvdebian/T/ As long as you don't have a coherent interconnect it's needed. On intel discrete device atomics require device memory, so they need full hmm migration (and hence wont use this function even once we land intel gpu svm code in upstream). On integrated the gpu is tied into the coherency fabric, so there it's not needed. I think the more fundamental question with both this function here and with forced migration to device memory is that there's no guarantee it will work out. At least that's my understanding. And for this gpu device atomics without coherent interconnect idea to work, we'd need to be able to guarantee that we can make any page device exclusive. So from my side I have some pretty big question marks on this entire thing overall. Cheers, Sima -- Simona Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch