From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 441CEC0218A for ; Tue, 28 Jan 2025 20:15:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2C6A5280163; Tue, 28 Jan 2025 15:15:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2773D28013F; Tue, 28 Jan 2025 15:15:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 13EF8280163; Tue, 28 Jan 2025 15:15:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id EAC0C28013F for ; Tue, 28 Jan 2025 15:15:06 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 9C0EDA0479 for ; Tue, 28 Jan 2025 20:15:06 +0000 (UTC) X-FDA: 83057964612.20.ACC674A Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) by imf07.hostedemail.com (Postfix) with ESMTP id 4F0A840011 for ; Tue, 28 Jan 2025 20:15:04 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=ffwll.ch header.s=google header.b=EF8szXj5; dmarc=none; spf=none (imf07.hostedemail.com: domain of simona.vetter@ffwll.ch has no SPF policy when checking 209.85.128.54) smtp.mailfrom=simona.vetter@ffwll.ch ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738095304; a=rsa-sha256; cv=none; b=Eq59DikPMzfF8Pv0/Sxqa4BaVeSbNtZroKIvNkPWxGEhPvO+p6orHEv0HV8qfUElJA9KKQ LNCIqT2YNqOu/AWCnLQtZsndEhppUphEcv/oGm5pnq34ED9yY7YddxhDICGtjcg2YfnRI/ jqeN9PpW6MuSPSaDuT6bvTXq7OoI3v8= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=ffwll.ch header.s=google header.b=EF8szXj5; dmarc=none; spf=none (imf07.hostedemail.com: domain of simona.vetter@ffwll.ch has no SPF policy when checking 209.85.128.54) smtp.mailfrom=simona.vetter@ffwll.ch ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738095304; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Jfne+1E0DxYE+tkkNd2LaEnYSrvpDi+VKE9gDmVZgHA=; b=NHTHrJBnizqdODnnSbdMKds3Pszv5dQ0bCYLGm7Xis5oeXrmwfcdUgx3xyds9wh04WlwOj GHaVOo5H/w5aKt5dZL9eZu0dFxOeE/KnlSY1L3aeFdQD89qZlSk5qoLOmwRclfsx5bKo/+ 7mhZCnWT8i27BtbU5aEAPDPEa9hqXLI= Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-43618283d48so43696325e9.1 for ; Tue, 28 Jan 2025 12:15:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; t=1738095303; x=1738700103; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date:from:to:cc :subject:date:message-id:reply-to; bh=Jfne+1E0DxYE+tkkNd2LaEnYSrvpDi+VKE9gDmVZgHA=; b=EF8szXj5ddJmKBiSCJUocA2GC3eWb0ZtMJwwBvmUm6H9NqiPcpjDXRoG3wbTrGRoV2 xP8Hz+ap+koWzmEo7TWJX9plT512puoSTrygww08BiE+j+VQDQ5wT0EvP8kVLVCJfFvJ q70ipKNc4I9XfjtqeqOrLqIk5kM8/p7fVC1jg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738095303; x=1738700103; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Jfne+1E0DxYE+tkkNd2LaEnYSrvpDi+VKE9gDmVZgHA=; b=EIRVFPX1+aGkE3a0kBU7uSPW0fLfDb+K2Es3ETMTPZf/N8A+S1TY59rNcBhnS6cb+u K7XkemNQdetdohd5y2dkM/W0IOzr26QZwHwd2YDwQaHLLFt/aNvDvrE2aL14wnJojZlK NwD8LMEvYSzFJgM3++21m8XQFGBecYZxFjiTwLsqBGy/WpytlJXgBIus8wu4xrMt6Tjs jGLPGx9T0l+jEiTdhvL5j/ioEJ8U/m+aX0QQpTBXONDWZkQ2l2cns0IY+ftIAl52V/Db hec5SpfH+K3GhB5aHqFkJEVwaA5S4O3IXf4s5EACSDICG5nf/tYX5MlcmZeEZjhtCyAN 8aRw== X-Forwarded-Encrypted: i=1; AJvYcCWwxWgaevvyH6Syp/S7vfgyRE5zegIVu/00aHlaFt4pqr/0oz4hveTE6FEg8ABuaeaZDdwn7SSvqA==@kvack.org X-Gm-Message-State: AOJu0Yxg3W3s/f+eLuAPTPP/5gbKHrJQAgpoR118rZ+o7DU9e9fkR4Bg GhUpi+1LkgS++xRYdbQD5vS3PtogSRvoCdX2j3glCeSH3M9+i2L004uh8aMjDco= X-Gm-Gg: ASbGncs+GzO8wtfOyl0nPl9H+zNEkBtrz6asxxd45Q/YAcmFvZR9yiEDnR5BzSQ/9Rl t0GvEHEI9j3NuKKzxLlP9Z2GfmMVeVAbK6vooTHfgu86FsTo1pQYEWbt0aHl3A+wfSDsKsuokQS VLjJopkgoeKlCY2WAI1R1xhTajT5b1VVIuI9UbnXBUeuH7LKtpRXM1/lcpReZWYV8vaN2WNNLQ3 El24eRNKHCRPYh3T4HqVt0bMiy0f2cgU0q4xM756ErZymNe5e9Dsvf8eyBW7Udc7i3Ndm0UxcTy dBSj1CYu4aKXmpQBOHDtFn8TJIE= X-Google-Smtp-Source: AGHT+IFCpJVPpO9rfXvd1jMOU5Y8GXvqZMpsBrMYN7aXsmHR2MRFFuNi1zjP08VspD+cMw3VvnJFqQ== X-Received: by 2002:a05:600c:1f18:b0:436:1b08:4c78 with SMTP id 5b1f17b1804b1-438dc43093cmr2810575e9.31.1738095302513; Tue, 28 Jan 2025 12:15:02 -0800 (PST) Received: from phenom.ffwll.local ([2a02:168:57f4:0:5485:d4b2:c087:b497]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-38c2a176490sm14733002f8f.1.2025.01.28.12.15.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jan 2025 12:15:01 -0800 (PST) Date: Tue, 28 Jan 2025 21:14:59 +0100 From: Simona Vetter To: Alistair Popple Cc: David Hildenbrand , "linux-mm@kvack.org" , John Hubbard , nouveau@lists.freedesktop.org, Jason Gunthorpe , DRI Development , Karol Herbst , Lyude Paul , Danilo Krummrich Subject: Re: [Question] Are "device exclusive non-swap entries" / "SVM atomics in Nouveau" still getting used in practice? Message-ID: Mail-Followup-To: Alistair Popple , David Hildenbrand , "linux-mm@kvack.org" , John Hubbard , nouveau@lists.freedesktop.org, Jason Gunthorpe , DRI Development , Karol Herbst , Lyude Paul , Danilo Krummrich References: <346518a4-a090-4eaa-bc04-634388fd4ca3@redhat.com> <8c6f3838-f194-4a42-845d-10011192a234@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: Linux phenom 6.12.11-amd64 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 4F0A840011 X-Stat-Signature: hgfuq84ykfmkounzkk8uy3aaiocbp8gb X-HE-Tag: 1738095304-464049 X-HE-Meta: U2FsdGVkX19y/5uowuyYwdGtAz0Vcptw3nYQrtEK+fQGfalbcYA822XyYPhgrgHRfSt50tTsjG4o0lyl+pz3DDlR8iOGZucSGzWlgrEoeieJj3XOhCkOD9loQ6SImq9Jbtgv3XJxnrg6/aEiFwK6urmM8fAvveTZjO/l9g25xAF0mWeyIWzeBEQMui2cS9uf5bRq2mQdVDdkLPYa5u1P3v5yISHmWWLjCwTGSvo1Re3Z0ahOLmZ1lArgA8Yp0QmazR3vLocIAy6KaqNbIKU8yTJI+DwSRaNFWYOxrA6iDfmdF9hX8EknF2MiNB0LCjkGpNDkv7jvJ2ys2xNxfKzdUIuLYTu/2okZTe9Oxa48sI7IMrtzfZgH1Jho9aR/VB8z/Vpvt3CAq2w/SDqLzRZI+hiBv5uyQyGYPMxmjl3lDX8oOhejbhkV9akKIIUWd7Un/NpPO5ZfCHG+FbEwRX46gtaJre+gblVi75/yRAan3EIoEaUZ9UIVx/a6vR+tfXF4EFp0scG+z/iGs815QwNLCkli/pHup+b1ahbcDZfhifHqH0xMoWKRBVXrNA1vM1Wr5JmlJTx5ZrXYMzpeZnyZDjkdrIqsjIXcar/xYcE9jT0P5dKmFOUzKWE6U+j2zTUhdtEJ9N+52XomMISp5rZuU3eBmYorp/w6wtCW4fWjJfPrjO1alpdEY+F60R/FVwonehu5V8wBPoGn5Ch0z3iItCa0NPnNmibPfM0nvp39pXB9YjeErL0Ey9jzahaGYokm+ZWksWXNS5LRZlR3Gc+RvZEc6Faxbc1whQhq2P3/zEP245PZ4XGciInzqY/q1atMb4OKTLJ7J1QnWGvO9YbtfjCx8h+qY9lpEwW7+ahS9Lqey2U4jZSc9yMubiXR5g8T0LASYc1Rv1KDVfqNKNwFiNOoMdpWIW3TnI7DSwg0R7y6XDcs+bM/NhXGoAJJFcyigGzubGaqW+O3c7T56U5 FJBmQ7ri RNfgMHTzmtaY4fS0hLP7BGBq2gyy6FVJ5RdYyzI2jCUL3nJY3KYEptuQnlPQQ9z6W8zmG6YGqrilqBPHjrf0fjN20hsJ+qghpgbODohS1AwRwmG5jXf5c5s/sRoxutiqgYzTs7M3s7vEgAq3maBxXGGdgpJzhKTJmWICm/rDQyJdAAsTNLccBp0YcDcT2/B475vsyFudRMOEgbqQ7+l5R4WpRWoI/2bWCCxw0Yw3MKtJmpTZfCovPksa9jKQpLy5zYOE7SJT9QVDZzrWCRIjG6xsdIxVV+S6SGgdESeNftfLVVG+/UyVhVtwzm1cXK6eHmrWPN+XxWcHQuyXUwJp188nOxPU/MWpWs1jHo0T+q4XevdZ/hW6t/7PrVE1AIMog8uxz X-Bogosity: Ham, tests=bogofilter, spamicity=0.219582, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Jan 28, 2025 at 11:09:24AM +1100, Alistair Popple wrote: > On Fri, Jan 24, 2025 at 06:54:02PM +0100, David Hildenbrand wrote: > > > > > On integrated the gpu is tied into the coherency > > > > > fabric, so there it's not needed. > > > > > > > > > > I think the more fundamental question with both this function here and > > > > > with forced migration to device memory is that there's no guarantee it > > > > > will work out. > > > > > > > > Yes, in particular with device-exclusive, it doesn't really work with THP > > > > and is only limited to anonymous memory. I have patches to at least make it > > > > work reliably with THP. > > > > > > I should have crawled through the implementation first before replying. > > > Since it only looks at folio_mapcount() make_device_exclusive() should at > > > least in theory work reliably on anon memory, and not be impacted by > > > elevated refcounts due to migration/ksm/thp/whatever. > > > > Yes, there is -- in theory -- nothing blocking the conversion except the > > folio lock. That's different than page migration. > > Indeed - this was the entire motivation for make_device_exclusive() - that we > needed a way to reliably exclude CPU access that couldn't be blocked in the same > way page migration can (otherwise we could have just migrated to a device page, > even if that may have added unwanted overhead). The folio_trylock worries me a bit. I guess this is to avoid deadlocks when locking multiple folios, but I think at least on the first one we need an unconditional folio_lock to guarantee forward progress. Since atomics can't cross 4k boundaries (or the hw is just really broken) this should be enough to avoid being stuck in a livelock. I'm also not seeing any other reason why a folio_lock shouldn't work here, but then my understanding of mm/ stuff is really just scratching the surface. I did crawl through all the other code and it looks like everything else is unconditional locks. So looks all good and I didn't spot anything else that seemed problematic. Somewhat aside, I do wonder whether we really want to require callers to hold the mmap lock, or whether with all the work towards lockless fastpath that shouldn't instead just be an implementation detail. At least for the gpu hmm code I've seen I've tried to push hard towards a world were the gpu side does not rely on mmap_read_lock being held at all, to future proof this all. And currently we only have one caller of make_device_exclusive_range() so would be simple to do. -Sima -- Simona Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch