From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1BE3C48BF6 for ; Thu, 29 Feb 2024 22:04:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 369DC6B0099; Thu, 29 Feb 2024 17:04:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 319A86B009B; Thu, 29 Feb 2024 17:04:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1E0536B009C; Thu, 29 Feb 2024 17:04:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 1097B6B0099 for ; Thu, 29 Feb 2024 17:04:17 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 9F03BA0163 for ; Thu, 29 Feb 2024 22:04:16 +0000 (UTC) X-FDA: 81846220512.05.0ECA241 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf23.hostedemail.com (Postfix) with ESMTP id 704C914000A for ; Thu, 29 Feb 2024 22:04:13 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=BbLKpCEG; spf=pass (imf23.hostedemail.com: domain of alex.williamson@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=alex.williamson@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709244253; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MpSECzjDR3t/OBNOdQe1mhDTBvXnESartLRzf687GAk=; b=MdRDHFwQAuLkTHavYKKcZ/7no8/53F+vLWm5aKRrezI/oru4gbYDt9YvZvItq/HhRP2/9u OrNzVVJZpMvcjMewz8mqzTewLirhw6f+/r3B3krdAQqCe4aosFH7ELJJ/vv/le312Wp7Dx K2Y8iROlwGqbw5XyrA2jjEgXyIanbvk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709244253; a=rsa-sha256; cv=none; b=26q/t9tN03TleckFboKfiRRlAU7njnJYXIFE77fTY3xeBEgoUqpTXafogPYVv/wFolrnnb 3b+XNyYYmnXwSMRF4OL/qaIetjGsJlaJqFo4ymfGcC7738ZusZtm9Lg2h+niR/jIgvYHY+ 3CbKuM6C10zyRbs78pHqeqlsVPK1hRA= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=BbLKpCEG; spf=pass (imf23.hostedemail.com: domain of alex.williamson@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=alex.williamson@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1709244252; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MpSECzjDR3t/OBNOdQe1mhDTBvXnESartLRzf687GAk=; b=BbLKpCEG7QIFLWgoAICCeh3NL6hdU6nGPdPSWlkntsnueX/uFAhL2lkqZQ8aYiykLTAm6l j2T8ZU2/7aHtbpUqUki9jR99UXIGuQ6ayxjeCbcLnPxWrGMKPB/UnNBUQ3msOBH8Yi27Kc wojRugVTGYw3rDXXPP9x50kzTvZyJWg= Received: from mail-io1-f71.google.com (mail-io1-f71.google.com [209.85.166.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-271-zZClbyByPVW43Hi5bY_1Xw-1; Thu, 29 Feb 2024 17:04:11 -0500 X-MC-Unique: zZClbyByPVW43Hi5bY_1Xw-1 Received: by mail-io1-f71.google.com with SMTP id ca18e2360f4ac-7c784b01313so193457339f.0 for ; Thu, 29 Feb 2024 14:04:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709244249; x=1709849049; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aNG3+S9v1/vdCmBpDlDkhTu4/UoEKRmapR+0MWUUhG4=; b=YIyoZ0Ue4UTdJ3RbWKdPueaenF/O49JjzyfdNN44xqxhkX5yK/S9t3U53TXr2TXe+8 Eom1MLcPHokVtVDlK3m4W5U7BF7jGtoJ1Hm+Xj4mCsK9Q+fRjORQ89ULcEj6ynXJJYRd EEHTP7V+3kHDom1lb1pVXusa6XPV6uxmCdUmJI3q1D7znAeJRIlZdAu+kBAZgTm5ECqQ R0KwosFXxsrI4a/gI6wiukXGR0Dl4b7Th9shm8byb/ijfMOV+suxVhLD/IQDXjar8BDn cfjCIWIlNvebtATvU4sfLpdRYYFBxASwOrZ3ctObxogSSYarzk97ls56uZMKVNQmNzAG VyLQ== X-Forwarded-Encrypted: i=1; AJvYcCXee/YJbawyc57yFyV4Taoil92oxFmbqVuXsfsOHer7v8MVuHuVzT1XK1eYKdKlFi7ZVazaZaHXDKLy8OGt16sT2qM= X-Gm-Message-State: AOJu0YxoqnLFpV8EcJ0IzgaOAPEY+yzsb2Zo0qCCuvIJgTHW4os6fEs9 bRQjzsok/qWW+4mkEv7QvbA7Z8alFtS5OLaA65uGQ1rXE6xP7hLw/OzAaAzwj1bfL6eaY+cjd6J 3YhrBvd87PK8EcG3pO6L5AQEBk5AHCZkLttvW4/NlGLF1z/EE X-Received: by 2002:a5e:c80a:0:b0:7c7:b1e4:50b with SMTP id y10-20020a5ec80a000000b007c7b1e4050bmr288977iol.13.1709244249620; Thu, 29 Feb 2024 14:04:09 -0800 (PST) X-Google-Smtp-Source: AGHT+IEj18l3oxXB3QTSfTU/vUmW9+aA3E/E9I3F9nYwIu/isQya1T/o6ucp9pWRYhN/op7hidf3DQ== X-Received: by 2002:a5e:c80a:0:b0:7c7:b1e4:50b with SMTP id y10-20020a5ec80a000000b007c7b1e4050bmr288958iol.13.1709244249267; Thu, 29 Feb 2024 14:04:09 -0800 (PST) Received: from redhat.com ([38.15.36.11]) by smtp.gmail.com with ESMTPSA id bv11-20020a056638448b00b004713170def2sm512021jab.93.2024.02.29.14.04.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Feb 2024 14:04:08 -0800 (PST) Date: Thu, 29 Feb 2024 15:04:06 -0700 From: Alex Williamson To: David Hildenbrand Cc: Yisheng Xie , akpm@linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH] vfio/type1: unpin PageReserved page Message-ID: <20240229150406.4d41db01.alex.williamson@redhat.com> In-Reply-To: <20240227132556.17e87767.alex.williamson@redhat.com> References: <20240226160106.24222-1-ethan.xys@linux.alibaba.com> <20240226091438.1fc37957.alex.williamson@redhat.com> <20240226103238.75ad4b24.alex.williamson@redhat.com> <20240227132556.17e87767.alex.williamson@redhat.com> X-Mailer: Claws Mail 4.2.0 (GTK 3.24.41; x86_64-redhat-linux-gnu) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 704C914000A X-Rspam-User: X-Stat-Signature: sex97ishh3ysukm5x3qqkauhrcxr79d7 X-Rspamd-Server: rspam03 X-HE-Tag: 1709244253-333900 X-HE-Meta: U2FsdGVkX1957rU0MuQKiHukA6jjyZ1Xm4KO8dCZVo/htAHqKj6AKYUZj2Nj7nzXjzAdF3xYIXb/9pRjQ4CTwNfkuqYbMIXmH7y3CIlGjiOEcG7PRsTB9tDMgWtMUmU5O7GN3W22kHl3JpGXMKh3EuKO8mIPsUJwbU1WZH5C5ft74fBizfNq0dh57XPYSS/yVlpkLoWSO3TkJWmN9llkuUdktDWWQxNxRvZZ19QkyN6+0Xy9VyW7uusQpc0UdHU0vGsXYROLPG0GiVvlFVmhiFVTmAqWYGlHtyRSXtmPcJFJQO+O3grBJI1shvTGAscPgBCe7UTGv2KhobZhOpoxhPTf1h0nZRcacc4E6lMmZiPrZ6F5Urr9oUQ8Dz0ZVaot+Lb+mAMsdn8nipdqbm+41dxmfXbVgG2SVL/x1eLGdNfOd2IlX5iFXf9dGAAD/IsRUGJKs2ps5I3uVh4D+HPDDbJYTf+1+3s6eOrl0xRwgkxCdBrJQOw7bgCMLoalWgI+0Vani8hwVNVktiUWB3ALeuURlYus13IvsZivSRHymWRTcBc6T8rEie0x+5SofIxSy48S1Hwo3kLZPYYeM4kgX9VI7dxw3BEgXwod2Zr3lEAPQb4nbGL0MOgfP7CoyKqpLyGiDxkMKP7k6KYH6Ng76nGZqS5/vzKS1kdrV5WXr13dg9wtZkVcvwd9MEw3SjWBM11p+xumzBDWFVj3bgLH2CuaHQSPrk2SScAkSboyzlQhOikyEYq0CLyqzpf5aUIJLFENd8Q8m3KoAZiTplE/UcM/R38A8bXpVzhUPRQNpL78PAOCV4gvIAtSq2dy/DYENEb7vztVmNKA3x5MQiB/YUwDy5osxqZi7WDVTkwpzBnZ7NxzFJZTUfSvKRlnybIfMNLEVTcY5cgqvsp34HJTA+WbVMsuNyvQoVrneb1wOZZpz4stNBEJlo2Gqv0cuyAPgY3lddIqryp3XfRx8Im oJb4Y0cV JEcCs8HNEoTWhfoR60zAsSogFpx2jUO9bc9TkIKcwB1ADYHTJoY431RbsQc1GX60Nv8ypxNAPSKs6XOwq1pzTMoPz/3OvAwu2mMdx09SjCutuXvvu/TaRQJjofwXaNgTcJ6vo/zKLLyp7f0tEKch21V7GrfkNcXU0d1CINwJPtoIrLWjq/A4fTdqC9+hs//HzNB9/O8U7/PghNjwYDMnlaY5S9LwEaCoRyFFqRKOYUzFrjvnQ9ErtEY/ekhoVLN/krTdF95pufkRFSUjAVdnQx15iipiGy9/wZ5JmMVBCgb8NyPHGb9wVQJfmkKfSco5OLZuvWXNZSsBM7Wbu9UGdd47A32QBr7Jl3/C7RLCOJ12yP0cVQ8yt6UvCUu/lKcbTlMEpRu7qvAIG7RhwAZt7OC3Wrw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, 27 Feb 2024 13:25:56 -0700 Alex Williamson wrote: > On Tue, 27 Feb 2024 11:27:08 +0100 > David Hildenbrand wrote: >=20 > > On 26.02.24 18:32, Alex Williamson wrote: =20 > > > On Tue, 27 Feb 2024 01:14:54 +0800 > > > Yisheng Xie wrote: > > > =20 > > >> =E5=9C=A8 2024/2/27 00:14, Alex Williamson =E5=86=99=E9=81=93: =20 > > >>> On Tue, 27 Feb 2024 00:01:06 +0800 > > >>> Yisheng Xie wrote: > > >>> =20 > > >>>> We meet a warning as following: > > >>>> WARNING: CPU: 99 PID: 1766859 at mm/gup.c:209 try_grab_page.par= t.0+0xe8/0x1b0 > > >>>> CPU: 99 PID: 1766859 Comm: qemu-kvm Kdump: loaded Tainted: GOE = 5.10.134-008.2.x86_64 #1 =20 > > >>> = ^^^^^^^^ > > >>> > > >>> Does this issue reproduce on mainline? Thanks, =20 > > >> > > >> I have check the code of mainline, the logical seems the same as my > > >> version. > > >> > > >> so I think it can reproduce if i understand correctly. =20 > > >=20 > > > I obviously can't speak to what's in your 5.10.134-008.2 kernel, but = I > > > do know there's a very similar issue resolved in v6.0 mainline and > > > included in v5.10.146 of the stable tree. Please test. Thanks, = =20 > >=20 > > This commit, to be precise: > >=20 > > commit 873aefb376bbc0ed1dd2381ea1d6ec88106fdbd4 > > Author: Alex Williamson > > Date: Mon Aug 29 21:05:40 2022 -0600 > >=20 > > vfio/type1: Unpin zero pages > > =20 > > There's currently a reference count leak on the zero page. We inc= rement > > the reference via pin_user_pages_remote(), but the page is later h= andled > > as an invalid/reserved page, therefore it's not accounted against = the > > user and not unpinned by our put_pfn(). > > =20 > > Introducing special zero page handling in put_pfn() would resolve = the > > leak, but without accounting of the zero page, a single user could > > still create enough mappings to generate a reference count overflo= w. > > =20 > > The zero page is always resident, so for our purposes there's no r= eason > > to keep it pinned. Therefore, add a loop to walk pages returned f= rom > > pin_user_pages_remote() and unpin any zero pages. > >=20 > >=20 > > BUT > >=20 > > in the meantime, we also have > >=20 > > commit c8070b78751955e59b42457b974bea4a4fe00187 > > Author: David Howells > > Date: Fri May 26 22:41:40 2023 +0100 > >=20 > > mm: Don't pin ZERO_PAGE in pin_user_pages() > > =20 > > Make pin_user_pages*() leave a ZERO_PAGE unpinned if it extracts a= pointer > > to it from the page tables and make unpin_user_page*() correspondi= ngly > > ignore a ZERO_PAGE when unpinning. We don't want to risk overrunn= ing a > > zero page's refcount as we're only allowed ~2 million pins on it - > > something that userspace can conceivably trigger. > > =20 > > Add a pair of functions to test whether a page or a folio is a ZER= O_PAGE. > >=20 > >=20 > > So the unpin_user_page_* won't do anything with the shared zeropage. > >=20 > > (likely, we could revert 873aefb376bbc0ed1dd2381ea1d6ec88106fdbd4) =20 >=20 >=20 > Yes, according to the commit log it seems like the unpin is now just > wasted work since v6.5. Thanks! I dusted off an old unit test for mapping the zeropage through vfio and started working on posting a revert for 873aefb376bb but I actually found that this appears to be resolved even before c8070b787519. I bisected it to: commit 84209e87c6963f928194a890399e24e8ad299db1 Author: David Hildenbrand Date: Wed Nov 16 11:26:48 2022 +0100 mm/gup: reliable R/O long-term pinning in COW mappings =20 We already support reliable R/O pinning of anonymous memory. However, assume we end up pinning (R/O long-term) a pagecache page or the shared zeropage inside a writable private ("COW") mapping. The next write access will trigger a write-fault and replace the pinned page by an exclusive anonymous page in the process page tables to break COW: the pinned page no longer corresponds to the page mapped into the process' page table.=20 Now that FAULT_FLAG_UNSHARE can break COW on anything mapped into a COW mapping, let's properly break COW first before R/O long-term pinning something that's not an exclusive anon page inside a COW mapping. FAULT_FLAG_UNSHARE will break COW and map an exclusive anon page instead that can get pinned safely. =20 With this change, we can stop using FOLL_FORCE|FOLL_WRITE for reliable R/O long-term pinning in COW mappings. [...] Note 3: For users that use FOLL_LONGTERM right now without FOLL_WRITE, such as VFIO, we'd now no longer pin the shared zeropage. Instead, we'd populate exclusive anon pages that we can pin. There was a concern that this could affect the memlock limit of existing setups. For example, a VM running with VFIO could run into the memlock limit and fail to run. However, we essentially had the same behavior already in commit 17839856fd58 ("gup: document and work around "COW can break either way" issue") which got merged into some enterprise distros, and there were not any such complaints. So most probably, we're fine. IIUC, c8070b787519 fixes that we can no longer run up the refcount on the zeropage but with the above we're no longer using the zeropage anyway. I also confirm that after your commit above, my unit test that previously triggered this with a read-only, anonymous mmap generates no unpin hits in the vfio loop. Anyway, all that to say that I think we're doubly covered that removing the heinous workaround in vfio is overdue. Thanks, Alex