From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A5C2ECAAD2 for ; Mon, 29 Aug 2022 16:08:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 52C3A940009; Mon, 29 Aug 2022 12:08:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4D8E6940007; Mon, 29 Aug 2022 12:08:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3A438940009; Mon, 29 Aug 2022 12:08:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 2DDBE940007 for ; Mon, 29 Aug 2022 12:08:14 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 0B1721C5B00 for ; Mon, 29 Aug 2022 16:08:14 +0000 (UTC) X-FDA: 79853112108.06.3D3CAA5 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf20.hostedemail.com (Postfix) with ESMTP id 605B01C0047 for ; Mon, 29 Aug 2022 16:08:13 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id CB8A6227E4; Mon, 29 Aug 2022 16:08:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1661789291; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=PGbWwfYuYN9T84wUp5KX7hENmHOxLt+bOTLxbPFr6cs=; b=Nz1DGpzfoyENhZNtQaMGDJXT2EGuJi02N9O2TjnP3C7kjF7GBdJ9i0keBKz9wTGzpq4R+b J+1+lahpOmmUy/XQKckOlDAXrewROaSvkEy/OEUJJHRZE2k9l2bw3cN30Ksra6fe8nTL89 3VAAHD8HpHvlgddeyw67uD4L+0G/hfM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1661789291; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=PGbWwfYuYN9T84wUp5KX7hENmHOxLt+bOTLxbPFr6cs=; b=uWr6d3d+E9HF7DuQEGYclf7K4yBkjwT1b9FAlRObNlmpiMlQ6+mTxnqJxBgw5uD6CwbPQd AMW5jFIeY6HdVwBA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id E31071352A; Mon, 29 Aug 2022 16:08:10 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id vlVgN2rkDGNfGAAAMHmgww (envelope-from ); Mon, 29 Aug 2022 16:08:10 +0000 Received: by quack3.suse.cz (Postfix, from userid 1000) id 0F8C0A066D; Mon, 29 Aug 2022 18:08:08 +0200 (CEST) Date: Mon, 29 Aug 2022 18:08:08 +0200 From: Jan Kara To: John Hubbard Cc: Al Viro , Andrew Morton , Jens Axboe , Miklos Szeredi , Christoph Hellwig , "Darrick J . Wong" , Trond Myklebust , Anna Schumaker , Jan Kara , Logan Gunthorpe , linux-block@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-nfs@vger.kernel.org, linux-mm@kvack.org, LKML Subject: Re: [PATCH 5/6] NFS: direct-io: convert to FOLL_PIN pages Message-ID: <20220829160808.rwkkiuelipr3huxk@quack3> References: <20220827083607.2345453-1-jhubbard@nvidia.com> <20220827083607.2345453-6-jhubbard@nvidia.com> <353f18ac-0792-2cb7-6675-868d0bd41d3d@nvidia.com> <217b4a17-1355-06c5-291e-7980c0d3cea6@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <217b4a17-1355-06c5-291e-7980c0d3cea6@nvidia.com> ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=Nz1DGpzf; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=uWr6d3d+; spf=pass (imf20.hostedemail.com: domain of jack@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=jack@suse.cz; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1661789293; a=rsa-sha256; cv=none; b=eAZ9mO3p60Qz1zGcybxmZiPLMu6Tdj+rIyZUAOAbn6RJU5o0lVUs/d3Qt8bk/HuKL2BUBY gfxQKajm+cyXVsv5MLE5BOeyVzQ3EbR2MEI3TYFMs3DiXlsN1aHpCxBUdYper1yIoAwLNS ZR0rde035HSRvEObSB1yNY6IuE6G964= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1661789293; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PGbWwfYuYN9T84wUp5KX7hENmHOxLt+bOTLxbPFr6cs=; b=An6aMtWoTkMXpmVNXay/UoGgTgO+F+cgXwiff946zlkbbbNCACVIORyIzK3RoTmirvicd+ zcq260tHhBEzLb8ilkL+E+SWzKm/panHC9jWa54A3vxTqGSqKzUl/C/F5/W0Iauid6hUg/ YbiXIBXFMkIPfuqYfMZu/ZfA+w3i9iI= Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=Nz1DGpzf; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=uWr6d3d+; spf=pass (imf20.hostedemail.com: domain of jack@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=jack@suse.cz; dmarc=none X-Rspamd-Server: rspam06 X-Rspam-User: X-Stat-Signature: orzxz59zopxp5nfkwtpfcmyqi9jhhqsi X-Rspamd-Queue-Id: 605B01C0047 X-HE-Tag: 1661789293-773704 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sun 28-08-22 21:59:49, John Hubbard wrote: > On 8/27/22 17:39, Al Viro wrote: > > On Sun, Aug 28, 2022 at 01:38:57AM +0100, Al Viro wrote: > >> On Sat, Aug 27, 2022 at 04:55:18PM -0700, John Hubbard wrote: > >>> On 8/27/22 15:48, Al Viro wrote: > >>>> On Sat, Aug 27, 2022 at 01:36:06AM -0700, John Hubbard wrote: > >>>>> Convert the NFS Direct IO layer to use pin_user_pages_fast() and > >>>>> unpin_user_page(), instead of get_user_pages_fast() and put_page(). > >>>> > >>>> Again, this stuff can be hit with ITER_BVEC iterators > >>>> > >>>>> - result = iov_iter_get_pages_alloc2(iter, &pagevec, > >>>>> + result = dio_w_iov_iter_pin_pages_alloc(iter, &pagevec, > >>>>> rsize, &pgbase); > >>>> > >>>> and this will break on those. > >>> > >>> If anyone has an example handy, of a user space program that leads > >>> to this situation (O_DIRECT with ITER_BVEC), it would really help > >>> me reach enlightenment a lot quicker in this area. :) > >> > >> Er... splice(2) to O_DIRECT-opened file on e.g. ext4? Or > >> sendfile(2) to the same, for that matter... > > > > s/ext4/nfs/ to hit this particular codepath, obviously. > > OK, I have a solution to this that's pretty easy: > > 1) Get rid of the user_backed_iter(i) check in > dio_w_iov_iter_pin_pages() and dio_w_iov_iter_pin_pages_alloc(), and > > 2) At the call sites, match up the unpin calls appropriately. > > ...and apply a similar fix for the fuse conversion patch. > > However, the core block/bio conversion in patch 4 still does depend upon > a key assumption, which I got from a 2019 email discussion with > Christoph Hellwig and others here [1], which says: > > "All pages released by bio_release_pages should come from > get_get_user_pages...". > > I really hope that still holds true. Otherwise this whole thing is in > trouble. > > [1] https://lore.kernel.org/kvm/20190724053053.GA18330@infradead.org/ Well as far as I've checked that discussion, Christoph was aware of pipe pages etc. (i.e., bvecs) entering direct IO code. But he had some patches [2] which enabled GUP to work for bvecs as well (using the kernel mapping under the hood AFAICT from a quick glance at the series). I suppose we could also handle this in __iov_iter_get_pages_alloc() by grabbing pin reference instead of plain get_page() for the case of bvec iter. That way we should have only pinned pages in bio_release_pages() even for the bvec case. [2] http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/gup-bvec Honza -- Jan Kara SUSE Labs, CR