From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16F2DECAAD1 for ; Thu, 1 Sep 2022 00:38:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 40A6C6B0072; Wed, 31 Aug 2022 20:38:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3B9C06B0073; Wed, 31 Aug 2022 20:38:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 25A308D0001; Wed, 31 Aug 2022 20:38:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 165296B0072 for ; Wed, 31 Aug 2022 20:38:43 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id BBAFD1A0658 for ; Thu, 1 Sep 2022 00:38:42 +0000 (UTC) X-FDA: 79861656084.27.392100C Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [62.89.141.173]) by imf27.hostedemail.com (Postfix) with ESMTP id 2326440080 for ; Thu, 1 Sep 2022 00:38:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=pAUCRheD7HEjVi/ypvCmDmDlGNpLzHPc+8Ry1BLGL5Y=; b=Kkt/ouct9XIMw6tkOmsU6S4gh0 m0aJRIol5LWmRPlMQj6EEcUbBd6CL+yJgxrzADT1Wf4exlM1w2B2Y+elhLK/xDSiNb5IXf9nRPHgD QjwWzunxbAmTwk68UOmqRMuAV2798RKPDIsqzqeinPmWVr0BEr8hXs4WNMYw9ptH/PlsoIgQ2VtRa QfsfNpmxbm5W0JGnHVtKlwXEKt33ktMGCPx2QvIKidb3ZAUk4/ImLf5Z3IP3dx7GZWeM1jAx2DZ88 vGTWM4OPiRSVC5Pkow1cvRVGFwb/9rxFEgEAcWniGihBMrZSR6pHMtXNYHrf9fmx7BSpC/ZX1TGME nUdUtbPQ==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.95 #2 (Red Hat Linux)) id 1oTYDt-00AnMP-NE; Thu, 01 Sep 2022 00:38:21 +0000 Date: Thu, 1 Sep 2022 01:38:21 +0100 From: Al Viro To: Jan Kara Cc: John Hubbard , Andrew Morton , Jens Axboe , Miklos Szeredi , Christoph Hellwig , "Darrick J . Wong" , Trond Myklebust , Anna Schumaker , Logan Gunthorpe , linux-block@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-nfs@vger.kernel.org, linux-mm@kvack.org, LKML Subject: Re: [PATCH 5/6] NFS: direct-io: convert to FOLL_PIN pages Message-ID: References: <20220827083607.2345453-1-jhubbard@nvidia.com> <20220827083607.2345453-6-jhubbard@nvidia.com> <353f18ac-0792-2cb7-6675-868d0bd41d3d@nvidia.com> <217b4a17-1355-06c5-291e-7980c0d3cea6@nvidia.com> <20220829160808.rwkkiuelipr3huxk@quack3> <20220831094349.boln4jjajkdtykx3@quack3> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20220831094349.boln4jjajkdtykx3@quack3> ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=linux.org.uk header.s=zeniv-20220401 header.b="Kkt/ouct"; spf=none (imf27.hostedemail.com: domain of viro@ftp.linux.org.uk has no SPF policy when checking 62.89.141.173) smtp.mailfrom=viro@ftp.linux.org.uk; dmarc=pass (policy=none) header.from=zeniv.linux.org.uk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1661992722; a=rsa-sha256; cv=none; b=gVjXiLnkCc/kNANHVKhaldE/loLEliM+qddZ13baV04qyBIY6LXXXb+HY6BuVK7VobVJpc CLlKfryYiGe3l1tsu01VvQnFFwdF0Zv4u4Jz0HYMarWp2/kr56+2RnpCS+VgyGz5sA8BXM wQYOILeSwHVRZG4kyAPHHaOdWWZId/A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1661992722; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pAUCRheD7HEjVi/ypvCmDmDlGNpLzHPc+8Ry1BLGL5Y=; b=T0aw3wxSIknU6jZTtwFUY42728XZdUnGsin7lDd/ASnRHPE06a/6zwaBbZ/dbPMqawQYiU pEWv7gTprnfdo9nz/mZW5qOoqYrI81um9Dx7kBEjj8n8Tf/mR0C1vma21AQ1ZBoX0CG7kF Zn+7EUTnHzlFaANOtWZ2MxedeZsPvSY= X-Rspam-User: Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=linux.org.uk header.s=zeniv-20220401 header.b="Kkt/ouct"; spf=none (imf27.hostedemail.com: domain of viro@ftp.linux.org.uk has no SPF policy when checking 62.89.141.173) smtp.mailfrom=viro@ftp.linux.org.uk; dmarc=pass (policy=none) header.from=zeniv.linux.org.uk X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 2326440080 X-Stat-Signature: ytgyzc1e7ga9me6egmo6aarmtrkinbn6 X-HE-Tag: 1661992722-431957 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Aug 31, 2022 at 11:43:49AM +0200, Jan Kara wrote: > So after looking into that a bit more, I think a clean approach would be to > provide iov_iter_pin_pages2() and iov_iter_pages_alloc2(), under the hood > in __iov_iter_get_pages_alloc() make sure we use pin_user_page() instead of > get_page() in all the cases (using this in pipe_get_pages() and > iter_xarray_get_pages() is easy) and then make all bio handling use the > pinning variants for iters. I think at least iov_iter_is_pipe() case needs > to be handled as well because as I wrote above, pipe pages can enter direct > IO code e.g. for splice(2). > > Also I think that all iov_iter_get_pages2() (or the _alloc2 variant) users > actually do want the "pin page" semantics in the end (they are accessing > page contents) so eventually we should convert them all to > iov_iter_pin_pages2() and remove iov_iter_get_pages2() altogether. But this > will take some more conversion work with networking etc. so I'd start with > converting bios only. Not sure, TBH... FWIW, quite a few of the callers of iov_iter_get_pages2() do *NOT* need to grab any references for BVEC/XARRAY/PIPE cases. What's more, it would be bloody useful to have a variant that doesn't grab references for !iter->user_backed case - that could be usable for KVEC as well, simplifying several callers. Requirements: * recepients of those struct page * should have a way to make dropping the page refs conditional (obviously); bio machinery can be told to do so. * callers should *NOT* do something like "set an ITER_BVEC iter, with page references grabbed and stashed in bio_vec array, call async read_iter() and drop the references in array - the refs we grab in dio will serve" Note that for sync IO that pattern is fine whether we grab/drop anything inside read_iter(); for async we could take depopulating the bio_vec array to the IO completion or downstream of that. * the code dealing with the references returned by iov_iter_..._pages should *NOT* play silly buggers with refcounts - something like "I'll grab a reference, start DMA and report success; page will stay around until I get around to dropping the ref and callers don't need to wait for that" deep in the bowels of infinibad stack (or something equally tasteful) is seriously asking for trouble. Future plans from the last cycle included iov_iter_find_pages{,_alloc}() that would *not* grab references on anything other than IOVEC and UBUF (would advance the iterator, same as iov_iter_get_pages2(), though). Then iov_iter_get_...() would become a wrapper for that. After that - look into switching the users of ..._get_... to ..._find_.... Hadn't done much in that direction yet, though - need to redo the analysis first. That primitive might very well do FOLL_PIN instead of FOLL_GET for IOVEC and UBUF...