From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5649CECAAD1 for ; Sat, 27 Aug 2022 22:27:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7E8C1940008; Sat, 27 Aug 2022 18:27:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7723D940007; Sat, 27 Aug 2022 18:27:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 638DB940008; Sat, 27 Aug 2022 18:27:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 4DF28940007 for ; Sat, 27 Aug 2022 18:27:50 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 270161A073B for ; Sat, 27 Aug 2022 22:27:50 +0000 (UTC) X-FDA: 79846811100.23.8F31C6A Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf31.hostedemail.com (Postfix) with ESMTP id C31B820038 for ; Sat, 27 Aug 2022 22:27:48 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id DD5DC60EB1; Sat, 27 Aug 2022 22:27:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9CCCAC433D7; Sat, 27 Aug 2022 22:27:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1661639267; bh=mP4LJW5/Ex8VVFzSK0tQqGy9Pf6M8NRGpWXThg5YJsY=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=fPCCHHBOCVt1Xlfm2Z5i/Pjl8wfTghEDGD0JFOt4WwDmgOEkru94oGjVnHGqnx/ch gQUe2uK6/r2IS2bj5FoIuojGJWJ0Ygto2fKaREz/DqcTJmhucEE0ruirhkci7F9fps g2F/It6cmXai1gbLalpP+edxEzs+snNPwjjIR5co= Date: Sat, 27 Aug 2022 15:27:45 -0700 From: Andrew Morton To: John Hubbard Cc: Jens Axboe , Alexander Viro , Miklos Szeredi , Christoph Hellwig , "Darrick J . Wong" , Trond Myklebust , Anna Schumaker , Jan Kara , Logan Gunthorpe , , , , , , LKML Subject: Re: [PATCH 2/6] block: add dio_w_*() wrappers for pin, unpin user pages Message-Id: <20220827152745.3dcd05e98b3a4383af650a72@linux-foundation.org> In-Reply-To: <20220827083607.2345453-3-jhubbard@nvidia.com> References: <20220827083607.2345453-1-jhubbard@nvidia.com> <20220827083607.2345453-3-jhubbard@nvidia.com> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; x86_64-redhat-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit ARC-Authentication-Results: i=1; imf31.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=fPCCHHBO; spf=pass (imf31.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1661639268; a=rsa-sha256; cv=none; b=6hj8xDCX0YDgggq4otU/qDv606LZdF6jRnEM9/MvTN9DmNlJx5NASVHEXse3Bkzb4NsvIT S/0nAC0bQCOeX255UStdNn7xrgGZBdoQX3iMTZC4pb6QkLatqlbGjv73iTzKQULbM6moLX ib1FiBr/DvlbBfuYOFhisEI+0Rlp7oQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1661639268; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8cP3mBDu/85Gv5QwSreMph4asqxy+5lKbStd/YPNCjI=; b=OVSNHL1vPT85xplH5ZThHvEzNK1ae0NeCv8cIWjQMECz7XyfGm1hMLa3Mo4CnWOI/ENDB2 UTn7ozQJx5YkVFjbCxqqG3z6pCRLM6K3kTbpCwH8k83XhxZwYZdDutfjcZfdfli9RAbapL TFzuZGiBI82ESDj1JmMKE+Vtyj/DfpM= X-Rspam-User: X-Stat-Signature: p3skrhwgm7fyyix7e554ufemmqcdwxpt X-Rspamd-Queue-Id: C31B820038 Authentication-Results: imf31.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=fPCCHHBO; spf=pass (imf31.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam09 X-HE-Tag: 1661639268-240171 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sat, 27 Aug 2022 01:36:03 -0700 John Hubbard wrote: > Background: The Direct IO part of the block infrastructure is being > changed to use pin_user_page*() and unpin_user_page*() calls, in place > of a mix of get_user_pages_fast(), get_page(), and put_page(). These > have to be changed over all at the same time, for block, bio, and all > filesystems. However, most filesystems can be changed via iomap and core > filesystem routines, so let's get that in place, and then continue on > with converting the remaining filesystems (9P, CIFS) and anything else > that feeds pages into bio that ultimately get released via > bio_release_pages(). > > Add a new config parameter, CONFIG_BLK_USE_PIN_USER_PAGES_FOR_DIO, and > dio_w_*() wrapper functions. The dio_w_ prefix was chosen for > uniqueness, so as to ease a subsequent kernel-wide rename via > search-and-replace. Together, these allow the developer to choose > between these sets of routines, for Direct IO code paths: > > a) pin_user_pages_fast() > pin_user_page() > unpin_user_page() > > b) get_user_pages_fast() > get_page() > put_page() > > CONFIG_BLK_USE_PIN_USER_PAGES_FOR_DIO is a temporary setting, and may > be deleted once the conversion is complete. In the meantime, developers > can enable this in order to try out each filesystem. > > Please remember that these /proc/vmstat items (below) should normally > contain the same values as each other, except during the middle of > pin/unpin operations. As such, they can be helpful when monitoring test > runs: > > nr_foll_pin_acquired > nr_foll_pin_released > > ... > > +static inline void dio_w_unpin_user_pages(struct page **pages, > + unsigned long npages) > +{ > + unsigned long i; > + > + for (i = 0; i < npages; i++) > + put_page(pages[i]); > +} release_pages()? Might be faster if many of the pages are page_count()==1. (release_pages() was almost as simple as the above when I added it a million years ago. But then progress happened).