From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.0 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4744AC33CA1 for ; Wed, 22 Jan 2020 03:54:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0A2B12071E for ; Wed, 22 Jan 2020 03:54:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=kernel-dk.20150623.gappssmtp.com header.i=@kernel-dk.20150623.gappssmtp.com header.b="KSpesqrG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0A2B12071E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B4E106B0005; Tue, 21 Jan 2020 22:54:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B24B76B0008; Tue, 21 Jan 2020 22:54:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A3B2F6B000C; Tue, 21 Jan 2020 22:54:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0154.hostedemail.com [216.40.44.154]) by kanga.kvack.org (Postfix) with ESMTP id 8F4026B0005 for ; Tue, 21 Jan 2020 22:54:26 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 482DE181AC9B6 for ; Wed, 22 Jan 2020 03:54:26 +0000 (UTC) X-FDA: 76403902932.09.pear44_26aac1cd6b928 X-HE-Tag: pear44_26aac1cd6b928 X-Filterd-Recvd-Size: 5276 Received: from mail-pj1-f66.google.com (mail-pj1-f66.google.com [209.85.216.66]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Wed, 22 Jan 2020 03:54:25 +0000 (UTC) Received: by mail-pj1-f66.google.com with SMTP id s7so2752687pjc.0 for ; Tue, 21 Jan 2020 19:54:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=tPnmRjn8rUepJ/m/lTDYE4fiV9urBFYj8TZ2cwtsXFs=; b=KSpesqrGU/sSuhK+zAQJLJsqmx89sl5iclvzT2IK65oe+ZjRjveqivkcgyc6xXLAE0 Gf/qWWnkdcchnpxRbyL75OocYw0M7lcckQxf0maZKU0/nVyJTaGR3dsuM2qKMczjsBIc ULFRb4+eU4SNt+Hyjc2lGP3MDgAoye2RMno+KvcyR9ygPi3L6NHNyVViIA7Rr5CN2gH7 7WzbyZQdd0N3CJUUVKxSDmiXVf442Lw5fChMDtGFBYpEU0Q4y3vjqoeWy3N3VoMaKdoN b6NhczU6yMwj7hwllwt2/UhOi8Tt8ETnDMO86eqKSj8YVaDvIdAl7RI6Z+jo6Joz3iMe eOoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=tPnmRjn8rUepJ/m/lTDYE4fiV9urBFYj8TZ2cwtsXFs=; b=nQn98QbDQsFvb9A6veim5i+nD1LlI69KHjzcI/LlKOkPww+exg6J2vO21CLoGa33fP r3ekH5d37KFzJ/19a2Y7I9Qs/JEZbIQTtlmXc3pXLApFjqSd0uZ9rskAVK/Zd2FIUIZb OhULKpU0jsX0xOAjJWwYv4QfjEz1Z0zsNBTxgglrFBpkvWI7s8fewj7pDuiSXjUKawuB ynQKivz3gALTq4ndx6FityBF8ERuESlDBJ++25cc9LvrCzDbGCflyslQFWHAMRGau48+ UHrqBHO4dhbV34/qw7fvHiJ6Ogwq8gqQpq8Nu2i6XYjK/erntTdhacco6iHrz06yBW9H kpzA== X-Gm-Message-State: APjAAAVI+/hFRrpeTZhoByn0Ju1Z4sRdNbzrbK1qV82H6nF7maSZA5Bh ZFQ8YrdcSeBHvpxk3IGHX0wcfw== X-Google-Smtp-Source: APXvYqz1ej/0UETLM0i3GpTnbpg6jU/Jhvxq4kyaua6Pk55bLtlskFqoOIKje0M5VplsB7SIZWO/vg== X-Received: by 2002:a17:902:302:: with SMTP id 2mr9076370pld.58.1579665264275; Tue, 21 Jan 2020 19:54:24 -0800 (PST) Received: from [192.168.1.188] ([66.219.217.145]) by smtp.gmail.com with ESMTPSA id d3sm43468307pfn.113.2020.01.21.19.54.23 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 21 Jan 2020 19:54:23 -0800 (PST) Subject: Re: [LSF/MM/BPF TOPIC] Do not pin pages for various direct-io scheme To: jglisse@redhat.com, lsf-pc@lists.linux-foundation.org Cc: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, Benjamin LaHaise References: <20200122023100.75226-1-jglisse@redhat.com> From: Jens Axboe Message-ID: Date: Tue, 21 Jan 2020 20:54:22 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.4.1 MIME-Version: 1.0 In-Reply-To: <20200122023100.75226-1-jglisse@redhat.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 1/21/20 7:31 PM, jglisse@redhat.com wrote: > From: J=C3=A9r=C3=B4me Glisse >=20 > Direct I/O does pin memory through GUP (get user page) this does > block several mm activities like: > - compaction > - numa > - migration > ... >=20 > It is also troublesome if the pinned pages are actualy file back > pages that migth go under writeback. In which case the page can > not be write protected from direct-io point of view (see various > discussion about recent work on GUP [1]). This does happens for > instance if the virtual memory address use as buffer for read > operation is the outcome of an mmap of a regular file. >=20 >=20 > With direct-io or aio (asynchronous io) pages are pinned until > syscall completion (which depends on many factors: io size, > block device speed, ...). For io-uring pages can be pinned an > indifinite amount of time. >=20 >=20 > So i would like to convert direct io code (direct-io, aio and > io-uring) to obey mmu notifier and thus allow memory management > and writeback to work and behave like any other process memory. >=20 > For direct-io and aio this mostly gives a way to wait on syscall > completion. For io-uring this means that buffer might need to be > re-validated (ie looking up pages again to get the new set of > pages for the buffer). Impact for io-uring is the delay needed > to lookup new pages or wait on writeback (if necessary). This > would only happens _if_ an invalidation event happens, which it- > self should only happen under memory preissure or for NUMA > activities. >=20 > They are ways to minimize the impact (for instance by using the > mmu notifier type to ignore some invalidation cases). >=20 >=20 > So i would like to discuss all this during LSF, it is mostly a > filesystem discussion with strong tie to mm. I'd be interested in this topic, as it pertains to io_uring. The whole point of registered buffers is to avoid mapping overhead, and page references. If we add extra overhead per operation for that, well... I'm assuming the above is strictly for file mapped pages? Or also page migration? --=20 Jens Axboe