From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B7FEC432C2 for ; Tue, 24 Sep 2019 09:29:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D3BB3214D9 for ; Tue, 24 Sep 2019 09:29:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=yandex-team.ru header.i=@yandex-team.ru header.b="sFEY7awd" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D3BB3214D9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=yandex-team.ru Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6C8916B0006; Tue, 24 Sep 2019 05:29:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6784A6B0007; Tue, 24 Sep 2019 05:29:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5B4B76B0008; Tue, 24 Sep 2019 05:29:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0019.hostedemail.com [216.40.44.19]) by kanga.kvack.org (Postfix) with ESMTP id 386A86B0006 for ; Tue, 24 Sep 2019 05:29:40 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id C753E20BD2 for ; Tue, 24 Sep 2019 09:29:39 +0000 (UTC) X-FDA: 75969291678.20.game69_372553435c44e X-HE-Tag: game69_372553435c44e X-Filterd-Recvd-Size: 4373 Received: from forwardcorp1o.mail.yandex.net (forwardcorp1o.mail.yandex.net [95.108.205.193]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Tue, 24 Sep 2019 09:29:38 +0000 (UTC) Received: from mxbackcorp2j.mail.yandex.net (mxbackcorp2j.mail.yandex.net [IPv6:2a02:6b8:0:1619::119]) by forwardcorp1o.mail.yandex.net (Yandex) with ESMTP id 157712E0AFF; Tue, 24 Sep 2019 12:29:36 +0300 (MSK) Received: from vla5-2bf13a090f43.qloud-c.yandex.net (vla5-2bf13a090f43.qloud-c.yandex.net [2a02:6b8:c18:3411:0:640:2bf1:3a09]) by mxbackcorp2j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id QkwcPN4QKg-TZfSidj7; Tue, 24 Sep 2019 12:29:36 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1569317376; bh=YKkn4HVvP/+S323ny4wtqnadzUN5R/l/3Q2qi4I+C20=; h=In-Reply-To:Message-ID:From:Date:References:To:Subject:Cc; b=sFEY7awdEgAKvKAPX0qms5EE4rM19nmu3enZSr/t/63SKXi9HMwNLdgBSZ8KwOqXg D5i6QuSwldSgtTjOcisJ0R2KTibZ+JyE01OrUCapm8OxcESAwyMOAfNoodsokaw2P6 cMpXcarS4KylAWkIXppr9gv/FKtG45Hp63c8iOP4= Authentication-Results: mxbackcorp2j.mail.yandex.net; dkim=pass header.i=@yandex-team.ru Received: from dynamic-red.dhcp.yndx.net (dynamic-red.dhcp.yndx.net [2a02:6b8:0:40c:3d4d:a9cb:ef29:4bb1]) by vla5-2bf13a090f43.qloud-c.yandex.net (nwsmtp/Yandex) with ESMTPSA id xBph0X0UAr-TZIeLosP; Tue, 24 Sep 2019 12:29:35 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client certificate not present) Subject: Re: [PATCH v2] mm: implement write-behind policy for sequential file writes To: Linus Torvalds Cc: linux-fsdevel , Linux-MM , Linux Kernel Mailing List , Jens Axboe , Michal Hocko , Dave Chinner , Mel Gorman , Johannes Weiner , Tejun Heo References: <156896493723.4334.13340481207144634918.stgit@buzz> From: Konstantin Khlebnikov Message-ID: Date: Tue, 24 Sep 2019 12:29:34 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-CA Content-Transfer-Encoding: 7bit X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 21/09/2019 02.05, Linus Torvalds wrote: > On Fri, Sep 20, 2019 at 12:35 AM Konstantin Khlebnikov > wrote: >> >> This patch implements write-behind policy which tracks sequential writes >> and starts background writeback when file have enough dirty pages. > > Apart from a spelling error ("contigious"), my only reaction is that > I've wanted this for the multi-file writes, not just for single big > files. > > Yes, single big files may be a simpler and perhaps the "10% effort for > 90% of the gain", and thus the right thing to do, but I do wonder if > you've looked at simply extending it to cover multiple files when > people copy a whole directory (or unpack a tar-file, or similar). > > Now, I hear you say "those are so small these days that it doesn't > matter". And maybe you're right. But partiocularly for slow media, > triggering good streaming write behavior has been a problem in the > past. > > So I'm wondering whether the "writebehind" state should perhaps be > considered be a process state, rather than "struct file" state, and > also start triggering for writing smaller files. It's simple to extend existing state with per-task counter of sequential writes to detect patterns like unpacking tarball with small files. After reaching some threshold write-behind could flush files in at close. But in this case it's hard to wait previous writes to limit amount of requests and pages in writeback for each stream. Theoretically we could build chain of inodes for delaying and batching. > > Maybe this was already discussed and people decided that the big-file > case was so much easier that it wasn't worth worrying about > writebehind for multiple files. > > Linus >