From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC836C04E30 for ; Mon, 9 Dec 2019 14:00:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8564D2068E for ; Mon, 9 Dec 2019 14:00:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Z6ObHytN" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8564D2068E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E7E046B2703; Mon, 9 Dec 2019 09:00:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E2F746B2704; Mon, 9 Dec 2019 09:00:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D1DFF6B2705; Mon, 9 Dec 2019 09:00:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0249.hostedemail.com [216.40.44.249]) by kanga.kvack.org (Postfix) with ESMTP id BCA356B2703 for ; Mon, 9 Dec 2019 09:00:29 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 830FE6124 for ; Mon, 9 Dec 2019 14:00:29 +0000 (UTC) X-FDA: 76245762978.13.crack73_434bcdb4c028 X-HE-Tag: crack73_434bcdb4c028 X-Filterd-Recvd-Size: 4729 Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by imf22.hostedemail.com (Postfix) with ESMTP for ; Mon, 9 Dec 2019 14:00:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1575900028; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=VQdYChUYPRwBtiwoaKfcOJXruVCbc28umUKlhLFhzOc=; b=Z6ObHytNW58qw5kXS3We4mSww5VjjVn9xmOKaa2m62Q9V8xX9qsDTy/OwLujmMHs1vlObi UvShvuapyo0ojaiqts8O6BROAi7ZSRd37qRhz7LERIfDTvxFQvBEhR6aCLjiM/MGu2lRZB APhkruTGlYqLEVBn79tZrR8sPu3VSC0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-123-FziZ82VkOq-scynChaHufQ-1; Mon, 09 Dec 2019 09:00:22 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 741EC96BF1; Mon, 9 Dec 2019 14:00:21 +0000 (UTC) Received: from warthog.procyon.org.uk (ovpn-120-250.rdu2.redhat.com [10.10.120.250]) by smtp.corp.redhat.com (Postfix) with ESMTP id 809D45DA2C; Mon, 9 Dec 2019 14:00:20 +0000 (UTC) Organization: Red Hat UK Ltd. Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SI4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 3798903 From: David Howells To: lsf-pc@lists.linux-foundation.org cc: dhowells@redhat.com, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Subject: [LSF/MM/BPF TOPIC] How to make fscache/cachefiles read shaping play nicely with the VM? MIME-Version: 1.0 Content-ID: <9607.1575900019.1@warthog.procyon.org.uk> Date: Mon, 09 Dec 2019 14:00:19 +0000 Message-ID: <9608.1575900019@warthog.procyon.org.uk> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-MC-Unique: FziZ82VkOq-scynChaHufQ-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi, I've been rewriting fscache and cachefiles to massively simplify it and mak= e use of the kiocb interface to do direct-I/O to/from the netfs's pages which didn't exist when I first did this. Instead it has been attempting to moni= tor the page bit waitqueues to see when the backing filesystem's pages become u= p to date. =09https://lore.kernel.org/lkml/24942.1573667720@warthog.procyon.org.uk/ =09https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/lo= g/?h=3Dfscache-iter To make it more efficient, following other network filesystems implementati= ons outside of the Linux kernel, I'm attempting to move to requiring reads and writes to the cache in much bigger granules (fixed at 256KiB initially), wh= ich means that I can represent the presence of a granule of that much data with= a single bit. So far, I've done this for ->readpages(), ->readpage() and ->write_begin() = by taking the requested page or pages and expanding/contracting the set of pag= es as necessary so that the first (or only) actually requested page is in ther= e and both ends of the sequence are appropriate aligned. This, however, is at odds with the VM and *its* idea of how to do things - particularly for ->readpages(). The logic of my fscache_read_helper()[*] i= s applied after the VM's readahead logic, and the two don't necessarily see e= ye to eye at present. [*] This is in the patch named "fscache: Add read helper" in the above-mentioned git tree and "afs: Use new fscache I/O API" which has examples of using it. There are some things that need to be taken into consideration: (1) I might want to make the granule size variable both by file and over t= he length of a file. So for a file that's, say, <=3D512MiB in size, I mi= ght want 1 bit per 256KiB granule, but over 512MiB I might want to switch = to 1 bit per 1MiB granule. Or for files that large, just use 1MiB granul= es all the way through. (2) The granule size might also need vary by cache. (3) Some files I want to treat as monolithic. The file is either all ther= e or none of it is. Examples might be non-regular files such as symlink= s or directories. (4) These parameters might be tunable by the admin. So how best to make the VM deal with this? Is it better to integrate such logic into the VM or leave it on top? David