From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F01FC00454 for ; Wed, 11 Dec 2019 20:05:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 61B74206A5 for ; Wed, 11 Dec 2019 20:05:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20150623.gappssmtp.com header.i=@kernel-dk.20150623.gappssmtp.com header.b="icYVu+ip" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 61B74206A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 179F26B33A5; Wed, 11 Dec 2019 15:05:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 103696B33B6; Wed, 11 Dec 2019 15:05:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F0CF16B33B7; Wed, 11 Dec 2019 15:05:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0050.hostedemail.com [216.40.44.50]) by kanga.kvack.org (Postfix) with ESMTP id D6C436B33A5 for ; Wed, 11 Dec 2019 15:05:04 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id A481D181AEF15 for ; Wed, 11 Dec 2019 20:05:04 +0000 (UTC) X-FDA: 76253939328.08.fan21_6674657632c5d X-HE-Tag: fan21_6674657632c5d X-Filterd-Recvd-Size: 5632 Received: from mail-pj1-f68.google.com (mail-pj1-f68.google.com [209.85.216.68]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 20:05:03 +0000 (UTC) Received: by mail-pj1-f68.google.com with SMTP id g4so2250pjs.10 for ; Wed, 11 Dec 2019 12:05:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=subject:from:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=tsOWIFudRN3reWK5MlLc7h9kutRZZYV6QEDcmlofBfg=; b=icYVu+ipVyioTkIzX05lyZ/nDofSI9BiOFxVI5kxybYBYBXw7LjODIWhaKTkcea/nk Qk51hgCt/OC6xTOeiKXKK4U9IQ/v0DtnEQnmDYbZz+2e405zkiY6t2RuPdfMCtRZjVJ4 nQN5W6GuAL8hOr1VyNiCoXkA4kzZ/61mMKXzHyzaLlnRtBA/mqR7NBGgZ59znrF9T7up PiQhaUBRqpyGGhCoPJY8HhGBR/cWCpUpKPOfyCGF6DqfefG+wfAbvEP4gR5I8o3lyEm1 S4V9+2H+vQQ+wyNnqQQZInrv/XHJrEC6N930U5VAF7m846uC0vjDl/+JEuaDExPnMzuf +7Uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:references:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=tsOWIFudRN3reWK5MlLc7h9kutRZZYV6QEDcmlofBfg=; b=BdgWPL0JCRI58UuNjXSxHWBC+atZ1kxPIP+YLvNRRVI6lI+rpv6PWa4vJCOQYk+RhJ DlGBx0DJJhAU88rxUuWE8Yhru422ErRQ8fQxX8dtIpfD2IjJmtQCWvrxBIqVNqe6+y4V C89J6tXkKyTOMIxaEnMzEcRd+qoZjPOKaKl3WRAp3jUgYPaPUaEj+x6jS2vpEjzGXKoC 1OxY9dD6d42t5hnX6P7ntwunt2kYgzP8vZlVO8x43u3Y9LrGn+5hZKu1pvX5zlxNC6QC YBXFsPBnvuHdQtwkbb6WoPw48tA6emWMP6bdD0rann3cvS96lbqU0XKuNQBHmFBdgNd9 /oYQ== X-Gm-Message-State: APjAAAWBsF8jZxytY9d3dXtQRiiB3S0A85Ub0c4TD53UO17I7oH+Ii9P H2D3uReLYyJKhpYLTe/bwzFWfw== X-Google-Smtp-Source: APXvYqxXBwB0bh9VK6p2nHwY5RWgvHbUK/AdjrWcmYmgtNlHRRs0vYZ+qQZa4Rq9Y8kOzoY88Z9TRg== X-Received: by 2002:a17:90a:cc10:: with SMTP id b16mr5238863pju.55.1576094702218; Wed, 11 Dec 2019 12:05:02 -0800 (PST) Received: from ?IPv6:2620:10d:c081:1130::1014? ([2620:10d:c090:180::50da]) by smtp.gmail.com with ESMTPSA id i127sm3971751pfc.55.2019.12.11.12.05.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 11 Dec 2019 12:05:01 -0800 (PST) Subject: Re: [PATCHSET v3 0/5] Support for RWF_UNCACHED From: Jens Axboe To: Linus Torvalds Cc: Linux-MM , linux-fsdevel , linux-block , Matthew Wilcox , Chris Mason , Dave Chinner , Johannes Weiner References: <20191211152943.2933-1-axboe@kernel.dk> Message-ID: Date: Wed, 11 Dec 2019 13:04:59 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 12/11/19 12:34 PM, Jens Axboe wrote: > On 12/11/19 10:56 AM, Jens Axboe wrote: >>> But I think most of the regular IO call chains come through >>> "mark_page_accessed()". So _that_ is the part you want to avoid (and >>> maybe the workingset code). And that should be fairly straightforward, >>> I think. >> >> Sure, I can give that a go and see how that behaves. > > Before doing that, I ran a streamed read test instead of just random > reads, and the behavior is roughly the same. kswapd consumes a bit less > CPU, but it's still very active once the page cache has been filled. For > specifics on the setup, I deliberately boot the box with 32G of RAM, and > the dataset is 320G. My initial tests were with 1 320G file, but > Johannes complained about that so I went to 32 10G files instead. That's > what I'm currently using. > > For the random test case, top of profile for kswapd is: > > + 33.49% kswapd0 [kernel.vmlinux] [k] xas_create > + 7.93% kswapd0 [kernel.vmlinux] [k] __isolate_lru_page > + 7.18% kswapd0 [kernel.vmlinux] [k] unlock_page > + 5.90% kswapd0 [kernel.vmlinux] [k] free_pcppages_bulk > + 5.64% kswapd0 [kernel.vmlinux] [k] _raw_spin_lock_irqsave > + 5.57% kswapd0 [kernel.vmlinux] [k] shrink_page_list > + 3.48% kswapd0 [kernel.vmlinux] [k] __remove_mapping > + 3.35% kswapd0 [kernel.vmlinux] [k] isolate_lru_pages > + 3.14% kswapd0 [kernel.vmlinux] [k] __delete_from_page_cache Here's the profile for the !mark_page_accessed() run, looks very much the same: + 32.84% kswapd0 [kernel.vmlinux] [k] xas_create + 8.05% kswapd0 [kernel.vmlinux] [k] unlock_page + 7.68% kswapd0 [kernel.vmlinux] [k] __isolate_lru_page + 6.08% kswapd0 [kernel.vmlinux] [k] free_pcppages_bulk + 5.96% kswapd0 [kernel.vmlinux] [k] _raw_spin_lock_irqsave + 5.56% kswapd0 [kernel.vmlinux] [k] shrink_page_list + 4.02% kswapd0 [kernel.vmlinux] [k] __remove_mapping + 3.70% kswapd0 [kernel.vmlinux] [k] __delete_from_page_cache + 3.55% kswapd0 [kernel.vmlinux] [k] isolate_lru_pages -- Jens Axboe