From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.0 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7EC65C43603 for ; Wed, 11 Dec 2019 19:34:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 44C5420836 for ; Wed, 11 Dec 2019 19:34:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=kernel-dk.20150623.gappssmtp.com header.i=@kernel-dk.20150623.gappssmtp.com header.b="UEc8drWi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 44C5420836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D4FAC6B33A5; Wed, 11 Dec 2019 14:34:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D00976B33A6; Wed, 11 Dec 2019 14:34:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BEFB16B33A7; Wed, 11 Dec 2019 14:34:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0084.hostedemail.com [216.40.44.84]) by kanga.kvack.org (Postfix) with ESMTP id AB5DA6B33A5 for ; Wed, 11 Dec 2019 14:34:46 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 612E78249980 for ; Wed, 11 Dec 2019 19:34:46 +0000 (UTC) X-FDA: 76253862972.16.nest89_80e91988f844b X-HE-Tag: nest89_80e91988f844b X-Filterd-Recvd-Size: 5645 Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) by imf22.hostedemail.com (Postfix) with ESMTP for ; Wed, 11 Dec 2019 19:34:45 +0000 (UTC) Received: by mail-pg1-f193.google.com with SMTP id b137so11237155pga.6 for ; Wed, 11 Dec 2019 11:34:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=subject:from:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=oTBM3Y8vtSCzFzu0QQg+BVLSXCcLcCz6WCWWZBR21KA=; b=UEc8drWiceCBzQdP4WwAoRAD4V9C8sKEg1FLZDGDXLCkv8SbOs8zwnIk4j4teD70TQ zu3XDVEqS0EnJllQjOUM6+x7WDFpjiN4tCdoxtd96kpgrZuojVNnQ3WCdzO8hrIGimef fnHc55Fbo/bxYc1XfbgqN+dNMxRvtkS8JevaSnJTIpvmwnaB67TAKCMCf14UO4Bl4zu5 UqJm0hPSbnuKp8beqgNeMsf3+YE1GCzj/K0Jz7y78tfQBKFvsoND1kjiaOAfHBmqr9kL Hgm29eQ9eqkD1sv7X41r2IVl2JBbKD7xK/4WEFU32jHuSskS0ODdtjXYeUMqwtsz0s2e nQzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:references:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=oTBM3Y8vtSCzFzu0QQg+BVLSXCcLcCz6WCWWZBR21KA=; b=llKJ67zRrKjSRXci9XkQFMCektXbRZooIMCIP1LRgVDBnvo46vLvnKjOejz5jMyKqW pLczNpiPGiCYpl5azCdt31/hOvpvThXBO37kicdEU4G/mphhcM/3flw1fx0R7AsOpaLa IsweQ9euEO5vZSQvR8ZvLm2n7Sm8c85lR9+DbPRuxgym/CUIJa0WoZv67mqtGGK/TOCv 1i9IGDN73z6cDdAg+10s26ewbpAV7SUVABqZ+Nu6ky10fvk7l/mK6J8j0q65NJ26LCU+ 3NcgYRfWO8CiEfyR8WlLEexoptbeBlChxSCcZBN49Hb0Cgz4FMJbK3Zv64dV9xuYz8Gj 265g== X-Gm-Message-State: APjAAAWktTgZ/2x3+/WI4oTaQ4zx4u7RoJolyNr7yWVYO1NXi102XGZO kBCH69AW48nIWHaFLSDiyrqF4A== X-Google-Smtp-Source: APXvYqxakjCkHk+LpBa3tkB5k/nQDmnN7UkDoqLLam7NqycuO4Nc8u97kPkYrw0QIsVKOd/vBhCxbw== X-Received: by 2002:a63:66c6:: with SMTP id a189mr5515581pgc.401.1576092883494; Wed, 11 Dec 2019 11:34:43 -0800 (PST) Received: from ?IPv6:2620:10d:c081:1130::1014? ([2620:10d:c090:180::50da]) by smtp.gmail.com with ESMTPSA id p38sm3138206pjp.27.2019.12.11.11.34.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 11 Dec 2019 11:34:42 -0800 (PST) Subject: Re: [PATCHSET v3 0/5] Support for RWF_UNCACHED From: Jens Axboe To: Linus Torvalds Cc: Linux-MM , linux-fsdevel , linux-block , Matthew Wilcox , Chris Mason , Dave Chinner , Johannes Weiner References: <20191211152943.2933-1-axboe@kernel.dk> Message-ID: Date: Wed, 11 Dec 2019 12:34:40 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 12/11/19 10:56 AM, Jens Axboe wrote: >> But I think most of the regular IO call chains come through >> "mark_page_accessed()". So _that_ is the part you want to avoid (and >> maybe the workingset code). And that should be fairly straightforward, >> I think. >=20 > Sure, I can give that a go and see how that behaves. Before doing that, I ran a streamed read test instead of just random reads, and the behavior is roughly the same. kswapd consumes a bit less CPU, but it's still very active once the page cache has been filled. For specifics on the setup, I deliberately boot the box with 32G of RAM, and the dataset is 320G. My initial tests were with 1 320G file, but Johannes complained about that so I went to 32 10G files instead. That's what I'm currently using. For the random test case, top of profile for kswapd is: + 33.49% kswapd0 [kernel.vmlinux] [k] xas_create = =E2=97=86 + 7.93% kswapd0 [kernel.vmlinux] [k] __isolate_lru_page = =E2=96=92 + 7.18% kswapd0 [kernel.vmlinux] [k] unlock_page = =E2=96=92 + 5.90% kswapd0 [kernel.vmlinux] [k] free_pcppages_bulk = =E2=96=92 + 5.64% kswapd0 [kernel.vmlinux] [k] _raw_spin_lock_irqsave = =E2=96=92 + 5.57% kswapd0 [kernel.vmlinux] [k] shrink_page_list = =E2=96=92 + 3.48% kswapd0 [kernel.vmlinux] [k] __remove_mapping = =E2=96=92 + 3.35% kswapd0 [kernel.vmlinux] [k] isolate_lru_pages = =E2=96=92 + 3.14% kswapd0 [kernel.vmlinux] [k] __delete_from_page_cache = =E2=96=92 Next I ran with NOT calling mark_page_accessed() to see if that makes a difference. See patch below, I just applied this on top of this patchset and added a new RWF_NOACCESS flag for it for ease of teting. I verified that we are indeed skipping the mark_page_accessed() call in generic_file_buffered_read(). I can't tell a difference in the results, there's no discernable difference between NOT calling mark_page_accessed() or calling it. Behavior seems about the same, in terms of pre and post page cache full, and kswapd still churns a lot once the page cache is filled up. --=20 Jens Axboe