From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.5 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0563EC4363A for ; Thu, 29 Oct 2020 13:57:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BD5AA20724 for ; Thu, 29 Oct 2020 13:57:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20150623.gappssmtp.com header.i=@kernel-dk.20150623.gappssmtp.com header.b="hIKgX2Ik" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BD5AA20724 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D95AE6B006C; Thu, 29 Oct 2020 09:57:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D6E5A6B0070; Thu, 29 Oct 2020 09:57:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CAAA86B0071; Thu, 29 Oct 2020 09:57:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0038.hostedemail.com [216.40.44.38]) by kanga.kvack.org (Postfix) with ESMTP id 9E2906B006C for ; Thu, 29 Oct 2020 09:57:38 -0400 (EDT) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id C48A1180AD815 for ; Thu, 29 Oct 2020 13:57:37 +0000 (UTC) X-FDA: 77425115754.08.dime48_01111932728d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin08.hostedemail.com (Postfix) with ESMTP id 957CC1819E773 for ; Thu, 29 Oct 2020 13:57:37 +0000 (UTC) X-HE-Tag: dime48_01111932728d X-Filterd-Recvd-Size: 6003 Received: from mail-io1-f46.google.com (mail-io1-f46.google.com [209.85.166.46]) by imf30.hostedemail.com (Postfix) with ESMTP for ; Thu, 29 Oct 2020 13:57:36 +0000 (UTC) Received: by mail-io1-f46.google.com with SMTP id z5so3528537iob.1 for ; Thu, 29 Oct 2020 06:57:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=subject:from:to:cc:references:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=Z7vpxnucdkvDVvOKVxrK/myWJRkou9MkRWf4uTlvH1I=; b=hIKgX2IkHUfHumCcrIgyy1+kik4E8/KumxCHPudrJVnxBfMkc6MUEiXsBcUY2BMheJ iK9vJrYR4E2vsMR8pvnRXOkVspG0GLzd4x08VRp742FqG+bRQ97T+X4OCWl0csCeIahB yfpVZWJULsVFJ3rBnBrsR8aIY/ggAoXjoqTLllv0QHcf1SuEUXC0tcu7DDVjMtaQh+rt lplFEvtnYu8irzi6gd811Qtzp93MyhNREtnqa/Lo3VMdTDAfrEqLOxHDKXFpNXhYVOhb ZJo4eJHV5L2rzbAiAMeEUV/UuRlYep2YhPQae34KTvbTcDnvhMSeifC28lu026PEIN00 n45w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:references:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=Z7vpxnucdkvDVvOKVxrK/myWJRkou9MkRWf4uTlvH1I=; b=QcZ7xe1PrExQcksX3vnWqCnaxee1LTNWYH4zN1J/W1XXzY0g4JMx9uHObysGJbXCpv R24lY/eHJbN6W4VenlMlcA4bHzV8oM9OUA7ADIh4MrF7i+Bpm1cajMfTbp86U2p4l0HU FTLlOpDOU7v8rhih+PdP6E67d9TsFciyTPYmVQ2a3V61ANwPMmbYrozZ+LTDHXMb+ogv Y2pA9skD/W167ZVKkXzTZA92aZpIkgiaxzxKaINAReaaMKU/Pw5dC7fGXjZDGMRZBG1j SDGlnKwYoR144TAlK0WLRFvV+oZ+89j05IwxMSCHTCJ3cjP7Z46yafGjPeiDpghcHDSe vbHA== X-Gm-Message-State: AOAM530Fi6Dd5rufdQiHBsrqVAevdnqUdTYaQUiHGWhMvGhrFsP8x6xg bw7XR0UFogEefaqQoL+YzFa9LQ== X-Google-Smtp-Source: ABdhPJyfd8zir53PRJngOkxHTdVPsY6o51Bu0szyghm7OnimAu1IQdk8va/0FAXRLtpdjrSeDdgfOQ== X-Received: by 2002:a6b:3f88:: with SMTP id m130mr3521818ioa.78.1603979855639; Thu, 29 Oct 2020 06:57:35 -0700 (PDT) Received: from [192.168.1.30] ([65.144.74.34]) by smtp.gmail.com with ESMTPSA id p66sm3030768ilb.48.2020.10.29.06.57.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 29 Oct 2020 06:57:35 -0700 (PDT) Subject: Re: + fs-break-generic_file_buffered_read-up-into-multiple-functions.patch added to -mm tree From: Jens Axboe To: Andrew Morton , Matthew Wilcox Cc: linux-mm@kvack.org, kent.overstreet@gmail.com References: <20201025220817.XxXVE%akpm@linux-foundation.org> <20201027133551.GW20115@casper.infradead.org> <20201028152235.125172e46ae70ba4d7094469@linux-foundation.org> Message-ID: <543aa047-f334-cc40-8466-51bff70ac3b3@kernel.dk> Date: Thu, 29 Oct 2020 07:57:34 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 10/28/20 4:26 PM, Jens Axboe wrote: > On 10/28/20 4:22 PM, Andrew Morton wrote: >> On Tue, 27 Oct 2020 13:35:51 +0000 Matthew Wilcox wrote: >> >>> On Sun, Oct 25, 2020 at 03:08:17PM -0700, akpm@linux-foundation.org wrote: >>>> The patch titled >>>> Subject: mm/filemap/c: freak generic_file_buffered_read up into multiple functions >>>> has been added to the -mm tree. Its filename is >>>> fs-break-generic_file_buffered_read-up-into-multiple-functions.patch >>> >>> Can we back this out? It really makes the THP patchset unhappy. I think >>> we can do something like this afterwards, but doing it this way round is >>> ridiculously hard. >> >> Two concerns: >> >> : On my test box, 4k buffered random reads go from ~150k to ~250k iops, >> : and the improvements to big sequential reads are even bigger. >> >> That's a big improvement! We want that improvement. Throwing it away >> on behalf of an as-yet-unmerged feature patchset hurts. Can we expect that >> this improvement will be available post-that-patchset? And when? >> >> (This improvment is rather hard to believe, really - more details on the >> test environment would be useful. Can we expect that people will in >> general see similar benefits or was there something special about the >> testing?) > > I did see some wins when I tested this. I'll try and run some testing > tomorrow and report back. If there's something specifically you want to > see tested, let me know. I did some testing, unfortunately it's _very_ hard to produce somewhat consistent and good numbers as it quickly becomes a game of kswapd. Here's a basic case of 4 threads doing 32k random reads: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 462 root 20 0 0 0 0 R 65.5 0.0 0:08.02 kswapd0 2287 axboe 20 0 1303448 2176 1072 R 46.6 0.0 0:05.35 fio 2289 axboe 20 0 1303456 2196 1092 D 46.6 0.0 0:05.34 fio 2290 axboe 20 0 1303460 2216 1112 D 46.6 0.0 0:05.37 fio 2288 axboe 20 0 1303452 2224 1120 R 45.9 0.0 0:05.33 fio Sad face... Unfortunately once kswapd kicks in, performance also plummets. This box only has 32G of ram, and you can fill that in less than 10 seconds doing buffered reads like that. I ran 4k and 32k testing, and using 1 and 4 threads. But given the above sadness, it quickly ends up looking the same for me. What I noticed in my initial testing on Kent's patches (which was focused on correctness) was that a read+write verify workload had consistently better read throughput. -- Jens Axboe