From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2AE41C433F5 for ; Thu, 7 Oct 2021 07:09:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A48AF60F48 for ; Thu, 7 Oct 2021 07:09:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A48AF60F48 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id E04C2900002; Thu, 7 Oct 2021 03:09:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DB45D6B0071; Thu, 7 Oct 2021 03:09:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA373900002; Thu, 7 Oct 2021 03:09:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0189.hostedemail.com [216.40.44.189]) by kanga.kvack.org (Postfix) with ESMTP id B747E6B006C for ; Thu, 7 Oct 2021 03:09:05 -0400 (EDT) Received: from smtpin34.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 6C25832075 for ; Thu, 7 Oct 2021 07:09:05 +0000 (UTC) X-FDA: 78668764650.34.67FC1D2 Received: from mail-il1-f175.google.com (mail-il1-f175.google.com [209.85.166.175]) by imf13.hostedemail.com (Postfix) with ESMTP id 2AB6C10396A3 for ; Thu, 7 Oct 2021 07:09:05 +0000 (UTC) Received: by mail-il1-f175.google.com with SMTP id r9so5412279ile.5 for ; Thu, 07 Oct 2021 00:09:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Lh6BMcp+uUzvZjxUzMuKHNduPeio4nwpw4ngm8F5c4M=; b=ito+9wxxJsSniBqVgvfy0Im8WB5nkVDa5p0tgsRR1LGQkvbAUWiupHbLD3aPjItGS1 QMAMh95n+JQJmXmDjoAUo615dhhd8Ps0SjY1dSa3yebfZyoTZh/3/le19ctWpIqx4OBZ OxfpLECD+fjm9ti3gOrWdDKuTx7FCGTuRw57Q= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Lh6BMcp+uUzvZjxUzMuKHNduPeio4nwpw4ngm8F5c4M=; b=ETkEe7HkGtin7K3eT2RY1oGlhZkrAzdcgT0P/PD988r8T7IFg/5x86shdAIRH9VTB/ otAcg8vIB+ujZVXCKiXMWf2wrYMMFSA3Dw7XqSMBAN1jgtWEEvd6J/FMQwm4ZCCkI0W/ mdVvpPEklBgFEJI6itubZsvxu2dKl1eBfgG2Up1vj3opS1dGBwJdnTqr39/DIo+rNEs2 KxM1mFd5KGteueUs1ScXS2RDXl60mIItu/kP6zWCb1RKmk6juu1nytMLoEPWk/OrEQzJ OeapmC+0xfnJ/EHAcWRutf9qo0awWQaOOqxiRHKMIIW4teJKUhlsgHjE6EGyAYE/ysb0 dgTg== X-Gm-Message-State: AOAM531Gb2jf/vsjeOP68Il2LNWLxuFMYRtKns16DHoKyEyxwd9Z5w3i xgX5m5Y5ICjFHW8rEOwzoYwbcNR77dYfIxy/2opJzQ== X-Google-Smtp-Source: ABdhPJziFAd8HiPos5rSavGMT24V9aH/wfBhnTjbFd/7m7KJVQajzSHEhGBZttOWnddsBsW75ZDiBi+mtg9hNmpL0G8= X-Received: by 2002:a92:300c:: with SMTP id x12mr1920694ile.230.1633590544261; Thu, 07 Oct 2021 00:09:04 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Hsin-Yi Wang Date: Thu, 7 Oct 2021 15:08:38 +0800 Message-ID: Subject: Re: Readahead regressed with c1f6925e1091("mm: put readahead pages in cache earlier") on multicore arm64 platforms To: Matthew Wilcox Cc: Andrew Morton , William Kucharski , Christoph Hellwig , linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 2AB6C10396A3 X-Stat-Signature: jo8ny4p5xthk5muwftxck14qcbauk76f Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=ito+9wxx; spf=pass (imf13.hostedemail.com: domain of hsinyi@chromium.org designates 209.85.166.175 as permitted sender) smtp.mailfrom=hsinyi@chromium.org; dmarc=pass (policy=none) header.from=chromium.org X-Rspamd-Server: rspam06 X-HE-Tag: 1633590545-230277 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Oct 7, 2021 at 12:08 PM Hsin-Yi Wang wrote: > > On Wed, Oct 6, 2021 at 9:12 PM Matthew Wilcox wrote: > > > > On Wed, Oct 06, 2021 at 09:07:56PM +0800, Hsin-Yi Wang wrote: > > > On Wed, Oct 6, 2021 at 7:21 PM Matthew Wilcox wrote: > > > > > > > > On Wed, Oct 06, 2021 at 05:25:23PM +0800, Hsin-Yi Wang wrote: > > > > > Hi Matthew, > > > > > > > > > > We tested that the performance of readahead is regressed on multicore > > > > > arm64 platforms running on the 5.10 kernel. > > > > > - The platform we used: 8 cores (4x a53(small), 4x a73(big)) arm64 platform > > > > > - The command we used: ureadahead $FILE ($FILE is a 1MB+ pack file, > > > > > note that if the file size is small, it's not obvious to see the > > > > > regression) > > > > > > > > > > After we revert the commit c1f6925e1091("mm: put readahead pages in > > > > > cache earlier"), the readahead performance is back: > > > > > - time ureadahead $FILE: > > > > > - 5.10: 1m23.124s > > > > > - with c1f6925e1091 reverted: 0m3.323s > > > > > - other LTS kernel (eg. 5.4): 0m3.066s > > > > > > > > > > The slowest part is aops->readpage() in read_pages() called in > > > > > read_pages(ractl, &page_pool, false); (the 3rd in > > > > > page_cache_ra_unbounded()) > > > > > > > > What filesystem are you using? > > > > > > > ext4, block size 4096 > > > > That's confusing. ext4 shouldn't hit that path; it has a ->readahead > > address space operation. > > Sorry for the confusion, both readahead and readpage are called. > The ->readpage is called by vfs: vfs_fadvise. > (Full path) > read_pages This calls into squashfs_readpage(). The data pasted before is with SQUASHFS_DECOMP_SINGLE. However if using SQUASHFS_DECOMP_MULTI_PERCPU config: - 5.10: 1. real 0m1.692s, sys 0m4.188s 2. real 0m1.655s, sys 0m4.175s - 5.10 with c1f6925e1091 reverted: 1. real 0m1.549s, 0m3.616s 2. real 0m1.603s, 0m3.638s which is slightly better but the difference is not that much as using SQUASHFS_DECOMP_SINGLE. > page_cache_ra_unbounded > do_page_cache_ra > force_page_cache_ra > generic_fadvise > vfs_fadvise > ksys_readahead > __arm64_compat_sys_aarch32_readahead > el0_svc_common > do_el0_svc_compat > el0_svc_compat > el0_sync_compat_handler > el0_sync_compat > > The ->readahead is called by ext4: ext4_file_read_iter. But this part is fast. > (Full path) > read_pages This calls into ext4_readahead(). > page_cache_ra_unbounded > do_page_cache_ra > ondemand_readahead > page_cache_sync_ra > generic_file_buffered_read > generic_file_read_iter > ext4_file_read_iter > do_iter_readv_writev > do_iter_read > vfs_iter_read > loop_queue_work > kthread_worker_fn > loop_kthread_worker_fn > kthread > ret_from_fork