From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86EDFC433EF for ; Wed, 6 Oct 2021 09:25:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1A6C86117A for ; Wed, 6 Oct 2021 09:25:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 1A6C86117A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 58A3E6B006C; Wed, 6 Oct 2021 05:25:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 539056B0071; Wed, 6 Oct 2021 05:25:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 429306B0073; Wed, 6 Oct 2021 05:25:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0183.hostedemail.com [216.40.44.183]) by kanga.kvack.org (Postfix) with ESMTP id 2FB816B006C for ; Wed, 6 Oct 2021 05:25:50 -0400 (EDT) Received: from smtpin39.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id CD0BB8249980 for ; Wed, 6 Oct 2021 09:25:49 +0000 (UTC) X-FDA: 78665480418.39.BD5A2C7 Received: from mail-il1-f172.google.com (mail-il1-f172.google.com [209.85.166.172]) by imf14.hostedemail.com (Postfix) with ESMTP id 8CF8260037CF for ; Wed, 6 Oct 2021 09:25:49 +0000 (UTC) Received: by mail-il1-f172.google.com with SMTP id y17so2168493ilb.9 for ; Wed, 06 Oct 2021 02:25:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=mime-version:from:date:message-id:subject:to:cc; bh=Ebq8b1XUSxPIbHVO2Rmlta4kkO79xC3ypNhBwFLSQCw=; b=nYmughVB5Rnse0DU7LI5t8nxwKLgXc6tgy+OW2XMatlrb33M0VwkSudZzMClE2pf8f /krEhjMm56bfc/gbKP0zTFQ+XD7VvrmvMbIXJMBe6pK1R+qCc3+dqBoe+QPlGLTeb0sY tW7ghL4mf2syC0B9Oxjc9QVWJW7mnnOoqFW/k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=Ebq8b1XUSxPIbHVO2Rmlta4kkO79xC3ypNhBwFLSQCw=; b=N6m1ZScoFCVesenG0NPDWOP+3w76Lao9rKNBQEMSz3prUJRRTHuSOdXwM5SKmlECkb xwSbHDdKxrldzlSbnyHDZIutVU8g/sTqBIGrBMKG4SCAzfz6RZrZgLpKFzBOcukr1Mj7 s/PoMh0nNnnAs01pxx6PCnYHCQZwxbjpcL7IhYhAVh1gpp97BXWARqgMhda4TYubCk24 S8x5EVD93mnMRbU1VHrgZQzhFa8+S+upfg8hYhwlLFD6zc561SBWEszHCqCKAjhGy//y /ZvasObjZFBb34XrJHzw0NjQYqEXht8Qqg9jRktBSVrZJBgquWFVUCl6xFPT34veKaLd 9pBg== X-Gm-Message-State: AOAM532c+/09wbFx4SDepb77UzrbivgoYvwQO3jSu33GCl+HJxHXplld DLEKwcnyR7w4UAjkKgIJ7BtNPZTF5Qytav4Ha7Ks2Q== X-Google-Smtp-Source: ABdhPJxzSrRNhO3UrWNbJvsmDqFbjId4kWb7ydCibUJwAzZitbzph006rM7K0ICCp+cIQQ6zj7Mm3PRzUj+ml8HAlg8= X-Received: by 2002:a92:b301:: with SMTP id p1mr6153023ilh.10.1633512348767; Wed, 06 Oct 2021 02:25:48 -0700 (PDT) MIME-Version: 1.0 From: Hsin-Yi Wang Date: Wed, 6 Oct 2021 17:25:23 +0800 Message-ID: Subject: Readahead regressed with c1f6925e1091("mm: put readahead pages in cache earlier") on multicore arm64 platforms To: "Matthew Wilcox (Oracle)" Cc: Andrew Morton , William Kucharski , Christoph Hellwig , linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 8CF8260037CF X-Stat-Signature: 64b1qcn4i1as8n6t5pz5ytdfqyzq8gas Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=nYmughVB; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf14.hostedemail.com: domain of hsinyi@chromium.org designates 209.85.166.172 as permitted sender) smtp.mailfrom=hsinyi@chromium.org X-HE-Tag: 1633512349-389059 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000035, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Matthew, We tested that the performance of readahead is regressed on multicore arm64 platforms running on the 5.10 kernel. - The platform we used: 8 cores (4x a53(small), 4x a73(big)) arm64 platform - The command we used: ureadahead $FILE ($FILE is a 1MB+ pack file, note that if the file size is small, it's not obvious to see the regression) After we revert the commit c1f6925e1091("mm: put readahead pages in cache earlier"), the readahead performance is back: - time ureadahead $FILE: - 5.10: 1m23.124s - with c1f6925e1091 reverted: 0m3.323s - other LTS kernel (eg. 5.4): 0m3.066s The slowest part is aops->readpage() in read_pages() called in read_pages(ractl, &page_pool, false); (the 3rd in page_cache_ra_unbounded()) static void read_pages(struct readahead_control *rac, struct list_head *pages, bool skip_page) { ... if (aops->readahead) { ... } else if (aops->readpages) { ... } else { while ((page = readahead_page(rac))) { aops->readpage(rac->file, page); // most of the time is spent on this line put_page(page); } } ... } We also found following metrics that are relevant: - time ureadahead $FILE: - 5.10 - taskset ureadahead to a small core: 0m7.411s - taskset ureadahead to a big core: 0m5.982s compared to the original 1m23s, pining the ureadahead task on a single core also solves the gap. Do you have any idea why moving pages to cache earlier then doing page read later will cause such a difference? Thanks, Hsin-Yi