From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48B45C433B4 for ; Wed, 14 Apr 2021 20:08:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DF83D60FEF for ; Wed, 14 Apr 2021 20:08:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DF83D60FEF Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5FEC86B0073; Wed, 14 Apr 2021 16:08:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5AEAC6B0074; Wed, 14 Apr 2021 16:08:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 450B96B0075; Wed, 14 Apr 2021 16:08:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0215.hostedemail.com [216.40.44.215]) by kanga.kvack.org (Postfix) with ESMTP id 2BE496B0073 for ; Wed, 14 Apr 2021 16:08:34 -0400 (EDT) Received: from smtpin35.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id DF83F180189FB for ; Wed, 14 Apr 2021 20:08:33 +0000 (UTC) X-FDA: 78032060106.35.FF2F423 Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) by imf16.hostedemail.com (Postfix) with ESMTP id DAFD980192E5 for ; Wed, 14 Apr 2021 20:08:32 +0000 (UTC) Received: by mail-wm1-f52.google.com with SMTP id z24-20020a1cf4180000b029012463a9027fso11160823wma.5 for ; Wed, 14 Apr 2021 13:08:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=dCkJ3L6wxS3AzadSsXtpvo5PeGe9Mmz+fl2xNX9Gte0=; b=VRFDLYUYYllSp7YVuiG8fJNtTM9SQhAmDOG4yLYbySp9D+R+snCUOxK/6p35n59Szz kcmbos1L8lzvhNgy3qiX9OOjlCRl5E8Bbqd5bxq7+GhTrGUEv4PoSMB3kioHkimQob7u /b24BCi6bR+B3SEM2MrtkYHnar2UeMNwEgRy+PZro8xiRzuAjlsdABqUM10ditulXlDa JyQFQlWGeEWimfMKn/uFhkxg8Sq6h9xm9BFaoRvXxyMTod/99Bf7JrEr6RS/XD/zFqsZ O5kWEiS9kLT+a4lJBiquMp/+MjlBn381TGPCAxGtpxKj9KX+Zj3U7hTLjh8Ep89PMlgk +sGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=dCkJ3L6wxS3AzadSsXtpvo5PeGe9Mmz+fl2xNX9Gte0=; b=D1DsBgCBfZiV3aTwaeAtQuY82MiZSOes//c08H1efN6SoiktQPhEMn2Om0UqE/qi3G f8v1Wbfb0v300k++Htlerliv+5rns/UHu3nHOe76+xsWnA9HQJYVBuBJlM/biBu/BfLk CnZZOWyy7fDGKvZwBhIFzzik0M7T9QZ/m2c04VSEdEPvNFzcKbROKwt0ZGyDC1/IpTlX 5GXt6mUHWmXQTr8cEape+ochdJHlg7C8J8zn72rtuD/L+Ce536igIqjogRXxJBMaN0dU cvwjfRP6GlOBFetoLd1nVPWsrB8iEt2Glxl/2n+kDVvzhV4PxhiRvHZWCMGgelYWAgP0 rk1g== X-Gm-Message-State: AOAM531KncF9b3FnHToo8sD9AeRx1Q5eelxT/iLb+iXOnuW3+S8x0LBJ hQqZOhRyEHZ+rZLJXiFisN7lSniihU3ovPg9pm8Pog== X-Google-Smtp-Source: ABdhPJyY2SsCoD/iPtzHo1ege18rCyoKZd1+jp9ZVgvOEQJvYhkgHuRWHI30WtYRv1mN7MwYW2ls5eWLmcMlzyFWf58= X-Received: by 2002:a7b:cf12:: with SMTP id l18mr4554076wmg.37.1618430912201; Wed, 14 Apr 2021 13:08:32 -0700 (PDT) MIME-Version: 1.0 References: <20210413075155.32652-1-sjpark@amazon.de> <3ddd4f8a-8e51-662b-df11-a63a0e75b2bc@kernel.dk> <20210413231436.GF63242@dread.disaster.area> <20210414155130.GU3762101@tassilo.jf.intel.com> In-Reply-To: From: Yu Zhao Date: Wed, 14 Apr 2021 14:08:20 -0600 Message-ID: Subject: Re: [PATCH v2 00/16] Multigenerational LRU Framework To: Rik van Riel Cc: Andi Kleen , Dave Chinner , Jens Axboe , SeongJae Park , Linux-MM , Andrew Morton , Benjamin Manes , Dave Hansen , Hillf Danton , Johannes Weiner , Jonathan Corbet , Joonsoo Kim , Matthew Wilcox , Mel Gorman , Miaohe Lin , Michael Larabel , Michal Hocko , Michel Lespinasse , Roman Gushchin , Rong Chen , SeongJae Park , Tim Chen , Vlastimil Babka , Yang Shi , Ying Huang , Zi Yan , linux-kernel , lkp@lists.01.org, Kernel Page Reclaim v2 Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: gs9w4ct7q5hioddjc711szs3ju1qhm73 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: DAFD980192E5 Received-SPF: none (google.com>: No applicable sender policy available) receiver=imf16; identity=mailfrom; envelope-from=""; helo=mail-wm1-f52.google.com; client-ip=209.85.128.52 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618430912-313330 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Apr 14, 2021 at 1:42 PM Rik van Riel wrote: > > On Wed, 2021-04-14 at 13:14 -0600, Yu Zhao wrote: > > On Wed, Apr 14, 2021 at 9:59 AM Rik van Riel > > wrote: > > > On Wed, 2021-04-14 at 08:51 -0700, Andi Kleen wrote: > > > > > 2) It will not scan PTE tables under non-leaf PMD entries > > > > > that > > > > > do not > > > > > have the accessed bit set, when > > > > > CONFIG_HAVE_ARCH_PARENT_PMD_YOUNG=y. > > > > > > > > This assumes that workloads have reasonable locality. Could > > > > there > > > > be a worst case where only one or two pages in each PTE are used, > > > > so this PTE skipping trick doesn't work? > > > > > > Databases with large shared memory segments shared between > > > many processes come to mind as a real-world example of a > > > worst case scenario. > > > > Well, I don't think you two are talking about the same thing. Andi > > was > > focusing on sparsity. Your example seems to be about sharing, i.e., > > ihgh mapcount. Of course both can happen at the same time, as I > > tested > > here: > > https://lore.kernel.org/linux-mm/YHFuL%2FDdtiml4biw@google.com/#t > > > > I'm skeptical that shared memory used by databases is that sparse, > > i.e., one page per PTE table, because the extremely low locality > > would > > heavily penalize their performance. But my knowledge in databases is > > close to zero. So feel free to enlighten me or just ignore what I > > said. > > A database may have a 200GB shared memory segment, > and a worker task that gets spun up to handle a > query might access only 1MB of memory to answer > that query. > > That memory could be from anywhere inside the > shared memory segment. Maybe some of the accesses > are more dense, and others more sparse, who knows? > > A lot of the locality > will depend on how memory > space inside the shared memory segment is reclaimed > and recycled inside the database. Thanks. Yeah, I guess we'll just need to see more benchmarks from the database realm. Stay tuned :)