From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4591AC432BE for ; Thu, 2 Sep 2021 03:25:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CDA92610A4 for ; Thu, 2 Sep 2021 03:25:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org CDA92610A4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id E725F8D0002; Wed, 1 Sep 2021 23:25:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E22FA8D0001; Wed, 1 Sep 2021 23:25:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D10768D0002; Wed, 1 Sep 2021 23:25:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0047.hostedemail.com [216.40.44.47]) by kanga.kvack.org (Postfix) with ESMTP id C2F9C8D0001 for ; Wed, 1 Sep 2021 23:25:43 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 7D7F118211CF6 for ; Thu, 2 Sep 2021 03:25:43 +0000 (UTC) X-FDA: 78541193766.11.E191B6F Received: from mail-pg1-f178.google.com (mail-pg1-f178.google.com [209.85.215.178]) by imf08.hostedemail.com (Postfix) with ESMTP id 23EA93000098 for ; Thu, 2 Sep 2021 03:25:43 +0000 (UTC) Received: by mail-pg1-f178.google.com with SMTP id t1so493300pgv.3 for ; Wed, 01 Sep 2021 20:25:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=date:from:subject:to:cc:references:in-reply-to:mime-version :message-id:content-transfer-encoding; bh=Hdf7I2a0TwdT9/hf+sdOXkC7oi98e/8luB2lwlTVGw8=; b=ao5rBVuN0+0lULIb33de3tPsOUWNRGXExvnOSbodha/awyoHM9WLwz9pXtV2x7k+w0 WmIDdakhfW5/rw/7LzPaPCJ/7zisXfUh1dxCNGwHcvT+k1LtTth2Opu72YU/9dqibl2Q bwMtuf6+TItNc0RAKOCYlJqR97QsNz8OBqJsR/AJCY3emyurvyKuiZ7fB/tA3m8RBYor GRvG3uAFgHHo+g2VADVI8DkBO8ZRHnu4DATR6/6flAbTONUJ0xq7FqCtiOH4gcBFska3 5MhiTL2w7rOM5J/ZvsYwS+uuBSxbp3XVlZgJg4KBlY6muZFHcN2ZD3Ec4pXRVAWeRnYf 2tBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:subject:to:cc:references:in-reply-to :mime-version:message-id:content-transfer-encoding; bh=Hdf7I2a0TwdT9/hf+sdOXkC7oi98e/8luB2lwlTVGw8=; b=iz0gwQD3m2NQjFHCHgdgWOjsD8pyQRbWNLIA3/sPbcBSiZxTLl+CM6IxJZOaquiwpY 6mlw04yp/rCA73Df1oEh970eiglNKxvfjJmHdGJ/g3GD0FfI7LDWg/hKzD+e+x4QvaAN 9DOQdYGjnFqc0PDMjuk56sbAyRRdIrxvY/BKtva5cL+ESG2klEQI8WDMEmsrIeGAMqwX zt8Yl8/uYqUWgrkLyUgDPs9MidgdUSu/m9w+PJL9Elks1OUaYsHjCkBlNS1TqDQqUkYy VQ6YHdXcZPkIj4lzt6x0oVjn6CUYHZVd/TWb08xnnYsdONf5no1+9aZIUENssNC7A3GK ypWg== X-Gm-Message-State: AOAM530WTLjsDPxUgFmEh5QxvqR4Pn7SsssrN2aBhuj711v1gg3E6zzq gYKJ/uCs8/NmfCBbgBiD/T0= X-Google-Smtp-Source: ABdhPJwZ4QizbvpLRKy6rIt2yM+imsF/wi8Dqc8V1XetpB+cdH1Krbou7JfpujSg4375FmbenWs+WA== X-Received: by 2002:a05:6a00:1ac7:b0:3e2:2d05:3b31 with SMTP id f7-20020a056a001ac700b003e22d053b31mr1206537pfv.2.1630553142069; Wed, 01 Sep 2021 20:25:42 -0700 (PDT) Received: from localhost (220-244-72-10.tpgi.com.au. [220.244.72.10]) by smtp.gmail.com with ESMTPSA id l11sm359345pjg.22.2021.09.01.20.25.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Sep 2021 20:25:41 -0700 (PDT) Date: Thu, 02 Sep 2021 13:25:36 +1000 From: Nicholas Piggin Subject: Re: Is it possible to implement the per-node page cache for programs/libraries? To: Shijie Huang , Matthew Wilcox Cc: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, song.bao.hua@hisilicon.com, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, Frank Wang References: In-Reply-To: MIME-Version: 1.0 Message-Id: <1630552995.2mupnzoqzs.astroid@bobo.none> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=ao5rBVuN; spf=pass (imf08.hostedemail.com: domain of npiggin@gmail.com designates 209.85.215.178 as permitted sender) smtp.mailfrom=npiggin@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 23EA93000098 X-Stat-Signature: t5kwbir1gkjb66msizqxq48ppka9rf3f X-HE-Tag: 1630553143-528312 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Excerpts from Matthew Wilcox's message of September 1, 2021 1:25 pm: > On Wed, Sep 01, 2021 at 11:07:41AM +0800, Shijie Huang wrote: >> In the NUMA, we only have one page cache for each file. For the >> program/shared libraries, the >> remote-access delays longer then the local-access. >>=20 >> So, is it possible to implement the per-node page cache for >> programs/libraries? >=20 > At this point, we have no way to support text replication within a > process. So what you're suggesting (if implemented) would work for > processes which limit themselves to a single node. That is, if you > have a system with CPUs 0-3 on node 0 and CPUs 4-7 on node 1, a process > which only works on node 0 or only works on node 1 will get text on the > appropriate node. >=20 > If there's a process which runs on both nodes 0 and 1, there's no support > for per-node PGDs. So it will get a mix of pages from nodes 0 and 1, > and that doesn't necessarily seem like a big win. I haven't yet dived > into how hard it would be to make mm->pgd a per-node allocation. >=20 > I have been thinking about this a bit; one of our internal performance > teams flagged the potential performance win to me a few months ago. > I don't have a concrete design for text replication yet; there have been > various attempts over the years, but none were particularly compelling. What was not compelling about it? https://lists.openwall.net/linux-kernel/2007/07/27/112 What are the other attempts? Thanks, Nick