From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF674C6FA81 for ; Fri, 2 Sep 2022 05:09:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 418F8800B2; Fri, 2 Sep 2022 01:09:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3C83C8008D; Fri, 2 Sep 2022 01:09:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2900D800B2; Fri, 2 Sep 2022 01:09:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 1802B8008D for ; Fri, 2 Sep 2022 01:09:26 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id DA01340944 for ; Fri, 2 Sep 2022 05:09:25 +0000 (UTC) X-FDA: 79865967090.25.8FAFBF2 Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) by imf05.hostedemail.com (Postfix) with ESMTP id 85CD810004D for ; Fri, 2 Sep 2022 05:09:25 +0000 (UTC) Received: by mail-pj1-f50.google.com with SMTP id i5-20020a17090a2a0500b001fd8708ffdfso4453412pjd.2 for ; Thu, 01 Sep 2022 22:09:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date; bh=PmZ+jeHheCTitnwYvU2IM5Mh5fQ9dWuFRDxNtLNgxGM=; b=kGybGFCn0r9hnHDD+XlTAUGy+rtoNiikm2mL5bvqn+0h4z1q4yzz3Ns7ajlCHdrdD7 g0dI/EU/xfSodqQS67E+p+MEscy/R40NUQ6JBx/HWVFnHy3NznELkLnyoGTOVzfThNfN XtUV1XgOogkXv8XVLpMF05vdye3kpjHgo0mQCxh2ik+qs/Fir1/o/Fs0PIyRUbXul2GF WpSSXSA+kfvF8yT/K0X1P7doQGU6kb6+o2xNezqlefMyIE6/nIfSKGiucjP/yRhloqs7 J42GoAmP2NFJbOIkChC90ot3MlQapEROsuhYzYMwxk2VNmJGTAj+NBikTfRICRZDlX2v B4gQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date; bh=PmZ+jeHheCTitnwYvU2IM5Mh5fQ9dWuFRDxNtLNgxGM=; b=k5sjTsTdlQ9DTHsa91700mP27YMLuxRc1AQA8ptEjdgE+IPpKwHm6Sv+XnVfY4hugi mRRL4RTTdWuedfRZRhgy65Q/x+96dhLojR+lCaUTsoMzDwZoxbnTta9WAgeqI3OLMZKz RzG59UQ6NHwU/WzSfHvUm7GVjTnJ6Fhue94G60O5kPb4KSSBlIBw0uIGh+ugJOAYy9p0 0DHuG9u8HsvTAhIyr7R3ohTmYZMnQp5gMX7EJk2doaAK8tfr7R2qqNNmHxrEFQBwKPfy bSEOcVxbFV9VmxYeEpgCKcmf/C7bOpq1iUWuFXvhPJGzruJjLB7+o0rAh/gw9yiEmFeN aGGA== X-Gm-Message-State: ACgBeo0fMw1o5REOXl22J0YqX36x803tIP5ZyvXfNRV7CKz+NCYMylSv 4hvzOBA9LC0BVwNIbM34MY+MUAk7DSpANT8b3rmw7A== X-Google-Smtp-Source: AA6agR4GqC333D5J7oqjHPpAb5xShJJ+b+rwcZLQGvuq6reZtSV2d3zGioXXI/2CJ1KNCtxqT/JIReg7dxj+9v2Mqek= X-Received: by 2002:a17:90b:3142:b0:1fa:f533:3191 with SMTP id ip2-20020a17090b314200b001faf5333191mr2851172pjb.193.1662095364249; Thu, 01 Sep 2022 22:09:24 -0700 (PDT) MIME-Version: 1.0 References: <20220830081736.119281-1-aneesh.kumar@linux.ibm.com> <87tu5rzigc.fsf@yhuang6-desk2.ccr.corp.intel.com> <87pmgezkhp.fsf@yhuang6-desk2.ccr.corp.intel.com> In-Reply-To: <87pmgezkhp.fsf@yhuang6-desk2.ccr.corp.intel.com> From: Wei Xu Date: Thu, 1 Sep 2022 22:09:13 -0700 Message-ID: Subject: Re: [PATCH v3 updated] mm/demotion: Expose memory tier details via sysfs To: "Huang, Ying" Cc: Aneesh Kumar K V , Johannes Weiner , Linux MM , Andrew Morton , Yang Shi , Davidlohr Bueso , Tim C Chen , Michal Hocko , Linux Kernel Mailing List , Hesham Almatary , Dave Hansen , Jonathan Cameron , Alistair Popple , Dan Williams , jvgediya.oss@gmail.com, Bharata B Rao , Greg Thelen Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1662095365; a=rsa-sha256; cv=none; b=earUkABp908G06Nm1cngjwFq0a89DNTi9OHP6PxK+jtpQAh2pIELVYa2IK2IbwgGy+7YSN /3PcFTkcsu+haeJmD1h0RjeETb9ytCAetmFOUlYUDRRpeFF2J5J9+sg+uYy96xwhIkokDu hIbJjgjCyPs+svJ9EK3GnK26UjH3JQk= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=kGybGFCn; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf05.hostedemail.com: domain of weixugc@google.com designates 209.85.216.50 as permitted sender) smtp.mailfrom=weixugc@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1662095365; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PmZ+jeHheCTitnwYvU2IM5Mh5fQ9dWuFRDxNtLNgxGM=; b=aVbMOhWILdjB3c03fs9MItfqhr3+TOidUKfSd+6B0H0+BRzjN+pSfGf6bufbmA66n5A5Jf Cx4j2vTkPEZW95gEU9u95mRsaFT9Jf9BLR80pEuoLtRK9CWGWIsyNryUBpRaJpTO+ibEq6 WCjVljXWzA5axXK1WVrponQ9trApwUM= X-Rspamd-Server: rspam09 Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=kGybGFCn; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf05.hostedemail.com: domain of weixugc@google.com designates 209.85.216.50 as permitted sender) smtp.mailfrom=weixugc@google.com X-Rspam-User: X-Stat-Signature: 6qu4ksboinqgi5z6w8bydhgz5c37ic53 X-Rspamd-Queue-Id: 85CD810004D X-HE-Tag: 1662095365-242374 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Sep 1, 2022 at 5:33 PM Huang, Ying wrote: > > Aneesh Kumar K V writes: > > > On 9/1/22 12:31 PM, Huang, Ying wrote: > >> "Aneesh Kumar K.V" writes: > >> > >>> This patch adds /sys/devices/virtual/memory_tiering/ where all memory= tier > >>> related details can be found. All allocated memory tiers will be list= ed > >>> there as /sys/devices/virtual/memory_tiering/memory_tierN/ > >>> > >>> The nodes which are part of a specific memory tier can be listed via > >>> /sys/devices/virtual/memory_tiering/memory_tierN/nodes > >> > >> I think "memory_tier" is a better subsystem/bus name than > >> memory_tiering. Because we have a set of memory_tierN devices inside. > >> "memory_tier" sounds more natural. I know this is subjective, just my > >> preference. > >> > >>> > >>> A directory hierarchy looks like > >>> :/sys/devices/virtual/memory_tiering$ tree memory_tier4/ > >>> memory_tier4/ > >>> =E2=94=9C=E2=94=80=E2=94=80 nodes > >>> =E2=94=9C=E2=94=80=E2=94=80 subsystem -> ../../../../bus/memory_tieri= ng > >>> =E2=94=94=E2=94=80=E2=94=80 uevent > >>> > >>> All toptier nodes are listed via > >>> /sys/devices/virtual/memory_tiering/toptier_nodes > >>> > >>> :/sys/devices/virtual/memory_tiering$ cat toptier_nodes > >>> 0,2 > >>> :/sys/devices/virtual/memory_tiering$ cat memory_tier4/nodes > >>> 0,2 > >> > >> I don't think that it is a good idea to show toptier information in us= er > >> space interface. Because it is just a in kernel implementation > >> details. Now, we only promote pages from !toptier to toptier. But > >> there may be multiple memory tiers in toptier and !toptier, we may > >> change the implementation in the future. For example, we may promote > >> pages from DRAM to HBM in the future. > >> > > > > > > In the case you describe above and others, we will always have a list o= f > > NUMA nodes from which memory promotion is not done. > > /sys/devices/virtual/memory_tiering/toptier_nodes shows that list. > > I don't think we will need that interface if we don't restrict promotion > in the future. For example, he can just check the memory tier with > smallest number. > > TBH, I don't know why do we need that interface. What is it for? We > don't want to expose unnecessary information to restrict our in kernel > implementation in the future. > > So, please remove that interface at least before we discussing it > thoroughly. I have asked for this interface to allow the userspace to query a list of top-tier nodes as the targets of userspace-driven promotions. The idea is that demotion can gradually go down tier by tier, but we promote hot pages directly to the top-tier and bypass the immediate tiers. Certainly, this can be viewed as a policy choice. Given that now we have a clearly defined memory tier hierarchy in sysfs and the toptier_nodes content can be constructed from this memory tier hierarchy and other information from the node sysfs interfaces, I am fine if we want to remove toptier_nodes and keep the current memory tier sysfs interfaces to the minimal. Wei Xu > >> Do we need a way to show the default memory tier in sysfs? That is, t= he > >> memory tier that the DRAM nodes belong to. > >> > > > > I will hold adding that until we have support for modifying memory tier= details from > > userspace. That is when userspace would want to know about the default = memory tier. > > > > For now, the user interface is a simpler hierarchy of memory tiers, it'= s associated > > nodes and the list of nodes from which promotion is not done. > > OK. > > Best Regards, > Huang, Ying