From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FCF0C433EF for ; Wed, 25 May 2022 15:33:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B52E58D0005; Wed, 25 May 2022 11:33:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B25B38D0002; Wed, 25 May 2022 11:33:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C7EC8D0005; Wed, 25 May 2022 11:33:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 8D0318D0002 for ; Wed, 25 May 2022 11:33:56 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 5EC28359AE for ; Wed, 25 May 2022 15:33:56 +0000 (UTC) X-FDA: 79504660872.09.F015973 Received: from mail-vk1-f179.google.com (mail-vk1-f179.google.com [209.85.221.179]) by imf29.hostedemail.com (Postfix) with ESMTP id BD390120015 for ; Wed, 25 May 2022 15:33:44 +0000 (UTC) Received: by mail-vk1-f179.google.com with SMTP id b81so10064523vkf.1 for ; Wed, 25 May 2022 08:33:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=L/HY259bVjGUdiARIFEb7gVit9w/Zkrf9kFmEOh5p/U=; b=Br7XN5sO4nabySMnczCeP5za9dMJNdr9gwLuivgSSSnA50/FkjJwB96mDayUI8QRES hO/toJToNOC6dV58kDbdPK5ItMV8R0isjxTMf2HJ2HIVhuAb0tx6aWJbEzCZgiurJj4S 8IRHPIYQjxxtpmRzL0jjEQza+3NTJwxlVn33rL9E0wVOCDA7o4MsUbehJxJipL/8H/64 7jnBCIQjfuVEfFDHxd0ZyPtKsoiyJ8EPTs3JwkbjZd4zNUS0LGi0TThTp0srN66qrMUL j9/vWXZGsBaQP6JAbzjVacpgNqLaVZ2+x2Q93KUiELXkWJf6GS9jC0Fq/uvSxkmKJlvy BXvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=L/HY259bVjGUdiARIFEb7gVit9w/Zkrf9kFmEOh5p/U=; b=bjZo2Jrzt7ia6ab4vfcYSuwnVqL1ujmIwYCSyPm8bXAVWCz4fSmpth5KVcCpNe/3K5 1itKq1mLVyEC1XLMF5cDtQBtaSVVkQYuUIQmYCimo8GMtYaih3xW/RJC7/KGqXC3SoVT 43dKGt1Ciy9Xj6ZWIsP/nhV2w1vRN/JpP19bUEZpsiSkWJ6FgO9PVG1cnm9F+HBdkjdn A1B9yq40xTVhGhu54Nq+y+bUxhelrTzwzEXbmGaDcq7ewjFJkQtoCyX1LjPXdxr/uTEr UQwR7nEa2Bt+YdY+9FFOWZA1IGF7VIDEAhHG1yp8El5vJQiFh19eRvtvtGFhtGaxffEL tyTQ== X-Gm-Message-State: AOAM530+iYlMs4pvd1wu14geAlp/Fm9dwYPTIh5FrCqpCWf8hSlb846Q VjTq/Z4gbnW7pI2uVHZuW90DVhbp43NdVeGWKv6rFw== X-Google-Smtp-Source: ABdhPJxsBryrDmT00sVFdPgNC8WS5tL2SlJYLuVJpna+dMdvH9lsPuVChzXc3nFL/7AEPsCRF0ASCrzJxFpRB4uUMV0= X-Received: by 2002:a05:6122:17a1:b0:357:840c:48c1 with SMTP id o33-20020a05612217a100b00357840c48c1mr7129196vkf.38.1653492834969; Wed, 25 May 2022 08:33:54 -0700 (PDT) MIME-Version: 1.0 References: <20220512160010.00005bc4@Huawei.com> <6b7c472b50049592cde912f04ca47c696caa2227.camel@intel.com> <6ce724e5c67d4f7530457897fa08d0a8ba5dd6d0.camel@intel.com> <594046f8-9ab3-786a-fc48-8a61f1238f52@linux.ibm.com> <2d06932c-7d80-94c7-89ab-78375e58d3cc@redhat.com> In-Reply-To: <2d06932c-7d80-94c7-89ab-78375e58d3cc@redhat.com> From: Wei Xu Date: Wed, 25 May 2022 08:33:43 -0700 Message-ID: Subject: Re: RFC: Memory Tiering Kernel Interfaces (v2) To: =?UTF-8?Q?Mika_Penttil=C3=A4?= Cc: Aneesh Kumar K V , Ying Huang , Jonathan Cameron , Andrew Morton , Greg Thelen , Yang Shi , Linux Kernel Mailing List , Jagdish Gediya , Michal Hocko , Tim C Chen , Dave Hansen , Alistair Popple , Baolin Wang , Feng Tang , Davidlohr Bueso , Dan Williams , David Rientjes , Linux MM , Brice Goglin , Hesham Almatary Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: 589xdssmngjowtaibfhjfosjbiyktu98 X-Rspam-User: Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Br7XN5sO; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf29.hostedemail.com: domain of weixugc@google.com designates 209.85.221.179 as permitted sender) smtp.mailfrom=weixugc@google.com X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: BD390120015 X-HE-Tag: 1653492824-752405 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, May 25, 2022 at 4:37 AM Mika Penttil=C3=A4 wr= ote: > > > > On 25.5.2022 13.01, Aneesh Kumar K V wrote: > > On 5/25/22 2:33 PM, Ying Huang wrote: > >> On Tue, 2022-05-24 at 22:32 -0700, Wei Xu wrote: > >>> On Tue, May 24, 2022 at 1:24 AM Ying Huang wro= te: > >>>> > >>>> On Tue, 2022-05-24 at 00:04 -0700, Wei Xu wrote: > >>>>> On Thu, May 19, 2022 at 8:06 PM Ying Huang > >>>>> wrote: > >>>>>> > > > > ... > > > >> > >> OK. Just to confirm. Does this mean that we will have fixed device I= D, > >> for example, > >> > >> GPU memtier255 > >> DRAM (with CPU) memtier0 > >> PMEM memtier1 > >> > >> When we add a new memtier, it can be memtier254, or memter2? The rank > >> value will determine the real demotion order. > >> > >> I think you may need to send v3 to make sure everyone is at the same > >> page. > >> > > > > What we have implemented which we will send as RFC shortly is below. > > > > cd /sys/dekvaneesh@ubuntu-guest:~$ cd /sys/devices/system/ > > kvaneesh@ubuntu-guest:/sys/devices/system$ pwd > > /sys/devices/system > > kvaneesh@ubuntu-guest:/sys/devices/system$ ls > > clockevents clocksource container cpu edac memory memtier mpic > > node power > > kvaneesh@ubuntu-guest:/sys/devices/system$ cd memtier/ > > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ pwd > > /sys/devices/system/memtier > > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ ls > > default_rank max_rank memtier1 power uevent > > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ cat default_rank > > 1 > > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ cat max_rank > > 3 > > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ cd memtier1/ > > kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ ls > > nodelist power rank subsystem uevent > > kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ cat nodelis= t > > 0-3 > > kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ cat rank > > 1 > > kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ cd > > ../../node/node1/ > > kvaneesh@ubuntu-guest:/sys/devices/system/node/node1$ cat memtier > > 1 > > kvaneesh@ubuntu-guest:/sys/devices/system/node/node1$ > > root@ubuntu-guest:/sys/devices/system/node/node1# echo 0 > memtier > > root@ubuntu-guest:/sys/devices/system/node/node1# cat memtier > > 0 > > root@ubuntu-guest:/sys/devices/system/node/node1# cd ../../memtier/ > > root@ubuntu-guest:/sys/devices/system/memtier# ls > > default_rank max_rank memtier0 memtier1 power uevent > > root@ubuntu-guest:/sys/devices/system/memtier# cd memtier0/ > > root@ubuntu-guest:/sys/devices/system/memtier/memtier0# cat nodelist > > 1 > > root@ubuntu-guest:/sys/devices/system/memtier/memtier0# cat rank > > 0 > > root@ubuntu-guest:/sys/devices/system/memtier/memtier0# echo 4 > rank > > bash: rank: Permission denied > > root@ubuntu-guest:/sys/devices/system/memtier/memtier0# > > > > Just to confirm, unlike today's demotion code, the demotion target > allocation is planned to honor mempolicies? Yes, though there will be some limitations in the beginning, specifically for per-thread mempolicy. > > --Mika > >