From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9429AC433EF for ; Thu, 12 May 2022 21:32:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C4DAC6B0073; Thu, 12 May 2022 17:32:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BFB256B0075; Thu, 12 May 2022 17:32:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A9C1E6B0078; Thu, 12 May 2022 17:32:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9A7FC6B0073 for ; Thu, 12 May 2022 17:32:01 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 70402608E9 for ; Thu, 12 May 2022 21:32:01 +0000 (UTC) X-FDA: 79458388842.23.5EF24E4 Received: from mail-ua1-f42.google.com (mail-ua1-f42.google.com [209.85.222.42]) by imf30.hostedemail.com (Postfix) with ESMTP id B12C3800AC for ; Thu, 12 May 2022 21:31:41 +0000 (UTC) Received: by mail-ua1-f42.google.com with SMTP id i16so2518731uat.5 for ; Thu, 12 May 2022 14:32:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=AAEvdTjpMiH02kk60TytbztnqV1ZmA0Yb2xj1ek1p2c=; b=AQd8ugEBqMeVcQ3c+P4owmAOI3jptgmmv/yVY0Hzubgs1BBkmPok2U3o4M4aCk+f9G bEmzXF7EAuns52hfOWdAIBe2eeGl6SJHg2wuaNag32vrrilFFSFqQw9E+vCOmlJuaYJL znmmUGgr18xmZC9qrjprWUDXibliidJx8DxzkUX7xZfi1wajzi/WYEIikt9z8aUiT5cp pMqJnOziS0on8jMnrq2brRt0OI3EtBN33RdO1PHJCzBAhjL7EjDjPEJmdwGSwheiA/tK 9cWCtEnahNxMgy0+sd0rvPJWcdzATvKC6vFKqbHb9wxLzNWiXts5Bx8pmjdUhPnDXY27 pdOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=AAEvdTjpMiH02kk60TytbztnqV1ZmA0Yb2xj1ek1p2c=; b=bLSFPfPCNE5lk4MYxSjuBQZ7JkFh1ZRLJRFmThQJPKiRVDXyVmtZc4NqwTzXTi40Cu KPIG/DzCoMdCEhyoUFIwg2sqveQvyZMDlnGc2Dpdqw1CDE4TtN8yY45POIjpPZACQItr Qq/eT6h6iLUzMd+6AdY6hbVoZlPaY0ymr68BQPCZgMI+l7Sz++jAzeNITJhVtLQTb3gR DWyMc7uA2fRkZGhXlG95DzsEiIFavlmfoAE/PJrrpZctfCR8bgsjSVt14chBaC+Mq1Nx dWfrvvmrU2yijhM9JiKHInXGYA5hF4mJ4DtYl8ZvuYgVTZU3MyqSza3An6PDa0zx8eGt iQNg== X-Gm-Message-State: AOAM530pSfYdLsMoeqPDBvOb8DWJdvLdWdpzfYJvdgY+rMsxgvRyCbBS bNI7rShkXoGiNTy3WXgtUXTC5YU53adFGlnXeSeykg== X-Google-Smtp-Source: ABdhPJzUBFca9v1LokdN/wm4ymV/l1Ks2bnzIIoADv2YZ78qVG92Y28zMfMniJ4L1epdqiiQt2gndkyf7XXgySKXvH8= X-Received: by 2002:ab0:e14:0:b0:360:e13:e5d7 with SMTP id g20-20020ab00e14000000b003600e13e5d7mr1229790uak.95.1652391120104; Thu, 12 May 2022 14:32:00 -0700 (PDT) MIME-Version: 1.0 References: <56b41ce6922ed5f640d9bd46a603fa27576532a9.camel@intel.com> <87y1z7jj85.fsf@linux.ibm.com> <333438929a8680317bdae02da921d1e50fb9b0b3.camel@linux.intel.com> In-Reply-To: <333438929a8680317bdae02da921d1e50fb9b0b3.camel@linux.intel.com> From: Wei Xu Date: Thu, 12 May 2022 14:31:49 -0700 Message-ID: Subject: Re: RFC: Memory Tiering Kernel Interfaces (v2) To: Tim Chen Cc: "Aneesh Kumar K.V" , "ying.huang@intel.com" , Andrew Morton , Greg Thelen , Yang Shi , Linux Kernel Mailing List , Jagdish Gediya , Michal Hocko , Tim C Chen , Dave Hansen , Alistair Popple , Baolin Wang , Feng Tang , Jonathan Cameron , Davidlohr Bueso , Dan Williams , David Rientjes , Linux MM , Brice Goglin , Hesham Almatary Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: B12C3800AC X-Stat-Signature: xe7s1599rer8kchmmns8r8fmx188sgks X-Rspam-User: Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=AQd8ugEB; spf=pass (imf30.hostedemail.com: domain of weixugc@google.com designates 209.85.222.42 as permitted sender) smtp.mailfrom=weixugc@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam09 X-HE-Tag: 1652391101-686138 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, May 12, 2022 at 2:13 PM Tim Chen wrote: > > On Thu, 2022-05-12 at 01:15 -0700, Wei Xu wrote: > > > > I am OK with moving back the memory tier nodelist into node/. When > > there are more memory tier attributes needed, we can then create the > > memory tier subtree and replace the tier nodelist in node/ with > > symlinks. > > > > So the revised sysfs interfaces are: > > > > * /sys/devices/system/node/memory_tierN (read-only) > > > > where N = 0, 1, 2 > > > > Format: node_list > > > > * /sys/devices/system/node/nodeN/memory_tier (read/write) > > > > where N = 0, 1, ... > > > > Format: int or empty > > This looks good to me. Just wonder if having just 1 tier > lower than DRAM is sufficient. We could have wide performance > range for such secondary memories and is one tier sufficient for them? > > Tim The tier design can be extended to more than 3 tiers (e.g. via CONFIG_MAX_MEMORY_TIERS). MAX_MEMORY_TIERS is set to 3 for now because without enough memory device performance information provided by the firmware, it is difficult for the kernel to properly initialize the memory tier hierarchy beyond 3 tiers (GPU, DRAM, PMEM). We will have to resort to the userspace override to set up such many-tier systems.