From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C275ECAAA3 for ; Fri, 26 Aug 2022 09:16:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 679B6940007; Fri, 26 Aug 2022 05:16:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 629196B0075; Fri, 26 Aug 2022 05:16:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4F05C940007; Fri, 26 Aug 2022 05:16:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 4128E6B0074 for ; Fri, 26 Aug 2022 05:16:04 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 0F81FAB8EC for ; Fri, 26 Aug 2022 09:16:04 +0000 (UTC) X-FDA: 79841187048.17.5C66A23 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by imf25.hostedemail.com (Postfix) with ESMTP id B4E6FA0015 for ; Fri, 26 Aug 2022 09:16:03 +0000 (UTC) Received: by mail-pj1-f47.google.com with SMTP id t5so1099877pjs.0 for ; Fri, 26 Aug 2022 02:16:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc; bh=8JkET+M6JudLsbBuC9noyGBV8761Vdk3MiQ4Gj3TP64=; b=g6yLbjzWZuEFnACRvvSR6Adru4uTw/gGAP8IJNcaVUWwTnvbMLYZt3WL2LyN/FOJFp kBye54Rg63Bmpyz2mlYRrr85ZkCzdXYbAMa6WJlE/M+WQhd7kK3FdK4uQOl4PPmnKmWg CfBL/7ns8M+T1fwjHLb1H73YxxHhS1RF2DlgU4fkHg+hVoxqseAqK/AkgLaGhq9ZY9bc tQk/Q4zZrvmL3rCKHu+vstq5o3S4kw53SvWTOxrzgUCQceMpMukGRUD31q4g2AI/Lblt ANwz7+ntVAJCfFVV6cQxYoZ8p5uEd+FvOiK0Q31Zprnp4cgygJVx/uXHLLMc05UVTuup hd2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc; bh=8JkET+M6JudLsbBuC9noyGBV8761Vdk3MiQ4Gj3TP64=; b=ESo3+DdCX77JRsvcb1lvQeYo/Otdym6XQ6frMV9TDVmR+s8lpwercxE0MW6C07o5W/ I3VpykngfBG02nfDVLnq1DdzLfVgRKZlZzZW1fMrvS63fewp79HBA2J7PI1FJLmAnuVv Y866rIJp7AAwVHG9+qRrkpuDqYzO5NFzx1PaufHW3An0bYVjFyCqD3DUZvuob/A5V88K +tUCniGPElHkTN5nouWD/oimmdSAtE6gBsFnTKMFKSW3JvcoPz8kDL6pBClhqjCWSOAy mzEpZNdX7qUrisAW54R3ZAeOQvuOirq/Byy9fE4Ovh4xg3W7yiRjE/wEyR97yVYvJk5S Oeww== X-Gm-Message-State: ACgBeo2kVVzB7Xa/8lgLjKRO6FMN2GJLcfXMrp/CDHDJdX0PFmU4lfcC W7xTdz1OLyHpaD4qbGoFb3FjIZlenhWEgIYzDWo1uQ== X-Google-Smtp-Source: AA6agR6pLmtwtrIZjtq/uf4J+tCnWLD3/QKIIUwGVBXNRbfKwUf4NIYdl+0IKMavTxhCIcoJr1gggCaPHxCmXpc0w+8= X-Received: by 2002:a17:90b:1804:b0:1f5:946:6b6f with SMTP id lw4-20020a17090b180400b001f509466b6fmr3414480pjb.160.1661505362672; Fri, 26 Aug 2022 02:16:02 -0700 (PDT) MIME-Version: 1.0 References: <20220825092325.381517-1-aneesh.kumar@linux.ibm.com> <877d2v3h8s.fsf@yhuang6-desk2.ccr.corp.intel.com> <4abb84e8-8035-65aa-941a-98f0d7902c42@linux.ibm.com> In-Reply-To: <4abb84e8-8035-65aa-941a-98f0d7902c42@linux.ibm.com> From: Wei Xu Date: Fri, 26 Aug 2022 02:15:51 -0700 Message-ID: Subject: Re: [RFC PATCH 1/2] mm/demotion: Expose memory type details via sysfs To: Aneesh Kumar K V Cc: "Huang, Ying" , Linux MM , Andrew Morton , Yang Shi , Davidlohr Bueso , Tim C Chen , Michal Hocko , Linux Kernel Mailing List , Hesham Almatary , Dave Hansen , Jonathan Cameron , Alistair Popple , Dan Williams , Johannes Weiner , jvgediya.oss@gmail.com, Bharata B Rao Content-Type: text/plain; charset="UTF-8" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1661505363; a=rsa-sha256; cv=none; b=UplRYP5BjB861bFamdbCEeTxHZvbRzFdrGvJj8711g3Q/AUYVXPDLFi55H44KzWVmOjIZp emp4jIxgcgBek4tfwIDiO/sveKjM14w7tZpwUA4pzg4nR4HbxqiMQbCW5plUJJg/tmgPAf AGd9XBI4geTlBqL3zfBbpd0vaj5WDfo= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=g6yLbjzW; spf=pass (imf25.hostedemail.com: domain of weixugc@google.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=weixugc@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1661505363; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8JkET+M6JudLsbBuC9noyGBV8761Vdk3MiQ4Gj3TP64=; b=hTiBPnEpY0tDI9uKMtR7Ke39Uuy+aVHNvntmYNsVhECBHV6u8FhXvEcbseylwsRKH9YDj2 xAA1BLmuivmGd2rudxpThoiiSMA8lVuSijmB3lLqccTWJm3+dz3kBija2nCZXBilJ8ji5R c1qiXQTM8fZRrxj6E5ODXRPtAYTQTfc= Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=g6yLbjzW; spf=pass (imf25.hostedemail.com: domain of weixugc@google.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=weixugc@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: 4unbi8cdcpd967hs1todaqaioeqfx64q X-Rspamd-Queue-Id: B4E6FA0015 X-HE-Tag: 1661505363-777665 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Aug 26, 2022 at 1:05 AM Aneesh Kumar K V wrote: > > On 8/26/22 1:30 PM, Wei Xu wrote: > > On Thu, Aug 25, 2022 at 8:00 PM Aneesh Kumar K V > > wrote: > >> > >> On 8/26/22 7:20 AM, Huang, Ying wrote: > >>> "Aneesh Kumar K.V" writes: > >>> > >>>> This patch adds /sys/devices/virtual/memtier/ where all memory tier related > >>>> details can be found. All allocated memory types will be listed there as > >>>> /sys/devices/virtual/memtier/memtypeN/ > >>> > >>> Another choice is to make memory types and memory tiers system devices. > >>> That is, > >>> > >>> /sys/devices/system/memory_type/memory_typeN > >>> /sys/devices/system/memory_tier/memory_tierN > >>> > >> > >> subsys_system_register() documentation says > >> > >> * Do not use this interface for anything new, it exists for compatibility > >> * with bad ideas only. New subsystems should use plain subsystems; and > >> * add the subsystem-wide attributes should be added to the subsystem > >> * directory itself and not some create fake root-device placed in > >> * /sys/devices/system/. > >> > >> memtier being a virtual device, I was under the impression that /sys/devices/virtual > >> is the recommended place. > >> > >>> That looks more natural to me. Because we already have "node" and > >>> "memory" devices there. Why don't you put memory types and memory tiers > >>> there? > >>> > >>> And, I think we shouldn't put "memory_type" in the "memory_tier" > >>> directory. "memory_type" isn't a part of "memory_tier". > >>> > >> > >> I was looking consolidating both memory tier and memory type into the same sysfs subsystem. > >> Your recommendation imply we create two subsystem memory_tier and memtype. I was > >> trying to avoid that. May be a generic term like "memory_tiering" can help to > >> consolidate all tiering related details there? > >> > > > > A generic term "memory_tiering" sounds good to me. > > > > Given that this will be a user-facing, stable kernel API, I think we'd > > better to only add what is most useful for userspace and don't have to > > mirror the kernel internal data structures in this interface. > > > > My understanding is that we haven't fully settled down on how to > > customize memory tiers from userspace. So we don't have to show > > memory_type yet, which is a kernel data structure at this point. > > > > The userspace does need to know what are the memory tiers and which > > NUMA nodes are included in each memory tier. How about we provide the > > "nodelist" interface for each memory tier as in the original proposal? > > > > The userspace would also like to know which memory tiers/nodes belong > > to the top tiers (the promotion targets). We can provide a "toptiers" > > or "toptiers_nodelist" interface to report that. > > > > How about also including abstract distance range of a memory tier? > That will be useful to derive the hierarchy. With the base abstract distance in the memtier name, do we need to show the abstract distance range if we don't customize memory tiers? > > Both should still be useful even if we decide to add memory_type for > > memory tier customization. > > > > -aneesh