From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D13A8CCA47C for ; Tue, 7 Jun 2022 20:18:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 48B9C6B007B; Tue, 7 Jun 2022 16:18:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 43B766B0081; Tue, 7 Jun 2022 16:18:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 303D56B0088; Tue, 7 Jun 2022 16:18:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 213206B007B for ; Tue, 7 Jun 2022 16:18:44 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 999B281049 for ; Tue, 7 Jun 2022 20:18:43 +0000 (UTC) X-FDA: 79552552926.16.89FA781 Received: from mail-ua1-f48.google.com (mail-ua1-f48.google.com [209.85.222.48]) by imf04.hostedemail.com (Postfix) with ESMTP id 524B540041 for ; Tue, 7 Jun 2022 20:18:43 +0000 (UTC) Received: by mail-ua1-f48.google.com with SMTP id o8so6206804uap.6 for ; Tue, 07 Jun 2022 13:18:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=TnwEpfB2bSSlI/sDocPm/aLibJux3ex7zyjvAuXKleM=; b=Q970Q+TTRk2mNBIP+W7nqQ1owOGjhQkURSJAN/rlGK7DCp3E8IuFXHlQ6omQurFMML P00hc7svC+IK1CNZOppZboVBoaIGFJYRFdJonvAhEDdzFhkJZiMGPTaIRdh3oBjNXitU bQSDjgOStKKB+M0qguoq133Jm0jlrHr+nXhCe9V2351Xfl7wTlbSxLV2U0CKWCejpvIw 3wR6dNTZKgdzOY/1uSwwbhWlqS/f25Ht0Xja29KhmJ5Xmztq3pdy+gSGpg5V6BKVZngK wWU7YFIoWgrOPOsJROn2B6qwjFEBqs77UXKyIFNNGNXLBtypzXYg/IIMOilCDFfmxES/ a/qA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=TnwEpfB2bSSlI/sDocPm/aLibJux3ex7zyjvAuXKleM=; b=LXLTuex1D8mFr1uZ4KhFOWr6Gj3DeX2Me41exnukZwl1+somjG3gTXcWwl3sNEf8z8 H48epI26wMstQ4WATTa80ck+iuHAx5n1Jnq2pIr72dyD5YjBD/hfr4CZ2zfYtDo1qN8Q pVihB2ScLvU3c21XTDaqX+l98z4yiWRAG5sFy34HYe187fIEJkoaXwE7DExeofEq+aQd NLqKd2WArVmnDhYA/HyTo7AlMmxirbePnShJfUuYZoy2axbXnPfEqFNqAI8qc8SeXhpm /JOpnGspom2TfakOzrrLA266gl/ExUULkHkTcRWUw5/YrYz/nup8GAlC6JRYjfSSX1K2 VPkw== X-Gm-Message-State: AOAM530+YGTNqRBXOdzPePTCUujcZTFqqFSjbIm5dFy9w7nOEBVk3BZR zoD09iVu2It3/KMbj444iUUe0fE3k81BZvS+uWFz2g== X-Google-Smtp-Source: ABdhPJxVJB6Dm8XEawl4PpBTwoDSrTFqr21rPVQiWO7fECO+iwYALaOcxZffBDNGyU6ZOPxOwnW7Awb3DeiEEuO+UNk= X-Received: by 2002:ab0:349a:0:b0:35c:b898:a733 with SMTP id c26-20020ab0349a000000b0035cb898a733mr34099691uar.85.1654633122327; Tue, 07 Jun 2022 13:18:42 -0700 (PDT) MIME-Version: 1.0 References: <20220603134237.131362-1-aneesh.kumar@linux.ibm.com> <20220603134237.131362-2-aneesh.kumar@linux.ibm.com> <92649c9a6e0b6931b34aeaaf22c0a1e874484b7f.camel@linux.intel.com> In-Reply-To: <92649c9a6e0b6931b34aeaaf22c0a1e874484b7f.camel@linux.intel.com> From: Wei Xu Date: Tue, 7 Jun 2022 13:18:31 -0700 Message-ID: Subject: Re: [PATCH v5 1/9] mm/demotion: Add support for explicit memory tiers To: Tim Chen Cc: "Aneesh Kumar K.V" , Linux MM , Andrew Morton , Huang Ying , Greg Thelen , Yang Shi , Davidlohr Bueso , Tim C Chen , Brice Goglin , Michal Hocko , Linux Kernel Mailing List , Hesham Almatary , Dave Hansen , Jonathan Cameron , Alistair Popple , Dan Williams , Feng Tang , Jagdish Gediya , Baolin Wang , David Rientjes Content-Type: text/plain; charset="UTF-8" Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Q970Q+TT; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf04.hostedemail.com: domain of weixugc@google.com designates 209.85.222.48 as permitted sender) smtp.mailfrom=weixugc@google.com X-Stat-Signature: zaxx9nqmcnr4st1jm88a7kw5i5bzamax X-Rspam-User: X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 524B540041 X-HE-Tag: 1654633123-894690 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jun 7, 2022 at 11:43 AM Tim Chen wrote: > > On Fri, 2022-06-03 at 19:12 +0530, Aneesh Kumar K.V wrote: > > > > > > The nodes which are part of a specific memory tier can be listed > > via > > /sys/devices/system/memtier/memtierN/nodelist > > > > "Rank" is an opaque value. Its absolute value doesn't have any > > special meaning. But the rank values of different memtiers can be > > compared with each other to determine the memory tier order. > > > > For example, if we have 3 memtiers: memtier0, memtier1, memiter2, and > > their rank values are 300, 200, 100, then the memory tier order is: > > memtier0 -> memtier2 -> memtier1, > > Why is memtier2 (rank 100) higher than memtier1 (rank 200)? Seems like > the order should be memtier0 -> memtier1 -> memtier2? > (rank 300) (rank 200) (rank 100) I think this is a copy-and-modify typo from my original memory tiering kernel interface RFC (v4, https://lore.kernel.org/linux-mm/CAAPL-u9Wv+nH1VOZTj=9p9S70Y3Qz3+63EkqncRDdHfubsrjfw@mail.gmail.com/T/): where the rank values are 100, 10, 50 (i.e the rank of memtier2 is higher than memtier1). > > where memtier0 is the highest tier > > and memtier1 is the lowest tier. > > I think memtier2 is the lowest as it has the lowest rank value. > > > > The rank value of each memtier should be unique. > > > > > > + > > +static void memory_tier_device_release(struct device *dev) > > +{ > > + struct memory_tier *tier = to_memory_tier(dev); > > + > > Do we need some ref counts on memory_tier? > If there is another device still using the same memtier, > free below could cause problem. > > > + kfree(tier); > > +} > > + > > > ... > > +static struct memory_tier *register_memory_tier(unsigned int tier) > > +{ > > + int error; > > + struct memory_tier *memtier; > > + > > + if (tier >= MAX_MEMORY_TIERS) > > + return NULL; > > + > > + memtier = kzalloc(sizeof(struct memory_tier), GFP_KERNEL); > > + if (!memtier) > > + return NULL; > > + > > + memtier->dev.id = tier; > > + memtier->rank = get_rank_from_tier(tier); > > + memtier->dev.bus = &memory_tier_subsys; > > + memtier->dev.release = memory_tier_device_release; > > + memtier->dev.groups = memory_tier_dev_groups; > > + > > Should you take the mem_tier_lock before you insert to > memtier-list? > > > + insert_memory_tier(memtier); > > + > > + error = device_register(&memtier->dev); > > + if (error) { > > + list_del(&memtier->list); > > + put_device(&memtier->dev); > > + return NULL; > > + } > > + return memtier; > > +} > > + > > +__maybe_unused // temporay to prevent warnings during bisects > > +static void unregister_memory_tier(struct memory_tier *memtier) > > +{ > > I think we should take mem_tier_lock before modifying memtier->list. > > > + list_del(&memtier->list); > > + device_unregister(&memtier->dev); > > +} > > + > > > > Thanks. > > Tim > >