From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C142C433EF for ; Wed, 8 Jun 2022 06:07:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 39F0B6B0071; Wed, 8 Jun 2022 02:07:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 327896B0072; Wed, 8 Jun 2022 02:07:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1A1F66B0073; Wed, 8 Jun 2022 02:07:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 057796B0071 for ; Wed, 8 Jun 2022 02:07:08 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id D22018069C for ; Wed, 8 Jun 2022 06:07:07 +0000 (UTC) X-FDA: 79554035694.07.E2B67FE Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by imf22.hostedemail.com (Postfix) with ESMTP id 97EB2C0046 for ; Wed, 8 Jun 2022 06:07:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1654668425; x=1686204425; h=message-id:subject:from:to:cc:date:in-reply-to: references:mime-version:content-transfer-encoding; bh=uVvtxf05SKDsySGKn1SFfLh2lQyAvaiLbpKHel9kyOU=; b=NdK2K5fej0YDpgK6kiKPa0y4UGqTuOV80RiDzKTUTmMf0jE6/fCXT+N1 YnBSG9+Io+guzTFT3nU/0jVtTYnYBbIWSmgjdNYoMKJBWZnm2Lk0Tbtj9 KKiqAJQ0G/zpgkPtkLuWq5d3f0LLqCdI8uwrxt5waN8tpgcd6MmuqNCCN 43dGsF4KSUoYoKwycBpaMyrde5p7IIj/PlTiLoRE6byU/x3sFWbDcmGDu N0ObQEp9NQCLx13jOauUhAjE2Oqhbu8EdSNTaHPmdLF+7kqc33hkhvF3N 2kAwvaJ+MQOXq15+uVniLypbOgoG3PQL8HdNAQqfH1DR4I144BfTjLC9N A==; X-IronPort-AV: E=McAfee;i="6400,9594,10371"; a="338551268" X-IronPort-AV: E=Sophos;i="5.91,285,1647327600"; d="scan'208";a="338551268" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Jun 2022 23:06:27 -0700 X-IronPort-AV: E=Sophos;i="5.91,285,1647327600"; d="scan'208";a="826760381" Received: from xding11-mobl.ccr.corp.intel.com ([10.254.214.239]) by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Jun 2022 23:06:22 -0700 Message-ID: <6d72cab6badb003d996eaf5454939e552d2ba67d.camel@intel.com> Subject: Re: [PATCH v5 1/9] mm/demotion: Add support for explicit memory tiers From: Ying Huang To: Aneesh Kumar K V , Tim Chen , linux-mm@kvack.org, akpm@linux-foundation.org Cc: Wei Xu , Greg Thelen , Yang Shi , Davidlohr Bueso , Tim C Chen , Brice Goglin , Michal Hocko , Linux Kernel Mailing List , Hesham Almatary , Dave Hansen , Jonathan Cameron , Alistair Popple , Dan Williams , Feng Tang , Jagdish Gediya , Baolin Wang , David Rientjes Date: Wed, 08 Jun 2022 14:06:20 +0800 In-Reply-To: <8a42d52c-6275-4798-19c0-dfc530c04b0e@linux.ibm.com> References: <20220603134237.131362-1-aneesh.kumar@linux.ibm.com> <20220603134237.131362-2-aneesh.kumar@linux.ibm.com> <92649c9a6e0b6931b34aeaaf22c0a1e874484b7f.camel@linux.intel.com> <8a42d52c-6275-4798-19c0-dfc530c04b0e@linux.ibm.com> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 97EB2C0046 X-Stat-Signature: 6xe5ag5csbnjhbq75t5h5o9eja7x8o7h Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=NdK2K5fe; dmarc=pass (policy=none) header.from=intel.com; spf=none (imf22.hostedemail.com: domain of ying.huang@intel.com has no SPF policy when checking 134.134.136.31) smtp.mailfrom=ying.huang@intel.com X-HE-Tag: 1654668425-908857 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, 2022-06-08 at 10:00 +0530, Aneesh Kumar K V wrote: > On 6/8/22 12:13 AM, Tim Chen wrote: > > On Fri, 2022-06-03 at 19:12 +0530, Aneesh Kumar K.V wrote: > > > > > > > > > The nodes which are part of a specific memory tier can be listed > > > via > > > /sys/devices/system/memtier/memtierN/nodelist > > > > > > "Rank" is an opaque value. Its absolute value doesn't have any > > > special meaning. But the rank values of different memtiers can be > > > compared with each other to determine the memory tier order. > > > > > > For example, if we have 3 memtiers: memtier0, memtier1, memiter2, and > > > their rank values are 300, 200, 100, then the memory tier order is: > > > memtier0 -> memtier2 -> memtier1, > > > > Why is memtier2 (rank 100) higher than memtier1 (rank 200)? Seems like > > the order should be memtier0 -> memtier1 -> memtier2? > >                      (rank 300) (rank 200) (rank 100) > > > > > where memtier0 is the highest tier > > > and memtier1 is the lowest tier. > > > > I think memtier2 is the lowest as it has the lowest rank value. > > > typo error. Will fix that in the next update > > > > > > > The rank value of each memtier should be unique. > > > > > > > > > + > > > +static void memory_tier_device_release(struct device *dev) > > > +{ > > > + struct memory_tier *tier = to_memory_tier(dev); > > > + > > > > Do we need some ref counts on memory_tier? > > If there is another device still using the same memtier, > > free below could cause problem. > > > > > + kfree(tier); > > > +} > > > + > > > > > ... > > > +static struct memory_tier *register_memory_tier(unsigned int tier) > > > +{ > > > + int error; > > > + struct memory_tier *memtier; > > > + > > > + if (tier >= MAX_MEMORY_TIERS) > > > + return NULL; > > > + > > > + memtier = kzalloc(sizeof(struct memory_tier), GFP_KERNEL); > > > + if (!memtier) > > > + return NULL; > > > + > > > + memtier->dev.id = tier; > > > + memtier->rank = get_rank_from_tier(tier); > > > + memtier->dev.bus = &memory_tier_subsys; > > > + memtier->dev.release = memory_tier_device_release; > > > + memtier->dev.groups = memory_tier_dev_groups; > > > + > > > > Should you take the mem_tier_lock before you insert to > > memtier-list? > > > Both register_memory_tier and unregister_memory_tier get called with > memory_tier_lock held. Then please add locking requirements to the comments above these functions. Best Regards, Huang, Ying > > > > > + insert_memory_tier(memtier); > > > + > > > + error = device_register(&memtier->dev); > > > + if (error) { > > > + list_del(&memtier->list); > > > + put_device(&memtier->dev); > > > + return NULL; > > > + } > > > + return memtier; > > > +} > > > + > > > +__maybe_unused // temporay to prevent warnings during bisects > > > +static void unregister_memory_tier(struct memory_tier *memtier) > > > +{ > > > > I think we should take mem_tier_lock before modifying memtier->list. > > > > unregister_memory_tier get called with memory_tier_lock held. > > > > + list_del(&memtier->list); > > > + device_unregister(&memtier->dev); > > > +} > > > + > > > > > -aneesh