From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4762CC00140 for ; Wed, 3 Aug 2022 03:18:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D96EC6B0072; Tue, 2 Aug 2022 23:18:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D46C18E0002; Tue, 2 Aug 2022 23:18:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BE7368E0001; Tue, 2 Aug 2022 23:18:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id AD5266B0072 for ; Tue, 2 Aug 2022 23:18:43 -0400 (EDT) Received: from smtpin31.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 785711C668C for ; Wed, 3 Aug 2022 03:18:43 +0000 (UTC) X-FDA: 79756824126.31.2A3B7EB Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by imf27.hostedemail.com (Postfix) with ESMTP id 0A4AE40114 for ; Wed, 3 Aug 2022 03:18:41 +0000 (UTC) Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2733Fwdp025452; Wed, 3 Aug 2022 03:18:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : in-reply-to : references : date : message-id : mime-version : content-type; s=pp1; bh=QjcNgPogLbvM2Fn56mi7nrWF2hFA8ulaHD5phxyXE14=; b=XXYfIaDWrvvzZGwES7qwXT+30jL8724kbMfZRlIotNORsl71MTVvLarZEOcDUWa2NWEF 7YQ7ragwVzGxxZFEXvLmekRrPd9X7R/94NnkjyyD1RDpucS/lPGm8nef0+vcpslqoGed 1IDZquAiUWlUlUWWxRu2+NLeA+VN5UAS/0bVz6KN+vbN34Eakfuq+VKvO7+5zO7iUdBs XoqV4NvCgag76Rrp4umHPbaaXiHnZ/rqOWO/BLT+NJuQnQhZVnWicCQWH9qk7+Fhe4sc 3eIXH4YEQn1wsY0DG3MigvpdhPPnmpGIpxziMW+hQ8Tkli7QfjUQjyUUlOrd3H0XoOpQ eQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3hqgy280xr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 03 Aug 2022 03:18:29 +0000 Received: from m0098420.ppops.net (m0098420.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2733ISQK029903; Wed, 3 Aug 2022 03:18:29 GMT Received: from ppma03wdc.us.ibm.com (ba.79.3fa9.ip4.static.sl-reverse.com [169.63.121.186]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3hqgy280xh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 03 Aug 2022 03:18:28 +0000 Received: from pps.filterd (ppma03wdc.us.ibm.com [127.0.0.1]) by ppma03wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 27336C0r001985; Wed, 3 Aug 2022 03:18:28 GMT Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by ppma03wdc.us.ibm.com with ESMTP id 3hmv99ds4x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 03 Aug 2022 03:18:28 +0000 Received: from b03ledav006.gho.boulder.ibm.com (b03ledav006.gho.boulder.ibm.com [9.17.130.237]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2733IRi941156984 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 3 Aug 2022 03:18:27 GMT Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3FC1FC6057; Wed, 3 Aug 2022 03:18:27 +0000 (GMT) Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BFC11C6055; Wed, 3 Aug 2022 03:18:21 +0000 (GMT) Received: from skywalker.linux.ibm.com (unknown [9.43.62.91]) by b03ledav006.gho.boulder.ibm.com (Postfix) with ESMTP; Wed, 3 Aug 2022 03:18:21 +0000 (GMT) X-Mailer: emacs 29.0.50 (via feedmail 11-beta-1 I) From: "Aneesh Kumar K.V" To: "Huang, Ying" Cc: linux-mm@kvack.org, akpm@linux-foundation.org, Wei Xu , Yang Shi , Davidlohr Bueso , Tim C Chen , Michal Hocko , Linux Kernel Mailing List , Hesham Almatary , Dave Hansen , Jonathan Cameron , Alistair Popple , Dan Williams , Johannes Weiner , jvgediya.oss@gmail.com Subject: Re: [PATCH v10 5/8] mm/demotion: Build demotion targets based on explicit memory tiers In-Reply-To: <87h733uyc8.fsf@yhuang6-desk2.ccr.corp.intel.com> References: <20220720025920.1373558-1-aneesh.kumar@linux.ibm.com> <20220720025920.1373558-6-aneesh.kumar@linux.ibm.com> <871qu8wc6c.fsf@yhuang6-desk2.ccr.corp.intel.com> <9f14814c-cb08-8032-caff-edcd0594ad41@linux.ibm.com> <87h733uyc8.fsf@yhuang6-desk2.ccr.corp.intel.com> Date: Wed, 03 Aug 2022 08:48:19 +0530 Message-ID: <878ro6rp50.fsf@linux.ibm.com> MIME-Version: 1.0 Content-Type: text/plain X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 99ovNddF_lANY-HwnyZu8JMSOUYxsvHV X-Proofpoint-ORIG-GUID: Yqul1cIHAeXJaD41LjHv8Jn9K-doIERY X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-03_01,2022-08-02_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 clxscore=1015 suspectscore=0 mlxscore=0 phishscore=0 mlxlogscore=999 spamscore=0 priorityscore=1501 malwarescore=0 impostorscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2206140000 definitions=main-2208030014 ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=XXYfIaDW; spf=pass (imf27.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659496722; a=rsa-sha256; cv=none; b=Ump4tgSeXykM0Ngfe3zucS/qIbyi7KhtvH5hFYiTVmptXKHIFzgnx1W2NXFeW/oSSFDveT y0zmBETcJs+d2Uwkt61xM+KaPJPq9VwlAeel0D/lyEwdhz/xc0qUEomgPgD7ZWiPzSIntv NCPpOrbpSQjeho402rWZdLMzttqE2O0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659496722; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QjcNgPogLbvM2Fn56mi7nrWF2hFA8ulaHD5phxyXE14=; b=PTNZL3+4YMPNG4cBulXzp8Zxg6hkYYukjhuGemp2zJsEkEal2rmTJ6r2fBUhgdEKEt/a5d vsGUAz34/Qq8enDXS1lQwkANbtCgrCgWX+dcbOmfGDuX5IhpJtkgwSEQ1XO5QaW1ITGEIe McJX02aSteoQV0S+EM/4rf75QaO6yqc= X-Stat-Signature: bdzsnfikjq4sj57mkwf67zsisjpdhk4m X-Rspamd-Queue-Id: 0A4AE40114 Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=XXYfIaDW; spf=pass (imf27.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1659496721-929425 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: "Huang, Ying" writes: > Aneesh Kumar K V writes: > >> On 7/26/22 1:14 PM, Huang, Ying wrote: >>> "Aneesh Kumar K.V" writes: .... >>> + >>>> static void init_node_memory_tier(int node) >>>> { >>>> int perf_level; >>>> @@ -84,11 +285,19 @@ static void init_node_memory_tier(int node) >>>> mutex_lock(&memory_tier_lock); >>>> >>>> memtier = __node_get_memory_tier(node); >>>> + /* >>>> + * if node is already part of the tier proceed with the >>>> + * current tier value, because we might want to establish >>>> + * new migration paths now. The node might be added to a tier >>>> + * before it was made part of N_MEMORY, hence estabilish_migration_targets >>>> + * will have skipped this node. >>>> + */ >>>> if (!memtier) { >>>> perf_level = node_devices[node]->perf_level; >>>> memtier = find_create_memory_tier(perf_level); >>>> node_set(node, memtier->nodelist); >>>> } >>>> + establish_migration_targets(); >>> >>> Why combines memory tiers establishing with demotion targets building? >>> I think that it's better to separate them. For example, if we move a >>> set of NUMA node from one memory tier to another memory tier, we only >>> need to run establish_migration_targets() once after moving all nodes. >>> >> >> Yes agree. I am not sure I followed your comment here. >> >> Demotion target rebuilding is a separate helper. Any update to memory tiers needs rebuilding >> of demotion targets. Also any change in node mask of memory tier needs >> demotion target rebuild. Can you clarify the code change you are suggesting here? > > I think we should call establish_migration_targets() in > migrate_on_reclaim_callback() directly. As the example I mentioned > above, sometimes, we don't need to call establish_migration_targets() > for each node changing. > We need to hold memory_tier_lock while updating node's memory tier and rebuilding demotion targets. All of that is done in the same function here. An update node memory tier that allow for updating multiple node's memory tier together would do what you mentioned above under memory_tier_lock ie, update all the nodes memory tier and then call establish_migration_targets() once. -aneesh