From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0FB0C54798 for ; Tue, 5 Mar 2024 06:23:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 824B56B007D; Tue, 5 Mar 2024 01:23:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7D4B36B0080; Tue, 5 Mar 2024 01:23:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 69BA56B0085; Tue, 5 Mar 2024 01:23:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5A8B86B007D for ; Tue, 5 Mar 2024 01:23:13 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 339631C0346 for ; Tue, 5 Mar 2024 06:23:13 +0000 (UTC) X-FDA: 81861993066.24.1E3F49A Received: from mail-vk1-f182.google.com (mail-vk1-f182.google.com [209.85.221.182]) by imf21.hostedemail.com (Postfix) with ESMTP id 910741C0003 for ; Tue, 5 Mar 2024 06:23:11 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=GVeTY3sx; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf21.hostedemail.com: domain of horenchuang@bytedance.com designates 209.85.221.182 as permitted sender) smtp.mailfrom=horenchuang@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709619791; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MOnvvH2W03oK21Enygd77Kz3RD653oFonWkuoJk8SUU=; b=fNe15+GPbjdx7aVc63luql+08MXTXH/Je9v8BQHxY+sKtCXfUXWJEfKq8wPQo4Kj3o3LGm kA+jy2nBP1Pj3tHkI3iwNI9GX4suyha6IA0RvXZufTRXWAJxYQRX6UeCyp0t2e900ftEhI vpaVqJ6ESbZiEKc/tOYYn08bRoXkfXY= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=GVeTY3sx; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf21.hostedemail.com: domain of horenchuang@bytedance.com designates 209.85.221.182 as permitted sender) smtp.mailfrom=horenchuang@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709619791; a=rsa-sha256; cv=none; b=6Ud2qT/+V8rZ/MLK427MNfsU8N7TmdTAXxAi1WfCTzK4d28MIasulGaYre4G/x4y2IcfHB Vz+14jJVTwugncUtkW4BzYStRwx06deDd6J8LRtDkxsocXGrhbmR75ScEQNWw0lAB/kOJM 6cPcXoDyKbCKmrCp0sd4NyxxKW4nYUc= Received: by mail-vk1-f182.google.com with SMTP id 71dfb90a1353d-4cb26623dc1so155448e0c.2 for ; Mon, 04 Mar 2024 22:23:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1709619790; x=1710224590; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=MOnvvH2W03oK21Enygd77Kz3RD653oFonWkuoJk8SUU=; b=GVeTY3sxsTA8UZZnXnQmIJNW3x8PV3vw6mbPehfoV+ALOJvdYhsClajvxzJ/iqibhH XQfgwqTMOC5pfHqLbdqWIZo4TQTw7IU7pcDAUCQoqsBBsvPd0rlxfEtraKagoaHD4/0g keb8GRZ4fTBVUJoqwN+Em0LHSzDrbLHoWJu4bQWs/6CH3xZgA43YhD7Dq9onL+JZx5Or lsF1MWnF3HsCMPDm/9Pw4H2CKR3NXfRUJzISxHeXxK/o/3JY72g7TlQYWLGba7MljwPt WlNvzCufPjzow0ie6aYgQkrnEoacWJHHVL9DPwWrG40TJEU0q/rZTiofT94Vnpo7hP2h CI5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709619790; x=1710224590; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MOnvvH2W03oK21Enygd77Kz3RD653oFonWkuoJk8SUU=; b=EtYKIqSnHIvopheLW0+d65IbT467KK+9PyRa/7Tek0uliZPMnbNbL2tnUtHpRjDYK/ OUjSPXhjYzMpMuAMGONnQAhmvN4fJtu4/j7JBX7KSe5hEnm2xXeBEncj0zqnDkZkrNPt 7XS+5v/2HSzF2DodmzznUN8mMcgEwhXAwTltLopZ8QUh6GDjhFmnW1Ood5+NaRFNzJmT rkdVHbeZqJVkvXdCqs82z9KEqvFz26U6zHz330qF/B5nQKgxfz1njcuNif65PKiR88iL tPEu7PP5uKguXictMLATWkm9UjYSFMS6HKuotZexUeNdu2XW7Pr+0kKEarJQBPayN5a9 5sFg== X-Forwarded-Encrypted: i=1; AJvYcCVRd2qU+cdZKqCFs/zJehaNTNgqPNcbbhMA/MYiEEsrY9XXZTfbTdst2EYTAQnGzFX2ogAO0tMco9UvOu0tm6XmjP0= X-Gm-Message-State: AOJu0YyfJK1dtzA6qLIXwSH+kEciNK9JnMHl+MfgrkMTF44KzOCHqIWM fFdn8+hmY9cEO0JcQE5p1/XfoAadmpBz/k9GTE695XRRG+Fit8CgX+AFjez2ly7C78nqpA4nOcm UvmOqu+qv99GEHsMmhH/zMDd7TxDnvUVNAzBCFA== X-Google-Smtp-Source: AGHT+IF8wFchH2Tpcwl+m9+vC++9MiGH8pjEs4cet64MI10UrFTvnJG9+E2iM+RbR2/QpLJ5rn/dajElMcAyZ2RW5JY= X-Received: by 2002:a05:6122:da7:b0:4ca:615e:1b61 with SMTP id bc39-20020a0561220da700b004ca615e1b61mr914463vkb.10.1709619790595; Mon, 04 Mar 2024 22:23:10 -0800 (PST) MIME-Version: 1.0 References: <20240301082248.3456086-1-horenchuang@bytedance.com> <87frx6btqp.fsf@yhuang6-desk2.ccr.corp.intel.com> In-Reply-To: <87frx6btqp.fsf@yhuang6-desk2.ccr.corp.intel.com> From: "Ho-Ren (Jack) Chuang" Date: Mon, 4 Mar 2024 22:22:59 -0800 Message-ID: Subject: Re: [External] Re: [PATCH v1 0/1] Improved Memory Tier Creation for CPUless NUMA Nodes To: "Huang, Ying" Cc: Gregory Price , aneesh.kumar@linux.ibm.com, mhocko@suse.com, tj@kernel.org, john@jagalactic.com, Eishan Mirakhur , Vinicius Tavares Petrucci , Ravis OpenSrc , Alistair Popple , "Rafael J. Wysocki" , Len Brown , Andrew Morton , Dave Jiang , Dan Williams , Jonathan Cameron , linux-acpi@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, "Ho-Ren (Jack) Chuang" , "Ho-Ren (Jack) Chuang" , linux-cxl@vger.kernel.org, qemu-devel@nongnu.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Stat-Signature: 7tiux4tca9rqakj6au7hfk3ks3obzaig X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 910741C0003 X-HE-Tag: 1709619791-435067 X-HE-Meta: U2FsdGVkX19agdO/dl6cdkHDAVdRpm4lxv3Ar3obscy8gCh3137PnT4M4l2TgwSuno7nI++WjbcYNAd/fg4LJyXCSYNQUwsKVUnHBKfRFqLgTll74g0LB/M1DLMR+41urpoVq6F/WzXgB2MXC2qyGOojSCrojXFSbT1lQvx0jvONOev8dzUNq4kYOV9GpApMl2E96CdnyyLhLpYYwhm4snQ0YtfMe4Bl60oMkOEVuzVDO/yka0u7qoV/hac37ZO/bMmAwbByllJuDB1a2kpyNKFMxL3PygN8sw666ea4jXvFGm+TpwoXkNK3Dg9prD7nS3HUCweFgFpq/GVbgexZLzgkkeH/qFyxWbnNi6/+dt8LMFXyCH8BBcg3aJro5w7+cnNHBVX9aqfFe+Fyr76DPZ1Wa1xv+sgDGA9PmlO7a6a7UutR2iNiFpJuJMCn2TFAxF3lLmC99HuOHVJm09YVc9SoScEuWOQPHR5woLE+MbMXoOuYLuQ7XYSO5JvJFkW1g8fWUc7wEbEo6d5HvhF4FJAO3s/5K4INgBBc+ZgKZojIcytdO1JBIJLOmIEslBRS6LyZ29WMnPXON3TwLJih5HQSyeqyKab8CLJwFNTsEY5uRO1mAvNeq+84ghCeCyMgyMULvjnshDs0K4eBjT3V87qYobxp09XQGOhcfvhZfqNvDZElCgK6TkrxXE2u9Tu/ELDfa0qXEejQ6LqnfCPI0PtUFYk7fdu2DtBbAr4m+mMjRTFhRMB/dLIkMpNXbg5PmWp7QRiLGIDJsuXRh/yta1T0gSretKAkDtPahO94MqlWU44Kr2knQP26P70IfvMxbtO1UshRphXHlH1llmKGnP2n9YzMM8D/8NdZ331AD5IUycd8R6R4j6u8k3wiy7WTdI3nvIwLOyRwZ/gFGIC/ZBAY9E6hP5BNSMWTYefC5kHsnWiQwIVmnvnLC6pVeN8x8WAcOAIX+JrleRwu0no uLg645kA DDb0bTJLkLCMJL4xoVpD8CNyzCULctFk1h2LQR/xZl1SJAdmZJVp8s7EqdzINLJaGPkdyDg1FmmRbqyg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sun, Mar 3, 2024 at 6:47=E2=80=AFPM Huang, Ying w= rote: > > "Ho-Ren (Jack) Chuang" writes: > > > The memory tiering component in the kernel is functionally useless for > > CPUless memory/non-DRAM devices like CXL1.1 type3 memory because the no= des > > are lumped together in the DRAM tier. > > https://lore.kernel.org/linux-mm/PH0PR08MB7955E9F08CCB64F23963B5C3A860A= @PH0PR08MB7955.namprd08.prod.outlook.com/T/ > > I think that it's unfair to call it "useless". Yes, it doesn't work if > the CXL memory device are not enumerate via drivers/dax/kmem.c. So, > please be specific about in which cases it doesn't work instead of too > general "useless". > Thank you and I didn't mean anything specific. I simply reused phrases we discussed earlier in the previous patchset. I will change them to the following in v2= : "At boot time, current memory tiering assigns all detected memory nodes to the same DRAM tier. This results in CPUless memory/non-DRAM devices, such as CXL1.1 type3 memory, being unable to be assigned to the correct memory tier, leading to the inability to migrate pages between different types of memory= ." Please see if this looks more specific. > > This patchset automatically resolves the issues. It delays the initiali= zation > > of memory tiers for CPUless NUMA nodes until they obtain HMAT informati= on > > at boot time, eliminating the need for user intervention. > > If no HMAT specified, it falls back to using `default_dram_type`. > > > > Example usecase: > > We have CXL memory on the host, and we create VMs with a new system mem= ory > > device backed by host CXL memory. We inject CXL memory performance attr= ibutes > > through QEMU, and the guest now sees memory nodes with performance attr= ibutes > > in HMAT. With this change, we enable the guest kernel to construct > > the correct memory tiering for the memory nodes. > > > > Ho-Ren (Jack) Chuang (1): > > memory tier: acpi/hmat: create CPUless memory tiers after obtaining > > HMAT info > > > > drivers/acpi/numa/hmat.c | 3 ++ > > include/linux/memory-tiers.h | 6 +++ > > mm/memory-tiers.c | 76 ++++++++++++++++++++++++++++++++---- > > 3 files changed, 77 insertions(+), 8 deletions(-) > > -- > Best Regards, > Huang, Ying --=20 --- Best regards, Ho-Ren (Jack) Chuang =E8=8E=8A=E8=B3=80=E4=BB=BB