From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8766C4167B for ; Tue, 5 Dec 2023 09:02:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4831A6B0082; Tue, 5 Dec 2023 04:02:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 433656B0087; Tue, 5 Dec 2023 04:02:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2D46B6B0088; Tue, 5 Dec 2023 04:02:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 193366B0082 for ; Tue, 5 Dec 2023 04:02:43 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E1F0814021A for ; Tue, 5 Dec 2023 09:02:42 +0000 (UTC) X-FDA: 81532174164.10.60ADBFF Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by imf09.hostedemail.com (Postfix) with ESMTP id 998E914000A for ; Tue, 5 Dec 2023 09:02:38 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=micron.com header.s=selector2 header.b=XG0L0iU9; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=micron.com; spf=pass (imf09.hostedemail.com: domain of sthanneeru.opensrc@micron.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=sthanneeru.opensrc@micron.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1701766958; a=rsa-sha256; cv=pass; b=Wa/NBgJY/xs0bazPUfxYUA08RIfpY7DLQCvMEpTgCHtEHVXDC9sgxSujjEk6ZpLc02BvEh CBdUKpBBm6uiqWYDQOwCtgcADYI4RY1J91SY4S++BS1iYRVF695ncLWQxQt2wm2xK0fsnN Lmm0Oz+6YOJ9kTLc9sdc1dr/0rzI2T4= ARC-Authentication-Results: i=2; imf09.hostedemail.com; dkim=pass header.d=micron.com header.s=selector2 header.b=XG0L0iU9; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=micron.com; spf=pass (imf09.hostedemail.com: domain of sthanneeru.opensrc@micron.com designates 40.107.236.41 as permitted sender) smtp.mailfrom=sthanneeru.opensrc@micron.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701766958; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QYY1dlKz7XRZSHjkJle7feKn+k9Cgkr7NBlAOAoWdKE=; b=Sbqsg8L065SGf/1EsEQly/rIrjyd8S6R9Q+SySw/s1xC/FBy38U7KSzA24sDUZ/diUiFi+ JPR7gbfSjG8zGK7mL+NbNBn2qWxNGWUovWCjEgDEgalcaWgtOv731iems04VUQcIsv3zWW lGhLz5W0D+E/55X0oyjZqxqTEXjb9tA= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ifRV/QVGiHyNAipob3b5IUUpXQFMZ3EbgnVsZvkYcg0bo9s1rMiXxMDxPsChpNFD4e7vQMICU05B4S5AMl7brktA7FUDVy5PvqVeHDuH6hoxt5GI6uFJwTBgCQXCGKyDkKygpclMCnKKChf//Ub2ETtsQBcxwE4Wk5nXL1QA7UzB5/7ni5o3Z+0SyBnR+DpjE+sfSPPfUowAI0F+bfc+m+5je0rxXI6iP/B4jnbT2fi9XWqgGO4npWhWTbxEUufqoUTmUO1kKs0HBaERvfBKBPx40+28HQkh67IcPD4hY9x5UAydsUDCvB1nKy1LbFQZ/zk0Rn4I1fkVUy2V+Pp+qg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QYY1dlKz7XRZSHjkJle7feKn+k9Cgkr7NBlAOAoWdKE=; b=AFTRGE+gD5wQjxe2HYRFqXd2XSnAySLUW/yo9LlIvl1mfdUtxsuv1QjHqLsEMgfFj+iaePxVWYSHfhRrt1C8fSG6AG+eKX7PWXM7UWSKHDxtfuU+4bLDTFNeP6zohpgZKnSAN5lZVAqCI6ommIiW7I7opPH5s0nVmH2rRiLtOGjGgYnCqPvj326OIz+yu/+E0Zw0kCqKLKRPAG0pDO9FUR4yNn47zEEAgxfPCNV9NoixJghYwdmCOv4oKdHRaS06WvBFDVcdJTRr06TTkISFHilXcuzTTHY0CWoPysiQExOYpob8VAHIR41w1CM4apvQvyO8SnGvBzLnoAexOPt8AA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 137.201.242.130) smtp.rcpttodomain=suse.com smtp.mailfrom=micron.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=micron.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=micron.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QYY1dlKz7XRZSHjkJle7feKn+k9Cgkr7NBlAOAoWdKE=; b=XG0L0iU9PeFPATRUe45PAharioBZEXUDoazY1daDrN2O3r8dc2refzE9BStsWZRwPrljHv5MOw96XWL0DbKYxyBbHagEGXlhcax4yOgZ2qW8UuVnuM9K0sA5pEZtotKXXT5XVHwbJqEUoWSgKia+n6s8igFysm8E9czWIBKUhMRjpykV5E9ndcFz9cHe4QKkfRbIk68P94Pa43j+sFYdevECBpQDsea4O0IPEyX3+5jFAh3vGnAMsSkMqqxj1mIbRHbDLvTBYHaZ4H+r9t+WxvK2dBTPfkxRtQzxyUI2Ja1nWzxgu6W0WrwNaL+YpjzOmIzmHIy9iAe4oUVLHiIiMg== Received: from BN9P221CA0014.NAMP221.PROD.OUTLOOK.COM (2603:10b6:408:10a::19) by CYXPR08MB9261.namprd08.prod.outlook.com (2603:10b6:930:d8::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.34; Tue, 5 Dec 2023 09:02:33 +0000 Received: from SN1PEPF0002BA50.namprd03.prod.outlook.com (2603:10b6:408:10a:cafe::a) by BN9P221CA0014.outlook.office365.com (2603:10b6:408:10a::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.33 via Frontend Transport; Tue, 5 Dec 2023 09:02:32 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 137.201.242.130) smtp.mailfrom=micron.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=micron.com; Received-SPF: Pass (protection.outlook.com: domain of micron.com designates 137.201.242.130 as permitted sender) receiver=protection.outlook.com; client-ip=137.201.242.130; helo=mail.micron.com; pr=C Received: from mail.micron.com (137.201.242.130) by SN1PEPF0002BA50.mail.protection.outlook.com (10.167.242.73) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7068.20 via Frontend Transport; Tue, 5 Dec 2023 09:02:32 +0000 Received: from BOW36EX19B.micron.com (137.201.85.154) by BOW36EX19B.micron.com (137.201.85.154) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.27; Tue, 5 Dec 2023 02:02:29 -0700 Received: from [10.3.101.85] (10.3.101.85) by RestrictedRelayBOW36EX19B.micron.com (137.201.85.154) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.27 via Frontend Transport; Tue, 5 Dec 2023 02:02:22 -0700 Message-ID: <8eae7d95-fc98-4ac7-8f83-d0caee00bc87@micron.com> Date: Tue, 5 Dec 2023 14:32:20 +0530 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [EXT] Re: [RFC PATCH 0/2] Node migration between memory tiers Content-Language: en-US To: Michal Hocko CC: , , , , , , , , , , , , , , , , , , References: <20231130220422.2033-1-sthanneeru.opensrc@micron.com> <1db561a9-6984-418d-9305-a2a5ece93696@micron.com> <2552828e-6865-4fa8-a9c4-8ed76dd85257@micron.com> From: Srinivasulu Thanneeru In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-MT-Whitelisted: matched X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002BA50:EE_|CYXPR08MB9261:EE_ X-MS-Office365-Filtering-Correlation-Id: 1e08090c-9afb-4ec5-04cf-08dbf570eaa5 X-LD-Processed: f38a5ecd-2813-4862-b11b-ac1d563c806f,ExtAddr X-EXT-ByPass: 1 X-MT-RULE-Whitelisted: Triggered X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: pH+rTWh1ZBt0vY8tPoHIkZfCRFBrYu4havX3fg1Mpd550AfzOtKtJt1MWopf2tQP4t1kFFkHj4uMWwprBigOBvYJQXR63bj/JG4pc2/OG3vS3VjUqPwHgeQTRzGXRGykiA91u19ryTI3HASeHS6LbxZjnJMb3xFT1zIOHSlowhBL5EQf3PqCgPpehLVcezgniJkxWtTaaV4nGoWaa6DwYtJFESXIPH2pMZJd+mUocTvS9TKaCns/FG87fmNJquL1PCdg9OpSlFGkIKwH6lB/lS9WBlxk9oGzovtbZ4fK3iOfCQWpkVr/zQMimbcr5jd7Q1D5xxV0FIOABpVRwAqZ09Pi6Eh3Mv6s3+u5wXC6my5HAE8X1xwKruGvae/LDyw0hyMBy2vC4sNFXmCOh5QEWPlsQnnmvpG5pID97QX+OsMw0x//3RFRLbOlloqdyaese1ZR2/nl2+EJdIyYag3rbuRk2Pk43qonPUUDaoedFgJQfc6rpS0hAngzqgJXwmLUhPvbHhJT+GEmBA0umvdiZnIp7E2t8WBeCGJGl4htQr2EnV9ovfzPWKc/A+E8TK1BogroNYaluDVIkl+vIvE6HCl6ZCWDVZa8tJNooUpdfvDR7JaWEUeQQJ8QftXUVfnyylU99xSzkOPvJyCb2gKDUhM2Tu2TgMUDTsxe0KqSPyXEiZapD3OO0Yg6yLRdwrXSkO3iUuY4ccl2a6NqysKpCDBkRwjJQQNC/JKA24hb7OW46CfzWApodkshVwo2p8tPKNqdU2l4b5e8WIQybZXL0CfsB0PoeG6SfklnOaIyq8c/gtGWTDaB6j4ssgJpXB9pombGMUsahmQ2912nsX15NgrGCLNJ8UK+UArxmQb2Lvw= X-Forefront-Antispam-Report: CIP:137.201.242.130;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.micron.com;PTR:masquerade.micron.com;CAT:NONE;SFS:(13230031)(4636009)(396003)(136003)(376002)(39860400002)(346002)(230922051799003)(230173577357003)(230273577357003)(186009)(82310400011)(64100799003)(451199024)(1800799012)(40470700004)(36840700001)(46966006)(40460700003)(5660300002)(7416002)(2906002)(36860700001)(31696002)(86362001)(36756003)(356005)(7636003)(41300700001)(82740400003)(47076005)(53546011)(2616005)(26005)(478600001)(8676002)(4326008)(54906003)(6916009)(70586007)(70206006)(16576012)(336012)(426003)(316002)(83380400001)(8936002)(31686004)(40480700001)(3940600001)(43740500002);DIR:OUT;SFP:1101; X-OriginatorOrg: micron.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2023 09:02:32.4101 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1e08090c-9afb-4ec5-04cf-08dbf570eaa5 X-MS-Exchange-CrossTenant-Id: f38a5ecd-2813-4862-b11b-ac1d563c806f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f38a5ecd-2813-4862-b11b-ac1d563c806f;Ip=[137.201.242.130];Helo=[mail.micron.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002BA50.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYXPR08MB9261 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 998E914000A X-Stat-Signature: gh6w41er9hzewhgwkm8jekkahd75w9dn X-HE-Tag: 1701766958-290269 X-HE-Meta: U2FsdGVkX18+U6k9xC853bDSvml52oH/NYHzp4g/57J0F9kdoCl0IalrZvG+ldqIq3EeXJfzdp1d3+1J3DVacP0jG0jThBpd3rTwU7uKJHZz2kMH4psVXdlFTypvOEpIz9BwFUymH/rohzJy1w9ozQSbzgK5vxvb/aqlQfU9txeOjr71iydupnFMWmn6Q1D/Idwzy9t/3ZrRdY55dC8Q5wgb5gmSXzmzvi7nAqEI98yUbT5fJnX+QK3lnqP/Q5vpjgXgH5nE7iKB50TlB5j1OIf2jB1haPc6Ogj6HzO2Jm0TrHo+OHFyGBiD5fuujIKVnZcUcCCYdrFs+4RFR3bRkShRtMykfoKEX6tmXVO4QhWuz5ajyn/RWc0qiiZFDlAo51ksXUbpn8npLuHAcyf2datnd+hurCBE9k8nc/Gy3nRTik0mQuYIflmHRA+GQGcIEQKilBb1gqwkIJLC2Rthl/mbRuptmJoVDsf+l5vnYm0tKXUiqilfBjqTLjPRhYSYAU9/7CzCRw7BWbiwuQR3pbKvutITZ7GxjqIlt8KJ9Y+QliOfm8fnRCb6KixxojBOvafPxH2l/Xt6nLE67tB6JhJRJPhg7hBWUzHoVcytgvIE0kOCY8HyKtqU0uDCeckrA9YwCR608N12+wETBVdkbdBVO6kkBLH0ote23romNfphARLZkMFCuMMLFhhiSW8yQlGbTxTZkZaPJ9ZVtlkfrvlpaF5aakISkW5Meg6iwemYRtw8IdyCf6R1pOcoOxRsWoiF8VoOmuMQLp8zc2O30o2Pj/TzbsLBAdb7xVA2WiSjHG6djlus8py28GWfdTEb16edsE7wLboNiRso4wUq0eyBpMh0w+xmjNI7mB+O+NwAUQ+ayP0iRbXIw1O2Uh9mPSSAsZ8g1VWIhlV5Fcl8XkfSs5fTzNUD+4EV238URwJcRpCvk0Es0nvc7v0Y1bkxWluV0nLbgGaDg6X4zZW XL5lWaIO f8jFCh4dlYrDpTFaWxP+mMqbhuNUgopsCeLrgxw5Hp/qeGfGqI3G9ZwlPraMo5dwdKBouzC9K4jH1WnhXs3r38Nrms+mY+lrDZd/lhR0tAzeYDVSGNG5/DyhGD8pkoCtKf25Blkmaulx4bnqoTt55z4QweIhECdmNleL9IrCks19Ovygo1B+uWcxzg6PIIqfB+T3wNyBFbOO4hlEBVm43ixlEE1nDLY0hRZccLwD7Rg4zih4gdXItwQLeDNAa6kwi62JNPhNRhm7nCt9FGW5COOtZNDW0D7exeRQoxq+QoHtOMIcJskfj19tyVv7/5n/clpEnlFUvX72tKKNeuH1UjZ2Epgvf6gFqALbQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 12/5/2023 2:21 PM, Michal Hocko wrote: > CAUTION: EXTERNAL EMAIL. Do not click links or open attachments unless you recognize the sender and were expecting this message. > > > On Tue 05-12-23 14:12:17, Srinivasulu Thanneeru wrote: >> >> >> On 12/5/2023 2:05 PM, Michal Hocko wrote: >>> CAUTION: EXTERNAL EMAIL. Do not click links or open attachments unless you recognize the sender and were expecting this message. >>> >>> >>> On Tue 05-12-23 01:26:07, Srinivasulu Thanneeru wrote: >>>> >>>> >>>> On 12/4/2023 9:13 PM, Michal Hocko wrote: >>>>> CAUTION: EXTERNAL EMAIL. Do not click links or open attachments unless you recognize the sender and were expecting this message. >>>>> >>>>> >>>>> On Fri 01-12-23 03:34:20, sthanneeru.opensrc@micron.com wrote: >>>>>> From: Srinivasulu Thanneeru >>>>>> >>>>>> The memory tiers feature allows nodes with similar memory types >>>>>> or performance characteristics to be grouped together in a >>>>>> memory tier. However, there is currently no provision for >>>>>> moving a node from one tier to another on demand. >>>>> >>>>> Could you expand on why this is really needed/necessary? What is the >>>>> actual usecase? >>>> >>>> Hi Michal Hock, >>>> >>>> Following two use-cases we have observed. >>>> 1. It is not accurate to group similar memory types in the same tier, >>>> because even similar memory types may have different speed grades. >>> >>> Presumably they are grouped based on a HW configuration. Does that mean >>> that the configuration is wrong? Are you trying to workaround that by >>> this interface? >>> >>>> 2. Some systems boots up with CXL devices and DRAM on the same memory-tier, >>>> we need a way to move the CXL nodes to the correct tier from the user space. >>> >>> Again, could you expand a bit more and explain why this cannot be >>> configured automatically? >> >> Yes, in both cases above, if hardware not automatically populated properly, >> in that case this interface would help to correct it from user space. >> >> We had observed case-2 in our setups. > > How hard it is to address this at the HW level? > > Btw. this is really important piece of context that should be part of > the changelog. Quite honestly introducing user interfaces solely to > workaround HW issues seems a rather weak justification. Are there any > usecases you can think of where this would be useful? I'm not sure how difficult to fix it in the hardware. Sure, i will capture the use-cases in the change log, will be sending V2 by changing interface from adistance_offset to memtier_overwrite to avoid complicated math for finding offset at user-level. Thank you Michal Hocko for the feedback. > -- > Michal Hocko > SUSE Labs