From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69B2AC282D1 for ; Fri, 7 Mar 2025 01:12:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 42EE3280002; Thu, 6 Mar 2025 20:11:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3E009280001; Thu, 6 Mar 2025 20:11:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2A783280002; Thu, 6 Mar 2025 20:11:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 11221280001 for ; Thu, 6 Mar 2025 20:11:57 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 79120A0FA5 for ; Fri, 7 Mar 2025 01:11:59 +0000 (UTC) X-FDA: 83192978358.06.57476E9 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by imf21.hostedemail.com (Postfix) with ESMTP id 6ABF21C0011 for ; Fri, 7 Mar 2025 01:11:55 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf21.hostedemail.com: domain of jonathan.cameron@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=jonathan.cameron@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741309916; a=rsa-sha256; cv=none; b=zeNCKAY4CBfcZL6lSZFWRwAuOffoD3wkzxr1x3P3XcsEGIQ5vEepbhaUrDGntMK1Trrrlv 0lAD0iWOXPYMzqaPG8sDsFa8t3uNQ3DVhoLtAK7OHiEIcZ05rG8CP1NbMg3GcdsdT3GkL7 kLQaKCx3utglQNelCOqjGwFu39Vu1WE= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf21.hostedemail.com: domain of jonathan.cameron@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=jonathan.cameron@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741309916; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vBYv0rqnUSDG6UdtRmuLKVRCR8PFUNwv3qO/Ou6TUno=; b=rMQF9lqbeQ8w/dr18y64V9jhYFofqx5lfB1uiXn5mKsREMtWpObSu/EZGsOyl6qftOg0xZ +nwSvGK/lALZb51Olg7jTvBj3eC3UelBbBSeeK9Qmi4NzTT0O+fiG+thM46I4wiGghTKsS y9SGPQMxIeWKOCKo6P8+MoXkwLOQevc= Received: from mail.maildlp.com (unknown [172.18.186.216]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4Z87Tf0gM1z6J6F2; Fri, 7 Mar 2025 09:08:54 +0800 (CST) Received: from frapeml500008.china.huawei.com (unknown [7.182.85.71]) by mail.maildlp.com (Postfix) with ESMTPS id 7A8C7140C98; Fri, 7 Mar 2025 09:11:53 +0800 (CST) Received: from localhost (10.48.43.65) by frapeml500008.china.huawei.com (7.182.85.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Fri, 7 Mar 2025 02:11:42 +0100 Date: Fri, 7 Mar 2025 09:11:37 +0800 From: Jonathan Cameron To: CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: Re: [PATCH 8/8] cxl/memfeature: Add CXL memory device memory sparing control feature Message-ID: <20250307091137.00006a0a@huawei.com> In-Reply-To: <20250227223816.2036-9-shiju.jose@huawei.com> References: <20250227223816.2036-1-shiju.jose@huawei.com> <20250227223816.2036-9-shiju.jose@huawei.com> X-Mailer: Claws Mail 4.3.0 (GTK 3.24.42; x86_64-w64-mingw32) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.48.43.65] X-ClientProxiedBy: lhrpeml500010.china.huawei.com (7.191.174.240) To frapeml500008.china.huawei.com (7.182.85.71) X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 6ABF21C0011 X-Stat-Signature: 6s9ukqky1ghxgtm4i6gjcy9gewimd3fu X-HE-Tag: 1741309915-957837 X-HE-Meta: U2FsdGVkX19ECt5DiPeOO46FyCnxLw4Av/Dcwx/127fDAUpn65F+0RivpW14tZe1nyf1Anacx3qSQZxTmxbq7+hLfY+v5Jc9PqKR6uNfrTfmLAw3/e34ohHHMiSjsczl8qpcdwZ8ggZxImCKP4H1vM7HJg5VhOfrAAvftWB6hacop3w3kNZ+tWWpyg5bgfPTj9khgFO2qTW11DI3t+Kj4dzrjWjGPVqscQId99KkgwTPgSvJ7NuaZgtK0Y9sin7GyV+BSax1AVHuL91BSm7XvrNZUansVwa701pTTBaefQq4b0um3cUxf51gBE8gaXvcnyvgClBfpLl3m43mBjGuCLNwTTinwB7Ix3AKraQId/ZiPjkK7Pl6Nq6KuJJ6bOrNFf/fxt9asHlRyRLKLeocHIgI+MWMCaHZ7vN4RK/BQ42tSbzLMDO0jxfcbF3s/eKxKL3p7e3idiJGIrto7ahpM0RdGA5x5+L6hAcolro3EUoPQjm3ypOTy93P6XmmuVyALNAt79dI48C5hc65cujH+M0ECC7rn3RtPwNflHn4I9b1W3pj+qsdSb6VWr7TVJhos9vynopGphPSN/GAczoP1zKdi7zrXBrqmNVVjM4iQTTP5ww3/DcEIaAPMqD9L8wGZ0/uIxKtDtwRZHmNqWIrwQBOEvupSyT8rLSrnRq4LO3w31XJrsrbKXPA4HHMAIx25egJuseH6ZlnDHFx6YvfK94sgPLXnTH2fgyhH4rFWUe3C55DSv0UQ/US89WWMLfoovKTK6CiShCwG7dHmq4SDCYs8gvf7tHpcphaLszISfrEnfdLMENCMBP0LGjc005JN6WDeVsVqyCWpQASTrVuqZcRba1OGsWWhkEr3tI1WCcohNGBKnNrClrg0Dea/2BizmHxV3JZH27y2RyPESXp1MHVJkMVCeO+5yiSaLA3/JgqzumbIypDouG8JBk26gGyrR05UnrGcBDKQw+72P3 QgTbckqL y002ZbYtfe7BZPb5VS6hmEHDIE7Nnvp4Bh33/zjan2hOzk7IFKu8p3/Tpppkz0E5pMtKNZDb1oQByma87u29tDLCjFAgRPGlZL7xgkqI86NzjZ1NklwJ38RHmA1rJHKgYRAMFax2kAbFc9Rt+zCejhvFRp3jZIniGYqiBL5b8SRQW3pkNk0IpEDVsVjPfciW86Cn4aIKod7muhBDNyir35RqMgF79YEjX/W2JWU5upHQTk10jeXIWBWVwzG/Dxc9t8e/4qO0VJoHc0ZWfxciLZ27bwEZ/CGj3j4Ka2lliTjseSGE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, 27 Feb 2025 22:38:15 +0000 wrote: > From: Shiju Jose > > Memory sparing is defined as a repair function that replaces a portion of > memory with a portion of functional memory at that same DPA. The subclasses > for this operation vary in terms of the scope of the sparing being > performed. The cacheline sparing subclass refers to a sparing action that > can replace a full cacheline. Row sparing is provided as an alternative to > PPR sparing functions and its scope is that of a single DDR row. > As per CXL r3.2 Table 8-125 foot note 1. Memory sparing is preferred over > PPR when possible. > Bank sparing allows an entire bank to be replaced. Rank sparing is defined > as an operation in which an entire DDR rank is replaced. > > Memory sparing maintenance operations may be supported by CXL devices > that implement CXL.mem protocol. A sparing maintenance operation requests > the CXL device to perform a repair operation on its media. > For example, a CXL device with DRAM components that support memory sparing > features may implement sparing maintenance operations. > > The host may issue a query command by setting query resources flag in the > input payload (CXL spec 3.2 Table 8-120) to determine availability of > sparing resources for a given address. In response to a query request, > the device shall report the resource availability by producing the memory > sparing event record (CXL spec 3.2 Table 8-60) in which the Channel, Rank, > Nibble Mask, Bank Group, Bank, Row, Column, Sub-Channel fields are a copy > of the values specified in the request. > > During the execution of a sparing maintenance operation, a CXL memory > device: > - may not retain data > - may not be able to process CXL.mem requests correctly. > These CXL memory device capabilities are specified by restriction flags > in the memory sparing feature readable attributes. > > When a CXL device identifies error on a memory component, the device > may inform the host about the need for a memory sparing maintenance > operation by using DRAM event record, where the 'maintenance needed' flag > may set. The event record contains some of the DPA, Channel, Rank, > Nibble Mask, Bank Group, Bank, Row, Column, Sub-Channel fields that > should be repaired. The userspace tool requests for maintenance operation > if the 'maintenance needed' flag set in the CXL DRAM error record. > > CXL spec 3.2 section 8.2.10.7.1.4 describes the device's memory sparing > maintenance operation feature. > > CXL spec 3.2 section 8.2.10.7.2.3 describes the memory sparing feature > discovery and configuration. > > Add support for controlling CXL memory device memory sparing feature. > Register with EDAC driver, which gets the memory repair attr descriptors > from the EDAC memory repair driver and exposes sysfs repair control > attributes for memory sparing to the userspace. For example CXL memory > sparing control for the CXL mem0 device is exposed in > /sys/bus/edac/devices/cxl_mem0/mem_repairX/ > > Use case > ======== > 1. CXL device identifies a failure in a memory component, report to > userspace in a CXL DRAM trace event with DPA and other attributes of > memory to repair such as channel, rank, nibble mask, bank Group, > bank, row, column, sub-channel. > > 2. Rasdaemon process the trace event and may issue query request in sysfs > check resources available for memory sparing if either of the following > conditions met. > - 'maintenance needed' flag set in the event record. > - 'threshold event' flag set for CVME threshold feature. > - If the previous case is not enough, may be when the number of corrected > error reported on a CXL.mem media to the user space exceeds an error > threshold set in the userspace policy. > > 3. Rasdaemon process the memory sparing trace event and issue repair > request for memory sparing. > > Kernel CXL driver shall report memory sparing event record to the userspace > with the resource availability in order rasdaemon to process the event > record and issue a repair request in sysfs for the memory sparing operation > in the CXL device. > > Note: Based on the feedbacks from the community 'query' sysfs attribute is > removed and reporting memory sparing error record to the userspace are not > supported. Instead userspace issues sparing operation and kernel does the > same to the CXL memory device, when 'maintenance needed' flag set in the > DRAM event record. > > Add checks to ensure the memory to be repaired is offline and if online, > then originates from a CXL DRAM error record reported in the current boot > before requesting a memory sparing operation on the device. > > Tested for memory sparing control feature with > "hw/cxl: Add memory sparing control feature" > Repository: "https://gitlab.com/shiju.jose/qemu.git" > Branch: cxl-ras-features-2024-10-24 > > Signed-off-by: Shiju Jose Similar comment to earlier on maybe using single line comments in more places rather than multiline. Perhaps worth doing that if you are respinning for other reasons. Reviewed-by: Jonathan Cameron > +static int cxl_mem_do_sparing_op(struct device *dev, > + struct cxl_mem_sparing_context *cxl_sparing_ctx, > + struct cxl_memdev_sparing_params *rd_params) > +{ > + struct cxl_memdev *cxlmd = cxl_sparing_ctx->cxlmd; > + struct cxl_memdev_sparing_in_payload sparing_pi; > + struct cxl_event_dram *rec = NULL; > + u16 validity_flags = 0; > + > + if (!rd_params->cap_safe_when_in_use) { > + /* > + * Memory to repair must be offline > + */ > + if (cxl_are_decoders_committed(cxlmd)) > + return -EBUSY; > + /* > + * offline, so good for repair > + */ More places as below where a single line comment would be fine and make a reader scroll a bit less. > +static int cxl_memdev_sparing_init(struct cxl_memdev *cxlmd, > + struct edac_dev_feature *ras_feature, > + const struct cxl_mem_sparing_desc *desc, > + u8 repair_inst) > +{ > + struct cxl_mem_sparing_context *cxl_sparing_ctx; > + struct cxl_memdev_sparing_params rd_params; > + struct cxl_feat_entry *feat_entry; > + int ret; > + > + feat_entry = cxl_get_feature_entry(cxlmd->cxlds, &desc->repair_uuid); > + if (IS_ERR(feat_entry)) > + return -EOPNOTSUPP; > + > + if (!(le32_to_cpu(feat_entry->flags) & CXL_FEATURE_F_CHANGEABLE)) > + return -EOPNOTSUPP; > + > + cxl_sparing_ctx = devm_kzalloc(&cxlmd->dev, sizeof(*cxl_sparing_ctx), > + GFP_KERNEL); > + if (!cxl_sparing_ctx) > + return -ENOMEM; > + > + *cxl_sparing_ctx = (struct cxl_mem_sparing_context) { > + .get_feat_size = le16_to_cpu(feat_entry->get_feat_size), > + .set_feat_size = le16_to_cpu(feat_entry->set_feat_size), > + .get_version = feat_entry->get_feat_ver, > + .set_version = feat_entry->set_feat_ver, > + .effects = le16_to_cpu(feat_entry->effects), > + .cxlmd = cxlmd, > + .repair_type = desc->repair_type, > + .granularity = desc->granularity, > + .instance = repair_inst++, > + }; > + uuid_copy(&cxl_sparing_ctx->repair_uuid, &desc->repair_uuid); > + > + /* > + * Read CXL device's sparing capabilities. a below. > + */ > + ret = cxl_mem_sparing_get_attrs(cxl_sparing_ctx, &rd_params); > + if (ret) > + return ret; > + > + /* > + * Set default value for persist_mode. > + */ If respining some of these comments don't need to be multiline.