From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 873ACC35FFF for ; Fri, 21 Mar 2025 10:03:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 07486280002; Fri, 21 Mar 2025 06:03:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 02492280001; Fri, 21 Mar 2025 06:03:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E2FB7280002; Fri, 21 Mar 2025 06:03:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id C639A280001 for ; Fri, 21 Mar 2025 06:03:15 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 2DE66BBB57 for ; Fri, 21 Mar 2025 10:03:16 +0000 (UTC) X-FDA: 83245120392.22.03ED262 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by imf22.hostedemail.com (Postfix) with ESMTP id 337E8C001B for ; Fri, 21 Mar 2025 10:03:12 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of jonathan.cameron@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=jonathan.cameron@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742551394; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9Jna7PX7dvWCoSRjzK5eXlIbxoWS47lYBmLeiB0jS2M=; b=bYNEC5F7WmY2AcBQYM/7XsdCMsURSMh/O6TY32DwL8alcXLbS+Gsls9Wk9fB6RJPGIVP7s xoLO1BeTB7wXHmhc7Qg5sud5dyDhEkjPiL2mTHz4E3esLEOv8AwKTb4qoTCd7m+alJmBIW aGiG8oUKFbwE19pQ/ln53FStLBr/vlI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742551394; a=rsa-sha256; cv=none; b=dIWtphceVSen7Y1pKNBlu+1v+0u/t0ZtbE/1md2OMfmMrgeodBmx//jB8JL9MSMAQcfk0I 9jWXJhIlyY9/ZXqaPIL5ygddSqdffcAYhtGCH4mDCFC5RN06DBt/NVmRhlqAy0vE37ykq2 zATVRAPbavK1dNEU+x22vFsrqzDmp0g= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of jonathan.cameron@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=jonathan.cameron@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from mail.maildlp.com (unknown [172.18.186.31]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4ZJyc82vtFz6K9V7; Fri, 21 Mar 2025 18:00:08 +0800 (CST) Received: from frapeml500008.china.huawei.com (unknown [7.182.85.71]) by mail.maildlp.com (Postfix) with ESMTPS id CC66A140146; Fri, 21 Mar 2025 18:03:08 +0800 (CST) Received: from localhost (10.203.177.66) by frapeml500008.china.huawei.com (7.182.85.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Fri, 21 Mar 2025 11:03:07 +0100 Date: Fri, 21 Mar 2025 10:03:05 +0000 From: Jonathan Cameron To: CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: Re: [PATCH v2 2/8] EDAC: Update documentation for the CXL memory patrol scrub control feature Message-ID: <20250321100305.000018d2@huawei.com> In-Reply-To: <20250320180450.539-3-shiju.jose@huawei.com> References: <20250320180450.539-1-shiju.jose@huawei.com> <20250320180450.539-3-shiju.jose@huawei.com> X-Mailer: Claws Mail 4.3.0 (GTK 3.24.42; x86_64-w64-mingw32) MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.203.177.66] X-ClientProxiedBy: lhrpeml500004.china.huawei.com (7.191.163.9) To frapeml500008.china.huawei.com (7.182.85.71) X-Stat-Signature: yir6zrijxz87mk6fxh6mkk8mt9d9jj64 X-Rspam-User: X-Rspamd-Queue-Id: 337E8C001B X-Rspamd-Server: rspam08 X-HE-Tag: 1742551392-578680 X-HE-Meta: U2FsdGVkX18R+ESy8DIvbcqX0HSboiGhwu5WUKbuQo7iZmdAqxYoJppt+F7WpXdWOSGKz3dq1UOQlXHb8AEW673zDyZ0D/BIppS61xpjCRst3oQccWaoPexhemqaDZb4EiHWufmyNBbHqtdprtBlgRIuHUxrICRJWFsqVywhwtfzzKPyixDauE7sk+gQ6TS/uclF3fJ2msYhsz/L8dUF11R2cDYNLFYmXix9f1ve54im+bxvDljSEvfbawsn8U3PMjWVDXqXubjVgDu62IjBUQ/zIbgRxeOXch4sRy9DI2JvmvKfiWo0kTMxCs6vl9eTPcy9uMkVZKuDghyUC4d/1Jumx7fciTKodNHAWWEED1KbtuWpL7TENMY/zZvFCS4mt/qZG4uRaATNn5vUJKDRYlZewtlup0zQLY6UPe8Pqno+B9ftihyIP8pmbjDZlckHKYju2H7C39uBV0JWHbOK+4jJZ1zihs8KDHqmc+4kD264f3lKmdjItAe1R29Tbwr7MFBwsi7gOWB1CQHQC2RzCvwbRkuyPoDi8pL9vygx42rs+QQJ/VBW3x/bUCmqUdPpxHorTsWzdHfiaWPIihauc4rmY//nZLZPrVsnneUUrcXIY11T6WhBjpt4xYsQlCmv5sxWGYjHioFIKGIRMJ6zwdYvVdvj+Ik+lBexqQshaRv1q3ALjeO6jlU0d3AOAyTs/mPyGgY/XxGVsrBSjNZO4D/sCiAsFBY3OFUapo4ZFD5mRIU8a3ygIvKz/baleDy3ybjm54ljuiC5t6TAU62EtzlEntJz6zgGar41A86zt6jBApELWKI9b3afaYMG+ZtUAS5kccKzIldhPtbJXKfqhpi+rqoiUpqTNzhPRyWrxujI6qNph66kSui7lqI52LAzc0Jvkeq25eeNgzi4KtBEGyF8NOacDSvEOnd8fpYeEnu1ESkH5HQZkPsleY+CdPZXRKIBM/nOrHJk2wtWsJk oPH0/cRZ QPKNv04KviQdCe3LosU0gC1TZkoA0omA1S/qSdhyD5xVncF0N3D8Rsw2amhVi2wNAh1f5n4hbEUxTsQotjX66NHh5zuN4TwIIwk/NI34jAs4HHy9xeLi82UEDfq10rmeT+bkCbGiWRQ2/Idzp7d81M0fqR0/Qr16zmSsOU6++uu4NOhxq92BWboe6gekj35x5/sTYS4W5HCX+adqtWZY09/er1Lo2ETH1vj+8ymiF5TfUsCOoU2Etpm0ttdWX1BaUej4ZVhyAD57wuww= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000020, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, 20 Mar 2025 18:04:39 +0000 wrote: > From: Shiju Jose >=20 > Update the Documentation/edac/scrub.rst to include descriptions and > policies for CXL memory device-based and CXL region-based patrol scrub > control. >=20 > Note: This may require inputs from CXL memory experts regarding > region-based scrubbing policies. So I suggested the region interfaces in the first place. It's all about usecases and 'why' we might increase the scrub rate. Ultimately the hardware is controlled in a device wide way, so we could have made it complex userspace problem to deal with it on a perf device. The region interfaces are there as a simplification not because they are strictly necessary. Anyhow, the use cases: 1) Scrubbing because a device is showing unexpectedly high errors. That control needs to be at device granularity. If one device in an interlea= ve set (backing a region) is dodgy, why make them all do more work? 2) Scrubbing may apply to memory that isn't online at all yet. Nice to know if we have a problem before we start using it! Likely this is setting system wide defaults on boot. 3) Scrubbing at higher rate because software has decided that we want more reliability for particular data. I've been calling this Differentiated Reliability. That data sits in a region which may cover part of multiple devices. The region interfaces are about supporting this use case. So now the question is what do we do if both interfaces are poked because someone cares simultaneously about 1 and 3? I'd suggest just laying out a set for rules on how to set the scrub rates for any mixture of requirements, rather than making the driver work out the optimum combination. =20 >=20 > Signed-off-by: Shiju Jose > --- > Documentation/edac/scrub.rst | 47 ++++++++++++++++++++++++++++++++++++ > 1 file changed, 47 insertions(+) >=20 > diff --git a/Documentation/edac/scrub.rst b/Documentation/edac/scrub.rst > index daab929cdba1..d1c02bd90090 100644 > --- a/Documentation/edac/scrub.rst > +++ b/Documentation/edac/scrub.rst > @@ -264,3 +264,51 @@ Sysfs files are documented in > `Documentation/ABI/testing/sysfs-edac-scrub` > =20 > `Documentation/ABI/testing/sysfs-edac-ecs` > + > +Examples > +-------- > + > +The usage takes the form shown in these examples: > + > +1. CXL memory device patrol scrubber > + > +1.1 Device based scrubbing > + > +CXL memory is exposed to memory management subsystem and ultimately user= space > +via CXL devices. > + > +For cases where hardware interleave controls do not directly map to regi= ons of > +Physical Address space, perhaps due to interleave the approach described= in=A0 > +1.2 Region based scrubbing section, which is specific to CXL regions sho= uld be > +followed. These sentences end up a bit unwieldy. Perhaps simply a forwards reference. When combining control via the device interfaces and region interfaces see 1.2 Region bases scrubbing. =20 > In those cases settings on the presented interface may interact with > +direct control via a device instance specific interface and care must be= taken. > + > +Sysfs files for scrubbing are documented in > +`Documentation/ABI/testing/sysfs-edac-scrub` > + > +1.2. Region based scrubbing > + > +CXL memory is exposed to memory management subsystem and ultimately user= space > +via CXL regions. CXL Regions represent mapped memory capacity in system > +physical address space. These can incorporate one or more parts of multi= ple CXL > +memory devices with traffic interleaved across them. The user may want t= o control > +the scrub rate via this more abstract region instead of having to figure= out the > +constituent devices and program them separately. The scrub rate for each= device > +covers the whole device. Thus if multiple regions use parts of that devi= ce then > +requests for scrubbing of other regions may result in a higher scrub rat= e than > +requested for this specific region. > + > +1. When user sets scrub rate for a memory region, the scrub rate for all= the CXL > + memory devices interleaved under that region is updated with the same= scrub > + rate.=20 Note that this may affect multiple regions. > + > +2. When user sets scrub rate for a memory device, only the scrub rate fo= r that > + memory devices is updated though device may be part of a memory regio= n and > + does not change scrub rate of other memory devices of that memory reg= ion. > + > +3. Scrub rate of a CXL memory device may be set via EDAC device or regio= n scrub > + interface simultaneously. Care must be taken to prevent a race condit= ion, or > + only region-based setting may be allowed. So is this saying if you want to mix and match, set region first then device next? Can we just lay out the rules to set up a weird mixture. We could add more smarts to the driver but do we care as mixing 1 and 3 above is pro= bably unusual? 1. Taking each region in turn from lowest desired scrub rate to highest and= set their scrub rates. Later regions may override the scrub rate on individ= ual devices (and hence potentially whole regions). 2. Take each device for which enhanced scrubbing is required (higher rate) = and set those scrub rates. This will override the scrub rates of individual= devices leaving any that are not specifically set to scrub at the maximum rate r= equired for any of the regions they are involved in backing. =20 > + > +Sysfs files for scrubbing are documented in > +`Documentation/ABI/testing/sysfs-edac-scrub`