From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF129C36002 for ; Mon, 24 Mar 2025 10:37:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 28DDF280002; Mon, 24 Mar 2025 06:37:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 216D6280001; Mon, 24 Mar 2025 06:37:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 093F0280002; Mon, 24 Mar 2025 06:37:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id DC23A280001 for ; Mon, 24 Mar 2025 06:37:12 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 0407FC18F5 for ; Mon, 24 Mar 2025 10:37:13 +0000 (UTC) X-FDA: 83256092388.25.242E303 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by imf10.hostedemail.com (Postfix) with ESMTP id 9F386C0008 for ; Mon, 24 Mar 2025 10:37:11 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=none; spf=pass (imf10.hostedemail.com: domain of shiju.jose@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=shiju.jose@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742812632; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YauxROHyNsp/qaKD2H8ihpWLARmPSE0WUfJnQezotT4=; b=ET3Dst/qhWEquHJ5LC9N7MVnLIH/xMYYRg3eALHZoxV1H58UdkuiwY3bCns+T1J8hmJ3OI bQEEy417Aa3c+swq3HVdwg8yeWR+5ui93/KGBkxJKbPu4/M0+ynteStmv9Rxy+ezPkiWNd mDnr/AyUT0PGNTnjQqx4pTbw835cR/I= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=none; spf=pass (imf10.hostedemail.com: domain of shiju.jose@huawei.com designates 185.176.79.56 as permitted sender) smtp.mailfrom=shiju.jose@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742812632; a=rsa-sha256; cv=none; b=ZfPjd75kj+59KVprc0ePK5YNkSAp0EYlCG3rGyd2KqKsZnmWVkiJHoqW5fuCyHO2XIuRtL ReRHq865WqzBkLXVfEPeunQTEIeimAFMCj8M78tNb07Ow5kPF/iXt1n4sBcU1XcI3xp3z+ HmN8HgJV6o71ux0gafCgD4OmfKIV/7w= Received: from mail.maildlp.com (unknown [172.18.186.31]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4ZLqHK4jGKz6L540; Mon, 24 Mar 2025 18:37:01 +0800 (CST) Received: from frapeml100007.china.huawei.com (unknown [7.182.85.133]) by mail.maildlp.com (Postfix) with ESMTPS id 94EAB14050C; Mon, 24 Mar 2025 18:37:08 +0800 (CST) Received: from frapeml500007.china.huawei.com (7.182.85.172) by frapeml100007.china.huawei.com (7.182.85.133) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 24 Mar 2025 11:37:08 +0100 Received: from frapeml500007.china.huawei.com ([7.182.85.172]) by frapeml500007.china.huawei.com ([7.182.85.172]) with mapi id 15.01.2507.039; Mon, 24 Mar 2025 11:37:08 +0100 From: Shiju Jose To: Jonathan Cameron CC: "linux-cxl@vger.kernel.org" , "dan.j.williams@intel.com" , "dave@stgolabs.net" , "dave.jiang@intel.com" , "alison.schofield@intel.com" , "vishal.l.verma@intel.com" , "ira.weiny@intel.com" , "david@redhat.com" , "Vilas.Sridharan@amd.com" , "linux-edac@vger.kernel.org" , "linux-acpi@vger.kernel.org" , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , "bp@alien8.de" , "tony.luck@intel.com" , "rafael@kernel.org" , "lenb@kernel.org" , "mchehab@kernel.org" , "leo.duran@amd.com" , "Yazen.Ghannam@amd.com" , "rientjes@google.com" , "jiaqiyan@google.com" , "Jon.Grimm@amd.com" , "dave.hansen@linux.intel.com" , "naoya.horiguchi@nec.com" , "james.morse@arm.com" , "jthoughton@google.com" , "somasundaram.a@hpe.com" , "erdemaktas@google.com" , "pgonda@google.com" , "duenwen@google.com" , "gthelen@google.com" , "wschwartz@amperecomputing.com" , "dferguson@amperecomputing.com" , "wbs@os.amperecomputing.com" , "nifan.cxl@gmail.com" , tanxiaofei , "Zengtao (B)" , Roberto Sassu , "kangkang.shen@futurewei.com" , wanghuiqiang , Linuxarm Subject: RE: [PATCH v2 2/8] EDAC: Update documentation for the CXL memory patrol scrub control feature Thread-Topic: [PATCH v2 2/8] EDAC: Update documentation for the CXL memory patrol scrub control feature Thread-Index: AQHbmcKs9aOZDT4yoEmIHT6+0TZ4QbN9TJeAgATOjJA= Date: Mon, 24 Mar 2025 10:37:08 +0000 Message-ID: References: <20250320180450.539-1-shiju.jose@huawei.com> <20250320180450.539-3-shiju.jose@huawei.com> <20250321100305.000018d2@huawei.com> In-Reply-To: <20250321100305.000018d2@huawei.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.48.154.177] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Rspamd-Queue-Id: 9F386C0008 X-Stat-Signature: xsctzrchjdb737ej5wg9ebmgsjfampmi X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1742812631-704840 X-HE-Meta: U2FsdGVkX1+vKc7v8j2Lx+wEbI4YtCIwknk5yV3KzvQxs4e2pCnqL0jDR/WK6g1onRdGjFEiVOz1CFWqeeChtI/SRzmH7rfUzLz1TnSUWjwKLp++pc0q1E0Anu7IM7byfERTKHZocXHnZIPyuJ7L+CS/ONQYBM6IotmFyYsUW7IHjP1HAcVUIHSPo3wF3E94b20nZ797opU1gItXkvbvT5d20ugURrLZFmCg5vhGUr2GUJTMHGrE9HO9RxUPoqLCZOB7oW1rh6aKQ4kMMYT538FvTeaR7L1MMNvkdYAU9edx97JgDNwNXrqUdDPydtOiLcgzruEH+GPrEaAHxZa5DADesog7hPAFB8hjS1yLQhbni37dNie50D0K2OBZWZG2EPbhVP1IO7EVmTCqbWiImldS0cgIumtcaIVjjxOKNFTfnNfNg5tlhfSQFh4OiMujyxigrqYQDrLYQQPVL/WVMJpVYVzdGXdEtJo3pD0HcqJXLyFdnx5JqTnm7hLAA3YNbiEKeKKtU3Ep2bMTjRcXUi81+LC6M+KV1j7JYcYOX0axjajzG7YBus9Ywg2i04rL+1LTKc0ikPejzcJgDMfRhudp3iUjimeBlgo0Yrb8WsCMLY3vQFJHQDC3Y1HTolr38EgwHpfH3NjLIiOvQojcqRJb6W1yQgT7ZCeuNWY4JdFzJk/p++eD90X41ZdcHVNpWP6Q4/uiuCH85+/zOFr0ZyoM2KuAdI1/7C+OjYcs2zkn8a5xvF2cI0pR0tzAWDZormsnn6jU9ruTxhQDG3mUYpB/cZd6QNTm2MyTkex6xiQGmaXjNrDAIg2THegjsbb1uoQIsLF6rX7vrsLGtlwWwU4lP7fh89NV1fupxsJvmIZVCOyTYFZL1vSXSG3fWgenE7PXfU+dwKAiIghj3/UTCV7rRSFH9dx7mM63yuvwOjuyr2MnUZY3lbr9e5cfU81bQpzJgY9+Sk2eXvXNIWA fyUv5nWT OCBTZh5Ug8cYE7trv8DYc99OwR36zlqkrTkZIM7/Q5bitwv59Z3mWPr1KuGvCVXbcZF66tz7FPBUCSS5jeup9KVq+RwQ3V5DggrxFn9fKtEjxMjfvnzPQEPBjSqBkdMcEuEi0NapPd7ubEe3wlzg6La3V8XyM5wul2akL/tpZzUbx4SRiGjWLzy6k8UAnEz/Pmt06QQivSa0RwtVieul5BCC52MqFF1FZiPwmQgH5RgnNMyVDoxSHjBGV+P4nVENxcL95xUsUb9pCUvfH8kQ+1DfNeH/Ph0RynXYJKoBOJOdf8HOt8bVwhvgIAWqVHyJ1Y4PcGSxKBqEm8mPUro7RHZESR9TDBVdLPHKTNYaBv1blDis6tjekjJ66BPGXtDi4L843DH/zbcM0uu8P4jeEqk4KmPw/XBl+MiOY3UsoseFd68fiqubeotN7dfhbk04u8INpw8xahai5k+lPj8Wrh5BKVN8gXDXlXIx7xq1cRveW0PnGdkkZ3PJwWUsLB265nqlZVWsCWJ2/3tHMlBKn10n/MejqBwxVisvZL2CeKqyuOZjgM1eY+LQpywlC4AxDFKGDPp4vQtDPAC1wmPqQNVtl0kyiVB4dl+kKzptdig6De7le9ZRe1T1XMHf7+uWTC1PQpGyMl5bLdP3e9mMdWV0ttpuZ7FjX2z1SxThCRGaGMiYFSswFZckn2OkhkANZXD46WAp3Z/bCWYsf/VUYgQH4nNcgYJ33Z3R3YIgeBmmUTf9UlnhQ7DlFvaxnVulFCDt+P4yF7/YK5ajqhi+n/wcenY6d+Y+P2NaY7ynBGdt3Xm2BGrlcHNV2YwnMP6S2uT2eErN55OTt5EI8MHg7EL9uaA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: >-----Original Message----- >From: Jonathan Cameron >Sent: 21 March 2025 10:03 >To: Shiju Jose >Cc: linux-cxl@vger.kernel.org; dan.j.williams@intel.com; dave@stgolabs.net= ; >dave.jiang@intel.com; alison.schofield@intel.com; vishal.l.verma@intel.com= ; >ira.weiny@intel.com; david@redhat.com; Vilas.Sridharan@amd.com; linux- >edac@vger.kernel.org; linux-acpi@vger.kernel.org; linux-mm@kvack.org; linu= x- >kernel@vger.kernel.org; bp@alien8.de; tony.luck@intel.com; rafael@kernel.o= rg; >lenb@kernel.org; mchehab@kernel.org; leo.duran@amd.com; >Yazen.Ghannam@amd.com; rientjes@google.com; jiaqiyan@google.com; >Jon.Grimm@amd.com; dave.hansen@linux.intel.com; >naoya.horiguchi@nec.com; james.morse@arm.com; jthoughton@google.com; >somasundaram.a@hpe.com; erdemaktas@google.com; pgonda@google.com; >duenwen@google.com; gthelen@google.com; >wschwartz@amperecomputing.com; dferguson@amperecomputing.com; >wbs@os.amperecomputing.com; nifan.cxl@gmail.com; tanxiaofei >; Zengtao (B) ; Roberto >Sassu ; kangkang.shen@futurewei.com; >wanghuiqiang ; Linuxarm > >Subject: Re: [PATCH v2 2/8] EDAC: Update documentation for the CXL memory >patrol scrub control feature > >On Thu, 20 Mar 2025 18:04:39 +0000 > wrote: > >> From: Shiju Jose >> >> Update the Documentation/edac/scrub.rst to include descriptions and >> policies for CXL memory device-based and CXL region-based patrol scrub >> control. >> >> Note: This may require inputs from CXL memory experts regarding >> region-based scrubbing policies. > >So I suggested the region interfaces in the first place. It's all about u= secases and >'why' we might increase the scrub rate. >Ultimately the hardware is controlled in a device wide way, so we could ha= ve >made it complex userspace problem to deal with it on a perf device. >The region interfaces are there as a simplification not because they are s= trictly >necessary. > >Anyhow, the use cases: > >1) Scrubbing because a device is showing unexpectedly high errors. That > control needs to be at device granularity. If one device in an interle= ave > set (backing a region) is dodgy, why make them all do more work? > >2) Scrubbing may apply to memory that isn't online at all yet. Nice to kn= ow > if we have a problem before we start using it! Likely this is setting > system wide defaults on boot. > >3) Scrubbing at higher rate because software has decided that we want > more reliability for particular data. I've been calling this > Differentiated Reliability. That data sits in a region which > may cover part of multiple devices. The region interfaces are about > supporting this use case. > >So now the question is what do we do if both interfaces are poked because >someone cares simultaneously about 1 and 3? > >I'd suggest just laying out a set for rules on how to set the scrub rates = for any >mixture of requirements, rather than making the driver work out the optimu= m >combination. > >> >> Signed-off-by: Shiju Jose >> --- >> Documentation/edac/scrub.rst | 47 >> ++++++++++++++++++++++++++++++++++++ >> 1 file changed, 47 insertions(+) >> >> diff --git a/Documentation/edac/scrub.rst >> b/Documentation/edac/scrub.rst index daab929cdba1..d1c02bd90090 100644 >> --- a/Documentation/edac/scrub.rst >> +++ b/Documentation/edac/scrub.rst >> @@ -264,3 +264,51 @@ Sysfs files are documented in >> `Documentation/ABI/testing/sysfs-edac-scrub` >> >> `Documentation/ABI/testing/sysfs-edac-ecs` >> + >> +Examples >> +-------- >> + >> +The usage takes the form shown in these examples: >> + >> +1. CXL memory device patrol scrubber >> + >> +1.1 Device based scrubbing >> + >> +CXL memory is exposed to memory management subsystem and ultimately >> +userspace via CXL devices. >> + >> +For cases where hardware interleave controls do not directly map to >> +regions of Physical Address space, perhaps due to interleave the >> +approach described in >> +1.2 Region based scrubbing section, which is specific to CXL regions >> +should be followed. > >These sentences end up a bit unwieldy. Perhaps simply a forwards reference= . > >When combining control via the device interfaces and region interfaces see >1.2 Region bases scrubbing. > > > >> In those cases settings on the presented interface may interact with >> +direct control via a device instance specific interface and care must b= e taken. >> + >> +Sysfs files for scrubbing are documented in >> +`Documentation/ABI/testing/sysfs-edac-scrub` >> + >> +1.2. Region based scrubbing >> + >> +CXL memory is exposed to memory management subsystem and ultimately >> +userspace via CXL regions. CXL Regions represent mapped memory >> +capacity in system physical address space. These can incorporate one >> +or more parts of multiple CXL memory devices with traffic interleaved >> +across them. The user may want to control the scrub rate via this >> +more abstract region instead of having to figure out the constituent >> +devices and program them separately. The scrub rate for each device >> +covers the whole device. Thus if multiple regions use parts of that >> +device then requests for scrubbing of other regions may result in a hig= her >scrub rate than requested for this specific region. >> + >> +1. When user sets scrub rate for a memory region, the scrub rate for al= l the >CXL >> + memory devices interleaved under that region is updated with the sam= e >scrub >> + rate. > >Note that this may affect multiple regions. > >> + >> +2. When user sets scrub rate for a memory device, only the scrub rate f= or >that >> + memory devices is updated though device may be part of a memory regi= on >and >> + does not change scrub rate of other memory devices of that memory >region. >> + >> +3. Scrub rate of a CXL memory device may be set via EDAC device or regi= on >scrub >> + interface simultaneously. Care must be taken to prevent a race condi= tion, >or >> + only region-based setting may be allowed. > >So is this saying if you want to mix and match, set region first then devi= ce next? >Can we just lay out the rules to set up a weird mixture. We could add mor= e >smarts to the driver but do we care as mixing 1 and 3 above is probably >unusual? > >1. Taking each region in turn from lowest desired scrub rate to highest an= d set > their scrub rates. Later regions may override the scrub rate on indivi= dual > devices (and hence potentially whole regions). > >2. Take each device for which enhanced scrubbing is required (higher rate)= and > set those scrub rates. This will override the scrub rates of individua= l devices > leaving any that are not specifically set to scrub at the maximum rate = required > for any of the regions they are involved in backing. Thanks. Will incorporate these info and rules in the next version. > > >> + >> +Sysfs files for scrubbing are documented in >> +`Documentation/ABI/testing/sysfs-edac-scrub` Shiju