From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1537ACA0FE7 for ; Tue, 26 Aug 2025 19:28:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4BC4D6B02D5; Tue, 26 Aug 2025 15:28:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 492546B02D6; Tue, 26 Aug 2025 15:28:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3A8696B02D7; Tue, 26 Aug 2025 15:28:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 230926B02D5 for ; Tue, 26 Aug 2025 15:28:42 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 9E31BC02AF for ; Tue, 26 Aug 2025 19:28:41 +0000 (UTC) X-FDA: 83819895642.03.5887C0F Received: from mx0b-002e3701.pphosted.com (mx0b-002e3701.pphosted.com [148.163.143.35]) by imf08.hostedemail.com (Postfix) with ESMTP id 8255716000F for ; Tue, 26 Aug 2025 19:28:34 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=temperror ("DNS error when getting key") header.d=hpe.com header.s=pps0720 header.b="g/kNUoMX"; dmarc=pass (policy=reject) header.from=hpe.com; spf=pass (imf08.hostedemail.com: domain of kyle.meyer@hpe.com designates 148.163.143.35 as permitted sender) smtp.mailfrom=kyle.meyer@hpe.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1756236519; a=rsa-sha256; cv=none; b=QRpvYRI7sCXjMZc2LPr7PkTWp4NTY2VVUYFBUo/a4VQq+3Y70q9vvW163M7bx41UDjkRPI r8YW4ZfjEn21i9FzF6ujcpxfMn2Pi6BadkfBU2viyUwEX75WcKZ58lLBXrEwEOuJzRF/9W vtSShinJ/Sf02V59ZmU398PnK4b4hTQ= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=temperror ("DNS error when getting key") header.d=hpe.com header.s=pps0720 header.b="g/kNUoMX"; dmarc=pass (policy=reject) header.from=hpe.com; spf=pass (imf08.hostedemail.com: domain of kyle.meyer@hpe.com designates 148.163.143.35 as permitted sender) smtp.mailfrom=kyle.meyer@hpe.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1756236519; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7JUHf7BGcTCGx0OjIaqQaW89aRdyWS24lXplepqdf0Y=; b=2OlFKWO5OPnhi8hE1Ow2KP21N47FFEuvX2e7BXd3TgqHc8kXg7eK5meSgQthOOhM0t7sB+ X2jaLqIvhAY3Xo3TSGpK47sngoKPpmpi4IfLIUhzx5xPlcNbfPfoCEdH7wUy8KFVqF3JAw GN1o7CgVKWoAuwLnnnXkHJ66ILEifhk= Received: from pps.filterd (m0134425.ppops.net [127.0.0.1]) by mx0b-002e3701.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57QEnWvt005250; Tue, 26 Aug 2025 19:27:54 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hpe.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pps0720; bh=7J UHf7BGcTCGx0OjIaqQaW89aRdyWS24lXplepqdf0Y=; b=g/kNUoMXTvnT5Ku/cb 9bBfSQn+TMvZQjL6YZWvSskTESSPnBEL9fumxirEoiFEUspBWyuXA1GzJDrj74Xc dzRbz2SgJcR+V47fNbUnvpGBedg9FHKzIHN71Mi/tx76JUAe6TBU/0e4F3omzvi2 hkHQp9xTEmRAGO0QnTF4RRLOQWPzASxGaHvnYp3P82miHcza+OH0BC1D8tij+9mo 89T1tGrQ6QydTAkrO+f61PCx6U8ZMH0wzBInAaBtW/RUoXnRkDQy57U/T9bqE0h6 BGhGoLYpd4V300MEUv4KCGlN/6a85iSKdZoqWhEEMJdWfDhR9tpYxNT8gu3ruTIR rwPw== Received: from p1lg14878.it.hpe.com ([16.230.97.204]) by mx0b-002e3701.pphosted.com (PPS) with ESMTPS id 48sf17au8q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 26 Aug 2025 19:27:54 +0000 (GMT) Received: from p1lg14886.dc01.its.hpecorp.net (unknown [10.119.18.237]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by p1lg14878.it.hpe.com (Postfix) with ESMTPS id 969BD130D8; Tue, 26 Aug 2025 19:27:52 +0000 (UTC) Received: from HPE-5CG20646DK.localdomain (unknown [16.231.227.39]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (Client did not present a certificate) by p1lg14886.dc01.its.hpecorp.net (Postfix) with ESMTPS id 6E097809D02; Tue, 26 Aug 2025 19:27:49 +0000 (UTC) Date: Tue, 26 Aug 2025 14:27:47 -0500 From: Kyle Meyer To: jane.chu@oracle.com Cc: Miaohe Lin , Jiaqi Yan , akpm@linux-foundation.org, david@redhat.com, tony.luck@intel.com, bp@alien8.de, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-edac@vger.kernel.org, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, nao.horiguchi@gmail.com, osalvador@suse.de, russ.anderson@hpe.com Subject: Re: [PATCH] mm/memory-failure: Do not call action_result() on already poisoned pages Message-ID: References: <20250821164445.14467-1-kyle.meyer@hpe.com> <14a0dd45-388d-7a32-5ee5-44e60277271a@huawei.com> <2bd5c32b-dfc4-4345-8cc8-bbda5acdc596@oracle.com> <714d066a-a06e-4b49-b66f-68952c6520d9@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <714d066a-a06e-4b49-b66f-68952c6520d9@oracle.com> X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODI2MDE3MSBTYWx0ZWRfX/iay3UD/QIpG ZhRWyAqGXSPnmUyBHzU1VEPLrTEjXCkS6onG9h7yiMGttrr6J022MV4K2q8IJNtEkqT7KkWi6bH Wn3BackBAowKiWca6Zaf0DtDwcthbjvVsoVAjvMubjVqvVAP7pMmWoF5Y6dj9ZSyB/V7rFOErb3 T30pVuTetvwN3vvy0mACUxYaAJG+iOmcbizstAB+VwSHmNqhOGipAgCJsE7tjzVgFzo0WonZGGl or35urLsDkN6nDxb3N0XPyx8f0M9a/yWa81iXIupaBgDGUPZ3IzCv6r0ja/BegIoM63j6jGci1d RbawfiBj8xG+rKRkaA+1/YApm1DaxkNkkJcxxN+Uc/hArcWmWp0tMW1BGSHcHJIZfQzXgrHgBq3 mBByblNwEcc4uQXp+0QMMiDXkwLaVrXHeeY/NpCmNhfclZFFTQ6DHcnfhObf1ySNoHxT+rwt X-Proofpoint-GUID: 1fcIsYQLzFsdnz0CmSebmgZueUEdY4x0 X-Proofpoint-ORIG-GUID: 1fcIsYQLzFsdnz0CmSebmgZueUEdY4x0 X-Authority-Analysis: v=2.4 cv=MZNsu4/f c=1 sm=1 tr=0 ts=68ae0aba cx=c_pps a=UObrlqRbTUrrdMEdGJ+KZA==:117 a=UObrlqRbTUrrdMEdGJ+KZA==:17 a=IkcTkHD0fZMA:10 a=2OwXVqhp2XgA:10 a=yPCof4ZbAAAA:8 a=MvuuwTCpAAAA:8 a=1XWaLZrsAAAA:8 a=77G3ujYeI9cyGRGMJesA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-HPE-SCL: -1 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-26_02,2025-08-26_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 phishscore=0 lowpriorityscore=0 mlxlogscore=999 bulkscore=0 adultscore=0 clxscore=1015 malwarescore=0 spamscore=0 mlxscore=0 suspectscore=0 priorityscore=1501 classifier=spam authscore=0 authtc=n/a authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2508260171 X-Rspamd-Queue-Id: 8255716000F X-Stat-Signature: bktmtgnh18y4abt517anxnstg5qhbwd1 X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1756236514-471235 X-HE-Meta: U2FsdGVkX1+Y3yDuDsra+DU+rDOZnPk2cPKLBZGXMtOrfZFD0MANNpdSPOz0eP+ULm66j7RNUymR9MgjgxqTfaKD//DyuPOayAwTVWqPU9chJbY+8oCISJydKOhJfuXYVDvRkwZnDzAgz+jC2ww1AKrR0lqE33o6tt1cdi9cLaxXRlcAQrBa/wshvzrnHr+ckKnlhDP3DagPveLdnYtZlFm26ZaYL1zGIM2L11Jv3M09PK0iYUsCm6/R77X7+L5bhT3Ty1VCMh8rizsCAr9pUvFk25DcpsD/Wf6lK97sHe4GfwwiNgdOxGyW7cG6O3MeG+UPizdqnaTeo3ZjavZvx8nbnWFQplyujzY4IusITrhDCC4jmtKUWA+sHd98iW+yKV42v7CE+zgJyL/A+hOEzz5lywOuxaRmJE1fBrcLWTMW5uHA2UUHuXVZnG/QcpbqwV5nV0ZUNvLTjZSEHqSP7RJxylqBuR4DnK3wds4lwge8TZoEm4cDbva7U26w7tSCngkh6j88yQw6d6SUu2jXHc6fwHatR9TT7sKPrG2Ef2WcesArU7unSjJnMd+R66J97LB9vKmdD86Pz4EChGmqFbHuc3qsg8MScfkmhG6j/uKo4DGdUZONfviRA/664bHSHLRnwH1hQpJFzwR7TRLIamqri9prqslNyaTvXMe+I4GcBKilbX7RuHqF5TfdhdEXet8cJR0k1stzVxzPM/nIe/yGlxF7xwajY/thhWZkl+Wpks/ENC0a7aAizLx3nImdQCesmde6sU3xuRvH6fXxIYoaJAAtHQJNpaDdYJTA11EdfmlMCTroeseacG6w94ExMnH1nKyVAT//4iUvpP5E12n5fcsf7nGocU4JncyIYIWAJlDAEcNmhFoVy+/rEdTg2sGGV6RxZp8CZlW5yGJNKLEnvhRs/x3kr0ccXUGuMP3g5n7WXUl6BoSsKqEd7x64mv12DOPHN1Xz66U2JBB fAyJMaKh gycA4HpXA41uUqU0uXw+wgxwACjl8MoppWaIkPCfQyiXTxuhxxzBYPLvzCgXEKvuNIGIe6rekIggnHhJcU4k1VG7ytnxAYnDZXyNculQhVDeMn04EqFfUcpplo/KOR2E/mty+uwdq4+40S3BjZUaWM4nyEpcIHhAiaWRDRFFLp47t0jUytRn/C2b4rCoTt0/ayrQGhZUJL+t2qeyJwc6qGWyl52AQnKHbgyYE/cUi71afI1g6jyTNHn3RoW+8Ty8QuQtyz889GSnCOE4i8NACu4OgbH/Yzs36UbPur6xS26UV9FR0LLO49eCRb2w+A49skBL/oGcPDrP3xmNI29MUHiOPMn5krYv8y1sYh5yb4iHYMFd+ydREhiMTdEeDllFEm69g9gWixk07MjbsweI/rpIRFCn1OtoIriyBAr5cNPBtH9EdL7JRJ1ARUYsVeGHyLEkt X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Aug 26, 2025 at 10:24:07AM -0700, jane.chu@oracle.com wrote: > > On 8/25/2025 6:56 PM, Kyle Meyer wrote: > > On Mon, Aug 25, 2025 at 03:36:54PM -0700, jane.chu@oracle.com wrote: > > > On 8/25/2025 9:09 AM, Kyle Meyer wrote: > > > > On Mon, Aug 25, 2025 at 11:04:43AM +0800, Miaohe Lin wrote: > > > > > On 2025/8/22 8:24, Jiaqi Yan wrote: > > > > > > On Thu, Aug 21, 2025 at 12:36 PM Kyle Meyer wrote: > > > > > > > > > > > > > > On Thu, Aug 21, 2025 at 11:23:48AM -0700, Jiaqi Yan wrote: > > > > > > > > On Thu, Aug 21, 2025 at 9:46 AM Kyle Meyer wrote: > > > > > > > > > > > > > > > > > > Calling action_result() on already poisoned pages causes issues: > > > > > > > > > > > > > > > > > > * The amount of hardware corrupted memory is incorrectly incremented. > > > > > > > > > * NUMA node memory failure statistics are incorrectly updated. > > > > > > > > > * Redundant "already poisoned" messages are printed. > > > > > > Assuming this means that the numbers reported from > > > /sys/devices/system/node/node*/memory_failure/* > > > do not match certain expectation, right? > > > > > > If so, could you clarify what is the expectation? > > > > Sure, and please let me know if I'm mistaken. > > > > Here's the description of total: > > > > What: /sys/devices/system/node/nodeX/memory_failure/total > > Date: January 2023 > > Contact: Jiaqi Yan > > Description: > > The total number of raw poisoned pages (pages containing > > corrupted data due to memory errors) on a NUMA node. > > > > That should emit the number of poisoned pages on NUMA node X. That's > > incremented each time update_per_node_mf_stats() is called. > > > > Here's the description of failed: > > > > What: /sys/devices/system/node/nodeX/memory_failure/failed > > Date: January 2023 > > Contact: Jiaqi Yan > > Description: > > Of the raw poisoned pages on a NUMA node, how many pages are > > failed by memory error recovery attempt. This usually means > > a key recovery operation failed. > > > > That should emit the number of poisoned pages on NUMA node X that could > > not be recovered because the attempt failed. That's incremented each time > > update_per_node_mf_stats() is called with MF_FAILED. > > > > We're currently calling action_result() with MF_FAILED each time we encounter > > a poisoned page (note: the huge page path is a bit different, we only call > > action_result() with MF_FAILED when MF_ACTION_REQUIRED is set). That, IMO, > > breaks the descriptions. We already incremented the per NUMA node MF statistics > > to account for that poisoned page. > > Thanks! My reading is that these numbers are best as hints, I won't take > them literately. As you alluded below, kill_accessing_process is applied > only if MF_ACTION_REQUIRED is set, though the page is already marked > poisoned. Besides, there can be bug that renders a poisoned page not being > properly isolated nor being properly categorized. If you're looking for > something precise, is there another way? maybe from firmware? Firmware records the number of memory errors that have been detected and reported, but it doesn't record how Linux responded to those memory errors. Checking the ring buffer, the amount of hardware corrupted memory, and the per NUMA node memory failure statistics is a simple way to determine how Linux responded. Since commit b8b9488d50b7, that has become unreliable. The same memory error may be reported by multiple sources and now each report increments the amount of hardware corrupted memory and the per NUMA node memory failure statistics. Isn't that a regression? The per NUMA node memory failure statistics might not always be 100% accurate, but this issue seems preventable. > > > > > > > > > > > > > > > > All agreed. > > > > > > > > > > > > > > > > > > > > > > > > > > Do not call action_result() on already poisoned pages and drop unused > > > > > > > > > MF_MSG_ALREADY_POISONED. > > > > > > > > > > > > > > > > Hi Kyle, > > > > > > > > > > > > > > > > Patch looks great to me, just one thought... > > > > > > > > > > Thanks both. > > > > > > > > > > > > > > > > > > > > > Alternatively, have you thought about keeping MF_MSG_ALREADY_POISONED > > > > > > > > but changing action_result for MF_MSG_ALREADY_POISONED? > > > > > > > > - don't num_poisoned_pages_inc(pfn) > > > > > > > > - don't update_per_node_mf_stats(pfn, result) > > > > > > > > - still pr_err("%#lx: recovery action for %s: %s\n", ...) > > > > > > > > - meanwhile remove "pr_err("%#lx: already hardware poisoned\n", pfn)" > > > > > > > > in memory_failure and try_memory_failure_hugetlb > > > > > > > > > > > > > > I did consider that approach but I was concerned about passing > > > > > > > MF_MSG_ALREADY_POISONED to action_result() with MF_FAILED. The message is a > > > > > > > bit misleading. > > > > > > > > > > > > Based on my reading the documentation for MF_* in static const char > > > > > > *action_name[]... > > > > > > > > > > > > Yeah, for file mapped pages, kernel may not have hole-punched or > > > > > > truncated it from the file mapping (shmem and hugetlbfs for example) > > > > > > but that still considered as MF_RECOVERED, so touching a page with > > > > > > HWPoison flag doesn't mean that page was failed to be recovered > > > > > > previously. > > > > > > > > > > > > For pages intended to be taken out of the buddy system, touching a > > > > > > page with HWPoison flag does imply it isn't isolated and hence > > > > > > MF_FAILED. > > > > > > > > > > There should be other cases that memory_failure failed to isolate the > > > > > hwpoisoned pages at first time due to various reasons. > > > > > > > > > > > > > > > > > In summary, seeing the HWPoison flag again doesn't necessarily > > > > > > indicate what the recovery result was previously; it only indicate > > > > > > kernel won't re-attempt to recover? > > > > > > > > > > Yes, kernel won't re-attempt to or just cannot recover. > > > > > > > > > > > > > > > > > > > > > > > > > How about introducing a new MF action result? Maybe MF_NONE? The message could > > > > > > > look something like: > > > > > > > > > > > > Adding MF_NONE sounds fine to me, as long as we correctly document its > > > > > > meaning, which can be subtle. > > > > > > > > > > Adding a new MF action result sounds good to me. But IMHO MF_NONE might not be that suitable > > > > > as kill_accessing_process might be called to kill proc in this case, so it's not "NONE". > > > > > > > > OK, would you like a separate MF action result for each case? Maybe > > > > MF_ALREADY_POISONED and MF_ALREADY_POISONED_KILLED? > > > > > > > > MF_ALREADY_POISONED can be the default and MF_ALREADY_POISONED_KILLED can be > > > > used when kill_accessing_process() returns -EHWPOISON. > > > > > > > > The log messages could look like... > > > > > > > > Memory failure: 0xXXXXXXXX: recovery action for already poisoned page: None > > > > and > > > > Memory failure: 0xXXXXXXXX: recovery action for already poisoned page: Process killed > > > > > > Agreed with Miaohe that "None" won't work. > > > > What action is M-F() taking to recover already poisoned pages that don't have > > MF_ACTION_REQUIRED set? > > The action taken toward poisoned page not under MF_ACTION_REQUIRED is > typically isolation, that is, remove the pte or mark the pte as poisoned > special swap entry, so that subsequent page fault is given a chance to > deliver a SIGBUS. That said, things might fail during the process, like > encountering GUP pinned THP page.> > > > "Process killed" sounds okay for MF_MSG_ALREADY_POISONED, but > > > we need to understand why "Failed" doesn't work for your usecase. > > > "Failed" means process is killed but page is not successfully isolated which > > > applies to MF_MSG_ALREADY_POISONED case as well. > > > > So that accessing process is killed. Why call action_result() with MF_FAILED? > > Doesn't that indicate we poisoned another page and the recovery attempt failed? > > What I recall is that, "recovery" is reserved for page that is clean, > isolated, and even by chance, unmapped. "failed" is reserved for page that > has been(or might not?) removed from the page table, page might be dirty, > certainly mapped, etc. A SIGBUS doesn't make recovery an automatic success. > > Others please correct me if I'm mistaken. Thank you very much for taking the time to explain everything. Thanks, Kyle Meyer