From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB190C27C6E for ; Fri, 14 Jun 2024 23:22:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C99716B015F; Fri, 14 Jun 2024 19:16:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BFBE06B018D; Fri, 14 Jun 2024 19:16:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B5D366B015F; Fri, 14 Jun 2024 19:16:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 06F766B0292 for ; Fri, 14 Jun 2024 19:16:12 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 97FF6C09B9 for ; Fri, 14 Jun 2024 23:16:12 +0000 (UTC) X-FDA: 82231054584.13.620FFCF Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) by imf18.hostedemail.com (Postfix) with ESMTP id C086E1C000A for ; Fri, 14 Jun 2024 23:16:10 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=MCdV1aRs; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf18.hostedemail.com: domain of jiaqiyan@google.com designates 209.85.221.54 as permitted sender) smtp.mailfrom=jiaqiyan@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718406968; a=rsa-sha256; cv=none; b=ZeZUhQoB974YusoogjaO+vFEfVF7BC93YHjfF1DlS5NCGLPVPe6vtF3FV+DH0gnUVnVqRZ xQKuJ/O/rQ0dHI/6xnboXsH6rxqnwLFMgZZbt3BREfaEZeuRcjpVXKjSB2xxIJImqlCWRp InYDjPA/yprZmnS7TBGVsejUE+rLt6U= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=MCdV1aRs; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf18.hostedemail.com: domain of jiaqiyan@google.com designates 209.85.221.54 as permitted sender) smtp.mailfrom=jiaqiyan@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718406968; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LahH6YvliG7eYlUbvikNbUEjbap7UqWjJYCEVPQs1io=; b=zj5nwyJ4pPixGsG4pTH35gAOJw/c+DLap0S/rF2eeYrXw9Nouev4fagu1WoFBetYXyuCmB BdD9PtkSz8xumGO3wxf2X3UQX3muUbqGAdg5sp7J3PBrrql85H7FpsboFy4LyFLU1Y06g3 WL8aUsreQjlpuqNYYFwGdkVOhUSE4ps= Received: by mail-wr1-f54.google.com with SMTP id ffacd0b85a97d-35f275c7286so2418381f8f.2 for ; Fri, 14 Jun 2024 16:16:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1718406969; x=1719011769; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=LahH6YvliG7eYlUbvikNbUEjbap7UqWjJYCEVPQs1io=; b=MCdV1aRs27mCWJPt0kFKO1UC1VTOrscYslVkisFCNc1fD9BCgxqCkvzrCtI2EGiwFt ou/cRDNpcZMnLmLJdy53RP7Qdv2RaMge/A22YAS+zKnQPfmQ3esN6zsSYhOiCT1kCFR1 DcxAKDW3nAwBGXBaGXnFJ7+NJGSjaH8I+gNHQVJb03rq2qhwkvPTPxAA2VieWR28he77 hq8GN2eL3Xf0ttr3LqtAuSF07lszSd1Ja88bMZjdCrDtl+5rpzRnWTbheItIgHQKSdiD MzaLrD4eFPXroYZAQz31DhSMtX5y1+XbrGaJMS2QaaWqwarH829n5rAAndPeqtNIyTIa y4KA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718406969; x=1719011769; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LahH6YvliG7eYlUbvikNbUEjbap7UqWjJYCEVPQs1io=; b=uliA/XD8nMkYXp8j9GMlx5hUfxFOI9El8ifqFz3UXJQT51pq2YQptvFptTKHZovT14 voSGqKH2VXcX2E0YgtxUOdWbG+uFaRUnAXil13k5VhbKT4472yHEZVdHdWEeyXGlEko6 /NNUCKcdBHDTKvdsdXcmeFadiZrZC3dgO/H5S9ZDROdSZ2HH+anB7lEB3+sQb5dE4WfJ 0LbByor9LLCvz/hkPTpnDwuYOgFRSlqVmiuUn4KZn+SFJOZ79JyMC9phD92n4NvkFYdG vC2HPJxVf9oB7S+sNlyLNfm1a2kPf917LMJ/TP1CTlQZ9mxjSI92wFGkl5izolE9FZD1 Urpg== X-Forwarded-Encrypted: i=1; AJvYcCUWruS1PKBvMdEl9qG16JzIPeb/d9JkzpRqm15BJVUH2wsFbwxocBxph+n8j35VgVTB6pHQjFbOHBvQNw6OIKnVb10= X-Gm-Message-State: AOJu0YzWg5NXNwOGltplt2oRXpp/WlRlwvIZFYHryZ/pAjNVWJZFy5ED JOLG6wAgyld14G+9bpRMnK63wGjM9c8sA4uIOSuwuC7qopbGHcGD6/o6+/9NZ18XILz/exu8vXk aAAZzQEVcWY1P+aj4FYhwnZgWdM7E4Z6v3fMe X-Google-Smtp-Source: AGHT+IGSGeWdWrHeyLBhi8ahJUOIBOV4p81XB0K8UtbxXZp0W1Hbsh5/A+lYxE2gXbWBKN4sXqq0XRi16MlYhpjbjow= X-Received: by 2002:a05:6000:4583:b0:360:8768:8dda with SMTP id ffacd0b85a97d-36087688e20mr695294f8f.7.1718406968805; Fri, 14 Jun 2024 16:16:08 -0700 (PDT) MIME-Version: 1.0 References: <20240611215544.2105970-1-jiaqiyan@google.com> <20240611215544.2105970-4-jiaqiyan@google.com> <9461874d-e2d6-25fc-813c-9c9bb0ad1aec@google.com> In-Reply-To: <9461874d-e2d6-25fc-813c-9c9bb0ad1aec@google.com> From: Jiaqi Yan Date: Fri, 14 Jun 2024 16:15:56 -0700 Message-ID: Subject: Re: [PATCH v2 3/3] docs: mm: add enable_soft_offline sysctl To: David Rientjes Cc: nao.horiguchi@gmail.com, linmiaohe@huawei.com, jane.chu@oracle.com, muchun.song@linux.dev, akpm@linux-foundation.org, shuah@kernel.org, corbet@lwn.net, osalvador@suse.de, duenwen@google.com, fvdl@google.com, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-doc@vger.kernel.org, Lance Yang Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: C086E1C000A X-Stat-Signature: wuacn1zy8k34u1hgw4mg4wxo55edt856 X-Rspam-User: X-HE-Tag: 1718406970-563528 X-HE-Meta: U2FsdGVkX18tVgbfXZQ4ucWp9Yln8Z8hwiORrvo7wF2SC+UP2kPaH6G58n/GhdE8FxG+tRNysdI8CYX/sRNsq5KCz/qSpft4gepacoENfy2i7sqMYb0WOAHWV44DF2mHBeNCvCkMr4mwMWH8mSzh63Uku/Q70LPe3KyfxP+II0UnldAnMYXJN59CFXS8A8Grr9Bwewv8Ugsal3xUyHKYuWZETvPbwsjFNCPMmhTH9OOoig13B7PAb1pSDCswswErxsNuFC7pVjzx+CRW0l4OWQVUa0EBEB70RbqGQo6JvFqz+CMSrDHslxj+ZnkUqG6gbWXP2KNlRK3rA5V3vCae+1qNnQOp+ad12J/pK2L9GcN7s4OaxnGSmaVs0mc26lijXrSzwGh4hqzyODyhx8BOWMSx7oihsnIpKoWmUNzt2XEhQ6aRQ4HpFO1bKBLfIo9VoIMslq3gaG/BK9ggegLtx1sWdcr0Ky+uGvQEvTq5NPDqu9HpnQkQ13cGofgyr4bXycoysdQpZbTr40Vtwy/aDcklx0LP/IwToS5DuBgvRpgQp9g0GsbOGCMRB0xmo8WjETZ1+ZVQaOucqWDWjmY7EgtdkJeOZMAMKsGdH3+jMT4lFTedc0SM/JdKcTjiYuijEk6NeXINS4xLxw0tmk0DEUCUfE05Bh4RsA8Ph4aqPL5Gw1Fw3NRbRmvTL0GaHum1scKSDfOm6YqTDABI27wx0Z0JwC4VqZCBABAC8M2owcmww6F0oIbJRO2DsHDQnScl/0YNNlT2GcwnzSETYKDRYoAWR58yuzdNYiVom1ZVOrkmdqPqZ7SBvbswk5B8bCeCrzMDfUE+32agNdK28QXfQmqzy8IDFiOaR6vdwsb2n6zKuviqx1aevsG4F/adhgaIy8C+DrgIppSbKmGMQepfgWfiQDw8LyRBeCNkM1t/So4LFT4JB4mfDBE0DaLUlzxkdXZi//VREHgP39wkl4h N+Q+S/eT kX3cvFrChx+xyUDiKbHHnfowXa0edr6W1S3Yy7jdeR7AHuij2yUvHbGl09pSAFapgV0bqTS8K5wOKXx1YcHrcVuRDKwyNthpXkE5N84t7znPkOGbUo+hxa8LCPAw3W5ay9xw6cAN6Bd0d2Xhc++p+bT+VhAP7mQ6K3oE32oDKivLWJCBReDbtcDlC6wUJRb4oEGM0i5k8Su8ygXCWt7foWz0iJJTeSGHOByQuihKGTgsp6LfsBmUOnBI92hyMNhNSJddkj/oKt+Uv7VIQ60nGPy92zDFiTHVRkuiiXHTJSeYFu8COf+ISd0pzyg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Thanks for your questions, David! On Tue, Jun 11, 2024 at 5:25=E2=80=AFPM David Rientjes wrote: > > On Tue, 11 Jun 2024, Jiaqi Yan wrote: > > > @@ -267,6 +268,20 @@ used:: > > These are informational only. They do not mean that anything is wrong > > with your system. To disable them, echo 4 (bit 2) into drop_caches. > > > > +enable_soft_offline > > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > +Control whether to soft offline memory pages that have (excessive) cor= rectable > > +memory errors. It is your call to choose between reliability (stay aw= ay from > > +fragile physical memory) vs performance (brought by HugeTLB or transpa= rent > > +hugepages). > > + > > Could you expand upon the relevance of HugeTLB or THP in this > documentation? I understand the need in some cases to soft offline memor= y > after a number of correctable memory errors, but it's not clear how the > performance implications plays into this. The paragraph below goes into = a To be accurate, I should say soft offlining transparent hugepage impacts performance, and soft offlining hugetlb hugepage impacts capacity. It may be clearer to first explain soft-offline's behaviors and implications, so that user knows what is the cost of soft-offline, then talks about the behavior of enable_soft_offline: Correctable memory errors are very common on servers. Soft-offline is ker= nel's handling for memory pages having (excessive) corrected memory errors. For different types of page, soft-offline has different behaviors / costs= . - For a raw error page, soft-offline migrates the in-use page's content t= o a new raw page. - For a page that is part of a transparent hugepage, soft-offline splits = the transparent hugepage into raw pages, then migrates only the raw error p= age. As a result, user is transparently backed by 1 less hugepage, impacting memory access performance. - For a page that is part of a HugeTLB hugepage, soft-offline first migra= tes the entire HugeTLB hugepage, during which a free hugepage will be consu= med as migration target. Then the original hugepage is dissolved into raw pages without compensation, reducing the capacity of the HugeTLB pool b= y 1. It is user's call to choose between reliability (staying away from fragil= e physical memory) vs performance / capacity implications in transparent an= d HugeTLB cases. > difference in the splitting behavior, are hugepage users the only ones > that should be concerned with this? If the cost of migrating a raw page is negligible, then yes, only hugepage users should be concerned and think about should they disable soft offline. > > > +When setting to 1, kernel attempts to soft offline the page when it th= inks > > +needed. For in-use page, page content will be migrated to a new page.= If > > +the oringinal hugepage is a HugeTLB hugepage, regardless of in-use or = free, > > s/oringinal/original/ To fix in v3. > > > +it will be dissolved into raw pages, and the capacity of the HugeTLB p= ool > > +will reduce by 1. If the original hugepage is a transparent hugepage,= it > > +will be split into raw pages. When setting to 0, kernel won't attempt= to > > +soft offline the page. Its default value is 1. > > > > This behavior is the same for all architectures? > Yes, enable_soft_offline has the same behavior for all architectures, and default=3D1. It may be worth mentioning that setting enable_soft_offline to 0 means: - If RAS Correctable Errors Collector is running, its request to soft offline pages will be ignored. - On ARM, the request to soft offline pages from GHES driver will be ignore= d. - On PARISC, the request to soft offline pages from Page Deallocation Table will be ignored. I can add these clarifications in v3 if they are valuable.