From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A7B1C4332F for ; Sun, 11 Dec 2022 21:52:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C9B4D8E0003; Sun, 11 Dec 2022 16:52:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C56388E0002; Sun, 11 Dec 2022 16:52:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AEB4C8E0003; Sun, 11 Dec 2022 16:52:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9C62D8E0002 for ; Sun, 11 Dec 2022 16:52:01 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 6F38412096B for ; Sun, 11 Dec 2022 21:52:01 +0000 (UTC) X-FDA: 80231373642.08.ED55E96 Received: from mail-ed1-f47.google.com (mail-ed1-f47.google.com [209.85.208.47]) by imf04.hostedemail.com (Postfix) with ESMTP id B74AA40004 for ; Sun, 11 Dec 2022 21:51:59 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Dv5yXSKe; spf=pass (imf04.hostedemail.com: domain of zokeefe@google.com designates 209.85.208.47 as permitted sender) smtp.mailfrom=zokeefe@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1670795519; a=rsa-sha256; cv=none; b=MttJrCnc15tPN8cQhsPMdS4dp2sgWJVkoQgAzL4ENyPWR9MWBj93GFd519OB4deFl5SzvY jr93Y5ZPdy1xWq5ZyQsJqlPAj9FenBkW51t41zXt1Sg9C8lUzXpw7lG8T0uU8x+yRYkyP/ KwtMPRGQG6dUZ2C75FZodFNOBPvj6tc= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Dv5yXSKe; spf=pass (imf04.hostedemail.com: domain of zokeefe@google.com designates 209.85.208.47 as permitted sender) smtp.mailfrom=zokeefe@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1670795519; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tAsBOke2BpP0AUx1Uo31pzqkvBQIOc+n/cFTJnRRyuQ=; b=XAq0tzFUyIaFscIjOUGItDBKKDpD35BspVOLUbgY+2kH3vCThVzesOCz+0DjncRJmVyB1A RJxMmEmPLo/C+3ezxkv100qIqoU/3B2uWA5ufEJPGE1deU56xGX6D1sTXdSgzNZHwSRhi3 8XZXMuTm7OdsbPTmI9+JGKDVa8ptNzk= Received: by mail-ed1-f47.google.com with SMTP id c66so10456943edf.5 for ; Sun, 11 Dec 2022 13:51:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=tAsBOke2BpP0AUx1Uo31pzqkvBQIOc+n/cFTJnRRyuQ=; b=Dv5yXSKeLNUyyGUFkrJkLTzYz94eTXhK+5oqRZKkpfmb7YaN8SuhThCSlF3uXvLnVc SOSfixiGs+btbhddbGbbsjSDuufY6X1bDP6/3b3P69hp6+uDClYilaZgovM+wbjhYJMJ fXhbmRe4IAMFJy+kRX6DTHF5pAubQLoK9Hrv2Ap9qmE+dyr2RzBRqRl0t4HxDz6patf7 17rLUoFVjSBz7K4dbwNjhM7yRW12wGG9hqtmqrvGkSvkV/FwZsN6+JaHjkiTxduNiL24 CvE9BTFwDFV5WwJ8CgAG5GaGCabjL+WfmhCyrO2bgerOOc70bQHmBgqQKTp8pVDpi8t6 vsQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tAsBOke2BpP0AUx1Uo31pzqkvBQIOc+n/cFTJnRRyuQ=; b=69bHbG363OMb0Qd6NNMW4Iaq/FbqDd9mUwxC/pws65xDkjaA4aM4yKdDjRXQDCq1xn 2yU+vjwINXi0FLDqiatw0o9bGxGck6JHH3yp02Ge99VGV/UrUyRtUJNB29x0tVXB5Ou7 QaJRKqU/+EWXBNLYS9wnl+93ep9WKIU+Kpi1NJBc5aL7t/Qklkc0xvIGAE/4fj5mCMia fk7d472xBGqEm5smgwE3MLTrKVc0roOT3zbc7Hh31NvQOynUxNhMxHqHDmo73IJnvydg bSRecENBoZmMwgnvceoCPoxlEWh3xu5FSl7irm8q7/O28ftjaDF5f0I6UqnVLA2t//2x DgRA== X-Gm-Message-State: ANoB5plrRQY2pULmevptvEeGAYToCcQJjW8RfkNnM4xct1vG1LD4pCUo G2/lFh7ogjLL0tByWF38bBny7ZRAbgLuPrNk1thB7w== X-Google-Smtp-Source: AA0mqf5yYWPfginE8ZXp76TtLxpZmPM/eTxW9uT+yBWXgTPTu9F1TwnKb7sgg7MwsT1SVXtrYK9VMkejB5BLWeThNfE= X-Received: by 2002:aa7:cd8d:0:b0:463:19ca:a573 with SMTP id x13-20020aa7cd8d000000b0046319caa573mr84471638edv.31.1670795517958; Sun, 11 Dec 2022 13:51:57 -0800 (PST) MIME-Version: 1.0 References: <20221021223300.3675201-1-zokeefe@google.com> <20221021223300.3675201-5-zokeefe@google.com> <374b1dcd-6a2c-a452-9c1b-9f5945df493b@gmail.com> In-Reply-To: <374b1dcd-6a2c-a452-9c1b-9f5945df493b@gmail.com> From: "Zach O'Keefe" Date: Sun, 11 Dec 2022 13:51:21 -0800 Message-ID: Subject: Re: [PATCH man-pages v3 4/4] madvise.2: add documentation for MADV_COLLAPSE To: Alejandro Colomar Cc: Yang Shi , linux-mm@kvack.org, linux-man@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: B74AA40004 X-Stat-Signature: 1wgaez8w7u9p479w5imp59fgbqcb6gg9 X-HE-Tag: 1670795519-756611 X-HE-Meta: U2FsdGVkX1/DtZ97PZ3l3MLUoE2BzmaksmcLGtMYa6X3zfdD0pOpHuaaq5PpYGqrV8Yn0J8tI3+ftgrundUemxsFirD+CFMz7/kgWwM2pyRdgHZkzlsV7mSoSb1JWtG+buJ0mDgrBPtvFWk8efqrwxXbfEVgHRg84tYkPkIO+H2MWPo+7vSD43xRNnKEAi2qbO+4BxVfBuoPzeG2y/eDHHe1OwO2U4xArGICKyXPS5UY30TUVpNWo2hHeVlFbTo4rLjqCVFgEx0m5N33WjpJooxXGXkR2/hieIdyYVkVdvtRHDo5JQ2G1gCPf3ONrRTyjodmBbbXwopbBuiLMMgy79kdACsRfcVt/fi+5od6bx1swulSOxZcxu5HuzeDaOxXZEu26T648OBTDqAx1lbFttSufS1DhJnhV1HJRiHfnFuPdLPXxY/2ahDI7d9Q/QuGaIqq+mCbkwnP751cpDKCv6Rn8/qhqOIplygVwM9i9d6ltut3dNKT9xjgf3cXmO6e64F3TQGcAAxfMAOlN1dgkl7eg68zsuVLticAxV4z/5UG2WMppqz1ViQV035a4x0fKq/JU9/uPtu4oi8NE4HrBe6J/D+Uns5JMs96BEC7q2dtDOtUkCXd8vyb7WECNvAZorvPKv4N702PwxlR+GEizRahF9XtkmcE+P/jHYvWeZaY52oYs3xCKvnRXFmyW7BKudEHSUaSgLk6xbtJUoJytV7K1QruPWLiQ2X12dhp7USSiaPgaAEjyLb86bycny+CUDuumpNF3TsTuu87waqPY+KNxJ1kZADhEwEqt0KyE9Kr4dMGFD96qAZL4i8bfUk6g1iROC2ObrViVIXgzf1p459pt1PFD7bpiCYo5ZcAyCG1OBLXpBtTg7n/LrnHAMW28Odg6qtE5IQX4eHdgU8QfB5TqkHFiFwDiugxKOSKTBm5GA2YwXxdNFsp8PF9GeuovgoGvj8or8jfsXgyc+T tfBFBhKa KvkzxFuoND6uyo5KXzz6vxB1hayWsr7hWGnmyFMMFXGCJ0v+G93SPvnSZpPpsZ4H2qQshFjX+u1ynwtVrqXag0WMZ4Ezy3rk1e+j7n7OMGLgprlrpG3pm23X/H9dip25JXwCx X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sun, Dec 11, 2022 at 9:59 AM Alejandro Colomar wrote: > > Hi Zach, Hey Alex, > On 10/22/22 00:33, Zach OKeefe wrote: > > From: Zach O'Keefe > > > > Linux 6.1 introduced MADV_COLLAPSE in upstream commit 7d8faaf15545 > > ("mm/madvise: introduce MADV_COLLAPSE sync hugepage collapse") and > > upstream commit 34488399fa08 ("mm/madvise: add file and shmem support t= o > > MADV_COLLAPSE"). Update the man-pages for madvise(2) and > > process_madvise(2). > > > > Link: https://lore.kernel.org/linux-mm/20220922224046.1143204-1-zokeefe= @google.com/ > > Link: https://lore.kernel.org/linux-mm/20220706235936.2197195-1-zokeefe= @google.com/ > > Signed-off-by: Zach O'Keefe > > Please see a few comments below. > Thanks for the mail. So, this patch was taken as commit b106cd5bf ("madvise.2: add documentation for MADV_COLLAPSE"). Some of your comments below were applied (I think, by you) as fixes pre-commit. However, there are some new comments (or ones that address the same lines, but in different ways). Is this mail to log ~ what changes were done, or is there anything actionable here on my side? Best, Zach Thanks for this. > Cheers, > > Alex > > > --- > > man2/madvise.2 | 90 +++++++++++++++++++++++++++++++++++++++++= - > > man2/process_madvise.2 | 10 +++++ > > 2 files changed, 98 insertions(+), 2 deletions(-) > > > > diff --git a/man2/madvise.2 b/man2/madvise.2 > > index df3413cc8..b03fc731d 100644 > > --- a/man2/madvise.2 > > +++ b/man2/madvise.2 > > @@ -385,9 +385,10 @@ set (see > > .BR prctl (2) ). > > .IP > > The > > -.B MADV_HUGEPAGE > > +.BR MADV_HUGEPAGE , > > +.BR MADV_NOHUGEPAGE , > > and > > -.B MADV_NOHUGEPAGE > > +.B MADV_COLLAPSE > > operations are available only if the kernel was configured with > > .B CONFIG_TRANSPARENT_HUGEPAGE > > and file/shmem memory is only supported if the kernel was configured = with > > @@ -400,6 +401,81 @@ and > > .I length > > will not be backed by transparent hugepages. > > .TP > > +.BR MADV_COLLAPSE " (since Linux 6.1)" > > +.\" commit 7d8faaf155454f8798ec56404faca29a82689c77 > > +.\" commit 34488399fa08faaf664743fa54b271eb6f9e1321 > > +Perform a best-effort synchronous collapse of the native pages mapped = by the > > Please use semantic line breaks. In this case, I'd break after "pages". > > man-pages(7): > Use semantic newlines > In the source of a manual page, new sentences should be started = on new > lines, long sentences should be split into lines at clause breaks= (com=E2=80=90 > mas, semicolons, colons, and so on), and long clauses should be = split > at phrase boundaries. This convention, sometimes known as "se= mantic > newlines", makes it easier to see the effect of patches, which = often > operate at the level of individual sentences, clauses, or phrases= . > > > +memory range into Transparent Huge Pages (THPs). > > +.B MADV_COLLAPSE > > +operates on the current state of memory of the calling process and mak= es no > > Here I'd break after "and". > > > +persistent changes or guarantees on how pages will be mapped, > > +constructed, > > +or faulted in the future. > > +.IP > > +.B MADV_COLLAPSE > > +supports private anonymous pages (see > > +.BR mmap (2)), > > +shmem pages, > > +and file-backed pages. > > +See > > +.B MADV_HUGEPAGE > > +for general information on memory requirements for THP. > > +If the range provided spans multiple VMAs, > > +the semantics of the collapse over each VMA is independent from the ot= hers. > > +If collapse of a given huge page-aligned/sized region fails, > > +the operation may continue to attempt collapsing the remainder of the > > Break after "collapsing". > > > +specified memory. > > +.B MADV_COLLAPSE > > +will automatically clamp the provided range to be hugepage-aligned. > > +.IP > > +All non-resident pages covered by the range will first be > > Break after "range". > > > +swapped/faulted-in, > > +before being copied onto a freshly allocated hugepage. > > +If the native pages compose the same PTE-mapped hugepage, > > +and are suitably aligned, > > +allocation of a new hugepage may be elided and collapse may happen > > Break before or after "and". > > > +in-place. > > +Unmapped pages will have their data directly initialized to 0 in the n= ew > > Break after "0". > > > +hugepage. > > +However, > > +for every eligible hugepage-aligned/sized region to be collapsed, > > +at least one page must currently be backed by physical memory. > > +.IP > > +.BR MADV_COLLAPSE > > s/BR/B/ > > > +is independent of any sysfs > > +(see > > +.BR sysfs (5)) > > +setting under > > +.IR /sys/kernel/mm/transparent_hugepage , > > +both in terms of determining THP eligibility, > > +and allocation semantics. > > +See Linux kernel source file > > +.I Documentation/admin\-guide/mm/transhuge.rst > > +for more information. > > +.BR MADV_COLLAPSE > > s/BR/B/ > > > +also ignores > > +.B huge=3D > > +tmpfs mount when operating on tmpfs files. > > +Allocation for the new hugepage may enter direct reclaim and/or compac= tion, > > +regardless of VMA flags > > +(though > > +.BR VM_NOHUGEPAGE > > s/BR/B/ > > > +is still respected). > > +.IP > > +When the system has multiple NUMA nodes, > > +the hugepage will be allocated from the node providing the most native > > Break after "from". > > > +pages. > > +.IP > > +If all hugepage-sized/aligned regions covered by the provided range we= re > > Prefer English rather than "/". > > > +either successfully collapsed, > > +or were already PMD-mapped THPs, > > +this operation will be deemed successful. > > +Note that this doesn=E2=80=99t guarantee anything about other possible= mappings of > > Break after "about". > > > +the memory. > > +Also note that many failures might have occurred since the operation m= ay > > +continue to collapse in the event collapse of a single hugepage-sized/= aligned > > Add some omitted "that" or something that will help readability to > non-native-English readers. > > And break at a better place. > > > +region fails. > > +.TP > > .BR MADV_DONTDUMP " (since Linux 3.4)" > > .\" commit 909af768e88867016f427264ae39d27a57b6a8ed > > .\" commit accb61fe7bb0f5c2a4102239e4981650f9048519 > > @@ -619,6 +695,11 @@ A kernel resource was temporarily unavailable. > > .B EBADF > > The map exists, but the area maps something that isn't a file. > > .TP > > +.B EBUSY > > +(for > > +.BR MADV_COLLAPSE ) > > +Could not charge hugepage to cgroup: cgroup limit exceeded. > > +.TP > > .B EFAULT > > .I advice > > is > > @@ -716,6 +797,11 @@ maximum resident set size. > > Not enough memory: paging in failed. > > .TP > > .B ENOMEM > > +(for > > +.BR MADV_COLLAPSE ) > > +Not enough memory: could not allocate hugepage. > > +.TP > > +.B ENOMEM > > Addresses in the specified range are not currently > > mapped, or are outside the address space of the process. > > .TP > > diff --git a/man2/process_madvise.2 b/man2/process_madvise.2 > > index 44d3b94e8..8b0ddccdd 100644 > > --- a/man2/process_madvise.2 > > +++ b/man2/process_madvise.2 > > @@ -73,6 +73,10 @@ argument is one of the following values: > > See > > .BR madvise (2). > > .TP > > +.B MADV_COLLAPSE > > +See > > +.BR madvise (2). > > +.TP > > .B MADV_PAGEOUT > > See > > .BR madvise (2). > > @@ -173,6 +177,12 @@ The caller does not have permission to access the = address space of the process > > .TP > > .B ESRCH > > The target process does not exist (i.e., it has terminated and been w= aited on). > > +.PP > > +See > > +.BR madvise (2) > > +for > > +.IR advice -specific > > +errors. > > .SH VERSIONS > > This system call first appeared in Linux 5.10. > > .\" commit ecb8ac8b1f146915aa6b96449b66dd48984caacc > > -- >