From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49735E9413A for ; Fri, 6 Oct 2023 22:28:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 81DB880010; Fri, 6 Oct 2023 18:28:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7CD8A80008; Fri, 6 Oct 2023 18:28:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6962E80010; Fri, 6 Oct 2023 18:28:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 5B38F80008 for ; Fri, 6 Oct 2023 18:28:41 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 31AE6A0221 for ; Fri, 6 Oct 2023 22:28:41 +0000 (UTC) X-FDA: 81316477242.25.2AC87A8 Received: from mail-qt1-f172.google.com (mail-qt1-f172.google.com [209.85.160.172]) by imf05.hostedemail.com (Postfix) with ESMTP id 6F5F2100002 for ; Fri, 6 Oct 2023 22:28:39 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=AlymEuHE; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf05.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.172 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696631319; a=rsa-sha256; cv=none; b=nLew1jFHtgoREy9dJEl8x3PzUHmqG0LDuvNOWWnGH5hqlaR0mUgNFXjqOxy9NHUEA2cW6i d6IUXnO9s+AbbBbtykeE+Q4vJ262z3coBy/1mFPqZH3DYFILYJnn4qK8i/A0/9bSrch8pz 2dqDlWAt7xCaOQOTP0gjJznW5abruA0= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=AlymEuHE; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf05.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.172 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696631319; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vNXo4Ck+cb5kMjLZ1w1gFPLnQ9WFu3luQ6DdPjjydeM=; b=mHmTXk8XqORSrLwQhPhHaLhHJMlV1smvdsLwI3pAJCvPSatxBuPxIyb7mchUDvTxCi6l3p bd8qn6KNf644TTYODpTz+Q/x+12dxmu0Ty+ODOGalE8VXmg7PJ5DQvXFhHREEzCpksDC1v SCZH2IyCdptn1+A1Y8RG10v/zpvC6t8= Received: by mail-qt1-f172.google.com with SMTP id d75a77b69052e-419886c7474so50431cf.1 for ; Fri, 06 Oct 2023 15:28:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1696631318; x=1697236118; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=vNXo4Ck+cb5kMjLZ1w1gFPLnQ9WFu3luQ6DdPjjydeM=; b=AlymEuHEn1sM+B1B9JHv/m431TmIYaFyKFJ4vxyIn2a9z/tI4N13tNMSHX7NxZJ/Ow DmKj1iijJrQzCe5xi0YvDvP08XbblcP4iJdHgQau+nnCHWgPYkmiFwZZ73peCeldcbUv r2kx4igiiIxPf1XjiDqKkvt0+Fg0r0fgGTu2Qgnz5CuPF2dtZiJDJjnPYNdi/n1tDevq 6K6r54wiZCReg1zcB37Qkl7TzJE+5v34iM6g/JNnIiLg+YNQXnQ7+p9jUwLkcXmuAdhJ FV3g3xNZWIkhYCeoXXMYeFoGrJS7OApDUqxN0Y4miEAfcXwYwFpojCyxXHg1i7yKmlTe pF/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696631318; x=1697236118; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vNXo4Ck+cb5kMjLZ1w1gFPLnQ9WFu3luQ6DdPjjydeM=; b=luGf/Vfcu43Pr5No9Eeb6Xpzuc8GcDFf4MdDfEPyDEpZ8qnrPs6HHvr3z/XDcELQAe pv5ab4lRSSSg85MrRsrUSTAw0J5q7jkztQH1IYWSL2prtI7WueDErZv3jhP61z1Orv2I ThYl8HoiY1Eq4XjoMVJugY9A5QtCWDH1McnGdaUL7YtE3rGxEvWckLI1OPUY0lCky+l6 XYKPSWZt6RHgDMOR5T9CwiF5rFAPIBcrIe5gXoi21AAxQItUAEZ5g3xp7aI12TBIpUeB gSpNvvRIu68LPk5nWMtdhmf19OVn+nv2CeDKUlBKhOasHMuenZ3cP09DaC/FSPzX1L0v QECg== X-Gm-Message-State: AOJu0YxqUUYWFBaALWXaZOSjKHzR0G09BRbzeOjf2unsulzUTE95EdRr +E7YeM1seRP+Zr1bRd4q+RoGayXN6EQ7Y583juPMgA== X-Google-Smtp-Source: AGHT+IFttWebdUSxJpbaQyrtnuXc6BVLXJiiyF08oBFcvtqaVZRda+96rYta3bQaMl8pMHaJou6x8IrBmegXGYQj1sg= X-Received: by 2002:ac8:5981:0:b0:3e0:c2dd:fd29 with SMTP id e1-20020ac85981000000b003e0c2ddfd29mr513044qte.4.1696631318364; Fri, 06 Oct 2023 15:28:38 -0700 (PDT) MIME-Version: 1.0 References: <20230929114421.3761121-1-ryan.roberts@arm.com> <20230929114421.3761121-7-ryan.roberts@arm.com> <2f64809e-0d0d-cc61-71ac-8d9b072efc3a@redhat.com> In-Reply-To: <2f64809e-0d0d-cc61-71ac-8d9b072efc3a@redhat.com> From: Yu Zhao Date: Fri, 6 Oct 2023 16:28:02 -0600 Message-ID: Subject: Re: [PATCH v6 6/9] mm: thp: Add "recommend" option for anon_orders To: David Hildenbrand , Ryan Roberts Cc: Andrew Morton , Matthew Wilcox , Yin Fengwei , Catalin Marinas , Anshuman Khandual , Yang Shi , "Huang, Ying" , Zi Yan , Luis Chamberlain , Itaru Kitayama , "Kirill A. Shutemov" , John Hubbard , David Rientjes , Vlastimil Babka , Hugh Dickins , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 6F5F2100002 X-Stat-Signature: urttg11fn4irar67qmmrpe9ty6zmtrix X-HE-Tag: 1696631319-518206 X-HE-Meta: U2FsdGVkX19MUo3DhaJ6bUyUgE9WzMrQLpj+/+fpYUcaw7VqNSAH9n6ASwEQCkzFihrAvPcu0KdXi4O7qwh/OpDSzYsAeTXRSM+xBDcVqJWfaqkQnpnkIxrcPfQVdRx74is87sHXpajPOyb3RxXlklJTRgI7dB3s/NtzLxPxYbGiqvNCW8wRGKLnSXvluCb7GhRIavkiD8XNJv6FZVAYe1FjU8CHB08abIKsQ/uB+5WO5wKNvWy5+lLegnVziGoI7snXgd+KRjicrs0Gv28058mZodPY83d5ErxrXljXTvayUPWrGv4m9+C3TfAN2JPB4wKh7pAysfKgJ0ppSP8NtcbOt5pNG4/J9G2/DLJc/3W5eCEWFCHP1u5PJrAkcZkPJCNQYNTVjfsDvH7lU1c+Jm0bdFnpI7kY1kjV5hVujwOE4DOoNYIet1yHfBDctZcNEr5/vwe/BknafqgqQYoDEFbeANjVDgANltjFp6MBXme7757oL2opKtu48Xxp8MdnNZezAnY2VFgWCqaLmxy6lFZSrv+dwjhqw4ixbiNyQk8RxpKDDeHVL3UM2XoORPuQqxQK44ABtnr2s44g6QzehDLAWX8RwyzermRGghUl6WmTxsxQNY1Unelm/DkBUm9CFSM0qTFPTHb2rn7tE3nYJ2uvu5uC+kJG08jRFI6lwIfd1kwOfd/uVqLq5Xs7K01fyZudO5Yl3bR7Y1uTjWzKaxo2h8G0wEXPrDk0n4WZ9UhI1bzCK4nYgjFXHreiMeNy3SiLZuYfkFr86mA+0Bf5LunN5EhFISvgriGv4NHwYB2DBsX0+2AKlst+C2lAgItz8xl/zjP5mKxugSvND+ZqEOMa/R3HcMxZONpHz5yMwWKYGHEDWQHB/Tey+FLyF0P/K0O8LyEEMXRNtUIa8wh4OYnfV1fI5AAQehhKJ3aLDIzMBZXafHbuTz/AO1q2pS3xb3X5AkTxYH7PnOEY7XT ir/LO1+2 GqZul3Dw/ww6tWzKbgLvJCOT0OtfEZ7BXyEV5Y70DTH0yGhk2tqBm+377jdLF/IpQWlnJn+DMbaRS1I2ZHXHZn8RQHfqoDqxfOeq7oj1yCQtArFsjoxfJtnjMgU5iJSn13Xmw7j6LfX2G/pqw8vMixI+ljAg4TZCuXfoe7h8SIzocrOE92FirMYcU0BtY/0n0JYFPy+eyTm+pDcNBI0SU1t6NeMKmv22pKWT0j2FmaWBX6L52koxTlhcfSooxDJyoqZgMC3Cun0l67NNK+ZV9L1SelOhlUxRYeXlGewrX6c3OZUY8bKqNzq4MuzHRxCzI8l7sEYNfKR8g0X0SVZ9Ssc8kHdz+FYRB4ZGWSLcoYUl7QzjsCjsGJhmgI8VR2tfhVXQ11Hb80QRrp2uVFGKM4Fg5CEW1d+snH/GNNV++jfHhy8+JlsedYkDOK+Cq4hoLJx3Gil83ttm8I8FSJAouQtXMyYDnUAudShifEO2pu+3PibvtaEveS/vgbamgiRJ0sNIn X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Oct 6, 2023 at 2:08=E2=80=AFPM David Hildenbrand = wrote: > > On 29.09.23 13:44, Ryan Roberts wrote: > > In addition to passing a bitfield of folio orders to enable for THP, > > allow the string "recommend" to be written, which has the effect of > > causing the system to enable the orders preferred by the architecture > > and by the mm. The user can see what these orders are by subsequently > > reading back the file. > > > > Note that these recommended orders are expected to be static for a give= n > > boot of the system, and so the keyword "auto" was deliberately not used= , > > as I want to reserve it for a possible future use where the "best" orde= r > > is chosen more dynamically at runtime. > > > > Recommended orders are determined as follows: > > - PMD_ORDER: The traditional THP size > > - arch_wants_pte_order() if implemented by the arch > > - PAGE_ALLOC_COSTLY_ORDER: The largest order kept on per-cpu free li= st > > > > arch_wants_pte_order() can be overridden by the architecture if desired= . > > Some architectures (e.g. arm64) can coalsece TLB entries if a contiguou= s > > set of ptes map physically contigious, naturally aligned memory, so thi= s > > mechanism allows the architecture to optimize as required. > > > > Here we add the default implementation of arch_wants_pte_order(), used > > when the architecture does not define it, which returns -1, implying > > that the HW has no preference. > > > > Signed-off-by: Ryan Roberts > > --- > > Documentation/admin-guide/mm/transhuge.rst | 4 ++++ > > include/linux/pgtable.h | 13 +++++++++++++ > > mm/huge_memory.c | 14 +++++++++++--- > > 3 files changed, 28 insertions(+), 3 deletions(-) > > > > diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation= /admin-guide/mm/transhuge.rst > > index 732c3b2f4ba8..d6363d4efa3a 100644 > > --- a/Documentation/admin-guide/mm/transhuge.rst > > +++ b/Documentation/admin-guide/mm/transhuge.rst > > @@ -187,6 +187,10 @@ pages (=3D16K if the page size is 4K). The example= above enables order-9 > > By enabling multiple orders, allocation of each order will be > > attempted, highest to lowest, until a successful allocation is made. > > If the PMD-order is unset, then no PMD-sized THPs will be allocated. > > +It is also possible to enable the recommended set of orders, which > > +will be optimized for the architecture and mm:: > > + > > + echo recommend >/sys/kernel/mm/transparent_hugepage/anon_orders > > > > The kernel will ignore any orders that it does not support so read th= e > > file back to determine which orders are enabled:: > > diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h > > index af7639c3b0a3..0e110ce57cc3 100644 > > --- a/include/linux/pgtable.h > > +++ b/include/linux/pgtable.h > > @@ -393,6 +393,19 @@ static inline void arch_check_zapped_pmd(struct vm= _area_struct *vma, > > } > > #endif > > > > +#ifndef arch_wants_pte_order > > +/* > > + * Returns preferred folio order for pte-mapped memory. Must be in ran= ge [0, > > + * PMD_ORDER) and must not be order-1 since THP requires large folios = to be at > > + * least order-2. Negative value implies that the HW has no preference= and mm > > + * will choose it's own default order. > > + */ > > +static inline int arch_wants_pte_order(void) > > +{ > > + return -1; > > +} > > +#endif > > + > > #ifndef __HAVE_ARCH_PTEP_GET_AND_CLEAR > > static inline pte_t ptep_get_and_clear(struct mm_struct *mm, > > unsigned long address, > > diff --git a/mm/huge_memory.c b/mm/huge_memory.c > > index bcecce769017..e2e2d3906a21 100644 > > --- a/mm/huge_memory.c > > +++ b/mm/huge_memory.c > > @@ -464,10 +464,18 @@ static ssize_t anon_orders_store(struct kobject *= kobj, > > int err; > > int ret =3D count; > > unsigned int orders; > > + int arch; > > > > - err =3D kstrtouint(buf, 0, &orders); > > - if (err) > > - ret =3D -EINVAL; > > + if (sysfs_streq(buf, "recommend")) { > > + arch =3D max(arch_wants_pte_order(), PAGE_ALLOC_COSTLY_OR= DER); > > + orders =3D BIT(arch); > > + orders |=3D BIT(PAGE_ALLOC_COSTLY_ORDER); > > + orders |=3D BIT(PMD_ORDER); > > + } else { > > + err =3D kstrtouint(buf, 0, &orders); > > + if (err) > > + ret =3D -EINVAL; > > + } > > > > if (ret > 0) { > > orders &=3D THP_ORDERS_ALL_ANON; > > :/ don't really like that. Regarding my proposal, one could have > something like that in an "auto" setting for the "enabled" value, or a > "recommended" setting [not sure]. Me either. Again this is something I call random -- we only discussed "auto", and yes, the commit message above explained why "recommended" here but it has never surfaced in previous discussions, has it? If so, this reinforces what I said here [1]. [1] https://lore.kernel.org/mm-commits/CAOUHufYEKx5_zxRJkeqrmnStFjR+pVQdpZ4= 0ATSTaxLA_iRPGw@mail.gmail.com/