From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7ACFEB64DC for ; Tue, 4 Jul 2023 02:19:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 437AE280050; Mon, 3 Jul 2023 22:19:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3E816280049; Mon, 3 Jul 2023 22:19:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2B090280050; Mon, 3 Jul 2023 22:19:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 1B496280049 for ; Mon, 3 Jul 2023 22:19:37 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id EC7941204E1 for ; Tue, 4 Jul 2023 02:19:36 +0000 (UTC) X-FDA: 80972323152.12.52D389A Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) by imf07.hostedemail.com (Postfix) with ESMTP id 1A1394000B for ; Tue, 4 Jul 2023 02:19:34 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="YanU/9Jp"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.176 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688437175; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BzYdNMzSEjup40eaGDHi1UPxNXPRVOvs9KVOLwuo1X8=; b=mVidXjJa0mxTTMVO4wOrsH5ePnTXXT+6FJChEXrB8rqys3SAzsYr/b0C2q2Rfg+TAyXzVc f6mCvTkaPYf36K/61quFdgbMn1Pn9IHtGknYLOkqWWke/GQav+BLHTvgEjMAkpIDP7Tln0 CE5YruXDZlSwhtszEprkOZkCqoYYcgc= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="YanU/9Jp"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.176 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688437175; a=rsa-sha256; cv=none; b=MXzVXqD8RqcIaRvfCqzfp5STaPqsrJbE15HQl1Q1Q4EraMDyUlGZL+iUlgfIRwIqCJaF9/ OHMxDEy9mf6sSf9Nht/UuuEEd3XagtFEG2nfUJNHM5ZDIsSCx9v8IZlr3GWSAZXAAzxHzB 1nQ2Jug+2rCTaqFWNSlur7jul2szpWQ= Received: by mail-qt1-f176.google.com with SMTP id d75a77b69052e-401f4408955so491701cf.1 for ; Mon, 03 Jul 2023 19:19:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688437174; x=1691029174; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=BzYdNMzSEjup40eaGDHi1UPxNXPRVOvs9KVOLwuo1X8=; b=YanU/9JpsSoIkstsYmvxwLf4Pdrd+kjk57QT+7u5Km8QdoY++Cscminmi7dNUCmVJN c1G3jQvnhhGmGj6GBKL3J9UuiWTFpAuIBdQCUNHV8eANm7KXF5nleI3yNlE0UMNJ1U3w hQOevI/XzfvRrzk+cwiZFyS3PPi7jqp8dHQUA/t/rL3C3lc/l0qdTkVheV4XcSvOUa/L /czQv+Yv3g9dFGe+fpbAq7qs/T/djlm8M1wAZJrbsB4s/7DBBEfdugZHtKfHn47zi+/H +8vR+lzQbzx+3OhnEbp8lbmKcJTg4gzamTde+B7QAwvJ6nJqj9bI3xdYSwJIEqGtGv3/ lyeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688437174; x=1691029174; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BzYdNMzSEjup40eaGDHi1UPxNXPRVOvs9KVOLwuo1X8=; b=TxQmi6oAVQyEyB450h65DqFDKcQSKkNx4JKV0hcKpQgECwbCOArgwigEfrKEiZQiWH IPr0EeaZ8kC7CdiY5qcMRo2KGe891eCIfT5IsuZb2vm7WSRXmBFCputVQvd6phmoNueP b/8JfMZUR6w4+4qcvprI5OlbJ6C7wXDNrVcJ5GhnP9jfH9GgNQX73U3SoipvhxtDlT3U BjDpoZWdczBo+0KfipkvTQyE75t0yVMzEv8c4HPDMvDnVqWZJwI5desNZuroP1TUxQRN mupwq5JzmvDUAjYrk8Z6G1E3qEmp2vBbVYRbzG9NJNW7p+nn7U2qNttUPFJ1ubTOoCJg xyRw== X-Gm-Message-State: ABy/qLYUAtn5SMyiHZIFgf7hmZXl04KwC3YSArFeH7EZcrImiH9avS6M 3fByL7+ia1gqilJRM7lAr36KVfgdXN6Z7Ib6OVxJNg== X-Google-Smtp-Source: APBJJlEcKfiNUvWa1AnjxU4r099qbHgdzmXqztQhjJj0kS84wyNjwTaQIL1OAimmRQDFKSkNbkpe8WpSjpyxfUuWJkQ= X-Received: by 2002:a05:622a:314:b0:3ef:302c:319e with SMTP id q20-20020a05622a031400b003ef302c319emr40179qtw.8.1688437174233; Mon, 03 Jul 2023 19:19:34 -0700 (PDT) MIME-Version: 1.0 References: <20230703135330.1865927-1-ryan.roberts@arm.com> In-Reply-To: <20230703135330.1865927-1-ryan.roberts@arm.com> From: Yu Zhao Date: Mon, 3 Jul 2023 20:18:50 -0600 Message-ID: Subject: Re: [PATCH v2 0/5] variable-order, large folios for anonymous memory To: Ryan Roberts Cc: Andrew Morton , Matthew Wilcox , "Kirill A. Shutemov" , Yin Fengwei , David Hildenbrand , Catalin Marinas , Will Deacon , Anshuman Khandual , Yang Shi , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 1A1394000B X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: fz78f6y41e9u7pxtz1sqekg5ywisif1n X-HE-Tag: 1688437174-822562 X-HE-Meta: U2FsdGVkX18dXJ7iR5Sri6PuNiZdv+tcRWz60GuKzFGwPJ/Tz4X7twn9o3LnkM4SInjuj41F/yARP3gg7BfhtHMFxMzDottyUt1qMrelVnV41xy9MzHvy58HJJnWKmVVGv/lFzvEcKTzFaGrbr7Db6PlzbLJJGorSNYRpDFbCJnIkUCiJhvQsAeY0F5b/xNFmf0bToFwCdUjhNaDwZExTeYMMiUyOtsR8pduZhCOjmNpbBDbZyUw6sdx0rnUZgufeVmGfSh1ss67GeTBvM1Johb+XtIrcBjfvxZcsy79Qr2mydsdCcjhEI5xMx7qzRF4hxqoIkn2Bi8FEuxqa9GM9ZV1q4r1Az9RLL5gXauKruo+UoVjmMcBQWSj4N5DlhuehXHj87PQD7yT555+idO5OvgqCMjrdASR0rDW53FM4FjwX730y1mD6xTlHt0JUGs/aSIFtGUyqWJpCS0+XuiZCdfCHnI/29jPYKUJXmpGsN7K+eq3wccJj3fYylnBhiMgLXWUFtrwfSc8z0Puheahu7amA0SJmDeVQLWDOtyFM8zzH459Alm6TqdcswWLfLye3+eiIn9IIGaT5C+/BXBbfP6tZ1ficE1r2bYVahpcOJVRvIp/NMOE7VQqDP9B48h6BAgF2LLonhugD2t4RtH73XDwaJYqgwpJTSwXxv7QwfPuLJDXi1Ql0BWJyvo2cOfmb7jixx+XanriKmzmJ7r7T+kWSg6lLsb6yflFWU7QEHQ17ALznDp8P1V+xvOXBVkr1GVJlq1w291lVtaNWiJ6J15J03iu4S8mwbSEIxNCUsKRnH9KdGTaKOAjvqeWgMVCdoauoY28qm01T6ZHB5iyr/SsXA6tCCgrl+Z6GHlV9E069u4RmK7enyOzXFDit5CqHtw7hWJdCfvn1kNeOvhq+hdwiWixk3DWSiPxiGydzaIayZlu8kwjOKJNfsd7SF0Na4JewpopBiJfycekj06 ThQPJ29J cTmWqAYoRaX8nVmUcKwEt2Ay0/q8ztWyj7f6cegh+2u4aJujYovhyLKSNBBr0PTviPvFZGXMJsOt70KjxZu8/kmllkqrWVRgr6DCBOLzdZnyI9oWy1lw9Ea18UVSNp6iiEg1dxFuzIPBtjkH38voTcmFcjiCT4CebyJqthlXSaMoFK3l9s94Pnh2eaexgkKGcyc2sTr5LewXMVy5C7YQZ45Js186tFaR90xLmkZju82E1HJOrrqho7tZ0qp0tTxd49L4it6/JIdDttUOnMOWITxF/yNok/EYjuUxcAGRvhafIwJom4OmlJwXAqgv1bGFbWG49 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Jul 3, 2023 at 7:53=E2=80=AFAM Ryan Roberts = wrote: > > Hi All, > > This is v2 of a series to implement variable order, large folios for anon= ymous > memory. The objective of this is to improve performance by allocating lar= ger > chunks of memory during anonymous page faults. See [1] for background. Thanks for the quick response! > I've significantly reworked and simplified the patch set based on comment= s from > Yu Zhao (thanks for all your feedback!). I've also renamed the feature to > VARIABLE_THP, on Yu's advice. > > The last patch is for arm64 to explicitly override the default > arch_wants_pte_order() and is intended as an example. If this series is a= ccepted > I suggest taking the first 4 patches through the mm tree and the arm64 ch= ange > could be handled through the arm64 tree separately. Neither has any build > dependency on the other. > > The one area where I haven't followed Yu's advice is in the determination= of the > size of folio to use. It was suggested that I have a single preferred lar= ge > order, and if it doesn't fit in the VMA (due to exceeding VMA bounds, or = there > being existing overlapping populated PTEs, etc) then fallback immediately= to > order-0. It turned out that this approach caused a performance regression= in the > Speedometer benchmark. I suppose it's regression against the v1, not the unpatched kernel. > With my v1 patch, there were significant quantities of > memory which could not be placed in the 64K bucket and were instead being > allocated for the 32K and 16K buckets. With the proposed simplification, = that > memory ended up using the 4K bucket, so page faults increased by 2.75x co= mpared > to the v1 patch (although due to the 64K bucket, this number is still a b= it > lower than the baseline). So instead, I continue to calculate a folio ord= er that > is somewhere between the preferred order and 0. (See below for more detai= ls). I suppose the benchmark wasn't running under memory pressure, which is uncommon for client devices. It could be easier the other way around: using 32/16KB shows regression whereas order-0 shows better performance under memory pressure. I'm not sure we should use v1 as the baseline. Unpatched kernel sounds more reasonable at this point. If 32/16KB is proven to be better in most scenarios including under memory pressure, we can reintroduce that policy. I highly doubt this is the case: we tried 16KB base page size on client devices, and overall, the regressions outweighs the benefits. > The patches are based on top of v6.4 plus Matthew Wilcox's set_ptes() ser= ies > [2], which is a hard dependency. I have a branch at [3]. It's not clear to me why [2] is a hard dependency. It seems to me we are getting close and I was hoping we could get into mm-unstable soon without depending on other series...