From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C897EEB64D9 for ; Wed, 5 Jul 2023 02:07:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2201D8D0001; Tue, 4 Jul 2023 22:07:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1D0BA6B0072; Tue, 4 Jul 2023 22:07:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 070C68D0001; Tue, 4 Jul 2023 22:07:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id EA2886B0071 for ; Tue, 4 Jul 2023 22:07:58 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id AD14F140423 for ; Wed, 5 Jul 2023 02:07:58 +0000 (UTC) X-FDA: 80975922636.30.F8F4209 Received: from mail-qt1-f182.google.com (mail-qt1-f182.google.com [209.85.160.182]) by imf07.hostedemail.com (Postfix) with ESMTP id E63F64000A for ; Wed, 5 Jul 2023 02:07:56 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=wAy8sTOX; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.182 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688522876; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=O0wAoKec+Pl35+rlh7VOhG01IZuhb1xkdl9OsSICnU4=; b=LRo3cYuos0pRc1XrNkur7DM2wTCPU/klgIHX6X/75cv1Gx8xtsKt3X8vnW0gKVe83XUvZ/ ybjmt90n7e+ai+tFWtJrlO4AqWmhgaULOOguOWh2u2UgE/Q/HXsJphKjuxtkVYTkMgMtWh EUa8MWgzo9Z2cJvJZh0TXQiYoBiUPVo= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=wAy8sTOX; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.182 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688522876; a=rsa-sha256; cv=none; b=lYdB6oNwyyBgI1tAoaVDXb05yqFGlyebHxdyLZzBgAIEtQKJiFBUmd7CvmxyU1HrglQ9rE aNCHSNvwFVvKHkHdk68hcMYzGsIZpoyl4Yfi/NPQVFps7sW82MBecDpb6lc3hdVJ/ZXXcS d1QAlPAkAMfW/eOpCkN60YWW6+XGrr0= Received: by mail-qt1-f182.google.com with SMTP id d75a77b69052e-401f4408955so732621cf.1 for ; Tue, 04 Jul 2023 19:07:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688522876; x=1691114876; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=O0wAoKec+Pl35+rlh7VOhG01IZuhb1xkdl9OsSICnU4=; b=wAy8sTOX1/jVffTuSno5nLY/g6Nds9KsWTTtt48Hwl5iJXJ4rND6+wb2RtOdTvqHGp wG/qWRcWihcQE7ssyUJUElWA+btPAGilKqV/NqS/hKQVM9M2yyEHQy59r9v/gVNXXMNm nBoNR8MIemlYUjlT6WrURGCdLEKirmzXV4TLbO0mQ0bVShVKbAfYhTyRvdoVN5FR+GFl x5iKLVVzYVZxG6YBzzPpHY0GpOy47RTVF1xPmvx+rGvp3+3FNh5o1hTOzPiZ9XT3P5AG dwRixmPu39Y38zlflxmG6E51X/bsWGxECu33E2SiohKx/JgXLLBnp7o4WUleFn1zuQmM Ergw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688522876; x=1691114876; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=O0wAoKec+Pl35+rlh7VOhG01IZuhb1xkdl9OsSICnU4=; b=HsygtVT3v9y6Ia6+vPyjVFdKdZL9ouSI7o+Rq/y5TbCBaSyxRvyAoerSxHHDxBGS69 fi1IPZbIlxoNDpsWlN3UwsAoKB+JnpantGMabaK35Z8pArRr5oVDr7/fiUX4Sn7WEZnD C3LCtYH0rqVv5ok8F7o9ULqzQM6hY+HvAs9U6Bj6+sIyCL5CUgt+ZV7870XX74fKA1Xy HDbdzx/PuQyWlp22vZgMpxuxML6IinVjAnlDEIFEAQ19nBWP667a50f0YgaxLiRO2RmK sOdD9b12ZPh2Imdtf8JkGH9mF9vDeXG4t7tLyXZy95nKfqPcn6A7IWpJPwOllbAiNAYw STbQ== X-Gm-Message-State: ABy/qLbpv9kRox/g4WQ+/mlD06jw2gwANzQ9ArVIOv1TFw0SM0g4WmBG SNBG1HEtqH4eHx24FxxWUOPoYBddx0h2Sh1dV+43rA== X-Google-Smtp-Source: APBJJlHtHWhqDwvFEDVpZ4G9KknXb3bW+OyzK+jOwsTwjjzc9ueG3UHDyD1AIbujZQYMnl5AcfvVcJcpxUv4tETsKPs= X-Received: by 2002:ac8:5fd3:0:b0:3f9:ab2c:88b9 with SMTP id k19-20020ac85fd3000000b003f9ab2c88b9mr13116qta.25.1688522875892; Tue, 04 Jul 2023 19:07:55 -0700 (PDT) MIME-Version: 1.0 References: <20230703135330.1865927-1-ryan.roberts@arm.com> <20230703135330.1865927-4-ryan.roberts@arm.com> In-Reply-To: From: Yu Zhao Date: Tue, 4 Jul 2023 20:07:19 -0600 Message-ID: Subject: Re: [PATCH v2 3/5] mm: Default implementation of arch_wants_pte_order() To: Ryan Roberts Cc: Andrew Morton , Matthew Wilcox , "Kirill A. Shutemov" , Yin Fengwei , David Hildenbrand , Catalin Marinas , Will Deacon , Anshuman Khandual , Yang Shi , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: E63F64000A X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: d9kifupnzzi1a8nf5r5pbt9rcx7uhry1 X-HE-Tag: 1688522876-928597 X-HE-Meta: U2FsdGVkX1+9uOOTI0PLXsBwqQC6WQKimI7Lv/BA1ndqjHJh62SZcFGqTEiGckrWWMPt+FjKq3EQpWqVW2ClRWzMPQ7bESZtRR0eq2YaBJbVcBJ+zTefwWT6mmaaLBrKCesgvbJwHcv6mcOuD2pGEBlg+cGlGmUbv1ztqCIb5G6+mrsGMjjxJb/JgJ7WfwgtApZew7eMs+lQazXZa9bYjfaxvQnvVczKZ5EPtvt/5Hxk37U/sF/UxlVrrhXRvrLWdelFyi8TczSv4eQ1Qw+Lr+Inl+XXcJuH901g07PPu1ha0PmC5V4KFa0aR18gnSX+Of42PR400xJdnsmt8rNFt+R+DaPEJ/8FIjUIpx+0n4LquvSuBs145dPeELypgp9AAfNqpPtAb0/RIaIkKIVsw26YdzPPZZVBA5TvJJqVp4xLtWz50Zueyf/NEicv84PDILhpCHarPndLes5bpPOT+1ygouu96bDqg+5CvXPbaL3pX8Lx4Y8hnv2RTObkg7J6NANBVpiII+NzG1TYpk2LK+IMIV1uD2Ze0co+TRTrrdRBtppnVymdj+XPcCufxNvZAQZaHoG+YADA4Nxmg+goGIV09Ib6UvI+p5uCf+rGrC794sKrqQRTmrM9z4HlZpHTpHBn5ncj6xj/jtF2y4jXEnah/x9bDSF+nfwHtarG92bbYbLhsOQo5S28YS0hEcRXh0jCzcEFyckhbmDkJYj9PzAXKMiyLhuumnxFzV80JX++WdrWltMT/Mj0kd+O3HSkiNHg+24joQwcVMhJSc2fIVGaJIasX1Ap9TtfjCgtLQM3tbSZIlSFKdHctJySsbR8hPSN6lvaHLaCV/o9GlQ7oTi+vDB93wmxzOwup4WXyJNaTU4OC9TrGMW7yI5M9xNoxrsoVfOG3A2979knlgAKQ06zq8dGqBx2wS7/cIXsuX6cNXBv4zfGG2VBuxr8zEx4fE6+xxGWprr7uqezbCO PSHzbTkz ebEinvzWqiPYh1jmIEtRnPw/gsi3rEJq7koPSToa9njgmyboGe+K6T7J/lnrdSclnwQkHYMwJLSk3Av7G7BdGrghfphgWn3nbpLVPfhhkMOYBjVlOvJEiUlYuYHU2cbs1einy5uOtR41XI/HZvFsq+1xPAgM6aD4Y31SQSOmYMZ9tKJou6ABz4yB+F/fi6tRJrE7+GKW4xLD9skMd0ctGl0KLYGbvQmXGibTPvyfrScCzxWWLfORZHiQBRdJG5cwa82uuwyhAl6I9gq5nODAqy2BUH+3tGRKhn5ZJ3pCQCA36f9VnpBG5apnc7HyhxKUsJzgF5eRRoioevJXGCSuhuuLjMw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jul 4, 2023 at 7:20=E2=80=AFAM Ryan Roberts = wrote: > > On 03/07/2023 20:50, Yu Zhao wrote: > > On Mon, Jul 3, 2023 at 7:53=E2=80=AFAM Ryan Roberts wrote: > >> > >> arch_wants_pte_order() can be overridden by the arch to return the > >> preferred folio order for pte-mapped memory. This is useful as some > >> architectures (e.g. arm64) can coalesce TLB entries when the physical > >> memory is suitably contiguous. > >> > >> The first user for this hint will be FLEXIBLE_THP, which aims to > >> allocate large folios for anonymous memory to reduce page faults and > >> other per-page operation costs. > >> > >> Here we add the default implementation of the function, used when the > >> architecture does not define it, which returns the order corresponding > >> to 64K. > > > > I don't really mind a non-zero default value. But people would ask why > > non-zero and why 64KB. Probably you could argue this is the large size > > all known archs support if they have TLB coalescing. For x86, AMD CPUs > > would want to override this. I'll leave it to Fengwei to decide > > whether Intel wants a different default value.> > > Also I don't like the vma parameter because it makes > > arch_wants_pte_order() a mix of hw preference and vma policy. From my > > POV, the function should be only about the former; the latter should > > be decided by arch-independent MM code. However, I can live with it if > > ARM MM people think this is really what you want. ATM, I'm skeptical > > they do. > > Here's the big picture for what I'm tryng to achieve: > > - In the common case, I'd like all programs to get a performance bump by > automatically and transparently using large anon folios - so no explicit > requirement on the process to opt-in. We all agree on this :) > - On arm64, in the above case, I'd like the preferred folio size to be 6= 4K; > from the (admittedly limitted) testing I've done that's about where the > performance knee is and it doesn't appear to increase the memory wastage = very > much. It also has the benefits that for 4K base pages this is the contpte= size > (order-4) so I can take full benefit of contpte mappings transparently to= the > process. And for 16K this is the HPA size (order-2). My highest priority is to get 16KB proven first because it would benefit both client and server devices. So it may be different from yours but I don't see any conflict. > - On arm64 when the process has marked the VMA for THP (or when > transparent_hugepage=3Dalways) but the VMA does not meet the requirements= for a > PMD-sized mapping (or we failed to allocate, ...) then I'd like to map us= ing > contpte. For 4K base pages this is 64K (order-4), for 16K this is 2M (ord= er-7) > and for 64K this is 2M (order-5). The 64K base page case is very importan= t since > the PMD size for that base page is 512MB which is almost impossible to al= locate > in practice. Which case (server or client) are you focusing on here? For our client devices, I can confidently say that 64KB has to be after 16KB, if it happens at all. For servers in general, I don't know of any major memory-intensive workloads that are not THP-aware, i.e., I don't think "VMA does not meet the requirements" is a concern.