From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C5D7C001DE for ; Fri, 4 Aug 2023 21:01:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A79F06B0071; Fri, 4 Aug 2023 17:01:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A32846B0072; Fri, 4 Aug 2023 17:01:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9182F6B0074; Fri, 4 Aug 2023 17:01:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 7BBCB6B0071 for ; Fri, 4 Aug 2023 17:01:12 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 09AF3161392 for ; Fri, 4 Aug 2023 21:01:12 +0000 (UTC) X-FDA: 81087642384.04.E66B5F6 Received: from mail-qt1-f181.google.com (mail-qt1-f181.google.com [209.85.160.181]) by imf20.hostedemail.com (Postfix) with ESMTP id 0F2EC1C001A for ; Fri, 4 Aug 2023 21:01:08 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=AOpo0iGE; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf20.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.181 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1691182869; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qIj55jcecVh0xUF4Aoj9xGp57UjUQUFnc7ZLpH5DAmI=; b=z9aE6E/1KCSmLaUpGgyCHhF7aWlJxmVxyqx8yngnQDx7SWXcXmrtDyo0mPtZhEvUUVsXyB MsTryTljtGUVYsnh6kVitZK35R9XIvQA6mxJp6SbW3L5ukAsr9BhXX2mu5hiM+0+fnAm9a kqISWfs38N+H+KzJFQXXuy9vJ22Dkww= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=AOpo0iGE; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf20.hostedemail.com: domain of yuzhao@google.com designates 209.85.160.181 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1691182869; a=rsa-sha256; cv=none; b=MC8ObzK+i/O/VSxQCWvY3E5piYj32I77egxhCtWZYnpBsShwslomX4CRe0v9YXa69gjoIm 35HZaUEC636VwSCIEXsLi2mC+vkOzRJD/AxQPgXGs86yaERsIs4nJRMkDD1OXTY/uQD8U2 ls/EYBkfwcGt9Q9qVQgt8NdW8YqGqHI= Received: by mail-qt1-f181.google.com with SMTP id d75a77b69052e-4036bd4fff1so24251cf.0 for ; Fri, 04 Aug 2023 14:01:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1691182868; x=1691787668; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=qIj55jcecVh0xUF4Aoj9xGp57UjUQUFnc7ZLpH5DAmI=; b=AOpo0iGE7WLVlwYqhqGpS4L6N4pxpkjCoQUl40dlWd3T8mkz0pSj/n4zFOlCEuUkXZ QeLBtEPqVzFXfJ8Qbh1c/b3dOnPIZpBu0uQkCgh6EJUjydmlPiESJ689AKEi/a6kmYoh k4GAvab3396RUY6vG88bVZgs26JmCAtS9A80csN1Z4Ivxdj9TghBfUWZDAvtSKx9oq3w p2vIW/cJ61PEHirr7+LWYzdjw6BXVOykSdx8RRFhjn2HEgeWkpaDi7GmQ8b0zDkbuOMr CwOdzBVbC+Gp1H6EGY/iKApLPCqBK6iiqmZ+Oc278QuI9BjrS6JEevKzlmNnRBWDTqCo rrEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691182868; x=1691787668; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qIj55jcecVh0xUF4Aoj9xGp57UjUQUFnc7ZLpH5DAmI=; b=YRYhHZ7Xpp5awokRdpeuXsRD97478j+O7PMZ775Xd+Szp7RrqGY/pMtnT9EwNz3sLQ cR79IH5KGtxgpdzfmXzFKN9rKYjy0R1yR0nx1Lkfmd8HlSuMpR2soK+xK5eLQx+EKxJu mSEZFV+vBE9KxHhOb4ulaExJZ+v176XD5mr3Toyte5KfVEmwv/HwI2qZ7jZpNGKV4QtP PfqmbuJWPxahv0Oj5ihzlnUJFoHlqJrJL46xgVh3hcsZhJfAQjjuG9xtcv8FqgoftHMN D2pDukl4fIHOoiIBIcHLVhrKuso0eLY1+k3KJ1lvNl6F1oO10R6a9ADun1wr8x4cbLrp 8WaA== X-Gm-Message-State: AOJu0YwSXzkBvO6LSBC2qlpoRsaDP7R3i8rI5KbiarZO81jITZJMNwLe YU8utQYdpo7j58+7oquZ0HgVswstwN5DowPOo9UEyQ== X-Google-Smtp-Source: AGHT+IGwOVqu/W7zHGnURPcrR7UU8BMLb7+UZ+exi6Efb4ae0sxYA6WOfslqP0jMXje5o1ZUl4kzAOto6atHEGT66e8= X-Received: by 2002:a05:622a:1805:b0:3f9:a770:7279 with SMTP id t5-20020a05622a180500b003f9a7707279mr22330qtc.9.1691182867906; Fri, 04 Aug 2023 14:01:07 -0700 (PDT) MIME-Version: 1.0 References: <20230726095146.2826796-1-ryan.roberts@arm.com> <20230726095146.2826796-3-ryan.roberts@arm.com> <5e595904-3dca-0e15-0769-7ed10975fd0d@arm.com> In-Reply-To: From: Yu Zhao Date: Fri, 4 Aug 2023 15:00:31 -0600 Message-ID: Subject: Re: [PATCH v4 2/5] mm: LARGE_ANON_FOLIO for improved performance To: David Hildenbrand Cc: Ryan Roberts , Andrew Morton , Matthew Wilcox , Yin Fengwei , Catalin Marinas , Will Deacon , Anshuman Khandual , Yang Shi , "Huang, Ying" , Zi Yan , Luis Chamberlain , Itaru Kitayama , "Kirill A. Shutemov" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 0F2EC1C001A X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: iupgiox9wsmshy8xq8mqmtbnbr5wbyu8 X-HE-Tag: 1691182868-199513 X-HE-Meta: U2FsdGVkX1+qRdvBOS2OKs37EdNPsSNtsPNtN4BXwM76MVjcUXlyl8KRwhxFOFJKLeuwjp4bdoD1AsVfH4kkAGRi7GpqrVyCMyE3q8cPloHnZatIKFyV9VmY26E3eVls6Sv5XCtSw5TCnPpUadJei80UE6F0aiFoOM2Y13WTUeWX5+WxYsHY/gjH6CIogC5z5Og5M1Jvj9r4hgnJFETsgN3Al8tJTBTtkT+/XT0G2InvbHymKS721qjtDNkQaUpAMGrFfchZUoTHze/WWfvrVcHa/xSVinAfaiqb7nXTpo2E7ACzXJt/x9DrwCRLXTuw7x5FWPtKEDZM8X2wQPHfmjn1MBBKZM9993G5BrcVFwvUfBoI/JR72ZmpANovz4QrELjh5BnvTA1vW/SQTb37AZlKt8EkZCImMIPF1MTPRIjgGCmzKe2NE12F6baYLs60EbGBeI/ZeVKKc77Vw/K7ZdgaTwa+wwZP4KoHOwmuf1gzMOqj4gUlNUhKkCZt5sJmRMyW349cA3hXXZEsdnwWBU/atwoHMH4G28QB5JEOruN2pHZx/Aj4K0YdFzF+tV2EeHop5tXLPX7ruRDGu/BxkuZa3eijJhqLQ+jzj3Ia7lyqc2fQPX0nHaEXoBiTkyEQh+h20NEP/oel5Su8nq1IHM/Aw78WsTatpPTW6k4zYGcF6jQMJPYVb5TaDdlKq+KkrjgwLe74Cv88UA63LrKTY6OF5FtB5BDaqtD3aT9EsXuUybhBv5ivHRSQedoHvmaQLn8bWsegRVve6CEp/5JAjL+UEnBQm8iCmQcxK9wEoYxlaBB7EXU7tV3ikf9j6ab5VFkJdvfePuIJQyRQYKI3epX5Hv+NQ9o2LRsdEY7HeGZbepq9mN8Psxt88tDlKqqWL5H8q4FZPBFxa4QVmOo0p5aHaRncXOYZWKvpFLifI5K+AgJeD//D7gCDiXIJEFu9y/rNUYsgyk2W/cWKgwl eh5iVB0A 7u8ReV/lMc4WYscGVTUg4kbV6b8crpImbgeLt3Cjp+G4t2n/oVR8TcyUjFN4vkjRC9iktB2aJg7wVynk0v0NlzWp8IULhbd6Icdt6cNVahJXI9wAs4e2o01wWgsYkG3r07ntc2Ng3aUROe0lV8vwRW3Fkgtzky9hYiuoqgYHbH3tcOqJoSiIp6VIg9qT4fGxPPkSZ7VXiq16DbPVemUanyCw4ub8QcEjITWnUxI3av4qlvxf5gk9TgrrTsaveBlOMtcHMWl2Zcq3CBiiIDifOezF0/Cs7x5tQ3jKKI4ONKntE6BCYmvOdcuMut6NFWZ6p2tXuMzLbRvieZFw01Gjut1nP2tdB2J7e3LwBm5zZaHk+TcaQPPy0v6I2lsKPCMULJiCV/tE9tOeSkOrHaphfGxzCYLOcXr4MjobsGpx5W9SXnWwpnGK5ISlwvQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Aug 4, 2023 at 2:23=E2=80=AFPM David Hildenbrand = wrote: > > On 04.08.23 10:27, Ryan Roberts wrote: > > On 04/08/2023 00:50, Yu Zhao wrote: > >> On Thu, Aug 3, 2023 at 6:43=E2=80=AFAM Ryan Roberts wrote: > >>> > >>> + Kirill > >>> > >>> On 26/07/2023 10:51, Ryan Roberts wrote: > >>>> Introduce LARGE_ANON_FOLIO feature, which allows anonymous memory to= be > >>>> allocated in large folios of a determined order. All pages of the la= rge > >>>> folio are pte-mapped during the same page fault, significantly reduc= ing > >>>> the number of page faults. The number of per-page operations (e.g. r= ef > >>>> counting, rmap management lru list management) are also significantl= y > >>>> reduced since those ops now become per-folio. > >>>> > >>>> The new behaviour is hidden behind the new LARGE_ANON_FOLIO Kconfig, > >>>> which defaults to disabled for now; The long term aim is for this to > >>>> defaut to enabled, but there are some risks around internal > >>>> fragmentation that need to be better understood first. > >>>> > >>>> When enabled, the folio order is determined as such: For a vma, proc= ess > >>>> or system that has explicitly disabled THP, we continue to allocate > >>>> order-0. THP is most likely disabled to avoid any possible internal > >>>> fragmentation so we honour that request. > >>>> > >>>> Otherwise, the return value of arch_wants_pte_order() is used. For v= mas > >>>> that have not explicitly opted-in to use transparent hugepages (e.g. > >>>> where thp=3Dmadvise and the vma does not have MADV_HUGEPAGE), then > >>>> arch_wants_pte_order() is limited to 64K (or PAGE_SIZE, whichever is > >>>> bigger). This allows for a performance boost without requiring any > >>>> explicit opt-in from the workload while limitting internal > >>>> fragmentation. > >>>> > >>>> If the preferred order can't be used (e.g. because the folio would > >>>> breach the bounds of the vma, or because ptes in the region are alre= ady > >>>> mapped) then we fall back to a suitable lower order; first > >>>> PAGE_ALLOC_COSTLY_ORDER, then order-0. > >>>> > >>> > >>> ... > >>> > >>>> +#define ANON_FOLIO_MAX_ORDER_UNHINTED \ > >>>> + (ilog2(max_t(unsigned long, SZ_64K, PAGE_SIZE)) - PAGE= _SHIFT) > >>>> + > >>>> +static int anon_folio_order(struct vm_area_struct *vma) > >>>> +{ > >>>> + int order; > >>>> + > >>>> + /* > >>>> + * If THP is explicitly disabled for either the vma, the proce= ss or the > >>>> + * system, then this is very likely intended to limit internal > >>>> + * fragmentation; in this case, don't attempt to allocate a la= rge > >>>> + * anonymous folio. > >>>> + * > >>>> + * Else, if the vma is eligible for thp, allocate a large foli= o of the > >>>> + * size preferred by the arch. Or if the arch requested a very= small > >>>> + * size or didn't request a size, then use PAGE_ALLOC_COSTLY_O= RDER, > >>>> + * which still meets the arch's requirements but means we stil= l take > >>>> + * advantage of SW optimizations (e.g. fewer page faults). > >>>> + * > >>>> + * Finally if thp is enabled but the vma isn't eligible, take = the > >>>> + * arch-preferred size and limit it to ANON_FOLIO_MAX_ORDER_UN= HINTED. > >>>> + * This ensures workloads that have not explicitly opted-in ta= ke benefit > >>>> + * while capping the potential for internal fragmentation. > >>>> + */ > >>>> + > >>>> + if ((vma->vm_flags & VM_NOHUGEPAGE) || > >>>> + test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags) || > >>>> + !hugepage_flags_enabled()) > >>>> + order =3D 0; > >>>> + else { > >>>> + order =3D max(arch_wants_pte_order(), PAGE_ALLOC_COSTL= Y_ORDER); > >>>> + > >>>> + if (!hugepage_vma_check(vma, vma->vm_flags, false, tru= e, true)) > >>>> + order =3D min(order, ANON_FOLIO_MAX_ORDER_UNHI= NTED); > >>>> + } > >>>> + > >>>> + return order; > >>>> +} > >>> > >>> > >>> Hi All, > >>> > >>> I'm writing up the conclusions that we arrived at during discussion i= n the THP > >>> meeting yesterday, regarding linkage with exiting THP ABIs. It would = be great if > >>> I can get explicit "agree" or disagree + rationale from at least Davi= d, Yu and > >>> Kirill. > >>> > >>> In summary; I think we are converging on the approach that is already= coded, but > >>> I'd like confirmation. > >>> > >>> > >>> > >>> The THP situation today > >>> ----------------------- > >>> > >>> - At system level: THP can be set to "never", "madvise" or "always" > >>> - At process level: THP can be "never" or "defer to system setting" > >>> - At VMA level: no-hint, MADV_HUGEPAGE, MADV_NOHUGEPAGE > >>> > >>> That gives us this table to describe how a page fault is handled, acc= ording to > >>> process state (columns) and vma flags (rows): > >>> > >>> | never | madvise | always > >>> ----------------|-----------|-----------|----------- > >>> no hint | S | S | THP>S > >>> MADV_HUGEPAGE | S | THP>S | THP>S > >>> MADV_NOHUGEPAGE | S | S | S > >>> > >>> Legend: > >>> S allocate single page (PTE-mapped) > >>> LAF allocate lage anon folio (PTE-mapped) > >>> THP allocate THP-sized folio (PMD-mapped) > >>>> fallback (usually because vma size/alignment insufficient for= folio) > >>> > >>> > >>> > >>> Principles for Large Anon Folios (LAF) > >>> -------------------------------------- > >>> > >>> David tells us there are use cases today (e.g. qemu live migration) w= hich use > >>> MADV_NOHUGEPAGE to mean "don't fill any PTEs that are not explicitly = faulted" > >>> and these use cases will break (i.e. functionally incorrect) if this = request is > >>> not honoured. > >> > >> I don't remember David saying this. I think he was referring to UFFD, > >> not MADV_NOHUGEPAGE, when discussing what we need to absolutely > >> respect. > > > > My understanding was that MADV_NOHUGEPAGE was being applied to regions = *before* > > UFFD was being registered, and the app relied on MADV_NOHUGEPAGE to not= back any > > unfaulted pages. It's not completely clear to me how not honouring > > MADV_NOHUGEPAGE would break things though. David? > > Sorry, I'm still lagging behind on some threads. > > Imagine the following for VM postcopy live migration: > > (1) Set MADV_NOHUGEPAGE on guest memory and discard all memory (e.g., > MADV_DONTNEED), to start with a clean slate. > (2) Migrates some pages during precopy from the source and stores them > into guest memory on the destination. Some of the memory locations > will have pages populated. > (3) At some point, decide to enable postcopy: enable userfaultfd on > guest memory. > (4) Discard *selected* pages again that have been dirtied in the > meantime on the source. These are pages that have been migrated > previously. > (5) Start running the VM on the destination. > (6) Anything that's not populated will trigger userfaultfd missing > faults. Then, you can request them from the source and place them. > > Assume you would populate more than required during 2), you can end up > not getting userfaultfd faults during 4) and corrupt your guest state. > It works if during (2) you migrated all guest memory, or if during 4) > you zap everything that still needs migr I see what you mean now. Thanks. Yes, in this case we have to interpret MADV_NOHUGEPAGE as nothing >4KB.