From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F5EFC64EB4 for ; Thu, 17 Aug 2023 21:13:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 81EC4940035; Thu, 17 Aug 2023 17:13:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7CEF5940009; Thu, 17 Aug 2023 17:13:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 696FB940035; Thu, 17 Aug 2023 17:13:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 5CB3F940009 for ; Thu, 17 Aug 2023 17:13:35 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 2F3054053E for ; Thu, 17 Aug 2023 21:13:35 +0000 (UTC) X-FDA: 81134847990.07.047C2CB Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) by imf15.hostedemail.com (Postfix) with ESMTP id 43422A0023 for ; Thu, 17 Aug 2023 21:13:33 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="Qm/4X2lt"; spf=pass (imf15.hostedemail.com: domain of zokeefe@google.com designates 209.85.128.53 as permitted sender) smtp.mailfrom=zokeefe@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692306813; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IdiC6L+abnI7yMaQpQLGz5ktCkS2ADhtdZrEzDNVLG8=; b=s4mjfYCeSURdmcLlp3Jhm/gbqthH6mwIPEUwqHAej+LYkdhE04eMhojD+W9vXEcwt2DWeL N10G0yvRVQtioanLtJq4iE1xsck32+OgDu+yh+bVLYdWSohrYJsZwbNxNb/pIlIU4E1e5b Z8yE5ro9AGZq5UNWOUzZXGaN7UFcKhs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692306813; a=rsa-sha256; cv=none; b=rIeGNrKFC9wr2npyGIVrKx+6WX3HzPKlsRSeS1le9MN/X0OlbJCatp2lehtTJOkvoGFOEJ pqJb3hRTHadhzPG/DtFsR4FekIfKLaxWlDQ1Wo1UDGg9T0TmMTn1FMCHj/KKGNwkVe7K+5 +PMVcoOhmzreTVH0/h/E7MCAvq8AEHc= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="Qm/4X2lt"; spf=pass (imf15.hostedemail.com: domain of zokeefe@google.com designates 209.85.128.53 as permitted sender) smtp.mailfrom=zokeefe@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-wm1-f53.google.com with SMTP id 5b1f17b1804b1-3fe32ec7201so8775e9.1 for ; Thu, 17 Aug 2023 14:13:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1692306812; x=1692911612; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=IdiC6L+abnI7yMaQpQLGz5ktCkS2ADhtdZrEzDNVLG8=; b=Qm/4X2ltNOuNAlJpd409MjBCzYTuAx+2qoDtpxCxzf4qxsBEFhk+xlVSwfp+UDzQVg TuOi0yv36C+6OqcwDFx4i7jOZ3QDF7MkwcEHjWg9lgb6qFy8a2K2KArY+pE3Ta15CpBJ QkrG/caC24wzjH4CgQqPCSnrwX8G2GBFz0lkslHFWaLXwZzWjvOfa1hbK7YX2HP9izU2 EznBqyWlCTBjZ0/hqNFeRfw9Oh0KxNknVNLyZ4RgUCveVF14OeDQDk+T/io7eS3nmXcu 2XU0oKikG1lcqu62bSJtbAzcmJ4QJopQ55+YXAz7OMjcWZB6babYixP87i+K26Id8e6S 6oHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692306812; x=1692911612; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IdiC6L+abnI7yMaQpQLGz5ktCkS2ADhtdZrEzDNVLG8=; b=SEqNcBGAgEuhJfyaI6UGRZpzP+xs5Nw0lT6oibX64fNvWivnq7PCoxxEyUBzartxrY R7+/Wq7SFRXi+xBX8Qzo2SB37D4Hvlmc3mHpUkM/DDvc0soL33JN/BRFG4vcK4jZu5F6 RHZYU36z+Xc6UEq0OKd8jHP2ro/A4gBDnV+KAtpweGJ4sf1Td/voi4+FlXPTGP/AOL5J 6H/NrUEMYlcQL7TC/JX3rFoh8YFUhBMVc05cGMyvpPCb9V7LoRpHBAzcTaOA1K2NNckU Kn+p95wXGkq+3INyhlLY/nlXedQSmTJWtHnsCv85PuzmGCpTIlZhGPBMrgD8PB8mCw5/ c5YA== X-Gm-Message-State: AOJu0YyepvaOMSLPT/r/AzoLey7YYjBWjom4KZzNQPBrOuOyEAbmkfmM fK/axUtSF+GLivxkhqNWp/R0RUJrwVUrzC0adiNquw== X-Google-Smtp-Source: AGHT+IH9NNddBZly34N9t4E6nIZ7GKWeYNNMS9wh7+N4YqplEB897dU/NG0Yfby50L3o03C0RH9TR06fJz5AOgz4SYM= X-Received: by 2002:a05:600c:3ca1:b0:3fe:cd3a:ef92 with SMTP id bg33-20020a05600c3ca100b003fecd3aef92mr2871wmb.6.1692306811600; Thu, 17 Aug 2023 14:13:31 -0700 (PDT) MIME-Version: 1.0 References: <20230812210053.2325091-1-zokeefe@google.com> In-Reply-To: From: "Zach O'Keefe" Date: Thu, 17 Aug 2023 14:12:54 -0700 Message-ID: Subject: Re: [EXTERNAL] [PATCH] mm/thp: fix "mm: thp: kill __transhuge_page_enabled()" To: Matthew Wilcox Cc: Saurabh Singh Sengar , Dan Williams , "linux-mm@kvack.org" , Yang Shi , "linux-kernel@vger.kernel.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 43422A0023 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: fdagzmcexogp3r3m7hrtxsifbc9ctpcw X-HE-Tag: 1692306813-923646 X-HE-Meta: U2FsdGVkX18SCFSrvIS/3BGaGJFW72+uf/dj+etQ5mbmO1CttmqPIhd9KANx14rC92+1BEqwwtKtrhIvDpbuRrqmhnEb7Gzplim8TEFOQ2ld5IV6khF6BKkOsGo8ap1BL7u5BYqp2i9JUDZZhJ12XZXOMeuIg/LzNw2R33620KIShjXcp8ZeyGx+MkXzotw1ZpoVA9I9D3gfpIS2FEYvGq/dsLqXUcBss84IGI228i+XqY011zFfi1EDKzugozBxByazVVB0d+NMunXR3+1YIA9oqTIikK771glkSejIRNM8c44W0KBBWjmWo5kCYten3VNnGrVEX+LbFx0Vno6PIbYtH1jr0QSQR+LQD0lqZ0EEwLK1dIMJeKdIaVmPJCrwHeegEmR5qmkigEyNAcY0fdx3Y7PovrdhIlWciQIufSr7uOLxxZs0QDDe128s3GfC+4CbPFLw9+y6v3B7U1W206xr/h4FC53xmIO23KokvOprZIrs7u0Zqtt4sDPcqmnoKYtj57phblhlMy9AOeBD6D0OF3iUM3No+vQA//YNbpJ0pfpZhsAKh+xxLN0dEuexUM/gqVdeMvbIgKrCqvdzuSPNgUrReeYByrISbdMfm1FoukhXtJiNYvAMsjAQ7nGmrJ8rQQsL3p+ItZbsHxQgQWMSlu7YCX1ax7kyBUEN5r+XnEqlZzwxaTotA9tNdpxhq5n7CUIB/dY0M3cj3HGApJDD2EvNmWb/AECrpOlgNtH8PTAA1hKNPx5UIyEr0Pt2ifKkPtaq6YWMhnA20RuPnn0ZGqYyLmtr7GSGeisO8ylFS3nwInZXgYXSRPp9aAxXW3701KYSYLsmjox+zsmxbiicOlWh56ZLwsW4tsLySFEz328MOqEki7oPtnFc/0YXJRb2I9oXa/PZfoJ03pA88pFyGXrVXmjnZwvxqvGQ+1PgV8naU545452X/U/chQqYX0apq9r35y1nYG13ESU 0mGBLN4H pOMcTokm4XjZX3Hh5BAAyaNqjZm4VI3g12vPHOtjAIKDvbiJeZd7BcaoH+1YoWPPuCLATVNg9U/KzvboU/Q9lOZIoexvDsytt42n7tdRShVR33tYDV/slwSfZT8us6iEhmJf721bBNDdsHomcsRIQEDHkbHSbflmAYqQEMdN1QHcJPFLf6/NHtgb+rTPu74+kYM/87V1pNNOCkq2HuOsqsQ8VSBFzLnCjkL06j1TjnZx/VUHHxbirJcN/YGZwFdOSlpqtajBN8f65TUApAhdPlq1TckjT8lFs0vOyr/lPYn18CvqDILb2qr1qj7iHlgax7j+Vuw/BSgzhqJ54EhWi91Cvb/pPeWummFOu0wO1u4/BlsMe3nvqgSRf6g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Aug 17, 2023 at 12:01=E2=80=AFPM Matthew Wilcox wrote: > > On Thu, Aug 17, 2023 at 11:13:36AM -0700, Zach O'Keefe wrote: > > > > IIUC then, there is a bug in smaps THPeligible code when > > > > CONFIG_READ_ONLY_THP_FOR_FS is not set. Not obvious, but apparently > > > > this config is (according to it's Kconfig desc) khugepaged-only, so= it > > > > should be fine for it to be disabled, yet allow > > > > do_sync_mmap_readahead() to install a pmd for file-backed memory. > > > > hugepage_vma_check() will need to be patched to fix this. > > > > > > I guess so ... > > > > The easiest and most satisfying way to handle this -- and I think we > > talked about this before -- is relaxing that complicated > > file_thp_enabled() check when the file's mapping supports large > > folios. I think that makes sense to me, though I don't know all the > > details fs-side. Will we need any hook to give fs the chance to update > > any internal state on collapse? > > If the filesystem has per-folio metadata, we need to give the filesystem > the chance to set that up. I've vaguaely been wondering about using the > ->migrate_folio callback for it. At the moment, I think it just refuses > to work if the folio isn't order-0. Ok, no free lunch here then. I'll give myself a reminder to come back here then and dig a little deeper. Thanks Matthew > > > > But I have a larger question for you: should we care about > > > > /sys/kernel/mm/transparent_hugepage/enabled for file-fault? We > > > > currently don't. Seems weird that we can transparently get a hugepa= ge > > > > when THP=3D"never". Also, if THP=3D"always", we might as well skip = the > > > > VM_HUGEPAGE check, and try the final pmd install (and save khugepag= ed > > > > the trouble of attempting it later). > > > > > > I deliberately ignored the humungous complexity of the THP options. > > > They're overgrown and make my brain hurt. [..] > > > > Same > > > > > [..] Instead, large folios are > > > adaptive; they observe the behaviour of the user program and choose b= ased > > > on history what to do. This is far superior to having a sysadmin tel= l > > > us what to do! > > > > I had written a bunch on this, but I arrived to the conclusion that > > (a) pmd-mapping here is ~ a free win, and (b) I'm not the best person > > to argue for these knobs, given MADV_COLLAPSE ignores them entirely :P > > > > ..But (sorry) what about MMF_DISABLE_THP? > > Yeah, we ignore that too. My rationale is -- as you said -- using the > PMDs is actually free, and it's really none of the app's business how > the page cache chooses to cache things. What should be done to be consistent with the collapse side here, for file/shmem if at all? Answering the question, "can this memory be backed by THPs" is becoming really complex, and that THPelligble smaps field is becoming increasingly more difficult to use.