From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27F91D49789 for ; Sun, 1 Dec 2024 21:22:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4538B6B0082; Sun, 1 Dec 2024 16:22:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3DC0E6B0083; Sun, 1 Dec 2024 16:22:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 256706B0085; Sun, 1 Dec 2024 16:22:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 0707B6B0082 for ; Sun, 1 Dec 2024 16:22:50 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id AC58D1613AA for ; Sun, 1 Dec 2024 21:22:49 +0000 (UTC) X-FDA: 82847664606.18.B55EE0A Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf05.hostedemail.com (Postfix) with ESMTP id 4D503100018 for ; Sun, 1 Dec 2024 21:22:26 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=GL7y15bN; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf05.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733088157; a=rsa-sha256; cv=none; b=dqPZW4MhKqE6KsfBaWLd8q4HLpN6AJE33tgN/WOAVoynh3HkTAj+Ahtu9bAaMa04XPpk6p bt1KcqOdf7ZpyiHG9oSztVjF+9rxym+3gE9E6noKj4w5bUZWbvIlFm9kcPuDG/uxAvBdv9 hs0bmNzuDmDAOL8idgAjBo2xA2VWIy4= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=GL7y15bN; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf05.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733088157; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=YM8DkKfRb8IpPKMV3s7yyQAleOKBXGqJpJ7QJnlSE3c=; b=ZSli0677gItMSa4hPNBZeqkNkHC3z9Ux7bQULibRsxcnyE41wldpyldfHPyCJfG+M8LpOO j0Cyo1j4AApq5MvvgbvLCp5UUa2DiHc4U05cJcHiZL+/tDXbHK728T1pHBJQEsfWdGjhFv HTyTa3aIe21hX3mO4CWf4rAdoL9NHjk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1733088166; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=YM8DkKfRb8IpPKMV3s7yyQAleOKBXGqJpJ7QJnlSE3c=; b=GL7y15bN0y5PncoiNefF2ncuAoSKi6Ud9VRLpCWVfpbTTGhx66xHEuGv6M7NVzdlvcPAy8 WTuMm1XQKMHXAFmKGASIcwSUK2VwkMWQYJNFFWgusRiE27Fi1NP2fO9CQi1MF7xKL5HADc gUY8eyMy/EigXYQ9rnR0YfjNo9z2hVA= Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-312-wll2-82ePNe5mXYT_n0THg-1; Sun, 01 Dec 2024 16:22:45 -0500 X-MC-Unique: wll2-82ePNe5mXYT_n0THg-1 X-Mimecast-MFC-AGG-ID: wll2-82ePNe5mXYT_n0THg Received: by mail-qt1-f200.google.com with SMTP id d75a77b69052e-466a500d7b5so56280921cf.1 for ; Sun, 01 Dec 2024 13:22:45 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733088165; x=1733692965; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=YM8DkKfRb8IpPKMV3s7yyQAleOKBXGqJpJ7QJnlSE3c=; b=sB5/D1dm0+3bzisGslUKhfQVzmlZvXEIQH7o8FClqOYY1fhbu5xRAdIr+obn8lz1Jx COT/DwtdCHxJvg59uwNeg/m2g7jLUHaUhMkb8Xt0c7sMjcVo9I7s7AXFEwKg2v/BiBRP pWf7bo91oZU6zssO3eXqr994SySv36O/BkkTswzGMeinU8t/dyeP/i+fzDWvQAEhSlPF gLBoLkOD/CYtF6gHh0D1zauobeVDCpSMmaxiVEW0kpM7SM2Wm5K4H0ULO6nD1VG9eeME YwVN1y66NK44k79pttfx57Ds6ANk2whvIi9AfTktyTpdllttXO3HPDwRRf/mYrCATDFY uhoA== X-Forwarded-Encrypted: i=1; AJvYcCULyI9eK+LCLJwL7MrAFxOsVH3opPHAVKhzUk7QIwnkDdK2A4qB+HlIz5nSMsH2FUpQ8od0WgQxLQ==@kvack.org X-Gm-Message-State: AOJu0Yy8bPY/GMSLmjpyHOMKoKDZyYOxGdfvua+RXGp3YI0Nyu6kBTCk VBEbAr/7s8GxnJ5/kGXewlFpd4XayWbCaxSWoeGFue2l0RPc6yRO653XJXDJK0Vwxexz8D5CKNK dsInF2DDizfh4ij//M5SjBE4/OL/8O7gM+XuyeFiSRfRlBEe8 X-Gm-Gg: ASbGncvPAz9YjL3u/kRgtlP9sE8OtYCK09sOnqhbrnwcNd6nQXJCTy/GZtrZPlWXm5p eWtbJn4iVBr49PZO2q28P2DFERq4OVcpJd91fqfWgX3NU2cakagIpgToLQGsM1UNSQRda2XAbXU J/pl8nchq+OGlfLWdw0Mf6r3fmp4x+VFdbdEYvFJjtkJgWYJaBYZAZbNxHqy3PiF09vjDDcI4QP dTxJf7fr5FL2NaT6LgS2RGi2QWOYlNBjPQznbyPJx+6x4gB5NT6bsuBR+l4TTlosYe6k6YOUdms yTacqr882pXJcuCA4GrXJc8b9g== X-Received: by 2002:a05:622a:5817:b0:461:3834:b632 with SMTP id d75a77b69052e-466b34d2ed9mr269144491cf.1.1733088165078; Sun, 01 Dec 2024 13:22:45 -0800 (PST) X-Google-Smtp-Source: AGHT+IEYMfFUKKqY8ldvyra2Aod+R5tafce9p1vcixhiu004Ft5YinVMtwa9M1qGrL758c7rfXnD3A== X-Received: by 2002:a05:622a:5817:b0:461:3834:b632 with SMTP id d75a77b69052e-466b34d2ed9mr269144211cf.1.1733088164736; Sun, 01 Dec 2024 13:22:44 -0800 (PST) Received: from x1n.redhat.com (pool-99-254-114-190.cpe.net.cable.rogers.com. [99.254.114.190]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-466c4249f0asm41278911cf.81.2024.12.01.13.22.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Dec 2024 13:22:43 -0800 (PST) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Rik van Riel , Breno Leitao , Andrew Morton , peterx@redhat.com, Muchun Song , Oscar Salvador , Roman Gushchin , Naoya Horiguchi , Ackerley Tng Subject: [PATCH 0/7] mm/hugetlb: Refactor hugetlb allocation resv accounting Date: Sun, 1 Dec 2024 16:22:33 -0500 Message-ID: <20241201212240.533824-1-peterx@redhat.com> X-Mailer: git-send-email 2.47.0 MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: xFJVcPWmNXglEEfSs94SB5-nexqVRM8tw2C9WtxR2Iw_1733088165 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit X-Stat-Signature: twoh8noc6aqgf4j13tqak73r34y1csqi X-Rspam-User: X-Rspamd-Queue-Id: 4D503100018 X-Rspamd-Server: rspam08 X-HE-Tag: 1733088146-433974 X-HE-Meta: U2FsdGVkX19LtbenqpwRwwgOKXgPA+Nu8oBQcaHsgcs+xekCDma2+iqMPHRUnXm66cD8sxMQI+2yWWxXMtbglJjiX+bP+H1mSMXzUdW77edZelkKrNSmQIpOVFdp7T9wsdDt5+B7Pp8MYUjrxZhWKm3PHBckvNXkmx0eHsE0COUtDPFyQ9Kce0rGIN2CQyT72zIibM7uq3AR1nwxtqxSW5R7sh0kAop4AQ7emNTqx1EjgzNhC9GTz4T5NqsSvsCZYT4qtJbgePBhZvCiqQ00kTwtAL0LnAOtYtdO3qCgyP5CLzbcOCPwBdXW+Lrp6hp8svCb8mKWFgyaoZENmFVGHWNqHlj8Dq6nq/9EOXVIftNU/ryPTqVajT/rkpCb3f5v6tULUkfvWJ63kmWPPW1b/MdlIzWNRKJI7EITyXMcRDKApe3hsl9cK0zEJEjVVkuYkDVLl2EA/eDZQadwFMb5M4HafzpyAxfWSN6N8TKeadssguHCEpiOjrOffLxFK4GAIlIluRBv6bN1BI/wT53iWesONJ0y5hLHundxgkM6knjOeP7mhcgpB0NHCUyJfUZTYW4NjV26cTEd9boXGRHBjCitwNZfpQmYFE11Bwo6xDrbdWj0zzU2kjOOrjsGbFxZRLbhP+lPFYfuFKu/q7+gH9vFTV/hJKqZEvTIZA0HgLiM6mBvsYZD4Jfk3zi8vIAxjdrSWTHh8B3hCmBUocIBITmchH9BcIsllXrxbekV2ABBlSkaNZuZv8ri2fwJMl4XMltSBXdRbv/zbmJXmKx9h2ZeWiyqS4XO7yn3kkYgjc9hbabstGkMz2x5TiCJy/89N2PSgiQ0isvd5NVd3edcdvTDgZaBKUNxydu/ePku+7x6LVG8zqwjE+dSmosAqh4hfMTE9RHpTnVeqSyYOchjQDc9kXO3eEcBTpYNGffMGLJgPRjf+rd4IGwuENb7mZU176J6MYJCPtlKF2WJBhB eIi5160c ljZit/DRrUYlJ6g3VPokxcasYVktiVQJ2MHQGNLe+DAJ9nUh7J9mGtJsZDZJxGt4kHQdGuMLclgGEftRkqLtfuYvGHAGtcehqqDKPFylzXf6B3NMUZnRenqEfOCo2JipPMZqspGNj/K7C815FvmyCEsjdW8BNyuf1ApTL/IdoIOgWkcc1cz6ppWQOzgf+2ftRq2rN3c8hjLBA/25H6GDIeEnLxDKs+eQQFK6/pdBgEIAkDPS+frZhiNTBL+jrrRiVmOqHTPFg47cgkXhpCq7W0nQpNThjZ69XKxBocgZyB0F8SxCfMtXqi2R8DaoNZ9odXxhePvessyN62BCbZV9F0A5iSxNgi/z1WwKXE5u/3yk2q6eiFMnetAdBqDEDC4TKf+VNzH72tHVu2wkWjY7cbaeaS6CnJzeGa6k4sgGmTS+GP+1ZDdkMaNaJD3KhRvItMA3dI4wCrJZ9mKw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: [based on akpm/mm-unstable, latest 8351c1503010, of Dec 1st 2024] This is a follow up on Ackerley's series here as replacement: https://lore.kernel.org/r/cover.1728684491.git.ackerleytng@google.com Ackerley, I wanted to reuse some of your patches, but when looking at this issue I found a bug which is described in patch 1. I'll need to have that patch to be the 1st patch of the series, and then I also found this is so far the best way to layout this whole set. It should have gone a bit further than what you tried to do in your series, but I assume many of your gmem 1G patches after that will still apply on top. The goal of this series is to cleanup hugetlb resv accounting, especially during folio allocation. It paves way for other users to allocate hugetlb folios out of either system reservations, or subpools (instead of hugetlbfs, as a file system). So for the longer term, maybe there's chance to use hugetlb to be separate concept v.s. hugetlbfs, which I hope would work. Going back to this small (not so much..) refactoring series. It touches the (probably.. hard to read for most) hugetlb resv code, try to make it more readable, decouple things so that it might be easier in the future to allocate the folios without hugetlb VMAs. Tests I've done: - I had a reproducer in patch 1 for the bug I found, this will start to work after patch 1 or the whole set applied. - Hugetlb regression tests (on x86_64 2MBs), includes: - All vmtests on hugetlbfs - libhugetlbfs test suite Note that I found libhugetlbfs test suites can fail on some of the tests, but it doesn't look like to be caused by this series, as I can get the same results when I run the test suites on either akpm/mm-stable or v6.12 tag. I didn't yet have time to look into all the issues, but the current guess is those issues are separate from this series. Comments welcomed, thanks. Peter Xu (7): mm/hugetlb: Fix avoid_reserve to allow taking folio from subpool mm/hugetlb: Stop using avoid_reserve flag in fork() mm/hugetlb: Rename avoid_reserve to cow_from_owner mm/hugetlb: Clean up map/global resv accounting when allocate mm/hugetlb: Simplify vma_has_reserves() mm/hugetlb: Drop vma_has_reserves() mm/hugetlb: Unify restore reserve accounting for new allocations fs/hugetlbfs/inode.c | 2 +- include/linux/hugetlb.h | 4 +- mm/hugetlb.c | 243 ++++++++++++++++++---------------------- 3 files changed, 110 insertions(+), 139 deletions(-) -- 2.47.0