From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84AB0C282D0 for ; Tue, 4 Mar 2025 11:49:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F0B766B0082; Tue, 4 Mar 2025 06:49:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EBC706B0083; Tue, 4 Mar 2025 06:49:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D84236B0085; Tue, 4 Mar 2025 06:49:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id BBE0F6B0082 for ; Tue, 4 Mar 2025 06:49:50 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 9C9435527A for ; Tue, 4 Mar 2025 11:49:48 +0000 (UTC) X-FDA: 83183699256.26.5FB3610 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by imf26.hostedemail.com (Postfix) with ESMTP id CA4D8140003 for ; Tue, 4 Mar 2025 11:49:46 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="A/fkfMd/"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf26.hostedemail.com: domain of hughd@google.com designates 209.85.214.182 as permitted sender) smtp.mailfrom=hughd@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741088986; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fPlb+RHpjvuAHo7jM9mxzGvumfLeY4uukzr0IOz2olo=; b=ajx0JtlU248cnHtuMQEmp8nAy6QMiStyfxks79ajgbkhLgZuoWB6Q965LceR2GW9HG+8gW B1y2YnlIqnqVULAEt5HYfVVh5PcorSHho2O/huEBKzyxALII0AADan/upOH24+mjnw43Pm qB0LzPIi9BNGDhBxeDPeSVcswKklGnw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741088986; a=rsa-sha256; cv=none; b=O2pMAJWUEimD8W67Kc7vJZTch7XHh4wrxU+7579LWkZTVPht0TV7WuJhxBUxnRY1cgxCsg 0UsLu7fKJPeVVVn1Y4QHVEym8VsovN/AaC2WPNDi1ZhG6JJf0YLSrbaYjmYNT/+kGNt9lz hDtAZax00p2gJV05Ocs3JZC8RNNw9rU= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="A/fkfMd/"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf26.hostedemail.com: domain of hughd@google.com designates 209.85.214.182 as permitted sender) smtp.mailfrom=hughd@google.com Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-223a7065ff8so56610895ad.0 for ; Tue, 04 Mar 2025 03:49:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1741088986; x=1741693786; darn=kvack.org; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=fPlb+RHpjvuAHo7jM9mxzGvumfLeY4uukzr0IOz2olo=; b=A/fkfMd/1afuF0LV+Z7Z8oAZvQ+aoe6p3trK+5HfSlknQci39xK0j0ovri7hf1ag9+ oOgw4UUUm2KdRRaRnr+tMw8qkdT/mC6L4ObvZ3KStAxXDq/kNmiiMlkJ71ld4qzCz5e+ Lt3nCKzwzJnLifcdwOxB8R2OZ/Mad1PNNddlj4YFWlV8oh6Lskqiu1qjPTrYwGvf34FN qfNsFMG+4QvAfIZ3kgB+nEvzfMQ3LaOWy915kJLp8+gFMFlT2pXGPzqUpX1jXwO0L5cP t2EhWbbhiXzOkbArWp4GWP88h5Inpqxe/n91GycWIXPOKyny7Xaee/pU6/II1ozoDBku BYzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741088986; x=1741693786; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=fPlb+RHpjvuAHo7jM9mxzGvumfLeY4uukzr0IOz2olo=; b=dlADMm+0hmMBefymzNYFuGZ6B0wXjTsJSF9+gKGA15KD0RBs19Pa98Gt1/SLrlKubh CVwZiT1s50JW/wrHLbfe1GX5boZL4QErs0JhEVzeit/5Vy3j5rAClLEH1u16vcch6RLU e7aHM/1DBF7zDhg7q+ztABBllrs3FSCSUFTJT6HDy22r1EWo5+mc9/xtVIXxZwhst5B5 e22MQSJ132guML2I7IW35aGoJxOciMF+luvIV05YQdsXOsxPqNY1hnFUIhcRW84aj2dY 85EtgMOTcXEqGIHuiDA+wqcMgt8j04xvLornCuGRvalRPC/40NY5zbRUXEo/qDtDCWHN 4CKQ== X-Gm-Message-State: AOJu0YxNa8isIKG/P9qYygEfQ72WaH88S6cT39VZ/m7yvmT/y8xJD/6g C+jTrWBelpwjeAEhmayiweQCkmiIWFUaqNiWd4FJepw9JEAAhS81wTFFcZBhog== X-Gm-Gg: ASbGncvcsJ0u5upYIo+ziQ2sSztcWLRrKyz5rf3HpV1AiLgmgziQ7nyx2RRam4YCi+b bK1pSDNX0zoge5Y/lFX2weW5LHFKsl99NqWj77kdL+poVGYudRHTzsiA0wjzz8zdQJuFTbaW4Fu qOgMHSgEEDE5TFINuOi0VpnP+MFq0qgC7xR1KFij9FfiIZuRdOBB3QE1BW+EIhASe7vMvwTyvVy A3exTuj7DxISyUsYl9kV3g1+eas9bBmui3EFEoGTT0uWtWHijgsidDrsWO8uXknCVxvtJiAbfpd DImnQdld2a/orquSmY6p6TBISqoM9JATPCbFcnsB8qyHqsWpO3Xw1LenMxNEu9igcPFCvUEMo5Q PMjb/B8FGzIcsm0SQeGuf8R66EM3G X-Google-Smtp-Source: AGHT+IHnUPMc2gQCmddqJdZbbCKqjAWO12cSIqW4RrCn3vAmDjlonjV/HmPGvIt1+pbSRHmaOhbgoA== X-Received: by 2002:a05:6a00:4f81:b0:736:5545:5b84 with SMTP id d2e1a72fcca58-7365545674fmr13070813b3a.3.1741088985318; Tue, 04 Mar 2025 03:49:45 -0800 (PST) Received: from darker.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-73638ab100asm6602885b3a.58.2025.03.04.03.49.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Mar 2025 03:49:44 -0800 (PST) Date: Tue, 4 Mar 2025 03:49:32 -0800 (PST) From: Hugh Dickins To: Zi Yan cc: linux-mm@kvack.org, Andrew Morton , "Kirill A . Shutemov" , "Matthew Wilcox (Oracle)" , Ryan Roberts , Hugh Dickins , David Hildenbrand , Yang Shi , Miaohe Lin , Kefeng Wang , Yu Zhao , John Hubbard , Baolin Wang , linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, Kairui Song , Liu Shixin Subject: Re: [PATCH v9 2/8] mm/huge_memory: add two new (not yet used) functions for folio_split() In-Reply-To: <20250226210032.2044041-3-ziy@nvidia.com> Message-ID: <2fae27fe-6e2e-3587-4b68-072118d80cf8@google.com> References: <20250226210032.2044041-1-ziy@nvidia.com> <20250226210032.2044041-3-ziy@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: CA4D8140003 X-Stat-Signature: s36pakysw1rbf8m6u31gik5wqhregde7 X-HE-Tag: 1741088986-463046 X-HE-Meta: U2FsdGVkX18wYqin+4D51I0pOWLlg2psOYO6yk0LpBYTf/8OeQNtX2pi1J4K1JhlhBwoA+Q3nLi6yYOLRhpkwgPVnRqUpy1y3MuTy+CG9NLxuzKoHB3bhPL+hBofR410qso66RDaJJEPbPwuoEUoO+2LjGp+FmENtj/+5Uc/81kctrUyuZ1ass7ub1xOm7yb2R13qRVa5CJm3Iawx76y3gozlV8Ew2BD7oZtyw2Jt/xF1NGdMoAvCmEoB+nUEpXPO+/cBD2Dv2f4UsKbuBHLUaPeWMCL2gqaKc1Xtq8XiA3ByRdwNDJ7A7xAPzfhvBZwCU6SQlhQEQYRi+4dDED2tip6kf2oR+wSiYgbcvgMOqCKhaQdeUFOqv3+9g5A4VMkF6774m5O4i62lDxQqpfUMKc9yrL7hfFC9fAtz7/ll1/gGIGN9rz0iVxXrz3jpvjacWqNTWfcEE06+mCP7j9uXsPk+m7jPBGshP7JbRC0IpNG3aOFeZiBcGrLyHwMNIhGGK2upSiwYKEbrZ+h1ew2chcnO43a6L05BNJUY+wzcBF60YOqEBM4McBYjlB13vo0ObWG/W6+ogp1/0hoxS5nSwO8SWxFLbDR6T7RGa1MQ7xBxR5JyNYwauuudRGcuagqNZ8lyjvghJnoxAHcPGH855j+4+X/oZxe0my2zHHkIJakiwzvd2yINGYm6C9CUKrNOsBhKsW08t3tmHyvpGKjxc54JZb1rqzjK3kiblFQicwh7/1U2VnKZxLx9nyYOQ7xY/lkQbDlqzTtRB4qXSQaQV9wBT0O4//YcOGNebcdZGUTee01GanrV3/0xJeEtLQIjh+pMR8s6/H3N2RqtEs+GpPCJlDR92CzEFpYHf3VSDJ7f2EymkonGIm4sIWP/EUeDKIVrrP5MCS2YVo229DNt14DWc+PbJQTqJKSXWoxaRgrLcuh/EC09ZhnZq5EbcC19cW8i6yGlLugFkreW2i f0GLqvfn GQeNY3Kuc5Rs4a1aQPSiIdRnhxFfpRMKfhJ4sNg0lJEdD+/ylZsUHmpJ5IRNu0CTtpKKrrXRpfH+gLFR+JfqMCXig6nduJIOrMEP4hnfZrQX/PPJ2maplkaAXjEeOVmHONFEIO3UgCq2x9QZ+n2LgPtlZuqp/YgXWB61B4r86MAGGeVz/t4fm9c6gq3UGQwHDh1yYcIP3ivQso7A4BcKpGdRJ7oJMOk7NYNzoKs6kG8cbG/7jrwZPF7enh61iueSv2fimJDHT37q7JhuSm19zZ0qY7S2NtV1JF01I9Oq81AuEgsjYzojAI/4n/td/tQtMR8TF9P6+xhF3OvA9ynMc/Eq35uSaP11q8Bz7+/UOtrFRSfsF8rBcJnKmjQeuMBYxjaNRSGZqZ+SYx9s+JwsfMzakF6w8Aae9U4w08PxY8i1NW88= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, 26 Feb 2025, Zi Yan wrote: > This is a preparation patch, both added functions are not used yet. > > The added __split_unmapped_folio() is able to split a folio with its > mapping removed in two manners: 1) uniform split (the existing way), and > 2) buddy allocator like split. > > The added __split_folio_to_order() can split a folio into any lower order. > For uniform split, __split_unmapped_folio() calls it once to split the > given folio to the new order. For buddy allocator split, > __split_unmapped_folio() calls it (folio_order - new_order) times and each > time splits the folio containing the given page to one lower order. > > Signed-off-by: Zi Yan Sorry, I'm tired and don't really want to be writing this yet, but the migrate "hotfix" has tipped my hand, and I need to get this out to you before more days pass. I'd been unable to complete even a single iteration of my "kernel builds on huge tmpfs while swapping to SSD" testing during this current 6.14-rc mm.git cycle (6.14-rc itself fine) - until the last week, when some important fixes have come in, so I'm no longer getting I/O errors from ext4-on-loop0-on-huge-tmpfs, and "Huh VM_FAULT_OOM leaked" warnings: good. But I still can't get beyond a few iterations, a few minutes: there's some corruption of user data, which usually manifests as a kernel build failing because fixdep couldn't find some truncated-on-the-left pathname. While it definitely bisected to your folio_split() series, it's quite possible that you're merely exposing an existing bug to wider use. I've spent the last few days trying to track this down, but still not succeeded: I'm still getting much the same corruption. But have been folding in various fixes as I found them, even though they have not solved the main problem at all. I'll return to trying to debug the corruption "tomorrow". I think (might be wrong, I'm in a rush) my mods are all to this "add two new (not yet used) functions for folio_split()" patch: please merge them in if you agree. 1. From source inspection, it looks like a folio_set_order() was missed. 2. Why is swapcache only checked when folio_test_anon? I can see that you've just copied that over from the old __split_huge_page(), but it seems wrong to me here and there - I guess a relic from before shmem could swap out a huge page. 3. Doing folio_next() inside the for(;;) is unsafe in those configs which have to look up zone etc, I got an oops from the "new_folio" loop; didn't hit an oops from the "release" loop but fixed that too. 4. While correcting anon versus mapping versus swap_cache, shortened the lines by avoiding origin_folio->mapping and &release->page. Signed-off-by: Hugh Dickins --- mm/huge_memory.c | 39 ++++++++++++++++++++------------------- 1 file changed, 20 insertions(+), 19 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 0e45937c0d91..9ce3906672b9 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3612,7 +3612,9 @@ static void __split_folio_to_order(struct folio *folio, int new_order) folio_xchg_last_cpupid(new_folio, folio_last_cpupid(folio)); } - if (!new_order) + if (new_order) + folio_set_order(folio, new_order); + else ClearPageCompound(&folio->page); } @@ -3682,7 +3684,9 @@ static int __split_unmapped_folio(struct folio *folio, int new_order, int ret = 0; bool stop_split = false; - if (folio_test_anon(folio) && folio_test_swapcache(folio)) { + if (folio_test_swapcache(folio)) { + VM_BUG_ON(mapping); + /* a swapcache folio can only be uniformly split to order-0 */ if (!uniform_split || new_order != 0) return -EINVAL; @@ -3750,9 +3754,8 @@ static int __split_unmapped_folio(struct folio *folio, int new_order, * is new_order, since the folio will be worked on in next * iteration. */ - for (release = folio, next = folio_next(folio); - release != end_folio; - release = next, next = folio_next(next)) { + for (release = folio; release != end_folio; release = next) { + next = folio_next(release); /* * for buddy allocator like split, the folio containing * page will be split next and should not be released, @@ -3784,32 +3787,31 @@ static int __split_unmapped_folio(struct folio *folio, int new_order, lru_add_page_tail(origin_folio, &release->page, lruvec, list); - /* Some pages can be beyond EOF: drop them from page cache */ + /* Some pages can be beyond EOF: drop them from cache */ if (release->index >= end) { - if (shmem_mapping(origin_folio->mapping)) + if (shmem_mapping(mapping)) nr_dropped += folio_nr_pages(release); else if (folio_test_clear_dirty(release)) folio_account_cleaned(release, - inode_to_wb(origin_folio->mapping->host)); + inode_to_wb(mapping->host)); __filemap_remove_folio(release, NULL); folio_put(release); - } else if (!folio_test_anon(release)) { - __xa_store(&origin_folio->mapping->i_pages, - release->index, &release->page, 0); + } else if (mapping) { + __xa_store(&mapping->i_pages, + release->index, release, 0); } else if (swap_cache) { __xa_store(&swap_cache->i_pages, swap_cache_index(release->swap), - &release->page, 0); + release, 0); } } } unlock_page_lruvec(lruvec); - if (folio_test_anon(origin_folio)) { - if (folio_test_swapcache(origin_folio)) - xa_unlock(&swap_cache->i_pages); - } else + if (swap_cache) + xa_unlock(&swap_cache->i_pages); + if (mapping) xa_unlock(&mapping->i_pages); /* Caller disabled irqs, so they are still disabled here */ @@ -3828,9 +3830,8 @@ static int __split_unmapped_folio(struct folio *folio, int new_order, * For buddy allocator like split, the first after-split folio is left * for caller to unlock. */ - for (new_folio = origin_folio, next = folio_next(origin_folio); - new_folio != next_folio; - new_folio = next, next = folio_next(next)) { + for (new_folio = origin_folio; new_folio != next_folio; new_folio = next) { + next = folio_next(new_folio); if (new_folio == page_folio(lock_at)) continue; -- 2.43.0