From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8CF3C61DA4 for ; Mon, 6 Feb 2023 06:26:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 16FC16B0074; Mon, 6 Feb 2023 01:26:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 120A66B0075; Mon, 6 Feb 2023 01:26:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F29B96B0078; Mon, 6 Feb 2023 01:26:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E2E236B0074 for ; Mon, 6 Feb 2023 01:26:27 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id B9899C0AE1 for ; Mon, 6 Feb 2023 06:26:27 +0000 (UTC) X-FDA: 80435882814.15.13CDA5B Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by imf06.hostedemail.com (Postfix) with ESMTP id 6CFAA180004 for ; Mon, 6 Feb 2023 06:26:23 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=AJUeTeMu; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf06.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675664786; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GLOH9wrODcfZhv9MjjNkH16hW6QEfXrp+tnYYMVcvLc=; b=qSKgFK5GL8NAO2NtxafME2L5xRW7xmHt4PdJBzbKdKTFUtucTLoqRoPTI1YrzRDx/Pw8kV lwYq7hMAMpyTNHfy25u15lzdAkki/zRTUL75zCseHkvWLxGlkqFOXwUs4jxJ4QD8jJYvKm 1tePV7yoTGAX69R/YUZlLeXYYZULW1g= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=AJUeTeMu; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf06.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675664786; a=rsa-sha256; cv=none; b=BuOEga9ntO4yD4dv9KEqfFc5froFBqObE4JIm+vtCEKuXAw8H48Cm1aBKAAR0UV9KxhISq 6MV+f2Rt+i0Af32SsQI9fiQKxdH2YAO55hdk2pdEWw4HhuRoN8YISj7dl3a0D77RR+SBCf iGm77q0YCVkDnMkoYKds097HTqMYm5M= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675664783; x=1707200783; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version; bh=CPUdhOmYI0PiN1vzkVZ1xV+YwRyaTa/BPUNr02/MIy8=; b=AJUeTeMumJhZ+0WiJGDW5INn3VUDmJNAHWy8BO+kzx1/wTOZpjpjD1mz CJusXe+1ogpAC+FE1fZNpm5yDpRem/b2+3xcmdLOUXZ4FOHiA8UmWXoaG /k+L7Ikm7ZfZJWHthO0IZlvR9/EkX6pZIubf7PoJkpSea3M8U9jFrVMBx sWVDBC/Rn7xB6ujeOFvM8e2+l04TiubqJ6D8C1aDbjOC/+jD9XdcQyEXn U/uqCGK/Sukff75jfQwI3UfD5WvTB2ili3Wc6QYqmaIbVWS2WFx6jHtgj YzupwMHVaTtgEdDPxqgbyyA7ldZooJHI0iWNTaPhb3gKzUJN4GtuoKV4d A==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="329154237" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="329154237" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 22:26:21 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="729913587" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="729913587" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 22:26:18 -0800 From: "Huang, Ying" To: Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: linux-mm@kvack.org, akpm@linux-foundation.org, Zi Yan , Yang Shi , Baolin Wang , Oscar Salvador , Matthew Wilcox , Bharata B Rao , Alistair Popple , haoxin , Minchan Kim Subject: Re: [BISECTED] first bad commit is c203c6d5b3f0597 ("migrate_pages: batch _unmap and _move") References: <87h6w5othj.fsf@yhuang6-desk2.ccr.corp.intel.com> <87wn4zmzd1.fsf@yhuang6-desk2.ccr.corp.intel.com> <874js2n65l.fsf@yhuang6-desk2.ccr.corp.intel.com> Date: Mon, 06 Feb 2023 14:25:19 +0800 In-Reply-To: (Hyeonggon Yoo's message of "Sun, 5 Feb 2023 14:38:44 +0000") Message-ID: <87ilgfjoog.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 6CFAA180004 X-Stat-Signature: a6ur17dg8n4wshzciq5g5fm5wgjdbmga X-HE-Tag: 1675664783-713800 X-HE-Meta: U2FsdGVkX1/VWEVJtVpgVC/kwXCYL37PBcbymYqfZK69x7JWtyfxuuYRu7NjCtGB8qsQxO0/r0nw8SKGcAHNLG7e3HjPrL28Etwbno9+QKx10RY/6ZB2y9taZ7rwO0HzKixu7CwY9BZmbpI3YkHdUu+nXWw2AFh5RfZHxALQiIiRovF/wzVv+PXoE8UTf0OojP7y/CidX1NSs5CoC9Kma1YyG+HEza7NL3oCncuDTDzAs9VGbmrkk8Ol0SX7JjvImpurxWvEz82b3xA1LBXW1U+zrPbhVBfzx9+KhD+QW4hNILZ+GUNnPIHRbgY+qRdikY1ckbVjFgFFso4m2CkPt9xnRdod254A8Iwv+U0hXPk+lIGNw+foEbOH78QJlLwuT+PvhsbPKzOthwAY38vq4V+uPyb5D/pen1ZRja9egOi+eB/9TG3txsukgmo1/FpKQqorqbaRdfJM0BSCI0flekmBCv3bMC7YLDbJOhmpwOpHzIHDpXNqg1p1DBRG8Auw+Ck43QSyiz0ioQWFva1QXMCK93Lec6VxnpSvbZgnmnc2Q6XbbfycDGHTYI/Tpkrnh8aneF5cGLLxqChMPZ+GnhttAJDqrV38HzXjjs5jvcYTcz7My/v5vSRShAknIDiUwxZ7dUVnzcBUY6oavxQeTvWi2U7qCcSfEUnSGjrKhRn/kKxj/QjIfAq+oEO19Pz5ZOBp6sbHw572S7hA0Q+mCO3NWGxWfZU0rTd81z1AfXx6IRy4xma62kyAyStSHiEkn8/tWEK/5Y1yY4jaJk/zeJ4Ir8pOe5slcj8AmgCT6DufPYnrsZjsMfUCSbq+GhzFEzEd58UzCxmIeHoVQce+HfaNv6IbAQeVH0voPtJX/K6Bysyg0Jv+yN4s1pOeVta6ZlfTILNUKxwdkJA730DL35VaWcBhn7uqFRmeX6S2eS4UIntjw0UhAgdreyTk3dUFOOP2ks2wqmW269I/4vH Lhbdxp16 BHkupFAfna0Z9LfyUImGPF38JG2/Ub1EQOiWIrSXh7vfIkevqmhDvB6lUMP+A0v/UkmV9NuDP1C8FAIfSO/Qv3J6r7ElPpqEDmtA75Cw9gGqorkgiv7PrqNv+lfrsoosmvZ3xxncU98zcIz1YuQIiCFgDmCADKLpm9mv40vqogQIsm7YNKMnSQVyoq6L9c1EftHmaG8uGZJGQhRdyvqQTLs7eB7qvsZcD/nC4mcQQt2k9AAV2uqp5posJv36fxXE0Ol0c6kP27oP6QCi26yfzMoCSx7bjJVBaC4DY8L2h1ZJXFVAj+LpgneYadqAxPHugfKlByC89k8fv5s/1J3IN8ckuvaFfd2FIUqztf/QKdNRevjWmhHisNFFLiHBousglKfqc6AdY4+fZyEqUZYTD7cJcpZHKX8RuF1YD X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hyeonggon Yoo <42.hyeyoo@gmail.com> writes: > On Fri, Feb 03, 2023 at 11:02:46PM +0800, Huang, Ying wrote: >> Hyeonggon Yoo <42.hyeyoo@gmail.com> writes: >> >> > On Fri, Feb 03, 2023 at 07:17:14AM +0800, Huang, Ying wrote: >> >> "Huang, Ying" writes: >> >> >> >> > Hi, Hyeonggon, >> >> > >> >> > Hyeonggon Yoo <42.hyeyoo@gmail.com> writes: >> >> > >> >> >> On Wed, Feb 01, 2023 at 01:09:10AM +0900, Hyeonggon Yoo wrote: >> >> >>> I've observed random list_del corruption on mm-unstable, >> >> >>> where HEAD is commit d69862e693c069f4 >> >> >>> ("mm/migrate: convert putback_movable_pages() to use folios"). >> >> >>> >> >> >>> The issue can be easily reproduced by stressing MM multiple times: >> >> >>> stress-ng --bigheap 0 --timeout 300 >> >> >>> >> >> >>> The compiler is gcc 12.2.1 and config, dmesg are included as attachment. >> >> >>> I will try to bisect but can't promise quick resolution :) >> >> >> >> >> >> >> >> >> The first bad commits appears to be: >> >> >> c203c6d5b3f0597 ("migrate_pages: batch _unmap and _move") >> >> >> >> >> >> the first bad commit _probably_ be earlier, but this is quite >> >> >> easy to reproduce so at this point I think above is the real bad commit. >> >> > >> >> > Thank you very much for reporting the bug. I'm in travel now but I will >> >> > try to find some time to reproduce and debug it. >> >> >> >> Still haven't reproduced the issue. But after reviewing the code, I >> >> found a bug in the code, which may cause list corruption. Can you try >> >> the debug patch below? >> > >> > Unfortunately my home server seems to be broken again :( >> > That means I only have access to VMs and not a real machine now. >> > >> > FYI it was not reproduced on KVM but reproduced on real machine. >> > Could you try checking on your machine with the config I attached? [1] >> >> Thank you very much for testing! >> >> > Sorry to bother your travel! >> >> Never mind. Your report helps me very much! >> >> > [1] https://marc.info/?l=linux-mm&m=167518135116956 >> >> I have reproduced the bug successfully! And I can reproduce the bug >> with the previous debug patch too, although the reproduction rate isn't >> high. >> >> And in my test, the following patch can fix the bug. >> >> It appears that zswap code will touch dst->lru during moving page. > > After setting swap space I was able to reproduce even on VM. > >> -------------------------8<---------------------------------- >> From b2e3f4aab16d8af0033286fde669b46e7467c7ec Mon Sep 17 00:00:00 2001 >> From: Huang Ying >> Date: Fri, 3 Feb 2023 22:03:24 +0800 >> Subject: [PATCH] dbg,migrate_pages: restore destination folio state before >> move >> >> --- >> mm/migrate.c | 15 ++++++++------- >> 1 file changed, 8 insertions(+), 7 deletions(-) > > > This fixes the bug on my test: > > Tested-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> > Thanks for such a quick fix! Thank you very much! >> >> diff --git a/mm/migrate.c b/mm/migrate.c >> index 143d96775b4d..fa7212330cb6 100644 >> --- a/mm/migrate.c >> +++ b/mm/migrate.c >> @@ -1225,13 +1225,19 @@ static int __migrate_folio_move(struct folio *src, struct folio *dst, >> int page_was_mapped = 0; >> struct anon_vma *anon_vma = NULL; >> bool is_lru = !__PageMovable(&src->page); >> + struct list_head *prev; >> >> __migrate_folio_extract(dst, &page_was_mapped, &anon_vma); >> + prev = dst->lru.prev; >> + list_del(&dst->lru); > > BTW may be silly questions, > > - How can zswap touch dst->lru during moving page, is there no lock > that prevents this to happen? > > - Does this race (?) happen only during moving page? > (I mean, why is it safe to perform list_del()/list_add() before and > after moving page?) This isn't a race condition. In the following code path, __migrate_folio_move() move_to_new_folio() mops->migrate_page() // z3fold_page_migrate() list_add(&newpage->lru, &pool->lru) newpage->lru will be changed during move_to_new_folio(). While the original code assumes that newpage->lru will not be changed. Best Regards, Huang, Ying >> >> rc = move_to_new_folio(dst, src, mode); >> >> - if (rc != -EAGAIN) >> - list_del(&dst->lru); >> + if (rc == -EAGAIN) { >> + list_add(&dst->lru, prev); >> + __migrate_folio_record(dst, page_was_mapped, anon_vma); >> + return rc; >> + } >> >> >> if (unlikely(!is_lru)) >> goto out_unlock_both; >> @@ -1251,11 +1257,6 @@ static int __migrate_folio_move(struct folio *src, struct folio *dst, >> lru_add_drain(); >> } >> >> - if (rc == -EAGAIN) { >> - __migrate_folio_record(dst, page_was_mapped, anon_vma); >> - return rc; >> - } >> - >> if (page_was_mapped) >> remove_migration_ptes(src, >> rc == MIGRATEPAGE_SUCCESS ? dst : src, false); >> -- >> 2.35.1