From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B76B1C433EF for ; Tue, 29 Mar 2022 04:02:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1C3A08D0002; Tue, 29 Mar 2022 00:02:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 173588D0001; Tue, 29 Mar 2022 00:02:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 063848D0002; Tue, 29 Mar 2022 00:02:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E74888D0001 for ; Tue, 29 Mar 2022 00:02:54 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 9A5F98249980 for ; Tue, 29 Mar 2022 04:02:54 +0000 (UTC) X-FDA: 79296077868.20.0DD6BE9 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf27.hostedemail.com (Postfix) with ESMTP id AB92B40002 for ; Tue, 29 Mar 2022 04:02:53 +0000 (UTC) Received: from kwepemi100014.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4KSG873GYPzgY7y; Tue, 29 Mar 2022 12:01:11 +0800 (CST) Received: from kwepemm600017.china.huawei.com (7.193.23.234) by kwepemi100014.china.huawei.com (7.221.188.106) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Tue, 29 Mar 2022 12:02:49 +0800 Received: from [10.174.179.234] (10.174.179.234) by kwepemm600017.china.huawei.com (7.193.23.234) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Tue, 29 Mar 2022 12:02:48 +0800 Message-ID: <484e856c-9a57-1696-8a23-75967ee7c291@huawei.com> Date: Tue, 29 Mar 2022 12:02:47 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.6.1 Subject: Re: [RFC v1 2/2] mm: khugepaged: recover from poisoned file-backed memory To: Jiaqi Yan , CC: , , , , , References: <20220323232929.3035443-1-jiaqiyan@google.com> <20220323232929.3035443-3-jiaqiyan@google.com> From: Tong Tiangen In-Reply-To: <20220323232929.3035443-3-jiaqiyan@google.com> Content-Type: text/plain; charset="UTF-8"; format=flowed X-Originating-IP: [10.174.179.234] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemm600017.china.huawei.com (7.193.23.234) X-CFilter-Loop: Reflected X-Rspam-User: X-Stat-Signature: z6ph77dismtiw4eriqr8ik1thdx6m8r7 Authentication-Results: imf27.hostedemail.com; dkim=none; spf=pass (imf27.hostedemail.com: domain of tongtiangen@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=tongtiangen@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: AB92B40002 X-HE-Tag: 1648526573-287418 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: =E5=9C=A8 2022/3/24 7:29, Jiaqi Yan =E5=86=99=E9=81=93: > Make collapse_file roll back when copying pages failed. > More concretely: > * extract copy operations into a separate loop > * postpone the updates for nr_none until both scan and copy succeeded > * postpone joining small xarray entries until both scan and copy > succeeded > * as for update operations to NR_XXX_THPS > * for SHMEM file, postpone until both scan and copy succeeded > * for other file, roll back if scan succeeded but copy failed >=20 > Signed-off-by: Jiaqi Yan > --- > include/linux/highmem.h | 18 ++++++++++ > mm/khugepaged.c | 75 +++++++++++++++++++++++++++-------------= - > 2 files changed, 67 insertions(+), 26 deletions(-) >=20 > diff --git a/include/linux/highmem.h b/include/linux/highmem.h > index 15d0aa4d349c..fc5aa221bdb5 100644 > --- a/include/linux/highmem.h > +++ b/include/linux/highmem.h > @@ -315,6 +315,24 @@ static inline void copy_highpage(struct page *to, = struct page *from) > kunmap_local(vfrom); > } > =20 > +/* > + * Machine check exception handled version of copy_highpage. > + * Return true if copying page content failed; otherwise false. > + */ > +static inline bool copy_highpage_mc(struct page *to, struct page *from= ) > +{ > + char *vfrom, *vto; > + unsigned long ret; > + > + vfrom =3D kmap_local_page(from); > + vto =3D kmap_local_page(to); > + ret =3D copy_mc_to_kernel(vto, vfrom, PAGE_SIZE); > + kunmap_local(vto); > + kunmap_local(vfrom); > + > + return ret > 0; > +} > + > #endif > =20 > static inline void memcpy_page(struct page *dst_page, size_t dst_off, > diff --git a/mm/khugepaged.c b/mm/khugepaged.c > index 84ed177f56ff..ed2b1cd4bbc6 100644 > --- a/mm/khugepaged.c > +++ b/mm/khugepaged.c > @@ -1708,12 +1708,13 @@ static void collapse_file(struct mm_struct *mm, > { > struct address_space *mapping =3D file->f_mapping; > gfp_t gfp; > - struct page *new_page; > + struct page *new_page, *page, *tmp; > pgoff_t index, end =3D start + HPAGE_PMD_NR; > LIST_HEAD(pagelist); > XA_STATE_ORDER(xas, &mapping->i_pages, start, HPAGE_PMD_ORDER); > int nr_none =3D 0, result =3D SCAN_SUCCEED; > bool is_shmem =3D shmem_file(file); > + bool copy_failed =3D false; > int nr; > =20 > VM_BUG_ON(!IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && !is_shmem); > @@ -1936,9 +1937,7 @@ static void collapse_file(struct mm_struct *mm, > } > nr =3D thp_nr_pages(new_page); > =20 > - if (is_shmem) > - __mod_lruvec_page_state(new_page, NR_SHMEM_THPS, nr); > - else { > + if (!is_shmem) { > __mod_lruvec_page_state(new_page, NR_FILE_THPS, nr); > filemap_nr_thps_inc(mapping); > /* > @@ -1956,34 +1955,39 @@ static void collapse_file(struct mm_struct *mm, > } > } > =20 > - if (nr_none) { > - __mod_lruvec_page_state(new_page, NR_FILE_PAGES, nr_none); > - if (is_shmem) > - __mod_lruvec_page_state(new_page, NR_SHMEM, nr_none); > - } > - > - /* Join all the small entries into a single multi-index entry */ > - xas_set_order(&xas, start, HPAGE_PMD_ORDER); > - xas_store(&xas, new_page); > xa_locked: > xas_unlock_irq(&xas); > xa_unlocked: > =20 > if (result =3D=3D SCAN_SUCCEED) { > - struct page *page, *tmp; > - > /* > * Replacing old pages with new one has succeeded, now we > - * need to copy the content and free the old pages. > + * attempt to copy the contents. > */ > index =3D start; > - list_for_each_entry_safe(page, tmp, &pagelist, lru) { > + list_for_each_entry(page, &pagelist, lru) { > while (index < page->index) { > clear_highpage(new_page + (index % HPAGE_PMD_NR)); > index++; > } > - copy_highpage(new_page + (page->index % HPAGE_PMD_NR), > - page); > + if (copy_highpage_mc(new_page + (page->index % HPAGE_PMD_NR), page)= ) { > + copy_failed =3D true; The 1st patch here used "copy_succeed =3D false", It is best that the=20 logic of the two positions can be unified. > + break; > + } > + index++; > + } > + while (!copy_failed && index < end) { > + clear_highpage(new_page + (page->index % HPAGE_PMD_NR)); > + index++; > + } > + } > + > + if (result =3D=3D SCAN_SUCCEED && !copy_failed) { > + /* > + * Copying old pages to huge one has succeeded, now we > + * need to free the old pages. > + */ > + list_for_each_entry_safe(page, tmp, &pagelist, lru) { > list_del(&page->lru); > page->mapping =3D NULL; > page_ref_unfreeze(page, 1); > @@ -1991,12 +1995,20 @@ static void collapse_file(struct mm_struct *mm, > ClearPageUnevictable(page); > unlock_page(page); > put_page(page); > - index++; > } > - while (index < end) { > - clear_highpage(new_page + (index % HPAGE_PMD_NR)); > - index++; > + > + xas_lock_irq(&xas); > + if (is_shmem) > + __mod_lruvec_page_state(new_page, NR_SHMEM_THPS, nr); > + if (nr_none) { > + __mod_lruvec_page_state(new_page, NR_FILE_PAGES, nr_none); > + if (is_shmem) > + __mod_lruvec_page_state(new_page, NR_SHMEM, nr_none); > } > + /* Join all the small entries into a single multi-index entry. */ > + xas_set_order(&xas, start, HPAGE_PMD_ORDER); > + xas_store(&xas, new_page); > + xas_unlock_irq(&xas); > =20 > SetPageUptodate(new_page); > page_ref_add(new_page, HPAGE_PMD_NR - 1); > @@ -2012,9 +2024,11 @@ static void collapse_file(struct mm_struct *mm, > =20 > khugepaged_pages_collapsed++; > } else { > - struct page *page; > - > - /* Something went wrong: roll back page cache changes */ > + /* > + * Something went wrong: > + * either result !=3D SCAN_SUCCEED or copy_failed, > + * roll back page cache changes > + */ > xas_lock_irq(&xas); > mapping->nrpages -=3D nr_none; > =20 > @@ -2047,6 +2061,15 @@ static void collapse_file(struct mm_struct *mm, > xas_lock_irq(&xas); > } > VM_BUG_ON(nr_none); > + /* > + * Undo the updates of thp_nr_pages(new_page) for non-SHMEM file, > + * which is not updated yet for SHMEM file. > + * These undos are not needed if result is not SCAN_SUCCEED. > + */ > + if (!is_shmem && result =3D=3D SCAN_SUCCEED) { > + __mod_lruvec_page_state(new_page, NR_FILE_THPS, -nr); > + filemap_nr_thps_dec(mapping); > + } > xas_unlock_irq(&xas); > =20 > new_page->mapping =3D NULL;