From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 904FBCA9ECB for ; Thu, 31 Oct 2019 18:10:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0DA1C20650 for ; Thu, 31 Oct 2019 18:10:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=fb.com header.i=@fb.com header.b="j2rLU1K4"; dkim=pass (1024-bit key) header.d=fb.onmicrosoft.com header.i=@fb.onmicrosoft.com header.b="Rt3z24QB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0DA1C20650 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=fb.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 56E596B0003; Thu, 31 Oct 2019 14:10:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4F7966B0005; Thu, 31 Oct 2019 14:10:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 397926B0006; Thu, 31 Oct 2019 14:10:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0230.hostedemail.com [216.40.44.230]) by kanga.kvack.org (Postfix) with ESMTP id 1349C6B0003 for ; Thu, 31 Oct 2019 14:10:44 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 95B9C8249980 for ; Thu, 31 Oct 2019 18:10:43 +0000 (UTC) X-FDA: 76104870366.25.dress77_4075cacf52e60 X-HE-Tag: dress77_4075cacf52e60 X-Filterd-Recvd-Size: 12661 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Thu, 31 Oct 2019 18:10:42 +0000 (UTC) Received: from pps.filterd (m0109333.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id x9VIAPvn029649; Thu, 31 Oct 2019 11:10:31 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=facebook; bh=qO75WSLv/4BY+TQnyeTXfbGMMMbu8lO30EZ30m5ZEpo=; b=j2rLU1K43C9NeSmU/9ti2XUdsPoFXNQyLqHdh86lE6aC9n31GhLM1zdAEFUiibSG7ykP 8Fks9rnOlPiODVJbQduTnEz7bF9Wt62o1+SSfkTQ+7J3W35HqjukHoUu6IOGlsGza61R yey7SopnCKjPt+/I0lu/pC2uD+vEofJEbXo= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com with ESMTP id 2w01c4s6wx-7 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Thu, 31 Oct 2019 11:10:31 -0700 Received: from ash-exopmbx201.TheFacebook.com (2620:10d:c0a8:83::8) by ash-exhub203.TheFacebook.com (2620:10d:c0a8:83::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Thu, 31 Oct 2019 11:10:29 -0700 Received: from ash-exhub101.TheFacebook.com (2620:10d:c0a8:82::e) by ash-exopmbx201.TheFacebook.com (2620:10d:c0a8:83::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Thu, 31 Oct 2019 11:10:29 -0700 Received: from NAM05-DM3-obe.outbound.protection.outlook.com (100.104.31.183) by o365-in.thefacebook.com (100.104.35.173) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5 via Frontend Transport; Thu, 31 Oct 2019 11:10:29 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=oWhPj8fNKnapa0IEzjjPFo7acb3ykBOvPZDN7oNqdO4DbISIUvVV4/z+KG/BWu/pV35sasUgfoUh04dL3BJTH95Zq0O8yIcxHC4Q9mMmAhYkK+pzQ9DGIpv/wHr+oKoP22RrDAJHSd+iK+9/PiSCgCSumRRWFAB/8/zXLwVW5aiTKktvuWCmDYEpQbillYeDfWjX+TT/ZK6SxsbXjpQw9s1hqh+6duOHCVZOMGSmbOgVroabebXc9jpBSvnC2xvZ3uqeszncs+VSik+jXa0FMDjSCsSdKTNl0VfXFKYLlG926kMqOBwRnVyoKEGt3aS1GeM040LPmBfcvhoh1m3MeQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qO75WSLv/4BY+TQnyeTXfbGMMMbu8lO30EZ30m5ZEpo=; b=cMpXbvsBWOnluYbY8poSgdOwCKzTseAi+KFB7pj2LHzKQS+XJmmtqZDlNvW8hODUe9cL4cioPPzBkDlE/DjFDWPUlc9Il6YO7ornja+c3saBc/EZVVTlD7ElO87JF+/sIPtHiHGepSYC2F7g8UV39cA5fFzbg6fKRQkizk8ko3KI1ySFr3uFDaNBMFz1XlmZi+dF2LomIcIxHqkNaCIKTO1JiyepUS9oJmsLsAyhx5qKN00MVhenzQMw9DrNXmohRaH88UAqxbnn1X9PsnkgQ81D1n9v/vq+dQL1dSqCsNhnZpcnKeF5Pd4V22JNvR+My3tUnnmfjbCQUkQVGkIzVA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=fb.com; dmarc=pass action=none header.from=fb.com; dkim=pass header.d=fb.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector2-fb-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qO75WSLv/4BY+TQnyeTXfbGMMMbu8lO30EZ30m5ZEpo=; b=Rt3z24QBRxlvpOqhIPTzhe0dS/UoppR0LvBYFzu/RlL/6eBf1r9JfSLhQiV2gpqfD7J5h1iyOPRJxKmI51hYYN1RKg7cQE+i+2v8A9JD4XQS+Lh5ufea2hzvivnbRcvN7MYBdRPb0Zw6McCbrqQVPYvAKf2bFjtHDovbWK2Nhso= Received: from MWHPR15MB1165.namprd15.prod.outlook.com (10.175.3.22) by MWHPR15MB1231.namprd15.prod.outlook.com (10.175.3.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2408.17; Thu, 31 Oct 2019 18:10:28 +0000 Received: from MWHPR15MB1165.namprd15.prod.outlook.com ([fe80::fdc8:5546:bace:15f5]) by MWHPR15MB1165.namprd15.prod.outlook.com ([fe80::fdc8:5546:bace:15f5%5]) with mapi id 15.20.2387.027; Thu, 31 Oct 2019 18:10:28 +0000 From: Song Liu To: Andrew Morton CC: Linux MM , Kernel Team , "Kirill A . Shutemov" , Hugh Dickins , William Kucharski , Johannes Weiner , Matthew Wilcox Subject: Re: [PATCH] mm/thp: flush file for !is_shmem PageDirty() case in collapse_file() Thread-Topic: [PATCH] mm/thp: flush file for !is_shmem PageDirty() case in collapse_file() Thread-Index: AQHVj12wRWDZ2/wcSkePDdk389uWy6dz/KCAgAERloA= Date: Thu, 31 Oct 2019 18:10:27 +0000 Message-ID: <6AA67FE2-762A-4A14-8C7A-4DC92C9913A4@fb.com> References: <20191030200736.3455046-1-songliubraving@fb.com> <20191030185115.ad780e74c66b6286789559fd@linux-foundation.org> In-Reply-To: <20191030185115.ad780e74c66b6286789559fd@linux-foundation.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: Apple Mail (2.3594.4.19) x-originating-ip: [2620:10d:c090:200::992f] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2f5f9063-1cb3-4fdd-ec1e-08d75e2d9c21 x-ms-traffictypediagnostic: MWHPR15MB1231: x-ms-exchange-purlcount: 1 x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 02070414A1 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(136003)(346002)(396003)(366004)(376002)(39860400002)(54534003)(189003)(199004)(66446008)(64756008)(81166006)(66556008)(46003)(305945005)(6436002)(8676002)(50226002)(81156014)(102836004)(6486002)(33656002)(7736002)(478600001)(76176011)(71190400001)(8936002)(5660300002)(25786009)(71200400001)(66476007)(86362001)(6306002)(966005)(229853002)(256004)(99286004)(36756003)(476003)(14444005)(316002)(2616005)(6246003)(6116002)(6506007)(54906003)(186003)(4326008)(14454004)(446003)(6916009)(66946007)(2906002)(486006)(11346002)(53546011)(6512007)(76116006);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR15MB1231;H:MWHPR15MB1165.namprd15.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: fb.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: lXHZ2f4DNQC/9gIhOJ5qg5npu0EbRGE1v0a7mikmugV4f6OD4Y7xVIsgjjKDFRe+Sz7iGOUW0fhrtCAKH8OxF9FlbsRV6H9TP6n5b6BE+Sjiz0B7WextvoykCjEJC/lwEJ9uhtmStTXl4WtYeYWZSX9Emwlv/SV1izaylHYAe6LB5rZZv1BqBB2Pa2UhxLQdGI2XooY15E+g/wUas4bu3se/X831oYL5b7nGByNsxLW5H8VNHeF2/0+tdIXuYzKOMSCAQ03DDRbX94eJ0hWfP6Ci8NAYDeCOVLMmMSBtrmJdEfGP/iPKvj+DmIP7bkFghoZyRjjvItSNqhDj1tX5GDiJCDBahbKkM8qEfJJ5VHaFbHL7+MNDL1yYWjlsP5J88Y+hucYpvrf82CxsXy+qcL1XwJlebngpOLr/ggAPXALnBRBZudvXgnIcEoNmraEIBGH80HngxTgUj1peF+dkOjbR5piDm9Rm1Y7I3XFhFIs= Content-Type: text/plain; charset="us-ascii" Content-ID: <7E935FF3EB757F46B5568A27E95AF9B2@namprd15.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 2f5f9063-1cb3-4fdd-ec1e-08d75e2d9c21 X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Oct 2019 18:10:27.9114 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 4THip2bBth+QBX+3M2RJRqsfvHo/5lXfwJxRc+A+UzmIiuaO0Oac6N4wbiANRh+9FN0DqoDYNUmNROUtHPNYBg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR15MB1231 X-OriginatorOrg: fb.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,1.0.8 definitions=2019-10-31_07:2019-10-30,2019-10-31 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 malwarescore=0 impostorscore=0 adultscore=0 suspectscore=0 phishscore=0 spamscore=0 mlxscore=0 priorityscore=1501 mlxlogscore=999 bulkscore=0 lowpriorityscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1908290000 definitions=main-1910310179 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Resending the mail, as I sent previous one in "Rich Text".=20 > On Oct 30, 2019, at 6:51 PM, Andrew Morton wr= ote: >=20 > On Wed, 30 Oct 2019 13:07:36 -0700 Song Liu wrote= : >=20 >> For non-shmem file THPs, khugepaged only collapses read only .text mappi= ng >> (VM_DENYWRITE). These pages should not be dirty except the case where th= e >> file hasn't been flushed since first write. >>=20 >> Call filemap_flush() in collapse_file() to accelerate the write back in >> such cases. >>=20 >> Also add warning if PageDirty() triggered for pages from readahead path. >>=20 >> Reported-and-tested-by: syzbot+efb9e48b9fbdc49bb34a@syzkaller.appspotmai= l.com >=20 > If sysbot reported something then we definitely want to be able to > review that report when reviewing the patch! So please do include a > Link: for that sort of thing. >=20 > A bit of sleuthing leads me to > http://lkml.kernel.org/r/000000000000c50fd70595fdd5b2@google.com >=20 > which shows that this patch is in fact a fix against > mmthp-recheck-each-page-before-collapsing-file-thp.patch. This very > important information wasn't in the changelog. >=20 > And this patch doesn't really fix the sysbot hang, does it? The patch > restores the flush which was removed in order to fix the sysbot hang. >=20 > In general, please do try to include all this sort of information when > preparing changelogs. Sorry for the confusion. This one is a bit tricky. syzbot reported hang issue with filemap_flush(), which is already fixed by previous patch which removes filemap_flush().=20 This patch tries to add filemap_flush() back, in a proper way. So this=20 patch is an improvement, not a fix. That's why I didn't include the=20 fixes tag.=20 However, I used syzbot to test this patch, by replying "#syz test: ...".=20 to the previous report (the one you found). I didn't cc the mail list for those tests. syzbot did find bug with an earlier version, and gave=20 green light for this version. This is the reason I would like to give=20 syzbot credit with the tag. Maybe I should just make it "Tested-by:=20 syzbot"?=20 >=20 >=20 >> --- a/mm/khugepaged.c >> +++ b/mm/khugepaged.c >> @@ -1601,6 +1601,33 @@ static void collapse_file(struct mm_struct *mm, >> result =3D SCAN_FAIL; >> goto xa_unlocked; >> } >> + if (WARN_ON_ONCE(PageDirty(page))) { >=20 > I'm not understanding what guarantees this. Can't another process > which has the file open for writing come in and dirty the page after > the readahead has completed and before this process locks the page? For non-shmem file, khugepaged only work for VM_DENYWRITE mappings.=20 Therefore, the page cannot be opened for write by another process. >=20 >> + /* >> + * page from readahead should not >> + * be dirty. Show warning if this >> + * somehow happens. >> + */ >> + result =3D SCAN_FAIL; >> + goto out_unlock; >> + } >> + } else if (PageDirty(page)) { >> + /* >> + * khugepaged only works on read-only fd, >> + * so this page is dirty because it hasn't >> + * been flushed since first write. There >> + * won't be new dirty pages. >> + * >> + * Trigger async flush here and hope the >> + * writeback is done when khugepaged >> + * revisits this page. >> + * >> + * This is a one-off situation. We are not >> + * forcing writeback in loop. >> + */ >> + xas_unlock_irq(&xas); >> + filemap_flush(mapping); >> + result =3D SCAN_FAIL; >> + goto xa_unlocked; >> } else if (trylock_page(page)) { >> get_page(page); >> xas_unlock_irq(&xas); >=20 > The patch mmthp-recheck-each-page-before-collapsing-file-thp.patch has > undergone quite a bit of churn so I don't think it should be mainlined > without more testing and review. But it fixes a significant issue. So > could the appropriate developers please take some time to recheck and > retest it all? We are running production tests with previous version (no filemap_flush).=20 syzbot also gives green light to that one. Maybe we should ship the=20 version without filemap_flush with 5.4, and this improvement with 5.5? This improvement also passed syzbot's test. If this looks good, we will=20 also add this to production tests.=20 Thanks, Song=