From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 121BDC83004 for ; Wed, 29 Apr 2020 13:37:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BECA3206D6 for ; Wed, 29 Apr 2020 13:37:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="mS4GHtyZ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BECA3206D6 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 50FE08E0001; Wed, 29 Apr 2020 09:37:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C91A38E0024; Wed, 29 Apr 2020 09:37:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 197758E000A; Wed, 29 Apr 2020 09:37:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0134.hostedemail.com [216.40.44.134]) by kanga.kvack.org (Postfix) with ESMTP id E13088E001F for ; Wed, 29 Apr 2020 09:37:00 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 8F35A181AEF00 for ; Wed, 29 Apr 2020 13:37:00 +0000 (UTC) X-FDA: 76760993400.16.brass06_3d2a5a461c05e X-HE-Tag: brass06_3d2a5a461c05e X-Filterd-Recvd-Size: 5180 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Wed, 29 Apr 2020 13:37:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=tyoiEDA4aqW5ranAPIIS6qQtQA5yS6rXGQxRGVOBB6U=; b=mS4GHtyZBEma/U97n2kGoKvCN4 uihpLwSbMpvtN78Td9rCZsJy24ZhDi39nb0f4DuBgT7dm0A+rZKdPmG3KNa3Df0H+grThrkbtfCQb swxNDNxX9eEFySUycDJniOWL8BXGubht8trSMNEOYGJ6NfDyp66CeNwi77HBNQEDw2VhVhkOJ9uIw M9Zpfytt4C1f2LYZ6iZWwprI4JB3pzdR6IeDBLH0hk76u0MXzs9XU+J2V/ufBX1Px9T4pmygj0Vki FQetRR3rC9FXsAXYER2iPCA9qRSdryPfcTvhwCZ2RXJEFpYxRLQ8qgiCCRyGkyTM9ym79exive9ki g9/3VlqQ==; Received: from willy by bombadil.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1jTmtX-0005v1-BZ; Wed, 29 Apr 2020 13:36:59 +0000 From: Matthew Wilcox To: linux-fsdevel@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 08/25] fs: Support THPs in zero_user_segments Date: Wed, 29 Apr 2020 06:36:40 -0700 Message-Id: <20200429133657.22632-9-willy@infradead.org> X-Mailer: git-send-email 2.21.1 In-Reply-To: <20200429133657.22632-1-willy@infradead.org> References: <20200429133657.22632-1-willy@infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" We can only kmap() one subpage of a THP at a time, so loop over all relevant subpages, skipping ones which don't need to be zeroed. This is too large to inline when THPs are enabled and we actually need highmem, so put it in highmem.c. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/highmem.h | 15 +++++++--- mm/highmem.c | 62 +++++++++++++++++++++++++++++++++++++++-- 2 files changed, 71 insertions(+), 6 deletions(-) diff --git a/include/linux/highmem.h b/include/linux/highmem.h index ea5cdbd8c2c3..74614903619d 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -215,13 +215,18 @@ static inline void clear_highpage(struct page *page= ) kunmap_atomic(kaddr); } =20 +#if defined(CONFIG_HIGHMEM) && defined(CONFIG_TRANSPARENT_HUGEPAGE) +void zero_user_segments(struct page *page, unsigned start1, unsigned end= 1, + unsigned start2, unsigned end2); +#else /* !HIGHMEM || !TRANSPARENT_HUGEPAGE */ static inline void zero_user_segments(struct page *page, - unsigned start1, unsigned end1, - unsigned start2, unsigned end2) + unsigned start1, unsigned end1, + unsigned start2, unsigned end2) { + unsigned long i; void *kaddr =3D kmap_atomic(page); =20 - BUG_ON(end1 > PAGE_SIZE || end2 > PAGE_SIZE); + BUG_ON(end1 > thp_size(page) || end2 > thp_size(page)); =20 if (end1 > start1) memset(kaddr + start1, 0, end1 - start1); @@ -230,8 +235,10 @@ static inline void zero_user_segments(struct page *p= age, memset(kaddr + start2, 0, end2 - start2); =20 kunmap_atomic(kaddr); - flush_dcache_page(page); + for (i =3D 0; i < hpage_nr_pages(page); i++) + flush_dcache_page(page + i); } +#endif /* !HIGHMEM || !TRANSPARENT_HUGEPAGE */ =20 static inline void zero_user_segment(struct page *page, unsigned start, unsigned end) diff --git a/mm/highmem.c b/mm/highmem.c index 64d8dea47dd1..3a85c66ef532 100644 --- a/mm/highmem.c +++ b/mm/highmem.c @@ -367,9 +367,67 @@ void kunmap_high(struct page *page) if (need_wakeup) wake_up(pkmap_map_wait); } - EXPORT_SYMBOL(kunmap_high); -#endif + +#ifdef CONFIG_TRANSPARENT_HUGEPAGE +void zero_user_segments(struct page *page, unsigned start1, unsigned end= 1, + unsigned start2, unsigned end2) +{ + unsigned int i; + + BUG_ON(end1 > thp_size(page) || end2 > thp_size(page)); + + for (i =3D 0; i < hpage_nr_pages(page); i++) { + void *kaddr; + unsigned this_end; + + if (end1 =3D=3D 0 && start2 >=3D PAGE_SIZE) { + start2 -=3D PAGE_SIZE; + end2 -=3D PAGE_SIZE; + continue; + } + + if (start1 >=3D PAGE_SIZE) { + start1 -=3D PAGE_SIZE; + end1 -=3D PAGE_SIZE; + if (start2) { + start2 -=3D PAGE_SIZE; + end2 -=3D PAGE_SIZE; + } + continue; + } + + kaddr =3D kmap_atomic(page + i); + + this_end =3D min_t(unsigned, end1, PAGE_SIZE); + if (end1 > start1) + memset(kaddr + start1, 0, this_end - start1); + end1 -=3D this_end; + start1 =3D 0; + + if (start2 >=3D PAGE_SIZE) { + start2 -=3D PAGE_SIZE; + end2 -=3D PAGE_SIZE; + } else { + this_end =3D min_t(unsigned, end2, PAGE_SIZE); + if (end2 > start2) + memset(kaddr + start2, 0, this_end - start2); + end2 -=3D this_end; + start2 =3D 0; + } + + kunmap_atomic(kaddr); + flush_dcache_page(page + i); + + if (!end1 && !end2) + break; + } + + BUG_ON((start1 | start2 | end1 | end2) !=3D 0); +} +EXPORT_SYMBOL(zero_user_segments); +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#endif /* CONFIG_HIGHMEM */ =20 #if defined(HASHED_PAGE_VIRTUAL) =20 --=20 2.26.2