From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 448F8C433E0 for ; Thu, 21 Jan 2021 04:21:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C2B5723884 for ; Thu, 21 Jan 2021 04:21:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C2B5723884 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0E8966B0008; Wed, 20 Jan 2021 23:21:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 099A36B000A; Wed, 20 Jan 2021 23:21:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F18056B000C; Wed, 20 Jan 2021 23:21:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id DDB136B0008 for ; Wed, 20 Jan 2021 23:21:53 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id A60EE180AD81F for ; Thu, 21 Jan 2021 04:21:53 +0000 (UTC) X-FDA: 77728484106.14.bells73_460156627560 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin14.hostedemail.com (Postfix) with ESMTP id 887DD18229818 for ; Thu, 21 Jan 2021 04:21:53 +0000 (UTC) X-HE-Tag: bells73_460156627560 X-Filterd-Recvd-Size: 5754 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Thu, 21 Jan 2021 04:21:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=lIjedZ5on/vPEaniCtRL6+F9x+cZv8e9hR//eN9xWhY=; b=qVujcRxHLReLbGixfiEFXTmmGT AKIa97hztVvCrtmPDj+koNh9n1Z2mx0HSMeYOrupi1kNqDO8ZBI42dvyAF5xeujCAKBecU9ajgtDE IxLxaDK8lTHcChDFQJxUdaT1pan/7aw6ikjKtY9qEVHdPzNPcU/RGImw3kRYSx4zYfYmheBg5ZiAN 48cQcgnF3kI2nx5CR2lvtaSSgp2G1ZWJ4TB8V0WiXZO4WJUReeZVvkSRs3iPY3ojsl25/pf9tsVr0 cMgRZn4fh9CbNse9SIeh7OUj5ySlE93rmM+1XBilNJOJQL+BIcg0YDls6YN7lNueF2KqR3d26XbU7 4uY24biA==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1l2RSB-00GbLi-Jd; Thu, 21 Jan 2021 04:20:28 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH v4 06/18] mm/filemap: Support readpage splitting a page Date: Thu, 21 Jan 2021 04:16:04 +0000 Message-Id: <20210121041616.3955703-7-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210121041616.3955703-1-willy@infradead.org> References: <20210121041616.3955703-1-willy@infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: For page splitting to succeed, the thread asking to split the page has to be the only one with a reference to the page. Calling wait_on_page_locked() while holding a reference to the page will effectively prevent this from happening with sufficient threads waiting on the same page. Use put_and_wait_on_page_locked() to sleep without holding a reference to the page, then retry the page lookup after the page is unlocked. Since we now get the page lock a little earlier in filemap_update_page(), we can eliminate a number of duplicate checks. The original intent (commit ebded02788b5 ("avoid unnecessary calls to lock_page when waiting for IO to complete during a read")) behind getting the page lock later was to avoid re-locking the page after it has been brought uptodate by another thread. We still avoid that because we go through the normal lookup path again after the winning thread has brought the page uptodate. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 76 ++++++++++++++++------------------------------------ 1 file changed, 23 insertions(+), 53 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index c8ec47f3c3a17..abbfa44d7c18a 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1372,14 +1372,6 @@ static int __wait_on_page_locked_async(struct page= *page, return ret; } =20 -static int wait_on_page_locked_async(struct page *page, - struct wait_page_queue *wait) -{ - if (!PageLocked(page)) - return 0; - return __wait_on_page_locked_async(compound_head(page), wait, false); -} - /** * put_and_wait_on_page_locked - Drop a reference and wait for it to be = unlocked * @page: The page to wait for. @@ -2276,64 +2268,42 @@ static struct page *filemap_update_page(struct ki= ocb *iocb, struct file *filp, struct inode *inode =3D mapping->host; int error; =20 - /* - * See comment in do_read_cache_page on why - * wait_on_page_locked is used to avoid unnecessarily - * serialisations and why it's safe. - */ if (iocb->ki_flags & IOCB_WAITQ) { - error =3D wait_on_page_locked_async(page, - iocb->ki_waitq); + error =3D lock_page_async(page, iocb->ki_waitq); + if (error) { + put_page(page); + return ERR_PTR(error); + } } else { - error =3D wait_on_page_locked_killable(page); - } - if (unlikely(error)) { - put_page(page); - return ERR_PTR(error); + if (!trylock_page(page)) { + put_and_wait_on_page_locked(page, TASK_KILLABLE); + return NULL; + } } - if (PageUptodate(page)) - return page; =20 + if (!page->mapping) + goto truncated; + if (PageUptodate(page)) + goto uptodate; if (inode->i_blkbits =3D=3D PAGE_SHIFT || !mapping->a_ops->is_partially_uptodate) - goto page_not_up_to_date; + goto readpage; /* pipes can't handle partially uptodate pages */ if (unlikely(iov_iter_is_pipe(iter))) - goto page_not_up_to_date; - if (!trylock_page(page)) - goto page_not_up_to_date; - /* Did it get truncated before we got the lock? */ - if (!page->mapping) - goto page_not_up_to_date_locked; + goto readpage; if (!mapping->a_ops->is_partially_uptodate(page, - pos & ~PAGE_MASK, count)) - goto page_not_up_to_date_locked; + pos & (thp_size(page) - 1), count)) + goto readpage; +uptodate: unlock_page(page); return page; =20 -page_not_up_to_date: - /* Get exclusive access to the page ... */ - error =3D lock_page_for_iocb(iocb, page); - if (unlikely(error)) { - put_page(page); - return ERR_PTR(error); - } - -page_not_up_to_date_locked: - /* Did it get truncated before we got the lock? */ - if (!page->mapping) { - unlock_page(page); - put_page(page); - return NULL; - } - - /* Did somebody else fill it already? */ - if (PageUptodate(page)) { - unlock_page(page); - return page; - } - +readpage: return filemap_read_page(iocb, filp, mapping, page); +truncated: + unlock_page(page); + put_page(page); + return NULL; } =20 static struct page *filemap_create_page(struct kiocb *iocb, --=20 2.29.2