From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDE54ECE58F for ; Mon, 7 Oct 2019 09:18:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 887F42173B for ; Mon, 7 Oct 2019 09:18:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 887F42173B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 122CA8E0003; Mon, 7 Oct 2019 05:18:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EDE878E0009; Mon, 7 Oct 2019 05:18:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D9C7C8E0006; Mon, 7 Oct 2019 05:18:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0038.hostedemail.com [216.40.44.38]) by kanga.kvack.org (Postfix) with ESMTP id B19F88E0005 for ; Mon, 7 Oct 2019 05:18:36 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 5BACC878D for ; Mon, 7 Oct 2019 09:18:36 +0000 (UTC) X-FDA: 76016438232.15.ice29_854878bf17b48 X-HE-Tag: ice29_854878bf17b48 X-Filterd-Recvd-Size: 6988 Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Mon, 7 Oct 2019 09:18:35 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 20F76ACAA; Mon, 7 Oct 2019 09:18:34 +0000 (UTC) From: Vlastimil Babka To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, Qian Cai , "Kirill A. Shutemov" , Matthew Wilcox , Mel Gorman , Michal Hocko , Vlastimil Babka , "Kirill A . Shutemov" Subject: [PATCH v3 1/3] mm, page_owner: fix off-by-one error in __set_page_owner_handle() Date: Mon, 7 Oct 2019 11:18:06 +0200 Message-Id: <20191007091808.7096-2-vbabka@suse.cz> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191007091808.7096-1-vbabka@suse.cz> References: <20191007091808.7096-1-vbabka@suse.cz> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: As noted by Kirill, commit 7e2f2a0cd17c ("mm, page_owner: record page own= er for each subpage") has introduced an off-by-one error in __set_page_owner_han= dle() when looking up page_ext for subpages. As a result, the head page page_ow= ner info is set twice, while for the last tail page, it's not set at all. Fix this and also make the code more efficient by advancing the page_ext pointer we already have, instead of calling lookup_page_ext() for each su= bpage. Since the full size of struct page_ext is not known at compile time, we c= an't use a simple page_ext++ statement, so introduce a page_ext_next() inline function for that. Reported-by: Kirill A. Shutemov Fixes: 7e2f2a0cd17c ("mm, page_owner: record page owner for each subpage"= ) Signed-off-by: Vlastimil Babka Acked-by: Kirill A. Shutemov --- include/linux/page_ext.h | 8 ++++++++ mm/page_ext.c | 23 +++++++++-------------- mm/page_owner.c | 15 +++++++-------- 3 files changed, 24 insertions(+), 22 deletions(-) diff --git a/include/linux/page_ext.h b/include/linux/page_ext.h index 682fd465df06..5e856512bafb 100644 --- a/include/linux/page_ext.h +++ b/include/linux/page_ext.h @@ -36,6 +36,7 @@ struct page_ext { unsigned long flags; }; =20 +extern unsigned long page_ext_size; extern void pgdat_page_ext_init(struct pglist_data *pgdat); =20 #ifdef CONFIG_SPARSEMEM @@ -52,6 +53,13 @@ static inline void page_ext_init(void) =20 struct page_ext *lookup_page_ext(const struct page *page); =20 +static inline struct page_ext *page_ext_next(struct page_ext *curr) +{ + void *next =3D curr; + next +=3D page_ext_size; + return next; +} + #else /* !CONFIG_PAGE_EXTENSION */ struct page_ext; =20 diff --git a/mm/page_ext.c b/mm/page_ext.c index 5f5769c7db3b..4ade843ff588 100644 --- a/mm/page_ext.c +++ b/mm/page_ext.c @@ -67,8 +67,9 @@ static struct page_ext_operations *page_ext_ops[] =3D { #endif }; =20 +unsigned long page_ext_size =3D sizeof(struct page_ext); + static unsigned long total_usage; -static unsigned long extra_mem; =20 static bool __init invoke_need_callbacks(void) { @@ -78,9 +79,8 @@ static bool __init invoke_need_callbacks(void) =20 for (i =3D 0; i < entries; i++) { if (page_ext_ops[i]->need && page_ext_ops[i]->need()) { - page_ext_ops[i]->offset =3D sizeof(struct page_ext) + - extra_mem; - extra_mem +=3D page_ext_ops[i]->size; + page_ext_ops[i]->offset =3D page_ext_size; + page_ext_size +=3D page_ext_ops[i]->size; need =3D true; } } @@ -99,14 +99,9 @@ static void __init invoke_init_callbacks(void) } } =20 -static unsigned long get_entry_size(void) -{ - return sizeof(struct page_ext) + extra_mem; -} - static inline struct page_ext *get_entry(void *base, unsigned long index= ) { - return base + get_entry_size() * index; + return base + page_ext_size * index; } =20 #if !defined(CONFIG_SPARSEMEM) @@ -156,7 +151,7 @@ static int __init alloc_node_page_ext(int nid) !IS_ALIGNED(node_end_pfn(nid), MAX_ORDER_NR_PAGES)) nr_pages +=3D MAX_ORDER_NR_PAGES; =20 - table_size =3D get_entry_size() * nr_pages; + table_size =3D page_ext_size * nr_pages; =20 base =3D memblock_alloc_try_nid( table_size, PAGE_SIZE, __pa(MAX_DMA_ADDRESS), @@ -234,7 +229,7 @@ static int __meminit init_section_page_ext(unsigned l= ong pfn, int nid) if (section->page_ext) return 0; =20 - table_size =3D get_entry_size() * PAGES_PER_SECTION; + table_size =3D page_ext_size * PAGES_PER_SECTION; base =3D alloc_page_ext(table_size, nid); =20 /* @@ -254,7 +249,7 @@ static int __meminit init_section_page_ext(unsigned l= ong pfn, int nid) * we need to apply a mask. */ pfn &=3D PAGE_SECTION_MASK; - section->page_ext =3D (void *)base - get_entry_size() * pfn; + section->page_ext =3D (void *)base - page_ext_size * pfn; total_usage +=3D table_size; return 0; } @@ -267,7 +262,7 @@ static void free_page_ext(void *addr) struct page *page =3D virt_to_page(addr); size_t table_size; =20 - table_size =3D get_entry_size() * PAGES_PER_SECTION; + table_size =3D page_ext_size * PAGES_PER_SECTION; =20 BUG_ON(PageReserved(page)); kmemleak_free(addr); diff --git a/mm/page_owner.c b/mm/page_owner.c index dee931184788..d3cf5d336ccf 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -156,10 +156,10 @@ void __reset_page_owner(struct page *page, unsigned= int order) handle =3D save_stack(GFP_NOWAIT | __GFP_NOWARN); #endif =20 + page_ext =3D lookup_page_ext(page); + if (unlikely(!page_ext)) + return; for (i =3D 0; i < (1 << order); i++) { - page_ext =3D lookup_page_ext(page + i); - if (unlikely(!page_ext)) - continue; __clear_bit(PAGE_EXT_OWNER_ACTIVE, &page_ext->flags); #ifdef CONFIG_DEBUG_PAGEALLOC if (debug_pagealloc_enabled()) { @@ -167,6 +167,7 @@ void __reset_page_owner(struct page *page, unsigned i= nt order) page_owner->free_handle =3D handle; } #endif + page_ext =3D page_ext_next(page_ext); } } =20 @@ -186,7 +187,7 @@ static inline void __set_page_owner_handle(struct pag= e *page, __set_bit(PAGE_EXT_OWNER, &page_ext->flags); __set_bit(PAGE_EXT_OWNER_ACTIVE, &page_ext->flags); =20 - page_ext =3D lookup_page_ext(page + i); + page_ext =3D page_ext_next(page_ext); } } =20 @@ -224,12 +225,10 @@ void __split_page_owner(struct page *page, unsigned= int order) if (unlikely(!page_ext)) return; =20 - page_owner =3D get_page_owner(page_ext); - page_owner->order =3D 0; - for (i =3D 1; i < (1 << order); i++) { - page_ext =3D lookup_page_ext(page + i); + for (i =3D 0; i < (1 << order); i++) { page_owner =3D get_page_owner(page_ext); page_owner->order =3D 0; + page_ext =3D page_ext_next(page_ext); } } =20 --=20 2.23.0