From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F18E9C433E0 for ; Thu, 18 Feb 2021 21:43:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4920560295 for ; Thu, 18 Feb 2021 21:43:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4920560295 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8D79B6B0005; Thu, 18 Feb 2021 16:43:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8610A6B006C; Thu, 18 Feb 2021 16:43:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6B36C6B006E; Thu, 18 Feb 2021 16:43:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0215.hostedemail.com [216.40.44.215]) by kanga.kvack.org (Postfix) with ESMTP id 4EC796B0005 for ; Thu, 18 Feb 2021 16:43:39 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 132FD180ACF6C for ; Thu, 18 Feb 2021 21:43:39 +0000 (UTC) X-FDA: 77832715758.17.6076CA4 Received: from aserp2120.oracle.com (aserp2120.oracle.com [141.146.126.78]) by imf27.hostedemail.com (Postfix) with ESMTP id B717A80192DA for ; Thu, 18 Feb 2021 21:43:34 +0000 (UTC) Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 11ILNggE152465; Thu, 18 Feb 2021 21:43:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : references : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=corp-2020-01-29; bh=9bDhzuwtJydPK3jDTF/9/JzVxW4gyP+1o+FR3dSelLo=; b=J91JPmuyCRWzpLsM7aly31rH1+URGwKo/4oqM1Mc2cPGXcJsON58KKnLyQfqf/F7rcLN 4/GfcBLWXN5BiToGIKlgRzI5x+Ebnu9oGGpurWeqBVwz7zM1hDcW7vbDukma7Hc2JCbG UYWhYHeB0T7lkKNl5yQ4U4XzMqwvRwZrwgZ5qMWiimuRxlvJt0qmV7PfsznUl7BAgOR5 BGTg/Re1clzYA9BaVwLlTKrC8/Y8HE7EKnOQHoOZknJCtDaqe66sccsdE3ZzA0y4G4qE IAbJAEq7HwSSReSHtiUx7b3tS+Wjf04d+IpLpIrvIGCxivshfolcNDF+0vqTp5FuWps1 ng== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 36pd9af0pf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 Feb 2021 21:43:33 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 11ILeBH9008615; Thu, 18 Feb 2021 21:43:32 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userp3030.oracle.com with ESMTP id 36prq129m5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 Feb 2021 21:43:32 +0000 Received: from abhmp0009.oracle.com (abhmp0009.oracle.com [141.146.116.15]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 11ILhRxe006761; Thu, 18 Feb 2021 21:43:27 GMT Received: from [192.168.2.112] (/50.38.35.18) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 18 Feb 2021 13:43:26 -0800 Subject: Re: [PATCH 1/2] hugetlb: fix update_and_free_page contig page struct assumption From: Mike Kravetz To: Jason Gunthorpe , Matthew Wilcox Cc: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Zi Yan , Davidlohr Bueso , "Kirill A . Shutemov" , Andrea Arcangeli , Oscar Salvador , Joao Martins , stable@vger.kernel.org References: <20210217184926.33567-1-mike.kravetz@oracle.com> <20210217110252.185c7f5cd5a87c3f7b0c0144@linux-foundation.org> <20210218144554.GS2858050@casper.infradead.org> <20210218172500.GA4718@ziepe.ca> Message-ID: Date: Thu, 18 Feb 2021 13:43:25 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.1.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Proofpoint-IMR: 1 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9899 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxlogscore=999 phishscore=0 adultscore=0 mlxscore=0 suspectscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102180179 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9899 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 clxscore=1015 impostorscore=0 mlxscore=0 phishscore=0 mlxlogscore=999 spamscore=0 bulkscore=0 priorityscore=1501 malwarescore=0 suspectscore=0 adultscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102180178 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: B717A80192DA X-Stat-Signature: frcnbo4bh9ogkt81t45ej9t8iobzsag3 Received-SPF: none (oracle.com>: No applicable sender policy available) receiver=imf27; identity=mailfrom; envelope-from=""; helo=aserp2120.oracle.com; client-ip=141.146.126.78 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1613684614-902071 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2/18/21 9:34 AM, Mike Kravetz wrote: > On 2/18/21 9:25 AM, Jason Gunthorpe wrote: >> On Thu, Feb 18, 2021 at 02:45:54PM +0000, Matthew Wilcox wrote: >>> On Wed, Feb 17, 2021 at 11:02:52AM -0800, Andrew Morton wrote: >>>> On Wed, 17 Feb 2021 10:49:25 -0800 Mike Kravetz wrote: >>>>> page structs are not guaranteed to be contiguous for gigantic pages. The >>>> >>>> June 2014. That's a long lurk time for a bug. I wonder if some later >>>> commit revealed it. >>> >>> I would suggest that gigantic pages have not seen much use. Certainly >>> performance with Intel CPUs on benchmarks that I've been involved with >>> showed lower performance with 1GB pages than with 2MB pages until quite >>> recently. >> >> I suggested in another thread that maybe it is time to consider >> dropping this "feature" >> >> If it has been slightly broken for 7 years it seems a good bet it >> isn't actually being used. >> >> The cost to fix GUP to be compatible with this will hurt normal >> GUP performance - and again, that nobody has hit this bug in GUP >> further suggests the feature isn't used.. > > I was thinking that we could detect these 'unusual' configurations and only > do the slower page struct walking in those cases. However, we would need to > do some research to make sure we have taken into account all possible config > options which can produce non-contiguous page structs. That should have zero > performance impact in the 'normal' cases. What about something like the following patch, and making all code that wants to scan gigantic page subpages use mem_map_next()? >From 95b0384bd5d7f0435546bdd3c01c478724ae0166 Mon Sep 17 00:00:00 2001 From: Mike Kravetz Date: Thu, 18 Feb 2021 13:35:02 -0800 Subject: [PATCH] mm: define PFN_PAGE_MAP_LINEAR to optimize gigantic page scans Signed-off-by: Mike Kravetz --- arch/ia64/include/asm/page.h | 1 + arch/m68k/include/asm/page_no.h | 1 + include/asm-generic/memory_model.h | 2 ++ mm/internal.h | 2 ++ 4 files changed, 6 insertions(+) diff --git a/arch/ia64/include/asm/page.h b/arch/ia64/include/asm/page.h index b69a5499d75b..8f4288862ec8 100644 --- a/arch/ia64/include/asm/page.h +++ b/arch/ia64/include/asm/page.h @@ -106,6 +106,7 @@ extern struct page *vmem_map; #ifdef CONFIG_DISCONTIGMEM # define page_to_pfn(page) ((unsigned long) (page - vmem_map)) # define pfn_to_page(pfn) (vmem_map + (pfn)) +# define PFN_PAGE_MAP_LINEAR # define __pfn_to_phys(pfn) PFN_PHYS(pfn) #else # include diff --git a/arch/m68k/include/asm/page_no.h b/arch/m68k/include/asm/page_no.h index 6bbe52025de3..cafc0731a42c 100644 --- a/arch/m68k/include/asm/page_no.h +++ b/arch/m68k/include/asm/page_no.h @@ -28,6 +28,7 @@ extern unsigned long memory_end; #define pfn_to_page(pfn) virt_to_page(pfn_to_virt(pfn)) #define page_to_pfn(page) virt_to_pfn(page_to_virt(page)) +#define PFN_PAGE_MAP_LINEAR #define pfn_valid(pfn) ((pfn) < max_mapnr) #define virt_addr_valid(kaddr) (((void *)(kaddr) >= (void *)PAGE_OFFSET) && \ diff --git a/include/asm-generic/memory_model.h b/include/asm-generic/memory_model.h index 7637fb46ba4f..8ac4c48dbf22 100644 --- a/include/asm-generic/memory_model.h +++ b/include/asm-generic/memory_model.h @@ -33,6 +33,7 @@ #define __pfn_to_page(pfn) (mem_map + ((pfn) - ARCH_PFN_OFFSET)) #define __page_to_pfn(page) ((unsigned long)((page) - mem_map) + \ ARCH_PFN_OFFSET) +#define PFN_PAGE_MAP_LINEAR #elif defined(CONFIG_DISCONTIGMEM) #define __pfn_to_page(pfn) \ @@ -53,6 +54,7 @@ /* memmap is virtually contiguous. */ #define __pfn_to_page(pfn) (vmemmap + (pfn)) #define __page_to_pfn(page) (unsigned long)((page) - vmemmap) +#define PFN_PAGE_MAP_LINEAR #elif defined(CONFIG_SPARSEMEM) /* diff --git a/mm/internal.h b/mm/internal.h index 25d2b2439f19..64cc5069047c 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -454,12 +454,14 @@ static inline struct page *mem_map_offset(struct page *base, int offset) static inline struct page *mem_map_next(struct page *iter, struct page *base, int offset) { +#ifndef PFN_PAGE_MAP_LINEAR if (unlikely((offset & (MAX_ORDER_NR_PAGES - 1)) == 0)) { unsigned long pfn = page_to_pfn(base) + offset; if (!pfn_valid(pfn)) return NULL; return pfn_to_page(pfn); } +#endif return iter + 1; } -- 2.29.2