From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71A74C43334 for ; Thu, 16 Jun 2022 03:35:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D37086B0072; Wed, 15 Jun 2022 23:35:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CE66E6B0073; Wed, 15 Jun 2022 23:35:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BD50B6B0074; Wed, 15 Jun 2022 23:35:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id AB9546B0072 for ; Wed, 15 Jun 2022 23:35:21 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 7B77734915 for ; Thu, 16 Jun 2022 03:35:21 +0000 (UTC) X-FDA: 79582683642.07.AD942B5 Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) by imf26.hostedemail.com (Postfix) with ESMTP id 4B892140080 for ; Thu, 16 Jun 2022 03:35:19 +0000 (UTC) X-Alimail-AntiSpam:AC=PASS;BC=-1|-1;BR=01201311R141e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046049;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=10;SR=0;TI=SMTPD_---0VGVdsvf_1655350515; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VGVdsvf_1655350515) by smtp.aliyun-inc.com; Thu, 16 Jun 2022 11:35:16 +0800 From: Baolin Wang To: mike.kravetz@oracle.com Cc: songmuchun@bytedance.com, akpm@linux-foundation.org, catalin.marinas@arm.com, will@kernel.org, anshuman.khandual@arm.com, baolin.wang@linux.alibaba.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH] arm64/hugetlb: Implement arm64 specific hugetlb_mask_last_hp Date: Thu, 16 Jun 2022 11:34:58 +0800 Message-Id: <7256dbe078d7231f45b0f47c2c52a3bd3aa10da7.1655350193.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655350521; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references; bh=rhDsRADDozE0niD1YpvDAsefpLPuQqyvPUzxhA1noZY=; b=GuWFm3RvyoSxD/2nlYRvq6ETL3bTgWBSQ1bOwJtCOUcnFHLpgc7qqQJLDsxagS7khqw6gk Wq6/8EflH2cHttn0EFVU7zGh1i2nXRiDGZnyDbjIXi88OsApv+1f99/86LMvEvn1IQepve xfVH5fu/nyeQlGxbFCc3XpZERqW18SA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655350521; a=rsa-sha256; cv=none; b=yJURA+hb6rM5PrqpDW/rEQ/CRHWZWGyOTRjNcX6hBgoH+fNmKZID7709/dpsbH4QAst4zo fVlgiL8dBsmIwAbKKUMX2p0iLZBVMJI+Om/P9Zp7Id26w/mPX9H/SZMy3PczNsIJ63cvGc GDQOxtXze5L9K0+NipH/E+Bo4k37f3A= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=alibaba.com; spf=pass (imf26.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.130 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com Authentication-Results: imf26.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=alibaba.com; spf=pass (imf26.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.130 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com X-Rspamd-Server: rspam12 X-Rspam-User: X-Stat-Signature: 37ggh3yr5od7hwu15abuu7psn4qgyx4u X-Rspamd-Queue-Id: 4B892140080 X-HE-Tag: 1655350519-248016 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The HugeTLB address ranges are linearly scanned during fork, unmap and remap operations, and the linear scan can skip to the end of range mapped by the page table page if hitting a non-present entry, which can help to speed linear scanning of the HugeTLB address ranges. So hugetlb_mask_last_hp() is introduced to help to update the address in the loop of HugeTLB linear scanning with getting the last huge page mapped by the associated page table page[1], when a non-present entry is encountered. Considering ARM64 specific cont-pte/pmd size HugeTLB, this patch implemented an ARM64 specific hugetlb_mask_last_hp() to help this case. [1] https://lore.kernel.org/linux-mm/20220527225849.284839-1-mike.kravetz@oracle.com/ Signed-off-by: Baolin Wang --- Note: this patch is based on the series: "hugetlb: speed up linear address scanning" from Mike. Mike, please fold it into your series. Thanks. --- arch/arm64/mm/hugetlbpage.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index e2a5ec9..958935c 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -368,6 +368,26 @@ pte_t *huge_pte_offset(struct mm_struct *mm, return NULL; } +unsigned long hugetlb_mask_last_hp(struct hstate *h) +{ + unsigned long hp_size = huge_page_size(h); + + switch (hp_size) { + case PUD_SIZE: + return PGDIR_SIZE - PUD_SIZE; + case CONT_PMD_SIZE: + return PUD_SIZE - CONT_PMD_SIZE; + case PMD_SIZE: + return PUD_SIZE - PMD_SIZE; + case CONT_PTE_SIZE: + return PMD_SIZE - CONT_PTE_SIZE; + default: + break; + } + + return ~0UL; +} + pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) { size_t pagesize = 1UL << shift; -- 1.8.3.1