From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E711CCCA473 for ; Fri, 24 Jun 2022 17:37:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DA2108E0249; Fri, 24 Jun 2022 13:37:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CDF4D8E0244; Fri, 24 Jun 2022 13:37:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A6C128E0249; Fri, 24 Jun 2022 13:37:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 7D5DD8E0244 for ; Fri, 24 Jun 2022 13:37:26 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 59DB1B70 for ; Fri, 24 Jun 2022 17:37:26 +0000 (UTC) X-FDA: 79613836092.05.9DC1C7B Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf11.hostedemail.com (Postfix) with ESMTP id E2D0A4002A for ; Fri, 24 Jun 2022 17:37:25 +0000 (UTC) Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-317b6ecba61so27047867b3.9 for ; Fri, 24 Jun 2022 10:37:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=5N1QqSZZ3sAe7uqZbPvcX1MwgZtWBTxE3Z4UbaM2Ubk=; b=q+mTw9aXiewEEPn69Lo/c9WP/AOoUrPORPuTtAp49sBVoij1KAB/iU1Xsmfcz2RONi 97ZReXD54tQvzGoEYGUk7bqiHAMjMBdSDeN0ZeRlmES9GnpySAERBNIAd4xcYCwSqWB9 7rTzJ5T7Icb9PLM1GoXpIL2LTZ8f3PW3xI9Uskzsm3fmgHe4+UJzlKEwXBwzjTskVRhP jO0DeJtpPQFYq6QRquH/FNG3s47JBo454bcpHh+Nv3bPhNDZ/+ST1no2b8ufY9BNObOU 06CUcUjjHu4D3yaQIgx4SDUgySvd1FhGM2vQVwgOwraU6Bc5o7byEvf4kBsFdHoToBy1 X8+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=5N1QqSZZ3sAe7uqZbPvcX1MwgZtWBTxE3Z4UbaM2Ubk=; b=vkusBbyUxlk5QJnPPUyUZSWWWAOaxshLnFosagI90D9nXzSczo+9Ff7lz4FvQZA0mT 8xHA284anyILFFQ3pNh4ojteWymyrz3BhYev2Tw1mwv2TBxc7U4OmSY1wlWBzwGQK9JQ +w0fy6mMgvEoh7FQeF88j+KugV6R+yZdq2OYuvOW638wcGQ43mkMS8UJ6+lzp7N7uKzm OOmYhPX5uEt7B+z5YDNKhWITbkIcp+1CrmhPE1cRlsL0vL87vQ0WI0rWwXIcFQc8WK8L EJUT1ghdZ8Sc3udx6D1PR7xQSqkQGZiXVamZ1W3CS93CbHC7dHyq2QC7PrlgPaCopcw9 cP8Q== X-Gm-Message-State: AJIora/W8UrE1fbhkUz3TFzNuMdpLjm0hb9JFnnXfsHDnwMEGdbzy/bO kJBii1Fiyne2hRvCSbPp+X5fWHlxstigJMxz X-Google-Smtp-Source: AGRyM1s7zrxwUtB89vtQHWN8oyQ64yZ//LRbGigtUbZGCP3Sqfb2xmvuSoQ6VmyGFOOMey8Bgk7vqBsKoi9rXZ66 X-Received: from jthoughton.c.googlers.com ([fda3:e722:ac3:cc00:14:4d90:c0a8:2a4f]) (user=jthoughton job=sendgmr) by 2002:a25:9103:0:b0:668:b7f8:b428 with SMTP id v3-20020a259103000000b00668b7f8b428mr309298ybl.270.1656092245324; Fri, 24 Jun 2022 10:37:25 -0700 (PDT) Date: Fri, 24 Jun 2022 17:36:41 +0000 In-Reply-To: <20220624173656.2033256-1-jthoughton@google.com> Message-Id: <20220624173656.2033256-12-jthoughton@google.com> Mime-Version: 1.0 References: <20220624173656.2033256-1-jthoughton@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [RFC PATCH 11/26] hugetlb: add hugetlb_walk_to to do PT walks From: James Houghton To: Mike Kravetz , Muchun Song , Peter Xu Cc: David Hildenbrand , David Rientjes , Axel Rasmussen , Mina Almasry , Jue Wang , Manish Mishra , "Dr . David Alan Gilbert" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, James Houghton Content-Type: text/plain; charset="UTF-8" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656092245; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5N1QqSZZ3sAe7uqZbPvcX1MwgZtWBTxE3Z4UbaM2Ubk=; b=KWrtP+3+TUpHUuAmAeoe1MPU0qS/Ylm443savQI6oSAZluPl8M/SibYqAAyU087U8bLgHG Y9yS8kKVtnWnlLGxIfmiiTwo+YMGKPA5CcOeLDospkhWFA2Gy/9P/bCoF54o7cba3jQOLA BbVA7V2UDfjTVCFJGyvFr8GnTkybHIs= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=q+mTw9aX; spf=pass (imf11.hostedemail.com: domain of 3Vfa1YgoKCDYblZgmYZlgfYggYdW.Ugedafmp-eecnSUc.gjY@flex--jthoughton.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3Vfa1YgoKCDYblZgmYZlgfYggYdW.Ugedafmp-eecnSUc.gjY@flex--jthoughton.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656092246; a=rsa-sha256; cv=none; b=sNvdvQMETG5lTvXZTAwywSv/bFx0taw6bya/Yg9ycuSA6UEtCCmQNGaNepJUjVRgRQsWIB zlxxiPQWGtHNILAwap+XtBwwCMlWGdHQkd6dE55VHeC4bqcKIp1Hko4Wj2kXJI9IjZ3r3e BdF2PXnKHW0HpyNKSxvin4ya8H5s9uA= Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=q+mTw9aX; spf=pass (imf11.hostedemail.com: domain of 3Vfa1YgoKCDYblZgmYZlgfYggYdW.Ugedafmp-eecnSUc.gjY@flex--jthoughton.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3Vfa1YgoKCDYblZgmYZlgfYggYdW.Ugedafmp-eecnSUc.gjY@flex--jthoughton.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspam-User: X-Rspamd-Server: rspam06 X-Stat-Signature: 97n597t99mqz9iyfctxboszeopsrn41s X-Rspamd-Queue-Id: E2D0A4002A X-HE-Tag: 1656092245-364778 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This adds it for architectures that use GENERAL_HUGETLB, including x86. Signed-off-by: James Houghton --- include/linux/hugetlb.h | 2 ++ mm/hugetlb.c | 45 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index e7a6b944d0cc..605aa19d8572 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -258,6 +258,8 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, unsigned long sz); pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr, unsigned long sz); +int hugetlb_walk_to(struct mm_struct *mm, struct hugetlb_pte *hpte, + unsigned long addr, unsigned long sz, bool stop_at_none); int huge_pmd_unshare(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long *addr, pte_t *ptep); void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma, diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 557b0afdb503..3ec2a921ee6f 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6981,6 +6981,51 @@ pte_t *huge_pte_offset(struct mm_struct *mm, return (pte_t *)pmd; } +int hugetlb_walk_to(struct mm_struct *mm, struct hugetlb_pte *hpte, + unsigned long addr, unsigned long sz, bool stop_at_none) +{ + pte_t *ptep; + + if (!hpte->ptep) { + pgd_t *pgd = pgd_offset(mm, addr); + + if (!pgd) + return -ENOMEM; + ptep = (pte_t *)p4d_alloc(mm, pgd, addr); + if (!ptep) + return -ENOMEM; + hugetlb_pte_populate(hpte, ptep, P4D_SHIFT); + } + + while (hugetlb_pte_size(hpte) > sz && + !hugetlb_pte_present_leaf(hpte) && + !(stop_at_none && hugetlb_pte_none(hpte))) { + if (hpte->shift == PMD_SHIFT) { + ptep = pte_alloc_map(mm, (pmd_t *)hpte->ptep, addr); + if (!ptep) + return -ENOMEM; + hpte->shift = PAGE_SHIFT; + hpte->ptep = ptep; + } else if (hpte->shift == PUD_SHIFT) { + ptep = (pte_t *)pmd_alloc(mm, (pud_t *)hpte->ptep, + addr); + if (!ptep) + return -ENOMEM; + hpte->shift = PMD_SHIFT; + hpte->ptep = ptep; + } else if (hpte->shift == P4D_SHIFT) { + ptep = (pte_t *)pud_alloc(mm, (p4d_t *)hpte->ptep, + addr); + if (!ptep) + return -ENOMEM; + hpte->shift = PUD_SHIFT; + hpte->ptep = ptep; + } else + BUG(); + } + return 0; +} + #endif /* CONFIG_ARCH_WANT_GENERAL_HUGETLB */ #ifdef CONFIG_HUGETLB_HIGH_GRANULARITY_MAPPING -- 2.37.0.rc0.161.g10f37bed90-goog