From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0BAB1C282DE for ; Thu, 13 Mar 2025 18:11:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 887D8280012; Thu, 13 Mar 2025 14:11:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 837D0280010; Thu, 13 Mar 2025 14:11:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6D832280012; Thu, 13 Mar 2025 14:11:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 48272280010 for ; Thu, 13 Mar 2025 14:11:50 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id CC402C03E0 for ; Thu, 13 Mar 2025 18:11:51 +0000 (UTC) X-FDA: 83217321222.20.2BE7B28 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf07.hostedemail.com (Postfix) with ESMTP id E1A334000A for ; Thu, 13 Mar 2025 18:11:49 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=rGPIhjPh; spf=pass (imf07.hostedemail.com: domain of 35B_TZwgKCLUeVXfhViWbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--jackmanb.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=35B_TZwgKCLUeVXfhViWbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--jackmanb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741889510; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9PPqJkUj7Pibb+TYCrq8XdoSiIxYoCw22sHO21dNMJ4=; b=ZpHv24yRfsOn9KZAH3gzbMEFJOTSZBQUYVR/LMoosvQGuscqmMfhFc2R4eBbJX7mDYajmn 2xAmGIMuFuqN5OvGfIaevRNWq30lSeWWTTzQMr3CJ48NsXELG3Bewatpvpq/cuc/Jk4G+U i5bN5RiME1VNRE6f3T0yxbRu7UuGs/I= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741889510; a=rsa-sha256; cv=none; b=SlDOzd16JPaunpCdg4bP4UEBOVswFKfC1qvD8foBC9ghMB/EiSrY2aXo5GuzGXGG5EaNS0 gMnVV6eMZeide1Rlv/Jm6NlHW+zIEOAO1KOKulMSr4ZOBNuvYk5kWsDtR4YoXdBkMhhag6 Sn5BbXb8x867WeAow5+PZLXLiP0glsQ= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=rGPIhjPh; spf=pass (imf07.hostedemail.com: domain of 35B_TZwgKCLUeVXfhViWbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--jackmanb.bounces.google.com designates 209.85.128.73 as permitted sender) smtp.mailfrom=35B_TZwgKCLUeVXfhViWbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--jackmanb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-43cf44b66f7so7384885e9.1 for ; Thu, 13 Mar 2025 11:11:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1741889508; x=1742494308; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=9PPqJkUj7Pibb+TYCrq8XdoSiIxYoCw22sHO21dNMJ4=; b=rGPIhjPheh9+w7NHoA0LZHxbw++gpH+3T/jjeH5ESXV9CRp5EzBC6L+J2MHDUQaMUZ F/bvFBVUI5q0/BgYtAiRG6UD1xtg/dVCo920ZgjbuujZbjAOgOFXNrcx+zb9BUIiFYs+ R3gEnHw4JBy+N/G1AcIFMUv0Ef2gHsq5o9ipDgVTv5BOku6v5PdWkHfclqkpAOUuRzpq +deckG+K0phvPzPSeD5y8fShEjYbSnCvKwm7UWT469Rzf6gFPWvPXBbTW5y8R3zggOlw iLinfEOjUYImWuzIbjKt9MxKNga93kC8Ebfg1HUa6KJ6WesP3iVfyTwNurjOR7JmTSIP g7XQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741889508; x=1742494308; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9PPqJkUj7Pibb+TYCrq8XdoSiIxYoCw22sHO21dNMJ4=; b=PVJf1ZkRP8gXoHc6Fa/dHQJPRs3nJzTYbjF9QASGMt6N1kbxDahkyd/dVJa19hP3mz fVnWfjDnPAGi73jaZnaYi4f7AMSX2wCWxtUzA+gDnUhieh3prUkcZs5MKFukK9x4f1Uv uMbIGIXUUBLClHU76l9IrOemqhTl8JrIWuXLnVWqZ0WCVoIvf6OQwU5YI3qoumv8Qxub Ok/wpo88v3DPhWzN4HknmaVv1bI7eSC9xASMmM6kZUDEz+Yck1GKtbL83s4qKfD4O03N wXI5whbyaTMtvlpR8cjTtvIzqnLAz5OW9Iq7x2UFmJrJJ40fFqWVswal2o4YVzVnCUxq jfoA== X-Forwarded-Encrypted: i=1; AJvYcCX/rqF91Uku8TWBa69jx0A6UnWvgwElRpRlc8qCtaBQDnins3o/XFxxWDcke4AAfKffahbm1h5wYg==@kvack.org X-Gm-Message-State: AOJu0YxyL1Xr89Yo3rGl/4WdZ90a0/WRzYw7JunSKnhRjk7XAR/oRbyS dFi6hFsdroapR7yEEEOu+6OxRsuT6tF0cZSKgsmtxJiMkgoQ4yWZz7n4mhO25jM0w7JGFFFjug3 xF+AcHu0VLg== X-Google-Smtp-Source: AGHT+IF8ERnTvg3YNcVI2WgDobZeTnOtswaGwtLGXdYxevgaK+phqTml6l4yAXJ9Dw4d5FjayQ84d9+Hw+R0XA== X-Received: from wmbfm8.prod.google.com ([2002:a05:600c:c08:b0:43b:c9cc:b9b3]) (user=jackmanb job=prod-delivery.src-stubby-dispatcher) by 2002:a5d:598c:0:b0:391:3988:1c7c with SMTP id ffacd0b85a97d-396c2105ea6mr522202f8f.24.1741889508450; Thu, 13 Mar 2025 11:11:48 -0700 (PDT) Date: Thu, 13 Mar 2025 18:11:23 +0000 In-Reply-To: <20250313-asi-page-alloc-v1-0-04972e046cea@google.com> Mime-Version: 1.0 References: <20250313-asi-page-alloc-v1-0-04972e046cea@google.com> X-Mailer: b4 0.14.2 Message-ID: <20250313-asi-page-alloc-v1-4-04972e046cea@google.com> Subject: [PATCH RFC 04/11] x86/mm/asi: Sync physmap into ASI_GLOBAL_NONSENSITIVE From: Brendan Jackman To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, Andrew Morton , David Rientjes , Vlastimil Babka , David Hildenbrand Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Mike Rapoport , Junaid Shahid , Reiji Watanabe , Patrick Bellasi , Brendan Jackman , Yosry Ahmed Content-Type: text/plain; charset="utf-8" X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: E1A334000A X-Stat-Signature: is8dk5i1qqgdd9mieh8qrii95zbthaci X-HE-Tag: 1741889509-829561 X-HE-Meta: U2FsdGVkX1+d92mlFDfGGtTRhu1CKY0xxMX04lYky9FKT7duGUhaoTOdzVYeX7KOQWdpMeTRB6DvXPqwYk50ieOp+EeCe7trh5TTUQuZiVXi/1dpd0bML8gu0d2T6UwHAbe8MFfcQQb+Y3VyutI+tJcGw2FRS6Ar8sZ/qnQbn0S8kCCFcgKcTBhQJvqSyAdceXp3iOdOQ3ItBsQ6fdObgaiNJdzFoehtQghs/zBiXKvQwPw5M9WXCx7i078ieBagHJrvOz64YyPfqKRg1lle8YBJ2smYosxZy4r/DljOH+jxjsrPbHeM2wxJWdrcEq6CtG2RQHkTN1TzY3RvbvrslvtWtt8WkfZ83nphNDDt945GzYD/U6EpZHzHwHzEu855+GdMV/9gO92Ybg5tZLI9q1306mRqKIdBYysK8KS1teCbTv0JLdFwRB+M7q7OSZo3QtQPBZgcGmVwxXJ/lHN6Xa9N26EK4SSm1kMNY7VZimNBIYfccIFjF6wBIhaqEsevjAAzlA/X1kfJVbaeJ9/WAs6vb+GUDxzPz0M0ukRZALTU/pg9I4EgPRpVHz7yaf90Nu80y44H2MjAMVrjHaTgEfT79u6lDRxOb5gNG5xFr14CfjAhsNEbNqFiDTgJNdJDQgElzxEPA5zRpmPHSAkP9mQjA/t/R9Up0M6q4ZqEN7FyP2gY6q3COphtTq7AWt8IweOuvRNlZgxShUkG+VgVkbKJbDCzZFeP4QHfgduyKYFTfXbgefo2kadjvR30vO++FOG/aDoB87ZZTFv8RNw9kKr/qaOFi8WjkIUb6QkR7Nk3CHHmQMi7UCfqZR2PBAXFpNfdTdgrscxjRydN2yzEBVGqf7gpWsO3oTeOSDDUc7YlMWximHp19SOoV+vEI+9hVy9N3fD/y53OWyqvOCkSmuJ0PX9mVyDsAPVl9jIFalPOieL4au83wNeuQFWUVXKGosNORoeaHjM4lKbjqJD d4wWarrK uPNebUWSbdFIFbk/MrVHJ+Kx7NbHZ8/69b4YRcnEm5vn5rtPOMahX/UKSidjK3iCcwGfW3ZxwjThuyDK9JdUHUoikqDsY4Aw7BYJ4tMN70mBtCxNBkFcELW9rQQdZZw25A5WUlPvreldewVf563U3svB+Z3n9wR1JIbBCBYbkF1nXu9p0bjyyfa09H0PA+v542oYa8DwPNlp4yWkDkM9TsyY5vFvQ0VYbWieSd44LQgFdUI6PB52mUTcWZPgeSiho2KuwC7nyT/dTloQ5hR+F+BKxsJD4/J0LNb9iACKZlz07CtGYQ03JHQ/MUHLFm+Zb94QFfYSa6of4CfLbBWF45tK9yeRRLgkqPp91v2zNm/JhE8E+JI+EhrPWoU22hdtpKIAXJ7zRskfq8olYKweQh9XFehsDjGe/cYvg6O0Ehk/lWeJaRmTsFQ4VLW9CnyEUuhbxCxsTlalwZnImBDh38HKtVf4rYNzf3j4Q5Bgw03AMipUOO/UN5ttPsvMxkJ9KObg0/XGEu8XambLn2rBrm6D44e3lphoPpitU2m4RhV3Bl8gh7BpO6p9v5ZbsSM8mopi/nCv57i3pgd9hcJ994pd5TTl42Jtr7GnE6czcOQ1rPmk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Mirror the physmap into the ASI pagetables, but with a maximum granularity that's guaranteed to allow changing pageblock sensitivity without having to allocate pagetables. Signed-off-by: Brendan Jackman --- arch/x86/mm/init_64.c | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index 8f75274fddd96b8285aff48493ebad93e30daebe..4ca6bb419b9643b0e72cb5b6da6d905f2b2be84b 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -7,6 +7,7 @@ * Copyright (C) 2002,2003 Andi Kleen */ +#include #include #include #include @@ -736,7 +737,8 @@ phys_pgd_init(pgd_t *pgd_page, unsigned long paddr_start, unsigned long paddr_en { unsigned long vaddr, vaddr_start, vaddr_end, vaddr_next, paddr_last; - *pgd_changed = false; + if (pgd_changed) + *pgd_changed = false; paddr_last = paddr_end; vaddr = (unsigned long)__va(paddr_start); @@ -770,9 +772,13 @@ phys_pgd_init(pgd_t *pgd_page, unsigned long paddr_start, unsigned long paddr_en (pud_t *) p4d, init); spin_unlock(&init_mm.page_table_lock); - *pgd_changed = true; + if (pgd_changed) + *pgd_changed = true; } + if (pgd_changed) + sync_global_pgds(vaddr_start, vaddr_end - 1); + return paddr_last; } @@ -784,9 +790,29 @@ __kernel_physical_mapping_init(unsigned long paddr_start, { bool pgd_changed; unsigned long paddr_last; + pgd_t *pgd_asi = asi_pgd(ASI_GLOBAL_NONSENSITIVE); paddr_last = phys_pgd_init(init_mm.pgd, paddr_start, paddr_end, page_size_mask, prot, init, &pgd_changed); + + /* + * Set up ASI's global-nonsensitive physmap. This needs to mapped at max + * 2M size so that regions can be mapped and unmapped at pageblock + * granularity without requiring allocations. + */ + if (pgd_asi) { + /* + * Since most memory is expected to end up sensitive, start with + * everything unmapped in this pagetable. The page allocator + * assumes that's the case. + */ + pgprot_t prot_np = __pgprot(pgprot_val(prot) & ~_PAGE_PRESENT); + + VM_BUG_ON((PAGE_SHIFT + pageblock_order) < page_level_shift(PG_LEVEL_2M)); + phys_pgd_init(pgd_asi, paddr_start, paddr_end, 1 << PG_LEVEL_2M, + prot_np, init, NULL); + } + if (pgd_changed) sync_global_pgds((unsigned long)__va(paddr_start), (unsigned long)__va(paddr_end) - 1); -- 2.49.0.rc1.451.g8f38331e32-goog