From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AAAAAC433F5 for ; Fri, 28 Jan 2022 20:59:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 31EDD6B00FA; Fri, 28 Jan 2022 15:59:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2CE656B0100; Fri, 28 Jan 2022 15:59:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 121AB6B0102; Fri, 28 Jan 2022 15:59:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0179.hostedemail.com [216.40.44.179]) by kanga.kvack.org (Postfix) with ESMTP id 023FC6B00FA for ; Fri, 28 Jan 2022 15:59:16 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id B4AA6181E1886 for ; Fri, 28 Jan 2022 20:59:15 +0000 (UTC) X-FDA: 79080911070.22.C003335 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by imf01.hostedemail.com (Postfix) with ESMTP id CE74B40032 for ; Fri, 28 Jan 2022 20:59:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1643403555; x=1674939555; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=sPAM/BFzre+OlxESQ1UNn23Z4mlITze1sx4aEFpsPcA=; b=dfeRAuJ3PbcX4RKV1it36t14srPSkBieqoPTHc8kO8DMQ/kS7P07qNKT 2nl0voxv7ZL63Dz4iPdlV9jFUpYuPWY4FWmU7yiS36TmmhAwdsLEj84l3 SdoCbzyhoqyaeB5JDey5/Rm8TpjCQAlBcP0Vjta9Ux2wf2xeg4m7a7i9G cWxPwpsvFpDkYo4nZc7SgPJAAcPaw6ftmO/WX59bfz9wLigze25uLqvnn ljENOo+zcoyaJOV2yJNY4YdNK+ps2YUrKHtzgD4QSMnIwFtR/pJbTk3su mwTztJ0EX00izuQhAKOsiFLf/IVaGSZRG7qEeTIY12eGTrALEuSAGfoe7 A==; X-IronPort-AV: E=McAfee;i="6200,9189,10241"; a="227865746" X-IronPort-AV: E=Sophos;i="5.88,324,1635231600"; d="scan'208";a="227865746" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jan 2022 12:59:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,324,1635231600"; d="scan'208";a="533622714" Received: from black.fi.intel.com ([10.237.72.28]) by fmsmga007.fm.intel.com with ESMTP; 28 Jan 2022 12:59:03 -0800 Received: by black.fi.intel.com (Postfix, from userid 1000) id D23C5478; Fri, 28 Jan 2022 22:59:09 +0200 (EET) From: "Kirill A. Shutemov" To: Borislav Petkov , Andy Lutomirski , Sean Christopherson , Andrew Morton , Joerg Roedel , Ard Biesheuvel Cc: Andi Kleen , Kuppuswamy Sathyanarayanan , David Rientjes , Vlastimil Babka , Tom Lendacky , Thomas Gleixner , Peter Zijlstra , Paolo Bonzini , Ingo Molnar , Varad Gautam , Dario Faggioli , Dave Hansen , Brijesh Singh , Mike Rapoport , David Hildenbrand , x86@kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, linux-efi@vger.kernel.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [PATCHv3 6/7] x86/mm: Provide helpers for unaccepted memory Date: Fri, 28 Jan 2022 23:59:05 +0300 Message-Id: <20220128205906.27503-7-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220128205906.27503-1-kirill.shutemov@linux.intel.com> References: <20220128205906.27503-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 X-Rspam-User: nil X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: CE74B40032 X-Stat-Signature: kemnz9xkgq6iemdx7t179k7cfxr58914 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=dfeRAuJ3; dmarc=pass (policy=none) header.from=intel.com; spf=none (imf01.hostedemail.com: domain of kirill.shutemov@linux.intel.com has no SPF policy when checking 192.55.52.151) smtp.mailfrom=kirill.shutemov@linux.intel.com X-HE-Tag: 1643403554-495044 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Core-mm requires few helpers to support unaccepted memory: - accept_memory() checks the range of addresses against the bitmap and accept memory if needed; - maybe_set_page_offline() checks the bitmap and marks a page with PageOffline() if memory acceptance required on the first allocation of the page. - accept_and_clear_page_offline() accepts memory for the page and clears PageOffline(). Signed-off-by: Kirill A. Shutemov --- arch/x86/include/asm/page.h | 5 ++ arch/x86/include/asm/unaccepted_memory.h | 3 + arch/x86/mm/Makefile | 2 + arch/x86/mm/unaccepted_memory.c | 90 ++++++++++++++++++++++++ 4 files changed, 100 insertions(+) create mode 100644 arch/x86/mm/unaccepted_memory.c diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page.h index 4d5810c8fab7..1e56d76ca474 100644 --- a/arch/x86/include/asm/page.h +++ b/arch/x86/include/asm/page.h @@ -19,6 +19,11 @@ struct page; =20 #include + +#ifdef CONFIG_UNACCEPTED_MEMORY +#include +#endif + extern struct range pfn_mapped[]; extern int nr_pfn_mapped; =20 diff --git a/arch/x86/include/asm/unaccepted_memory.h b/arch/x86/include/= asm/unaccepted_memory.h index f1f835d3cd78..0c0ea777809a 100644 --- a/arch/x86/include/asm/unaccepted_memory.h +++ b/arch/x86/include/asm/unaccepted_memory.h @@ -6,9 +6,12 @@ #include =20 struct boot_params; +struct page; =20 void mark_unaccepted(struct boot_params *params, u64 start, u64 num); =20 void accept_memory(phys_addr_t start, phys_addr_t end); =20 +void maybe_mark_page_unaccepted(struct page *page, unsigned int order); +void accept_page(struct page *page, unsigned int order); #endif diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile index fe3d3061fc11..e327f83e6bbf 100644 --- a/arch/x86/mm/Makefile +++ b/arch/x86/mm/Makefile @@ -60,3 +60,5 @@ obj-$(CONFIG_AMD_MEM_ENCRYPT) +=3D mem_encrypt_amd.o =20 obj-$(CONFIG_AMD_MEM_ENCRYPT) +=3D mem_encrypt_identity.o obj-$(CONFIG_AMD_MEM_ENCRYPT) +=3D mem_encrypt_boot.o + +obj-$(CONFIG_UNACCEPTED_MEMORY) +=3D unaccepted_memory.o diff --git a/arch/x86/mm/unaccepted_memory.c b/arch/x86/mm/unaccepted_mem= ory.c new file mode 100644 index 000000000000..adcac22dfe75 --- /dev/null +++ b/arch/x86/mm/unaccepted_memory.c @@ -0,0 +1,90 @@ +#include +#include +#include +#include + +#include +#include +#include + +static DEFINE_SPINLOCK(unaccepted_memory_lock); + +#define PMD_ORDER (PMD_SHIFT - PAGE_SHIFT) + +static void __accept_memory(phys_addr_t start, phys_addr_t end) +{ + unsigned long *unaccepted_memory; + unsigned int rs, re; + + unaccepted_memory =3D __va(boot_params.unaccepted_memory); + rs =3D start / PMD_SIZE; + + for_each_set_bitrange_from(rs, re, unaccepted_memory, + DIV_ROUND_UP(end, PMD_SIZE)) { + /* Platform-specific memory-acceptance call goes here */ + panic("Cannot accept memory"); + bitmap_clear(unaccepted_memory, rs, re - rs); + } +} + +void accept_memory(phys_addr_t start, phys_addr_t end) +{ + unsigned long flags; + if (!boot_params.unaccepted_memory) + return; + + spin_lock_irqsave(&unaccepted_memory_lock, flags); + __accept_memory(start, end); + spin_unlock_irqrestore(&unaccepted_memory_lock, flags); +} + +void __init maybe_mark_page_unaccepted(struct page *page, unsigned int o= rder) +{ + unsigned long *unaccepted_memory; + phys_addr_t addr =3D page_to_phys(page); + unsigned long flags; + bool unaccepted =3D false; + unsigned int i; + + if (!boot_params.unaccepted_memory) + return; + + unaccepted_memory =3D __va(boot_params.unaccepted_memory); + spin_lock_irqsave(&unaccepted_memory_lock, flags); + if (order < PMD_ORDER) { + BUG_ON(test_bit(addr / PMD_SIZE, unaccepted_memory)); + goto out; + } + + for (i =3D 0; i < (1 << (order - PMD_ORDER)); i++) { + if (test_bit(addr / PMD_SIZE + i, unaccepted_memory)) { + unaccepted =3D true; + break; + } + } + + /* At least part of page is uneccepted */ + if (unaccepted) + __SetPageBuddyUnaccepted(page); +out: + spin_unlock_irqrestore(&unaccepted_memory_lock, flags); +} + +void accept_page(struct page *page, unsigned int order) +{ + phys_addr_t addr =3D round_down(page_to_phys(page), PMD_SIZE); + int i; + + WARN_ON_ONCE(!boot_params.unaccepted_memory); + + page =3D pfn_to_page(addr >> PAGE_SHIFT); + if (order < PMD_ORDER) + order =3D PMD_ORDER; + + accept_memory(addr, addr + (PAGE_SIZE << order)); + + for (i =3D 0; i < (1 << order); i++) { + if (PageBuddyUnaccepted(page + i)) + __ClearPageBuddyUnaccepted(page + i); + } +} --=20 2.34.1