From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C899C4828F for ; Sat, 3 Feb 2024 01:32:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 60DD06B006E; Fri, 2 Feb 2024 20:32:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 596C36B0071; Fri, 2 Feb 2024 20:32:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3E9A86B0072; Fri, 2 Feb 2024 20:32:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 278756B006E for ; Fri, 2 Feb 2024 20:32:24 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id DA11C120988 for ; Sat, 3 Feb 2024 01:32:23 +0000 (UTC) X-FDA: 81748767366.11.FB37BE5 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf21.hostedemail.com (Postfix) with ESMTP id 131771C000F for ; Sat, 3 Feb 2024 01:32:20 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=YryTM5p8; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf21.hostedemail.com: domain of eugenis@google.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=eugenis@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706923941; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OEllp6xt+hG9OYbVYTRwXvaPtKG2vixg03RLhlAvHS8=; b=tfxxQImLcsGYBNmUV/S7r6OZ/TwVeAoxC6/9AQ+cqfZ08+e4iyTxscHa3leEEFLG1eE/ke YRhGbgtCI2yn2lIovAkbT7IfVcjf4elF1q9JPlbWpFPC4Nf6tXuCCqezHutuzSuTx0wrYs vkIhlHA1tvKrD5yDQxG/Pbv+FXvCYJE= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=YryTM5p8; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf21.hostedemail.com: domain of eugenis@google.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=eugenis@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706923941; a=rsa-sha256; cv=none; b=dswdaWDz115jkB6gTTMr6Ixby2ZZOdwPyU6K/NFa4SVYvNElAPY9L/4xEYjJLQXwl/kHV5 iTFhXA77KLxhRpXYBoFFx5TAAjErelyJ6WW0jo9Q1vcR63A8evrIQDtsjRADs9EJ2nzUUD 0pUx5x5JB1etVJVSGC/wDG+ImKeR86E= Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-1d93b982761so94725ad.0 for ; Fri, 02 Feb 2024 17:32:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706923940; x=1707528740; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=OEllp6xt+hG9OYbVYTRwXvaPtKG2vixg03RLhlAvHS8=; b=YryTM5p8srhPtZU7R5f2tU8SPIzzvMyqm0phA59netRaPRtclgqAAd3smF1F/SYKYu XOUQNUOLz9m8xafBKim88rqwbANleQkUrxY+kkv/RaPJhPfb6sWFiP9Ur+z3vF0fe/a+ efPXfv/QHQrrRFkdO5t1XF7EQF8g+9JfhH5eNUt4ZbAAslE8cbf285qq3z/F75W5hOu/ 4lFm8QwNL7ZT5mowMU7VfoqhHd/bnJyHD4hI9Df92Ea25nS7JJwhL34jVgSZifpq2Z3K j/ojNzpEKCexFjGWFCACaaVu2sCNKY2P+DbW+NPFjLEYIxUj5m8LAMZLfbGL5S3MjUY7 biVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706923940; x=1707528740; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OEllp6xt+hG9OYbVYTRwXvaPtKG2vixg03RLhlAvHS8=; b=q6F8uukN0r5LUqoLWKhndhkBxNuU/tjAx8ERdjsnihdHNwUViNm6mPAS8eLrLqTgBv sIxRM9Ce5D7kg39N0yaKTPKyJO5I4v+cJ27C/sBziFeo5y0D4Y2qBcPWv4OOMEf7A8r4 DEv4lGZ7zqQtSifvFcEv7ZWiB/aAaT2ozPDpInokDrJFc4/r1zmTMev5K3K+qCiDKy93 ITDBwVoGQHyUonBCc6RDP++SpQl4vviOpbrf7/WQRbiigbH9ZdQ3sxtsht0RmDNwdrCV R950hZi7Le0aa0afSQjU+i8+K2ktMcMHfKbR5DXFiSEBZp2d+yw7CrPzQ7x2C37y26IO ENsQ== X-Gm-Message-State: AOJu0Yzhv6kSqwVdEAk9InoIUI/F+r5c4xk5E+kdKAaAVqJx7H+zrz+d OCjvSw7sI1b+MyxMZA5gjooDfj7pGxNzrG1vguQPlEzVpFMCOvvPp+lz7Oi2RIRSE6KbTEOS0yZ uFd5R6aY3KOlv8EwuHyZ1h/yB9ylIdozrx7id X-Google-Smtp-Source: AGHT+IEyl82Hg3VFPZZlXyUwgpQogEWkAxahEhAwIFKbaGb5fy4bBRD0kVA10OreWVQXtVHsW/d+Xp4LHMHYfB4lY90= X-Received: by 2002:a17:903:22c2:b0:1d9:5229:989 with SMTP id y2-20020a17090322c200b001d952290989mr70672plg.10.1706923939451; Fri, 02 Feb 2024 17:32:19 -0800 (PST) MIME-Version: 1.0 References: <20240125164256.4147-1-alexandru.elisei@arm.com> <20240125164256.4147-29-alexandru.elisei@arm.com> In-Reply-To: From: Evgenii Stepanov Date: Fri, 2 Feb 2024 17:32:07 -0800 Message-ID: Subject: Re: [PATCH RFC v3 28/35] arm64: mte: swap: Handle tag restoring when missing tag storage To: Alexandru Elisei Cc: Peter Collingbourne , catalin.marinas@arm.com, will@kernel.org, oliver.upton@linux.dev, maz@kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, arnd@arndb.de, akpm@linux-foundation.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, mhiramat@kernel.org, rppt@kernel.org, hughd@google.com, steven.price@arm.com, anshuman.khandual@arm.com, vincenzo.frascino@arm.com, david@redhat.com, kcc@google.com, hyesoo.yu@samsung.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 131771C000F X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 8jg6j1xxanie3ii7fnzwpruknwogzf3c X-HE-Tag: 1706923940-736875 X-HE-Meta: U2FsdGVkX19ZZpa/70I/LLpFFKa3/07xqiB++pzR+ylT8n5dGG045q+NmhrNgmi6Eq3yZx+E5qRXY7XbzELnesfYd23myxqselZvzq0hkuS9ezPL8Ru6VUIR1eZKKDHrgmLdrLsheKdeNmqoZNQ2EoajtXiOTzXcskNPFZDmDgx6tDdJJqbj6cQTet/FQZZTaRF8XN6VvndhG9rRs5BA78JNgJH0L3IrK+24tBrCn4zVPjfZ0UFqhy/8vEHjbI9Uvr25JIBIHN0T08cVLqUuNAonuPd+aOcJ5irFU7HvaI5GJ9iDWJNB8eWvHi4yYCwMQ27XIS3ZFsoHpaWJpDsDZjl7tU+rLBmvlOORLPda8VvoVgQHRyaoQY9ckI8u6Z96EJb0kk6sOUDw2/XvOdg7penbvR/aFqN7tctJSp6eWjLrAeMz9eYNBtLKb/e1/uSqyQdaboxgg7kTZ14npC2HFwYj3QEBuTu8kQkgSPRjDLMKX1gbdySQovitVgn9FjOjn6Jsjk5Myd9dDkFbYHjZLMhuTUZaVkeOK0LoEssDB+5wihn2l+wASFKoSLwgsaOPCRLyarpSE8LUPD+MsKgrxc++ujeJ5umdV8IgPJlXxf/CxaYqB1LT5Y+oArvOP9ZrT01EPyZGR97XvUeIpn18W8vlXBX/15KGx8tFew5riUwuJmsBmu6PLFZikWkF9VatLjlnBL44cmLnZhnwL5Blwvm87O7snXVo86YTUcwNhNFs4zdAS9ekpNj88UnWrbLe1cmOvtkKKg1nUei+RKFGmk40fQpuMv8/InQsL4FC4MGf923Yb88CRgfT3nUXAh2hh2uo7gxOk9OYKfEtw52NvNWvWAEJWE6LQ1w0p+7DXdcAN8bAjbRt8pwXEQp3/c9Aa0sLtDp3qfe8uhbtaB3N8odM1LLLrFgYP6dCDpbiXeiWb2w5uJm/ylqqSnOORa6XF5lJvAoDbSaKU/sC+LS vP5oi5Z5 ZTc/rtnzz23v1MDcNquUMnea4VsDA2TM97Sn1cbsr7JK2TUNnKspai2KJgyrdH57cAUQVcUMSb4a0nxAjNh4i5Jr4brdixd3//KPK535qppHc1LI9tNWGVvirNBlr0ry48lm/f985LEEZskl0ilfnSOeC/eXZbfRe/L5DkuMT+xhcUOSviJPVQPodG3Ke1dXmCEVekOpS0R8zVtc0x9DH0Js8rkmdXQSwtHjicPCfyMGT2QLQ8Sn/s/62+IYa4A1uwZzkKP5uqA0L+40sFBYFY1+imYusNdycREP0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Feb 2, 2024 at 6:56=E2=80=AFAM Alexandru Elisei wrote: > > Hi Peter, > > On Thu, Feb 01, 2024 at 08:02:40PM -0800, Peter Collingbourne wrote: > > On Thu, Jan 25, 2024 at 8:45=E2=80=AFAM Alexandru Elisei > > wrote: > > > > > > Linux restores tags when a page is swapped in and there are tags asso= ciated > > > with the swap entry which the new page will replace. The saved tags a= re > > > restored even if the page will not be mapped as tagged, to protect ag= ainst > > > cases where the page is shared between different VMAs, and is tagged = in > > > some, but untagged in others. By using this approach, the process can= still > > > access the correct tags following an mprotect(PROT_MTE) on the non-MT= E > > > enabled VMA. > > > > > > But this poses a challenge for managing tag storage: in the scenario = above, > > > when a new page is allocated to be swapped in for the process where i= t will > > > be mapped as untagged, the corresponding tag storage block is not res= erved. > > > mte_restore_page_tags_by_swp_entry(), when it restores the saved tags= , will > > > overwrite data in the tag storage block associated with the new page, > > > leading to data corruption if the block is in use by a process. > > > > > > Get around this issue by saving the tags in a new xarray, this time i= ndexed > > > by the page pfn, and then restoring them when tag storage is reserved= for > > > the page. > > > > > > Signed-off-by: Alexandru Elisei > > > --- > > > > > > Changes since rfc v2: > > > > > > * Restore saved tags **before** setting the PG_tag_storage_reserved b= it to > > > eliminate a brief window of opportunity where userspace can access un= initialized > > > tags (Peter Collingbourne). > > > > > > arch/arm64/include/asm/mte_tag_storage.h | 8 ++ > > > arch/arm64/include/asm/pgtable.h | 11 +++ > > > arch/arm64/kernel/mte_tag_storage.c | 12 ++- > > > arch/arm64/mm/mteswap.c | 110 +++++++++++++++++++++= ++ > > > 4 files changed, 140 insertions(+), 1 deletion(-) > > > > > > diff --git a/arch/arm64/include/asm/mte_tag_storage.h b/arch/arm64/in= clude/asm/mte_tag_storage.h > > > index 50bdae94cf71..40590a8c3748 100644 > > > --- a/arch/arm64/include/asm/mte_tag_storage.h > > > +++ b/arch/arm64/include/asm/mte_tag_storage.h > > > @@ -36,6 +36,14 @@ bool page_is_tag_storage(struct page *page); > > > > > > vm_fault_t handle_folio_missing_tag_storage(struct folio *folio, str= uct vm_fault *vmf, > > > bool *map_pte); > > > +vm_fault_t mte_try_transfer_swap_tags(swp_entry_t entry, struct page= *page); > > > + > > > +void tags_by_pfn_lock(void); > > > +void tags_by_pfn_unlock(void); > > > + > > > +void *mte_erase_tags_for_pfn(unsigned long pfn); > > > +bool mte_save_tags_for_pfn(void *tags, unsigned long pfn); > > > +void mte_restore_tags_for_pfn(unsigned long start_pfn, int order); > > > #else > > > static inline bool tag_storage_enabled(void) > > > { > > > diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/as= m/pgtable.h > > > index 0174e292f890..87ae59436162 100644 > > > --- a/arch/arm64/include/asm/pgtable.h > > > +++ b/arch/arm64/include/asm/pgtable.h > > > @@ -1085,6 +1085,17 @@ static inline void arch_swap_invalidate_area(i= nt type) > > > mte_invalidate_tags_area_by_swp_entry(type); > > > } > > > > > > +#ifdef CONFIG_ARM64_MTE_TAG_STORAGE > > > +#define __HAVE_ARCH_SWAP_PREPARE_TO_RESTORE > > > +static inline vm_fault_t arch_swap_prepare_to_restore(swp_entry_t en= try, > > > + struct folio *f= olio) > > > +{ > > > + if (tag_storage_enabled()) > > > + return mte_try_transfer_swap_tags(entry, &folio->page= ); > > > + return 0; > > > +} > > > +#endif > > > + > > > #define __HAVE_ARCH_SWAP_RESTORE > > > static inline void arch_swap_restore(swp_entry_t entry, struct folio= *folio) > > > { > > > diff --git a/arch/arm64/kernel/mte_tag_storage.c b/arch/arm64/kernel/= mte_tag_storage.c > > > index afe2bb754879..ac7b9c9c585c 100644 > > > --- a/arch/arm64/kernel/mte_tag_storage.c > > > +++ b/arch/arm64/kernel/mte_tag_storage.c > > > @@ -567,6 +567,7 @@ int reserve_tag_storage(struct page *page, int or= der, gfp_t gfp) > > > } > > > } > > > > > > + mte_restore_tags_for_pfn(page_to_pfn(page), order); > > > page_set_tag_storage_reserved(page, order); > > > out_unlock: > > > mutex_unlock(&tag_blocks_lock); > > > @@ -595,7 +596,8 @@ void free_tag_storage(struct page *page, int orde= r) > > > struct tag_region *region; > > > unsigned long page_va; > > > unsigned long flags; > > > - int ret; > > > + void *tags; > > > + int i, ret; > > > > > > ret =3D tag_storage_find_block(page, &start_block, ®ion); > > > if (WARN_ONCE(ret, "Missing tag storage block for pfn 0x%lx",= page_to_pfn(page))) > > > @@ -605,6 +607,14 @@ void free_tag_storage(struct page *page, int ord= er) > > > /* Avoid writeback of dirty tag cache lines corrupting data. = */ > > > dcache_inval_tags_poc(page_va, page_va + (PAGE_SIZE << order)= ); > > > > > > + tags_by_pfn_lock(); > > > + for (i =3D 0; i < (1 << order); i++) { > > > + tags =3D mte_erase_tags_for_pfn(page_to_pfn(page + i)= ); > > > + if (unlikely(tags)) > > > + mte_free_tag_buf(tags); > > > + } > > > + tags_by_pfn_unlock(); > > > + > > > end_block =3D start_block + order_to_num_blocks(order, region= ->block_size_pages); > > > > > > xa_lock_irqsave(&tag_blocks_reserved, flags); > > > diff --git a/arch/arm64/mm/mteswap.c b/arch/arm64/mm/mteswap.c > > > index 2a43746b803f..e11495fa3c18 100644 > > > --- a/arch/arm64/mm/mteswap.c > > > +++ b/arch/arm64/mm/mteswap.c > > > @@ -20,6 +20,112 @@ void mte_free_tag_buf(void *buf) > > > kfree(buf); > > > } > > > > > > +#ifdef CONFIG_ARM64_MTE_TAG_STORAGE > > > +static DEFINE_XARRAY(tags_by_pfn); > > > + > > > +void tags_by_pfn_lock(void) > > > +{ > > > + xa_lock(&tags_by_pfn); > > > +} > > > + > > > +void tags_by_pfn_unlock(void) > > > +{ > > > + xa_unlock(&tags_by_pfn); > > > +} > > > + > > > +void *mte_erase_tags_for_pfn(unsigned long pfn) > > > +{ > > > + return __xa_erase(&tags_by_pfn, pfn); > > > +} > > > + > > > +bool mte_save_tags_for_pfn(void *tags, unsigned long pfn) > > > +{ > > > + void *entry; > > > + int ret; > > > + > > > + ret =3D xa_reserve(&tags_by_pfn, pfn, GFP_KERNEL); > > > > copy_highpage can be called from an atomic context, so it isn't > > currently valid to pass GFP_KERNEL here. > > > > To give one example of a possible atomic context call, copy_pte_range > > will take a PTE spinlock and can call copy_present_pte, which can call > > copy_present_page, which will call copy_user_highpage. > > > > To give another example, __buffer_migrate_folio can call > > spin_lock(&mapping->private_lock), then call folio_migrate_copy, which > > will call folio_copy. > > That is very unfortunate from my part. I distinctly remember looking > precisely at copy_page_range() to double check that it doesn't call > copy_*highpage() from an atomic context, I can only assume that I missed > that it's called with the ptl lock held. > > With your two examples, and the khugepaged case in patch #31 ("khugepaged= : > arm64: Don't collapse MTE enabled VMAs"), it's crystal clear that the > convention for copy_*highpage() is that the function cannot sleep. > > There are two issues here: allocating the buffer in memory where the tags > will be copied, and xarray allocating memory for a new entry. > > One fix would be to allocate an entire page with __GFP_ATOMIC, and use th= at > as a cache for tag buffers (storing the tags for a page uses 1/32th of a > page). From what little I know about xarray, xarray stores would still ha= ve > to be GFP_ATOMIC. This should fix the sleeping in atomic context bug. But > the issue I see with this is that a memory allocation can fail, while > copy_*highpage() cannot. Send a fatal signal to the process if memory > allocation fails? > > Another approach would be to preallocate memory in a preemptible context, > something like copy_*highpage_prepare(), but that would mean a lot more > work: finding all the places where copy_*highpage is used and add > copy_*highpage_prepare() outside the critical section, releasing the memo= ry > in case of failure (like in the copy_pte_range() case - maybe > copy_*highpage_end()?). That's a pretty big maintenance burden for the MM > code. Although maybe other architectures can find a use for it? > > And yet another approach is reserve the needed memory (for the buffer and > in the xarray) when the page is allocated, if it doesn't have tag storage > reserved, regardless of the page being allocated as tagged or not. Then i= n > set_pte_at() free this memory if it's unused. But this would mean reservi= ng > memory for possibly all memory allocations in the system (including for t= ag > storage pages) if userspace doesn't use tags at all, though not all pages > in the system will have this memory reserved at the same time. Pretty big > downside. > > Out of the three, I prefer the first, but it's definitely not perfect. I'= ll > try to think of something else, maybe I can come up with something better= . > > What are your thoughts? > > Thanks, > Alex > > > > > Peter > > > > > + if (ret) > > > + return true; > > > + > > > + tags_by_pfn_lock(); > > > + > > > + if (page_tag_storage_reserved(pfn_to_page(pfn))) { > > > + xa_release(&tags_by_pfn, pfn); > > > + tags_by_pfn_unlock(); > > > + return false; > > > + } > > > + > > > + entry =3D __xa_store(&tags_by_pfn, pfn, tags, GFP_ATOMIC); > > > + if (xa_is_err(entry)) { > > > + xa_release(&tags_by_pfn, pfn); > > > + goto out_unlock; > > > + } else if (entry) { > > > + mte_free_tag_buf(entry); > > > + } > > > + > > > +out_unlock: > > > + tags_by_pfn_unlock(); > > > + return true; > > > +} > > > + > > > +void mte_restore_tags_for_pfn(unsigned long start_pfn, int order) > > > +{ > > > + struct page *page =3D pfn_to_page(start_pfn); > > > + unsigned long pfn; > > > + void *tags; > > > + > > > + tags_by_pfn_lock(); > > > + > > > + for (pfn =3D start_pfn; pfn < start_pfn + (1 << order); pfn++= , page++) { > > > + tags =3D mte_erase_tags_for_pfn(pfn); > > > + if (unlikely(tags)) { > > > + /* > > > + * Mark the page as tagged so mte_sync_tags()= doesn't > > > + * clear the tags. > > > + */ > > > + WARN_ON_ONCE(!try_page_mte_tagging(page)); > > > + mte_copy_page_tags_from_buf(page_address(page= ), tags); > > > + set_page_mte_tagged(page); I hit a WARN_ON_ONCE inside `set_page_mte_tagged` at this call site, because the page does not have PG_tag_storage_reserved yet. Swap the order of calls in reserve_tag_storage? > > > + mte_free_tag_buf(tags); > > > + } > > > + } > > > + > > > + tags_by_pfn_unlock(); > > > +} > > > + > > > +/* > > > + * Note on locking: swap in/out is done with the folio locked, which= eliminates > > > + * races with mte_save/restore_page_tags_by_swp_entry. > > > + */ > > > +vm_fault_t mte_try_transfer_swap_tags(swp_entry_t entry, struct page= *page) > > > +{ > > > + void *swap_tags, *pfn_tags; > > > + bool saved; > > > + > > > + /* > > > + * mte_restore_page_tags_by_swp_entry() will take care of cop= ying the > > > + * tags over. > > > + */ > > > + if (likely(page_mte_tagged(page) || page_tag_storage_reserved= (page))) > > > + return 0; > > > + > > > + swap_tags =3D xa_load(&tags_by_swp_entry, entry.val); > > > + if (!swap_tags) > > > + return 0; > > > + > > > + pfn_tags =3D mte_allocate_tag_buf(); > > > + if (!pfn_tags) > > > + return VM_FAULT_OOM; > > > + > > > + memcpy(pfn_tags, swap_tags, MTE_PAGE_TAG_STORAGE_SIZE); > > > + saved =3D mte_save_tags_for_pfn(pfn_tags, page_to_pfn(page)); > > > + if (!saved) > > > + mte_free_tag_buf(pfn_tags); > > > + > > > + return 0; > > > +} > > > +#endif > > > + > > > int mte_save_page_tags_by_swp_entry(struct page *page) > > > { > > > void *tags, *ret; > > > @@ -54,6 +160,10 @@ void mte_restore_page_tags_by_swp_entry(swp_entry= _t entry, struct page *page) > > > if (!tags) > > > return; > > > > > > + /* Tags will be restored when tag storage is reserved. */ > > > + if (tag_storage_enabled() && unlikely(!page_tag_storage_reser= ved(page))) > > > + return; > > > + > > > if (try_page_mte_tagging(page)) { > > > mte_copy_page_tags_from_buf(page_address(page), tags)= ; > > > set_page_mte_tagged(page); > > > -- > > > 2.43.0 > > >