From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF72FC433FE for ; Mon, 28 Nov 2022 13:46:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2582C6B0072; Mon, 28 Nov 2022 08:46:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2096D6B0073; Mon, 28 Nov 2022 08:46:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0A9376B0074; Mon, 28 Nov 2022 08:46:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id EFD336B0072 for ; Mon, 28 Nov 2022 08:46:35 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id C866D1A0CEE for ; Mon, 28 Nov 2022 13:46:35 +0000 (UTC) X-FDA: 80182975950.25.1DF9F71 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf17.hostedemail.com (Postfix) with ESMTP id 6641340018 for ; Mon, 28 Nov 2022 13:46:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669643193; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3HDzAsqBZ7PX3t0Ys2WvADREK4jr7JLgdDTg5ZTPBkU=; b=FFLG8KAI5G0WO5WQevdKIDCwhwwDLRFs7MsvxeqoFFtthkN4kn+X9iuAqCBFEq1hhA2PxZ dug/DaAoMoVdaYyBJaQJtQu+xhn+vBLFHgivj38URnf/KPdw8W9JjTZp1kK/xEtGVD+mZv lbZvbaykmGb09qJLBXhDEMs+fNI6XAg= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-55-WUQvIOlBNpuBijwQO9W9Sg-1; Mon, 28 Nov 2022 08:46:22 -0500 X-MC-Unique: WUQvIOlBNpuBijwQO9W9Sg-1 Received: by mail-wm1-f69.google.com with SMTP id j2-20020a05600c1c0200b003cf7397fc9bso6301646wms.5 for ; Mon, 28 Nov 2022 05:46:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:subject:organization:from :references:cc:to:content-language:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=3HDzAsqBZ7PX3t0Ys2WvADREK4jr7JLgdDTg5ZTPBkU=; b=ud5sv+ZQKwRnTDAeErWDLnUjz2Qg4utV3MOTOB44ZfpImiW+bh+rtmfSSuEg4conjr 62BhEur6LTz+OgPcMASyN+0MmZRgg/SqcPeJW0WwezSw1b0si9qBBEUhh8xv97SGGBXD KwgU1dUFa8J/wATlS1Z1xieYwY5dDCenlkimwQ/FFS6Kck7ecvw83pCEZguewV6gO8j0 WJWsVrZzUHY4cJqml9mHjsv4YMAr/fqSFLkjKGYNh0KnVP9N0PfqSNcoEiu8rHIuMRYF EGz7fpAtrOsIDXsVBNC2KVg3bzsKbSstnZZoeBFl9C3W6UTF6Q1l5bVbiEB4tlgwacyP +ibQ== X-Gm-Message-State: ANoB5pm+0zePWR1vnLurQ5SuY3w4WlOszAPUIS6+Eny859gNI4TMwD6q tePGTgdf0hkxbl1DxpvnaJlkUam5E16BaP9NrzN03vru0VEvtgR0evqmK77rtHizIT4o68CvtNV C7fjVSrhTINQ= X-Received: by 2002:adf:d1ca:0:b0:242:fce:543b with SMTP id b10-20020adfd1ca000000b002420fce543bmr5919897wrd.244.1669643181440; Mon, 28 Nov 2022 05:46:21 -0800 (PST) X-Google-Smtp-Source: AA0mqf5d0b0bHCirMEZ9PQgiB4yBu033LT8ciePisgG8Py7+aHKQ3vBiZ5+w4wTctfHe3DjkQ7NmKw== X-Received: by 2002:adf:d1ca:0:b0:242:fce:543b with SMTP id b10-20020adfd1ca000000b002420fce543bmr5919872wrd.244.1669643181117; Mon, 28 Nov 2022 05:46:21 -0800 (PST) Received: from ?IPV6:2003:cb:c702:9000:3d6:e434:f8b4:80cf? (p200300cbc702900003d6e434f8b480cf.dip0.t-ipconnect.de. [2003:cb:c702:9000:3d6:e434:f8b4:80cf]) by smtp.gmail.com with ESMTPSA id g11-20020a05600c310b00b003cfd4e6400csm16715113wmo.19.2022.11.28.05.46.19 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 28 Nov 2022 05:46:20 -0800 (PST) Message-ID: Date: Mon, 28 Nov 2022 14:46:19 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.1 To: Jann Horn , security@kernel.org, Andrew Morton Cc: Yang Shi , Peter Xu , John Hubbard , linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20221125213714.4115729-1-jannh@google.com> <20221125213714.4115729-2-jannh@google.com> From: David Hildenbrand Organization: Red Hat Subject: Re: [PATCH v3 2/3] mm/khugepaged: Fix GUP-fast interaction by sending IPI In-Reply-To: <20221125213714.4115729-2-jannh@google.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1669643194; a=rsa-sha256; cv=none; b=T2Tv7AaoFMrq3GMGe0ywHzXL91PNF1hVZnw1I7rr5jaizh/36QsfxRENpx6fpY9Io20vdS Gg7YggIxZi6fKy2FPEOgKV48HO+qrxiWT450+Rc7O8dyb1zYfFfKclaom6/eTkASgj5jsX Fsioh981AE60317Fsc4KnPr9p690MRQ= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=FFLG8KAI; spf=pass (imf17.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1669643194; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3HDzAsqBZ7PX3t0Ys2WvADREK4jr7JLgdDTg5ZTPBkU=; b=NyACF3WDjPOQW1yvBKsj1KepZ3lKvUEQQMqWqO0zl7DJgWA/dwH9TM5/ckNEQnP7tPFO+8 xufqhWqBTsioZ77BLTra7R+4XuTuTW0gqe3qTURI35LGcWpZBdne0gcmH4z6v8HvAc+L0Z qTpYiXE80VluwLJiTGSA8EH0iqOSgHw= X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 6641340018 X-Rspam-User: Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=FFLG8KAI; spf=pass (imf17.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Stat-Signature: x6d1twkrd7eobmau75cyihfnkyety7co X-HE-Tag: 1669643194-930927 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 25.11.22 22:37, Jann Horn wrote: > Since commit 70cbc3cc78a99 ("mm: gup: fix the fast GUP race against THP > collapse"), the lockless_pages_from_mm() fastpath rechecks the pmd_t to > ensure that the page table was not removed by khugepaged in between. > > However, lockless_pages_from_mm() still requires that the page table is not > concurrently freed. That's an interesting point. For anon THPs, the page table won't get immediately freed, but instead will be deposited in the "pgtable list" stored alongside the THP. From there, it might get withdrawn (pgtable_trans_huge_withdraw()) and a) Reused as a page table when splitting the THP. That should be fine, no garbage in it, simply a page table again. b) Freed when zapping the THP (zap_deposited_table()). that would be bad. ... but I just realized that e.g., radix__pgtable_trans_huge_deposit uses actual page content to link the deposited page tables, which means we'd already storing garbage in there when depositing the page, not when freeing+reusing the page .... Maybe worth adding to the description. > Fix it by sending IPIs (if the architecture uses > semi-RCU-style page table freeing) before freeing/reusing page tables. > > Cc: stable@kernel.org > Fixes: ba76149f47d8 ("thp: khugepaged") > Signed-off-by: Jann Horn > --- > replaced the mmu_gather-based scheme with an RCU call as suggested by > Peter Xu > > include/asm-generic/tlb.h | 4 ++++ > mm/khugepaged.c | 2 ++ > mm/mmu_gather.c | 4 +--- > 3 files changed, 7 insertions(+), 3 deletions(-) > > diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h > index 492dce43236ea..cab7cfebf40bd 100644 > --- a/include/asm-generic/tlb.h > +++ b/include/asm-generic/tlb.h > @@ -222,12 +222,16 @@ extern void tlb_remove_table(struct mmu_gather *tlb, void *table); > #define tlb_needs_table_invalidate() (true) > #endif > > +void tlb_remove_table_sync_one(void); > + > #else > > #ifdef tlb_needs_table_invalidate > #error tlb_needs_table_invalidate() requires MMU_GATHER_RCU_TABLE_FREE > #endif > > +static inline void tlb_remove_table_sync_one(void) { } > + > #endif /* CONFIG_MMU_GATHER_RCU_TABLE_FREE */ > > > diff --git a/mm/khugepaged.c b/mm/khugepaged.c > index 674b111a24fa7..c3d3ce596bff7 100644 > --- a/mm/khugepaged.c > +++ b/mm/khugepaged.c > @@ -1057,6 +1057,7 @@ static int collapse_huge_page(struct mm_struct *mm, unsigned long address, > _pmd = pmdp_collapse_flush(vma, address, pmd); > spin_unlock(pmd_ptl); > mmu_notifier_invalidate_range_end(&range); > + tlb_remove_table_sync_one(); > > spin_lock(pte_ptl); > result = __collapse_huge_page_isolate(vma, address, pte, cc, > @@ -1415,6 +1416,7 @@ static void collapse_and_free_pmd(struct mm_struct *mm, struct vm_area_struct *v > lockdep_assert_held_write(&vma->anon_vma->root->rwsem); > > pmd = pmdp_collapse_flush(vma, addr, pmdp); > + tlb_remove_table_sync_one(); > mm_dec_nr_ptes(mm); > page_table_check_pte_clear_range(mm, addr, pmd); > pte_free(mm, pmd_pgtable(pmd)); > diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c > index add4244e5790d..3a2c3f8cad2fe 100644 > --- a/mm/mmu_gather.c > +++ b/mm/mmu_gather.c > @@ -153,7 +153,7 @@ static void tlb_remove_table_smp_sync(void *arg) > /* Simply deliver the interrupt */ > } > > -static void tlb_remove_table_sync_one(void) > +void tlb_remove_table_sync_one(void) > { > /* > * This isn't an RCU grace period and hence the page-tables cannot be > @@ -177,8 +177,6 @@ static void tlb_remove_table_free(struct mmu_table_batch *batch) > > #else /* !CONFIG_MMU_GATHER_RCU_TABLE_FREE */ > > -static void tlb_remove_table_sync_one(void) { } > - > static void tlb_remove_table_free(struct mmu_table_batch *batch) > { > __tlb_remove_table_free(batch); With CONFIG_MMU_GATHER_RCU_TABLE_FREE this will most certainly do the right thing. I assume with CONFIG_MMU_GATHER_RCU_TABLE_FREE, the assumption is that there will be an implicit IPI. That implicit IPI has to happen before we deposit. I assume that is expected to happen during pmdp_collapse_flush() ? -- Thanks, David / dhildenb