From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3E40C04A94 for ; Thu, 10 Aug 2023 13:16:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 78D8B6B0071; Thu, 10 Aug 2023 09:16:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 73DB56B0074; Thu, 10 Aug 2023 09:16:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6056F6B0075; Thu, 10 Aug 2023 09:16:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 4D7A96B0071 for ; Thu, 10 Aug 2023 09:16:58 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id E74B08090F for ; Thu, 10 Aug 2023 13:16:57 +0000 (UTC) X-FDA: 81108245274.12.8564EDC Received: from mail.loongson.cn (mail.loongson.cn [114.242.206.163]) by imf02.hostedemail.com (Postfix) with ESMTP id 48E4880006 for ; Thu, 10 Aug 2023 13:16:53 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; spf=pass (imf02.hostedemail.com: domain of maobibo@loongson.cn designates 114.242.206.163 as permitted sender) smtp.mailfrom=maobibo@loongson.cn; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1691673415; a=rsa-sha256; cv=none; b=oYsj+Epta0jDmQx1Wc/IBc5XC6FD0suyREYPNxS4OYmOWllaPnhcFXYXOSbewFhijquEGC HZvk4zCqP9ERUW8jWFxntYWa0h9QTT25synBHY/Ap7ywdhtoDodqg5f0Pbdtv5xL8eEy5W PCYU8f16LTUtCNLPdTl/pBEKqXgfXSg= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; spf=pass (imf02.hostedemail.com: domain of maobibo@loongson.cn designates 114.242.206.163 as permitted sender) smtp.mailfrom=maobibo@loongson.cn; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1691673415; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=N5G4ghcYozpZjNOyPaiZ0xkoR+BtTvp8ocfchwskmgs=; b=MO9/PYU0Sd4aSirRyI2PuPHPOqpOzM7YZZU4ogVaoJ6QMPh+fnMu9QDvuNJLl1GWOV28cZ cL2hgNMk7OMujIOn8NK8cO2WX4rMKH+XKO/4+9dpfxPvlliW8/Q8ac1hiWobOBUGJiyAeu UtmgetAoija9iCOq987fKLE9rlLZFKI= Received: from loongson.cn (unknown [10.20.42.170]) by gateway (Coremail) with SMTP id _____8CxfOpC49RkBrEUAA--.17584S3; Thu, 10 Aug 2023 21:16:50 +0800 (CST) Received: from [10.20.42.170] (unknown [10.20.42.170]) by localhost.localdomain (Coremail) with SMTP id AQAAf8Ax98xB49RkxTVTAA--.56519S3; Thu, 10 Aug 2023 21:16:49 +0800 (CST) Message-ID: <277ee023-dc94-6c23-20b2-7deba641f1b1@loongson.cn> Date: Thu, 10 Aug 2023 21:16:49 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: [RFC PATCH v2 5/5] KVM: Unmap pages only when it's indeed protected for NUMA migration Content-Language: en-US To: Yan Zhao , linux-mm@kvack.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: pbonzini@redhat.com, seanjc@google.com, mike.kravetz@oracle.com, apopple@nvidia.com, jgg@nvidia.com, rppt@kernel.org, akpm@linux-foundation.org, kevin.tian@intel.com, david@redhat.com References: <20230810085636.25914-1-yan.y.zhao@intel.com> <20230810090218.26244-1-yan.y.zhao@intel.com> From: bibo mao In-Reply-To: <20230810090218.26244-1-yan.y.zhao@intel.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-CM-TRANSID:AQAAf8Ax98xB49RkxTVTAA--.56519S3 X-CM-SenderInfo: xpdruxter6z05rqj20fqof0/ X-Coremail-Antispam: 1Uk129KBj93XoWxXr1xXFWDZw4kWFW3tFyDtwc_yoW5CrW8pF WDKrZ5GFsrX3yqgayjqa1vya43XrZ7Wa18Ja4fGr9xtFn0grnrJrW8KwnFvFykAr9YqF13 Zayjqr18u34UAagCm3ZEXasCq-sJn29KB7ZKAUJUUUU5529EdanIXcx71UUUUU7KY7ZEXa sCq-sGcSsGvfJ3Ic02F40EFcxC0VAKzVAqx4xG6I80ebIjqfuFe4nvWSU5nxnvy29KBjDU 0xBIdaVrnRJUUUvIb4IE77IF4wAFF20E14v26r1j6r4UM7CY07I20VC2zVCF04k26cxKx2 IYs7xG6rWj6s0DM7CIcVAFz4kK6r1Y6r17M28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48v e4kI8wA2z4x0Y4vE2Ix0cI8IcVAFwI0_Gr0_Xr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI 0_Gr0_Cr1l84ACjcxK6I8E87Iv67AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVCY1x0267AK xVW8Jr0_Cr1UM2AIxVAIcxkEcVAq07x20xvEncxIr21l57IF6xkI12xvs2x26I8E6xACxx 1l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1Y6r17McIj6I8E87Iv 67AKxVW8JVWxJwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IY64vIr41lc7I2V7IY0VAS07 AlzVAYIcxG8wCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02 F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_Jw0_GF ylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7Cj xVAFwI0_Jr0_Gr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r 4j6F4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x07jY SoJUUUUU= X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 48E4880006 X-Stat-Signature: iazoamif344nepdhfdw3ccs61zcjhuwo X-HE-Tag: 1691673413-59554 X-HE-Meta: U2FsdGVkX1/dwBwChZjetCfWmn7Z8lXek1I6YaongZvUiFlNl4WN3RV+LPq6wRqQ4biqGkaE9jU+y0Ucba+Xdq0d9nYWBcWwfC28k6ea0JCwlJ62hSqInciYdp6yVyNLZ51dVLZnRMrm1ouq1TlMDk+pUfI9/+OLsrTqirajkHIYWq16QzqE7zsrfCE8SfTYrPPHToagbt0z+M3SwMr+lsGsHISLAUsgoH/mbjODN/bJs/AYw/IQUVDqQrxFepB7EIE3rT+CvaA2+VNTI/hA5n056P0NAYXPcmR648KjZ4rfQUzpIhYnC8f0FF3swwjH2bBvVBwvJEIeBBS1lqBZ/AzppLrMaeFn3s4SMjT3h89r5ZgMNeauGsdkrl+2wpE4LdlKEFzff6m36wgd5n5ytRYJLPXziIck+31Ll5q8vscz63FJgpFtADAYsU+v2QDkrzZoIiJxZLkKiLKn5sIMTMNC7HORT+ztt/XHzXU1WhhHsR39pcPNWzd8cUcnUJNNMs7E9KqlQ+f/1lelRL966/GMolp7Gwh2Hd1HVXJLK8unYpVjbZny6koYUxHXJflT9RzQ9ew2FVNFMCdq8DB6CUN2njTCTKvOi46Boa6MOrltpTtv19/C+3FAYNA7qg2gaAAVsqcXQ7yqJDqrfX70wEa58gWterElAuYX+SDsSsghkh7lwkVYJZXmXskG6Lymtn1hE9j9Arr9uz4lz4TDUwbNFB9PEstQM1CE73+KIWz3OuIUBV/OdtVc7Z/QwGFRXS1WPIOBsLy8nb5xpnriSP4vS8hR9jdmTk0iXZKB0OZVbX6IDj38C1kd+SLFg2M0Q185UooQfF+zeKd5Zo2Vevb1+2gRxEFifiWC10F2lQ/HZA6dmCuPIHF5w3MbK9KVUsmECyaZltGh1/w1hsEjeY02LqmvRfw96FLsyFNRRutM//m/L0oZ07O0a/2+mIQ7bxsDgSiPAWnvWijmW5F E+WPNhDR IfBVsUFZkCvp85uMJSz5GWYre2Inj5i8wmAJvpBsUNR93jveF31+ANBLxYQOS67AE5pksUOC0fNZjU/I= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: 在 2023/8/10 17:02, Yan Zhao 写道: > Register to .numa_protect() callback in mmu notifier so that KVM can get > acurate information about when a page is PROT_NONE protected in primary > MMU and unmap it in secondary MMU accordingly. > > In KVM's .invalidate_range_start() handler, if the event is to notify that > the range may be protected to PROT_NONE for NUMA migration purpose, > don't do the unmapping in secondary MMU. Hold on until.numa_protect() > comes. > > Signed-off-by: Yan Zhao > --- > virt/kvm/kvm_main.c | 25 ++++++++++++++++++++++--- > 1 file changed, 22 insertions(+), 3 deletions(-) > > diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c > index dfbaafbe3a00..907444a1761b 100644 > --- a/virt/kvm/kvm_main.c > +++ b/virt/kvm/kvm_main.c > @@ -711,6 +711,20 @@ static void kvm_mmu_notifier_change_pte(struct mmu_notifier *mn, > kvm_handle_hva_range(mn, address, address + 1, pte, kvm_change_spte_gfn); > } > > +static void kvm_mmu_notifier_numa_protect(struct mmu_notifier *mn, > + struct mm_struct *mm, > + unsigned long start, > + unsigned long end) > +{ > + struct kvm *kvm = mmu_notifier_to_kvm(mn); > + > + WARN_ON_ONCE(!READ_ONCE(kvm->mn_active_invalidate_count)); > + if (!READ_ONCE(kvm->mmu_invalidate_in_progress)) > + return; > + > + kvm_handle_hva_range(mn, start, end, __pte(0), kvm_unmap_gfn_range); > +} numa balance will scan wide memory range, and there will be one time ipi notification with kvm_flush_remote_tlbs. With page level notification, it may bring out lots of flush remote tlb ipi notification. however numa balance notification, pmd table of vm maybe needs not be freed in kvm_unmap_gfn_range. Regards Bibo Mao > + > void kvm_mmu_invalidate_begin(struct kvm *kvm, unsigned long start, > unsigned long end) > { > @@ -744,14 +758,18 @@ static int kvm_mmu_notifier_invalidate_range_start(struct mmu_notifier *mn, > const struct mmu_notifier_range *range) > { > struct kvm *kvm = mmu_notifier_to_kvm(mn); > + bool is_numa = (range->event == MMU_NOTIFY_PROTECTION_VMA) && > + (range->flags & MMU_NOTIFIER_RANGE_NUMA); > const struct kvm_hva_range hva_range = { > .start = range->start, > .end = range->end, > .pte = __pte(0), > - .handler = kvm_unmap_gfn_range, > + .handler = !is_numa ? kvm_unmap_gfn_range : > + (void *)kvm_null_fn, > .on_lock = kvm_mmu_invalidate_begin, > - .on_unlock = kvm_arch_guest_memory_reclaimed, > - .flush_on_ret = true, > + .on_unlock = !is_numa ? kvm_arch_guest_memory_reclaimed : > + (void *)kvm_null_fn, > + .flush_on_ret = !is_numa ? true : false, > .may_block = mmu_notifier_range_blockable(range), > }; > > @@ -899,6 +917,7 @@ static const struct mmu_notifier_ops kvm_mmu_notifier_ops = { > .clear_young = kvm_mmu_notifier_clear_young, > .test_young = kvm_mmu_notifier_test_young, > .change_pte = kvm_mmu_notifier_change_pte, > + .numa_protect = kvm_mmu_notifier_numa_protect, > .release = kvm_mmu_notifier_release, > }; >