From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5BBDFC25B74 for ; Tue, 21 May 2024 23:30:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4C0926B0085; Tue, 21 May 2024 19:29:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4712C6B0088; Tue, 21 May 2024 19:29:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 35F6F6B0089; Tue, 21 May 2024 19:29:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 1376F6B0085 for ; Tue, 21 May 2024 19:29:59 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 77E9A1C21D5 for ; Tue, 21 May 2024 23:29:58 +0000 (UTC) X-FDA: 82143998076.06.31F1D55 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf01.hostedemail.com (Postfix) with ESMTP id BB0B440002 for ; Tue, 21 May 2024 23:29:56 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=NXIbOsdh; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of 3cy5NZgYKCD4xjfsohlttlqj.htrqnsz2-rrp0fhp.twl@flex--seanjc.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3cy5NZgYKCD4xjfsohlttlqj.htrqnsz2-rrp0fhp.twl@flex--seanjc.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716334196; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+BzI0I7st4rXq5MwmbAV5Tzi38yIZZhUt1WvSYOgULQ=; b=AV+HmzFoahGDUxXn8COR7zXAfpwy79S6FcePu1JNpaBcms6qfnhfymNL2bqf+wDyvSCpkB 0nJh9NZmW2g+pdcXKTPTWXH9Q/KO8FFLO/hYaOKnll/rGMQFLvqiQd1dnc6dk7T9FN/fjA qWymMDZivyeN6ckiiofLzaTeSGKk4IU= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=NXIbOsdh; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of 3cy5NZgYKCD4xjfsohlttlqj.htrqnsz2-rrp0fhp.twl@flex--seanjc.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3cy5NZgYKCD4xjfsohlttlqj.htrqnsz2-rrp0fhp.twl@flex--seanjc.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716334196; a=rsa-sha256; cv=none; b=0pTwUQ5MQw4qumGhjsvdoIka63KbUTNcw0Zf3aqFkCoOVKWD3BEp7LlIbW78ZiYdqJTu8G hLx2kOg3qy7bjCqAO0k/n8DIb1zyIQNu84pOswm/bq+hh8vydGL5qab0Kw+bGnpp+oI+8s y26J4n1bz3LVRHbzEGZZEl607Eo4n+w= Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-61be325413eso5029237b3.1 for ; Tue, 21 May 2024 16:29:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1716334196; x=1716938996; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=+BzI0I7st4rXq5MwmbAV5Tzi38yIZZhUt1WvSYOgULQ=; b=NXIbOsdhkNZtVTQ17brtG5TwjTqUo3/b9e1altbH8s2yC8IXQbD8A74PbpY8eNA36o z/6+l7vg0zp10pEz3Pd8bVtjyVskG/vv9sB0Df4AMI3ZG34PMkgohcziq0P7g574LJ+Q dysBejVbSNPMnD38wVXO8iTwFlag2gNzeJs2nxnMCUT2vizZhg8b4j5k+aPwtTA7iP6b quwJ6QlP/W7GH+8C+ZMb6jj8YpUTPK3CjxUm0XQNfuwXGzqVKF/eYUCR9b+t5AZflo9q y7mNbh8SR3BKTiuyWb8i+jMVw3tFXPn7Cb5Sb76c1D8CzT/TQidwjQgE3/yEynFyMkVF DOuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716334196; x=1716938996; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+BzI0I7st4rXq5MwmbAV5Tzi38yIZZhUt1WvSYOgULQ=; b=RTBf2R3MTiST0X/bqbdhWk/cwFZZfAd3soH/z7FGEhopaA4YgXUORnUjq3TtX/eWEu yNtqyTIRJaPNPx0nhwhDMzKSsoQG6pBdjBhBEXhTXKSVBEhZTc9Ki2YWPRUzxI+2Dm2G IgtdE1kAJi4SbMETTTHU15m9VTF3rv11j9FJKIwVjZD4atJlUAeQtZADACM/Y32/cktk SRao/0uBsWupYCtRCVkf09vELVs6SVJpDdk8MdpaJFRflvRbBvaBngIvnM9oIkDOM+46 ti7luR0qJupSy1HBNwIEy+19xUkyPN04wjPWE+3H0QhEdf3oopAdbyJzVICJ73AZvnwx M1PA== X-Forwarded-Encrypted: i=1; AJvYcCXBDrwJ/QuL46LDBBOWZfq8ZH6YZjRQom2D6UQUnADuS8AlLRY46VbRq7ImA5UqRzXQdxBfk3WwGvelkUKrT4+XJKw= X-Gm-Message-State: AOJu0YxterqW/2ArVM4H/iEA7IVu88v2ZCStV64JejS+cVX7NCn9gwZC UBjqFIEjAwUMjcGyb9FkX5hlovLp6+h5aTJ4WcI0r2kBYwbTKqmePgKZTjFKNkruvIIPFmeVWkD PCg== X-Google-Smtp-Source: AGHT+IFdSbE18dHmXUZm+Ed94A9LAnG/1FtV6QIW3gxa+x5R7ywczK/so7v774Y5MGmKju/Uxz5TPDu07oo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a81:49c3:0:b0:618:5009:cb71 with SMTP id 00721157ae682-627e4892a8amr1387677b3.5.1716334195799; Tue, 21 May 2024 16:29:55 -0700 (PDT) Date: Tue, 21 May 2024 16:29:54 -0700 In-Reply-To: <7a46456d6750ea682ba321ad09541fa81677b81a.camel@redhat.com> Mime-Version: 1.0 References: <7a46456d6750ea682ba321ad09541fa81677b81a.camel@redhat.com> Message-ID: Subject: Re: access_tracking_perf_test kvm selftest doesn't work when Multi-Gen LRU is in use From: Sean Christopherson To: Maxim Levitsky Cc: kvm@vger.kernel.org, Paolo Bonzini , Henry Huang , linux-mm@kvack.org Content-Type: text/plain; charset="us-ascii" X-Rspamd-Queue-Id: BB0B440002 X-Stat-Signature: p3gq9zj4fezk4wjb75rzq15ckq6sna89 X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1716334196-971275 X-HE-Meta: U2FsdGVkX1/nKdsWQqQxUS9DR+fnOYLkWZeUTAolkIVHF8Lt7dOgZB5vXGquWj0PZFx11TBea15OsxZ6RQy6xSjmqm+p2YfuxpqSzBA+WP/k1YzGJ9z7XCtBkMQFtEJszUoQKe138n/F2l0Pvzi82vTJwjZnqq8Ni815TZSmWl4PVTSqi7D+exNibMRt3AUDyvn3axk6FG+6uHghBi3kc2y0Yp59zUOm03k2r0CQscfTGFtotWmDxmKYq6NsDKzX4ElX65ejcqCO7mqgMBL8ZjkBIfpczLCY4KlXr7AFKc6ywXQxonirRWlPyEfBtpiQRoUNa4ttLTmNu0Q94yYLi9bZGxnwMmyl/Rmu6b1U3i+LGg2RfrWlsTE4lNp84kSWwCJTAvwU7lG2Ql+tHyfiDzREU0tG4atjSDVBFV5YNUTEsdZWkUCY/iz0FF10sNJLqVQH1ipAVnVIifVqsJk/y4Cz0SkKs/XaWJ3tYaEy/565AljXOYZlirS/4e/jp1y5ofjg+O5xLrYFiCOmblWB6W1R6MKBoytC4b6MGDbX3jNS7N9t87G5FvBLnLuf87pQv3Yqh85W7YsgOPsA6K2MRE0IJXMU5Eibya6iDjRlPwerVJ76wIV8By/Z0Y+SMpn9IeARO8PAOSJh4CpZ0FJrA5IYf+pHy0Zl64gj7A2uWkA8Jc0Tczqhy6NitS8qSdrsCr6ILELHssZfX5/2XPvydS/7M533CipobMCPrpIkgVsPis/bgQ0Ouy5gLiCwXX0eYn+kUly4EJeF750/ljHMeFNKGW3YtVv0g6p59HM8ZFcbwso7aHNo0j+uebLdKANW2IkZuk16c3zwfPMqazYJhofseTGEKCPXoOfzeoSNRUktgLzdAEbFoxEVO575d1NNWf4LOIBGVCi6r6zK9hZlgQgbQTNMG2a9JmEwHVxDMwlpd5DIbUbVFEyO+pq47PRN6Szx5BQb76nbi6NulSa 24ANly8Y LOAoAnU/5LkY9mOSfYPM/LKztun4SHJ3tBDNRTOwZuLhnQEx12jQbzR2ggn/AAA4H9bzqcHIa84uMn0LeQ8pSEg0RBxhlGprVTSbSa3kdS7whKatobX6+JXbS6caDHWc8o3ZJ+B/ZAFOc93W2Tx6Xc0O3cx20m3yQlnDNonveyW1uH/54BrTaRRtoE6nGKEQaT6O4rQv8R4kuizqaH2mEC60dpaasEZFPPZN18E+XotBKIKiff9ZOof11HTcAKxSSYe0aN0rLB/yxUIStUCaXknwvuJKmNU6ASq1NmtqTEaCM54UuSNwgrb+slZkOcQjbvEKWwGmUQTgXQBt2UfGDCklVmV2W/HstsjASwT7lj3rY74AHMMZSSuKX+LOoo0O4vySnV2zsSfw1UYaNFuSjb8DoL2ZEfLk3s06RKlZ5hklTZRYCYcotGey9oacAIKIU6RhV2Ch0MaPnT573qxRCJFxHS526Sq0L43Q2azDJYWga490PzUAQ1Dgjd+boxrPb3+YMo85lcvQormH1iYg6RoM/uyWAipnm5D/YCL+ojbNiIkslG6DJzJrysCy6sZGbPa+Zll2VO83L+SEMjWcvTH3nXEroZ8joxmx0kqbO1fOPO59StfwIzWXfQ1sRBjnblVhx X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, May 15, 2024, Maxim Levitsky wrote: > Small note on why we started seeing this failure on RHEL 9 and only on some machines: > > - RHEL9 has MGLRU enabled, RHEL8 doesn't. For a stopgap in KVM selftests, or possibly even a long term solution in case the decision is that page_idle will simply have different behavior for MGLRU, couldn't we tweak the test to not assert if MGRLU is enabled? E.g. refactor get_module_param_integer() and/or get_module_param() to add get_sysfs_value_integer() or so, and then do this? diff --git a/tools/testing/selftests/kvm/access_tracking_perf_test.c b/tools/testing/selftests/kvm/access_tracking_perf_test.c index 3c7defd34f56..1e759df36098 100644 --- a/tools/testing/selftests/kvm/access_tracking_perf_test.c +++ b/tools/testing/selftests/kvm/access_tracking_perf_test.c @@ -123,6 +123,11 @@ static void mark_page_idle(int page_idle_fd, uint64_t pfn) "Set page_idle bits for PFN 0x%" PRIx64, pfn); } +static bool is_lru_gen_enabled(void) +{ + return !!get_sysfs_value_integer("/sys/kernel/mm/lru_gen/enabled"); +} + static void mark_vcpu_memory_idle(struct kvm_vm *vm, struct memstress_vcpu_args *vcpu_args) { @@ -185,7 +190,8 @@ static void mark_vcpu_memory_idle(struct kvm_vm *vm, */ if (still_idle >= pages / 10) { #ifdef __x86_64__ - TEST_ASSERT(this_cpu_has(X86_FEATURE_HYPERVISOR), + TEST_ASSERT(this_cpu_has(X86_FEATURE_HYPERVISOR) || + is_lru_gen_enabled(), "vCPU%d: Too many pages still idle (%lu out of %lu)", vcpu_idx, still_idle, pages); #endif > - machine needs to have more than one NUMA node because NUMA balancing > (enabled by default) tries apparently to write protect the primary PTEs > of (all?) processes every few seconds, and that causes KVM to flush the secondary PTEs: > (at least with new tdp mmu) > > access_tracking-3448 [091] ....1.. 1380.244666: handle_changed_spte <-tdp_mmu_set_spte > access_tracking-3448 [091] ....1.. 1380.244667: > => cdc_driver_init > => handle_changed_spte > => tdp_mmu_set_spte > => tdp_mmu_zap_leafs > => kvm_tdp_mmu_unmap_gfn_range > => kvm_unmap_gfn_range > => kvm_mmu_notifier_invalidate_range_start > => __mmu_notifier_invalidate_range_start > => change_p4d_range > => change_protection > => change_prot_numa > => task_numa_work > => task_work_run > => exit_to_user_mode_prepare > => syscall_exit_to_user_mode > => do_syscall_64 > => entry_SYSCALL_64_after_hwframe > > It's a separate question, if the NUMA balancing should do this, or if NUMA > balancing should be enabled by default, FWIW, IMO, enabling NUMA balancing on a system whose primary purpose is to run VMs is bad idea. NUMA balancing operates under the assumption that a !PRESENT #PF is relatively cheap. When secondary MMUs are involved, that is simply not the case, e.g. to honor the mmu_notifer event, KVM zaps _and_ does a remote TLB flush. Even if we reworked KVM and/or the mmu_notifiers so that KVM didn't need to do such a heavy operation, the cost of page fault VM-Exit is significantly higher than the cost of a host #PF. > because there are other reasons that can force KVM to invalidate the > secondary mappings and trigger this issue. Ya.