From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11F31C433F5 for ; Thu, 28 Apr 2022 15:17:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3A5936B00A2; Thu, 28 Apr 2022 11:17:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 356386B00A3; Thu, 28 Apr 2022 11:17:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F5E56B00A4; Thu, 28 Apr 2022 11:17:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.25]) by kanga.kvack.org (Postfix) with ESMTP id 106D56B00A2 for ; Thu, 28 Apr 2022 11:17:14 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id DBE5F82412 for ; Thu, 28 Apr 2022 15:17:13 +0000 (UTC) X-FDA: 79406641146.27.9B74D63 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf03.hostedemail.com (Postfix) with ESMTP id 19DAC2005E for ; Thu, 28 Apr 2022 15:17:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1651159032; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sHEg8M8sspeeYGEbdvtWdwBiaF22012ZA01u4D76s0o=; b=DIAA9eXDe4EBs5P19W2U57CKOhZ68IiGDPs2Jn5BQncOUHZNwL+ZMQG9/la1h5dgZUsAoR I2zITQYCdNG6X+YsyRP+RHYWtWNwV6Tu6fklQ6FFn88LhXnNW9Mpr1509ezA0UJR4VFVZp CvOYv4JZGS+MAeqJtu+QOVOLNwCqYWk= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-150-vqRIQHi4NTCv9FK_0vOBWQ-1; Thu, 28 Apr 2022 11:17:11 -0400 X-MC-Unique: vqRIQHi4NTCv9FK_0vOBWQ-1 Received: by mail-wr1-f71.google.com with SMTP id s8-20020adf9788000000b0020adb01dc25so2039939wrb.20 for ; Thu, 28 Apr 2022 08:17:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent :content-language:to:cc:references:from:organization:subject :in-reply-to:content-transfer-encoding; bh=sHEg8M8sspeeYGEbdvtWdwBiaF22012ZA01u4D76s0o=; b=Pzwo929rNlPhaZ7eGyHRPTaEpP+GGxff8YXXfjRmNSnNGAy/AJAFrxh/B13FrSusUV IBWskLHm/MadoUX/MoU0mkQ/C211rK+miL/d3H+DRQQ0XOsDT2uv3bHSKeJfUjH28VK3 rQ/CEcYR56sUhlm7yTwVpcav1GPjV49YV7BpYonM1vBxrGMbxv9JmXvy+MybGxA529/E x1GWV21yUzpOPLxZtU/7P745mc95cyInXWLUcxo9wWUDPlI8KKi1DzROxngrWry6TfcE GnbTXsp7m9voOknAOz1plleen+8ka+0VWfu7bWeltsUlJp+WKQepMfQqf+Q1G8Q+35YT NUxg== X-Gm-Message-State: AOAM530X+JoSFjT1Odzpow8wcgSBnZNj4QRVcXfaSDeMSs/hdFuKjrr9 Jb0MZ1g9gZhhecK9VOIkaGkVFODtpO3CkFGPvrEhglI6E0zNluB5XBF065swuGlF13mpfAh2jmE GVY6dFevoKQ4= X-Received: by 2002:a05:6000:1e05:b0:20a:ecc7:41cf with SMTP id bj5-20020a0560001e0500b0020aecc741cfmr8115364wrb.102.1651159029913; Thu, 28 Apr 2022 08:17:09 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx5BQnX6mIdCOLdW46rgDYhg84e4FCHOWeThU4njV4qp4XrgDxqv1rwiDHi8KjnhsRdvQ2u9w== X-Received: by 2002:a05:6000:1e05:b0:20a:ecc7:41cf with SMTP id bj5-20020a0560001e0500b0020aecc741cfmr8115344wrb.102.1651159029619; Thu, 28 Apr 2022 08:17:09 -0700 (PDT) Received: from ?IPV6:2003:cb:c708:ef00:7443:a23c:26b8:b96? (p200300cbc708ef007443a23c26b80b96.dip0.t-ipconnect.de. [2003:cb:c708:ef00:7443:a23c:26b8:b96]) by smtp.gmail.com with ESMTPSA id z11-20020a05600c220b00b00393ffde5f5fsm4731134wml.36.2022.04.28.08.17.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 28 Apr 2022 08:17:08 -0700 (PDT) Message-ID: <3a441789-b3e4-236e-2e44-e7a1c7258a94@redhat.com> Date: Thu, 28 Apr 2022 17:17:07 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.6.2 To: Bibo Mao , Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yang Shi References: <20220317065024.2635069-1-maobibo@loongson.cn> From: David Hildenbrand Organization: Red Hat Subject: Re: [PATCH v3] mm/khugepaged: sched to numa node when collapse huge page In-Reply-To: <20220317065024.2635069-1-maobibo@loongson.cn> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Stat-Signature: kf6a8xgn4md3fjs9qbjz8fftsrf7t5yg X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 19DAC2005E Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=DIAA9eXD; spf=none (imf03.hostedemail.com: domain of david@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspam-User: X-HE-Tag: 1651159028-241795 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 17.03.22 07:50, Bibo Mao wrote: > collapse huge page will copy huge page from general small pages, > dest node is calculated from most one of source pages, however > THP daemon is not scheduled on dest node. The performance may be > poor since huge page copying across nodes, also cache is not used > for target node. With this patch, khugepaged daemon switches to > the same numa node with huge page. It saves copying time and makes > use of local cache better. > > With this patch, specint 2006 base performance is improved with 6% > on Loongson 3C5000L platform with 32 cores and 8 numa nodes. If it helps, that's nice as long as it doesn't hurt other cases. > > Signed-off-by: Bibo Mao > --- > changelog: > V2: remove node record for thp daemon > V3: remove unlikely statement > --- > mm/khugepaged.c | 8 ++++++++ > 1 file changed, 8 insertions(+) > > diff --git a/mm/khugepaged.c b/mm/khugepaged.c > index 131492fd1148..b3cf0885f5a2 100644 > --- a/mm/khugepaged.c > +++ b/mm/khugepaged.c > @@ -1066,6 +1066,7 @@ static void collapse_huge_page(struct mm_struct *mm, > struct vm_area_struct *vma; > struct mmu_notifier_range range; > gfp_t gfp; > + const struct cpumask *cpumask; > > VM_BUG_ON(address & ~HPAGE_PMD_MASK); > > @@ -1079,6 +1080,13 @@ static void collapse_huge_page(struct mm_struct *mm, > * that. We will recheck the vma after taking it again in write mode. > */ > mmap_read_unlock(mm); > + > + /* sched to specified node before huage page memory copy */ huage? I assume "huge" > + if (task_node(current) != node) { > + cpumask = cpumask_of_node(node); > + if (!cpumask_empty(cpumask)) > + set_cpus_allowed_ptr(current, cpumask); > + } I wonder if that will always be optimized out without NUMA and if we want to check for IS_ENABLED(CONFIG_NUMA). Regarding comments from others, I agree: I think what we'd actually want is something like "try to reschedule to one of these CPUs immediately. If they are all busy, just stay here. Also, I do wonder if there could already be scenarios where someone wants to let khugepaged run only on selected housekeeping CPUs (e.g., when pinning VCPUs in a VM to physical CPUs). It might even degrade the VM performance in that case if we schedule something unrelated on these CPUs. (I don't know which interfaces we might already have to configure housekeeping CPUs for kthreads). I can spot in kernel/kthread.c:kthread() set_cpus_allowed_ptr(current, housekeeping_cpumask(HK_TYPE_KTHREAD)); Hmmmmm ... -- Thanks, David / dhildenb