From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 238DFC433F5 for ; Thu, 28 Apr 2022 13:50:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4FB636B008C; Thu, 28 Apr 2022 09:50:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4AA176B0093; Thu, 28 Apr 2022 09:50:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 349046B0095; Thu, 28 Apr 2022 09:50:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.25]) by kanga.kvack.org (Postfix) with ESMTP id 27FB66B008C for ; Thu, 28 Apr 2022 09:50:18 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id AC9052A0D1 for ; Thu, 28 Apr 2022 13:50:17 +0000 (UTC) X-FDA: 79406422074.26.5ED6982 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf17.hostedemail.com (Postfix) with ESMTP id 7C17A4005B for ; Thu, 28 Apr 2022 13:50:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1651153816; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=eSrTpAY01c2sAeG8S3OyerXPiSesjlWTEqP60Y4j5NQ=; b=FBOD66HFobKXjoXgGGgK/8a+XDqHvcP59iabzMfGnRcnO0vMhZLe6Gewp8W8vbWzGQLPFh 3SfowZ3ijsqbF0zyh3kx4BwrbESpGtZ4E0fEtA9cWqP9m1QY1vn9jVFtLY0fGPgWpX/rzu D6t3TNeTyaL0Hly/CUmrhX2Ao8GyVuw= Received: from mail-io1-f72.google.com (mail-io1-f72.google.com [209.85.166.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-509-uYqPaeayMr2EBu0zUK-cjQ-1; Thu, 28 Apr 2022 09:50:15 -0400 X-MC-Unique: uYqPaeayMr2EBu0zUK-cjQ-1 Received: by mail-io1-f72.google.com with SMTP id r17-20020a0566022b9100b00654b99e71dbso4530778iov.3 for ; Thu, 28 Apr 2022 06:50:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=eSrTpAY01c2sAeG8S3OyerXPiSesjlWTEqP60Y4j5NQ=; b=8ECdnz5y311bn1VsQH33EW0YpsV/uhiEWSf0aDbPyi0w57s5pOjxB7LhHMKSZpPWcb f6zZOcRD3GmLHdbGxnzX3TBVHwZUCNyZsHNiCgA1r6+z1+c8mlLFfj3SKTlxTP+TmlCy yeGjphp9pcZ05aoNTuSjZLufiB6h78EruFooENE8SP7+gsv1UY6OXMCT2C3ntdJb6zvg s4gQzHs7tPddOeixWa09I+I8svc1PMO1oc0apUhN3ZNw5KL9Eir3b1o5CpCfSKdVYg5Z 70VcB3qa+bl8KqGah+Q6tlHZdKrlfVh8W26MdBhx+lgr3rrc/9AhXmAjhkh96TprR4bA XTMQ== X-Gm-Message-State: AOAM530FJe0VaIXhx1RK2s5fLSp/01n7USLVer+EYr4LG82Uav5YgB9R VUdbNdtvNN6MHO4iCsbHgjaamRtLlFmzjg6nhb0caCqaVVEk4WEtZJjySKPN/72//c9k7pOlSTH oksdmrpcnVEk= X-Received: by 2002:a05:6638:dc9:b0:323:68d6:e15a with SMTP id m9-20020a0566380dc900b0032368d6e15amr13889551jaj.238.1651153814152; Thu, 28 Apr 2022 06:50:14 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyBVVKwnStvWwJKB1GlEEK2+/gpmNiM5imf8h7vzirYHVExWGex8bY2fS3no0+WVvn+STav4Q== X-Received: by 2002:a05:6638:dc9:b0:323:68d6:e15a with SMTP id m9-20020a0566380dc900b0032368d6e15amr13889543jaj.238.1651153813906; Thu, 28 Apr 2022 06:50:13 -0700 (PDT) Received: from xz-m1.local (cpec09435e3e0ee-cmc09435e3e0ec.cpe.net.cable.rogers.com. [99.241.198.116]) by smtp.gmail.com with ESMTPSA id n4-20020a6b4104000000b006549fdd0780sm12944272ioa.51.2022.04.28.06.50.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 28 Apr 2022 06:50:13 -0700 (PDT) Date: Thu, 28 Apr 2022 09:50:12 -0400 From: Peter Xu To: Bibo Mao Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, David Hildenbrand , Yang Shi Subject: Re: [PATCH v3] mm/khugepaged: sched to numa node when collapse huge page Message-ID: References: <20220317065024.2635069-1-maobibo@loongson.cn> MIME-Version: 1.0 In-Reply-To: <20220317065024.2635069-1-maobibo@loongson.cn> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-Stat-Signature: xqt5emfh9m3g8ifx4s7rsxzknuync7f8 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 7C17A4005B Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=FBOD66HF; spf=none (imf17.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.129.124) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspam-User: X-HE-Tag: 1651153808-24092 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi, Bibo, On Thu, Mar 17, 2022 at 02:50:24AM -0400, Bibo Mao wrote: > collapse huge page will copy huge page from general small pages, > dest node is calculated from most one of source pages, however > THP daemon is not scheduled on dest node. The performance may be > poor since huge page copying across nodes, also cache is not used > for target node. With this patch, khugepaged daemon switches to > the same numa node with huge page. It saves copying time and makes > use of local cache better. > > With this patch, specint 2006 base performance is improved with 6% > on Loongson 3C5000L platform with 32 cores and 8 numa nodes. Totally not familiar with specint, so a pure question is whether it'll make a real difference in real-world workloads? As I assume in real world the memory affinity to the processors should change relatively slow on tuned systems, so even if khugepaged copied a bit slower then it'll not affect much on the real workload after the movement completes? The other question is if it makes sense, whether it's applicable to file thps too (collapse_file)? Thanks, > > Signed-off-by: Bibo Mao > --- > changelog: > V2: remove node record for thp daemon > V3: remove unlikely statement > --- > mm/khugepaged.c | 8 ++++++++ > 1 file changed, 8 insertions(+) > > diff --git a/mm/khugepaged.c b/mm/khugepaged.c > index 131492fd1148..b3cf0885f5a2 100644 > --- a/mm/khugepaged.c > +++ b/mm/khugepaged.c > @@ -1066,6 +1066,7 @@ static void collapse_huge_page(struct mm_struct *mm, > struct vm_area_struct *vma; > struct mmu_notifier_range range; > gfp_t gfp; > + const struct cpumask *cpumask; > > VM_BUG_ON(address & ~HPAGE_PMD_MASK); > > @@ -1079,6 +1080,13 @@ static void collapse_huge_page(struct mm_struct *mm, > * that. We will recheck the vma after taking it again in write mode. > */ > mmap_read_unlock(mm); > + > + /* sched to specified node before huage page memory copy */ > + if (task_node(current) != node) { > + cpumask = cpumask_of_node(node); > + if (!cpumask_empty(cpumask)) > + set_cpus_allowed_ptr(current, cpumask); > + } > new_page = khugepaged_alloc_page(hpage, gfp, node); > if (!new_page) { > result = SCAN_ALLOC_HUGE_PAGE_FAIL; -- Peter Xu