From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACBF0C54ED1 for ; Tue, 27 May 2025 09:05:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2DF176B0099; Tue, 27 May 2025 05:05:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 28FCE6B009A; Tue, 27 May 2025 05:05:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 17EE86B009B; Tue, 27 May 2025 05:05:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id ED7896B0099 for ; Tue, 27 May 2025 05:05:30 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id A79871CB3E0 for ; Tue, 27 May 2025 09:05:30 +0000 (UTC) X-FDA: 83488104420.22.55339D5 Received: from out30-98.freemail.mail.aliyun.com (out30-98.freemail.mail.aliyun.com [115.124.30.98]) by imf21.hostedemail.com (Postfix) with ESMTP id F0BE31C000A for ; Tue, 27 May 2025 09:05:27 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=CZvtfoAE; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf21.hostedemail.com: domain of ying.huang@linux.alibaba.com designates 115.124.30.98 as permitted sender) smtp.mailfrom=ying.huang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748336729; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=g3UmXVOxJL0IKWUMXDh3p6MkUMRqvLMyZGb15DpuS6c=; b=PDJmj6XxP+x98QSmvP3rC8x5q7BCaY/NfCIYis/q+LCLFQJHxuzIuzfIfItcBN1IwNoHz3 PrEf3FK83WvjYjXuezI2X+9MbHPHC6AW0yXJgyjHRTKKQGeNK8xrddtx6+qoDXleq8mRtQ VhE4wIYvy8yq35oz0qGJZD2gGtyXbsE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748336729; a=rsa-sha256; cv=none; b=bBFc0+ojAg4LKqad0/7sM8IulEnt2pFtj3Wco4xzprr0qlMa1CoezO3L8W4+gcDgM7cpjk vyarni6lpIWX2E+Ir+tG9bmYdhclK7iIlaWU0qa26kPzU6JsKVQFuj+CVW2e+rxzBI/UrT 8L5MPC0o3n7FcONmnwvlOJqthsTv0jg= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=CZvtfoAE; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf21.hostedemail.com: domain of ying.huang@linux.alibaba.com designates 115.124.30.98 as permitted sender) smtp.mailfrom=ying.huang@linux.alibaba.com DKIM-Signature:v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1748336724; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type; bh=g3UmXVOxJL0IKWUMXDh3p6MkUMRqvLMyZGb15DpuS6c=; b=CZvtfoAEwbUdC+ftJvS5UKsY+pPrkwZ4XUvZ9Ai01M+b2bbeUMR4UH09dE0wepRWwjPY8PPNFF1olTxFC6JgZiTU/YSnNlRHoX9f2SrCGmOkIGUy75Du/QL6g9+YgdKA95u6IF8oa16QpTAMKBb4nY9fXmtrhjoNgvyq4a/iP/Q= Received: from DESKTOP-5N7EMDA(mailfrom:ying.huang@linux.alibaba.com fp:SMTPD_---0WbzeqpI_1748336721 cluster:ay36) by smtp.aliyun-inc.com; Tue, 27 May 2025 17:05:22 +0800 From: "Huang, Ying" To: Bharata B Rao Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Jonathan.Cameron@huawei.com, dave.hansen@intel.com, gourry@gourry.net, hannes@cmpxchg.org, mgorman@techsingularity.net, mingo@redhat.com, peterz@infradead.org, raghavendra.kt@amd.com, riel@surriel.com, rientjes@google.com, sj@kernel.org, weixugc@google.com, willy@infradead.org, ziy@nvidia.com, dave@stgolabs.net, nifan.cxl@gmail.com, joshua.hahnjy@gmail.com, xuezhengchu@huawei.com, yiannis@zptcorp.com, akpm@linux-foundation.org, david@redhat.com Subject: Re: [RFC PATCH v0 0/2] Batch migration for NUMA balancing In-Reply-To: <55c51f34-b41d-49b2-96a2-dcc5f425f966@amd.com> (Bharata B. Rao's message of "Tue, 27 May 2025 14:23:27 +0530") References: <20250521080238.209678-1-bharata@amd.com> <87sekrbvyr.fsf@DESKTOP-5N7EMDA> <55c51f34-b41d-49b2-96a2-dcc5f425f966@amd.com> Date: Tue, 27 May 2025 17:05:21 +0800 Message-ID: <87cybua0fi.fsf@DESKTOP-5N7EMDA> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: F0BE31C000A X-Stat-Signature: okrnr1ggd8exap96u7763i7z7p4onbsw X-Rspam-User: X-HE-Tag: 1748336727-715594 X-HE-Meta: U2FsdGVkX18H6QWdc+fumzJ3Uaqo5PtiD8iY/5vl6MsSZ8+zOAkWBtQr1Gzu59Jln6W7HP4EWYuokdIcY6DogIkjeF+Qd1QRn7BdRM+RZCWMX32Sz31D7tWZmAsxB46TBQTc8xKLFXxEdnB1exc4THACiztxmA8L6MgCI29OlLM3j6jGlEXcPNvjqUkgzEWYCVwDLBpOEZTdRUD6C0/TaGgbtM5fcW3tO7aRYidGk6UmjEoHcS+7HTh04Yd8cU0dudPxS5JQFmq6ubL3W0HooK074RL46Vc3cUTmd0bbvVrysl8RWmD+fMR4RKeLws2AiXw52Z1lnO5IstTNvvDKJY6xOragBwUNtYh/mPtUqKsql1deaw2T8aoz8zTAoDp6uSK5NCr6/Os4gSJpIYZNVuE0qUDpQHIF+1EQlfiAGD5oCtKv5sOVuW5v+8zeDr1bd2v1m258z59m5gsjAlyD9nfRTdwGP5VU4tdQ6uSAyEMbNCrO3AdxjAI94CEKuwQCtzcyaY++Xbz06dCdIRGNhJOtjCQeab5UuVmWVxxVLDSDBiIQNMmKMY5CVokrZEtG8YrA+5rVGmQxodqaZgt2xHYcXW2YoZ30xaLll9cDNVPvR6UKA98acpj3AculFuu3gY7QkSWk+BcdKGFpzFGRH04+UbHIOOWrsz87cf8bXaZKax2kURnu1I8WaLFqUBnqe8+fSd/c78bvSi6MDgGBQcg308hISW5RIIMYkhZ0jlQV5LVXy4a0Ro3oQ9r4/XOkkDa9p5fVRPsY6ivLSr9IHZE+zzZ7lMd5Xgofoq5wuNXOymF79hqmnBkEDFMkHpBCe9bRdbkb3FFZg3wz2OQPX9ovyQMYXwn9DRnK6mLDRz8PiFQtWi7B1dUyY8nvb7pu8WHONxf8mAWyvO7545GeIE0caB3h3ivH5RGyI2nX5Gg7DkHvPzRM05uuoZbeJ7FZe8Zymck7w9Y5k0AriAY bNXf1/C1 vwzUx6Ljo2G3JX7HFoh9PpUYuST9zf20By/JAE2f9UkrSZ/uxRPqhqPLtrNwhpHSxPWV13QvwJ5E2PHFR/dBKfLnwLCwea2QJmOlfZRPDCYUk8c2pGpAkqm3NJ83YTzy45aFUm496Cyiam32bGwqwaZYFjVqmMc4eaPrPDQyIARO5wrj6Ae012BJn4oV4F/x/2GxG4ft9MZ+XFq3ta/TzN1BwfqVb9oE3czqjNgqnLLKLPJIv4PHKeRqwhHhp0vkBTPsyJO8O6WjWewg4Pw13mib2TdjYc3kjO9Z/S2njuJYgrxdDLp6PIhktOO8ivg3d8UbN1ZcP5t7GlCdGUl3kjiQMzfu1WJ6YCNKfnWgYGZfDRQyz8T1KBWUp2VTu0dVTHIrk X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Bharata B Rao writes: > On 26-May-25 2:16 PM, Huang, Ying wrote: >> Hi, Bharata, >> Bharata B Rao writes: >> >>> Hi, >>> >>> This is an attempt to convert the NUMA balancing to do batched >>> migration instead of migrating one folio at a time. The basic >>> idea is to collect (from hint fault handler) the folios to be >>> migrated in a list and batch-migrate them from task_work context. >>> More details about the specifics are present in patch 2/2. >>> >>> During LSFMM[1] and subsequent discussions in MM alignment calls[2], >>> it was suggested that separate migration threads to handle migration >>> or promotion request may be desirable. Existing NUMA balancing, hot >>> page promotion and other future promotion techniques could off-load >>> migration part to these threads. >> What is the expected benefit of the change? > > Initially it is about cleanliness and separation of migration into its > own thread/sub-system. > >> For code reuse, we can use migrate_misplaced_folio() or >> migrate_misplaced_folio_batch() in various promotion path. > > That's what I have done in this patchset at least. We thought we could > go full length and off-load migration to its own thread. Even if we migrate pages in another thread, the migrated pages will be unmapped, copied, remapped during migrating. That is, the workload threads may be stalled to wait for migrating. So, we need to measure the real benefit firstly. >> For workload latency influence, per my understanding, PTE scanning >> is >> much more serious than migration. Why not start from that? > > Raghu's PTE A bit scanning is one effort towards that (Removing PTE > scanning from task context. --- Best Regards, Huang, Ying