From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D18AAC54FB3 for ; Mon, 26 May 2025 08:33:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1E2FF6B0083; Mon, 26 May 2025 04:33:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1BA9D6B0088; Mon, 26 May 2025 04:33:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0F7E46B0089; Mon, 26 May 2025 04:33:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id E55D06B0083 for ; Mon, 26 May 2025 04:33:44 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E978C140684 for ; Mon, 26 May 2025 08:33:43 +0000 (UTC) X-FDA: 83484395526.12.FBA7E05 Received: from out30-112.freemail.mail.aliyun.com (out30-112.freemail.mail.aliyun.com [115.124.30.112]) by imf04.hostedemail.com (Postfix) with ESMTP id 086BC40002 for ; Mon, 26 May 2025 08:33:40 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=L3acg2qE; spf=pass (imf04.hostedemail.com: domain of ying.huang@linux.alibaba.com designates 115.124.30.112 as permitted sender) smtp.mailfrom=ying.huang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748248422; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VPwPW0avOtzhWIYNFKRbT9qL1dQ35N08OEEC01EhqzQ=; b=GgkSLxlKtk9sKLC/Z88C7dlFa7nxrekqQvISVCqqrSTp1vrtMQxFgdbkAFrMAvKwez58te vh6wJC32pOLH3mv+lGG3pM5Ew3ikS7Kxmw1pTwdYjYCcIwUhN2qin0ZE0mFJnVflhpWqsW BbMkGQ1QmvSVAkI6jX4UcjjDUbu0pfo= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=L3acg2qE; spf=pass (imf04.hostedemail.com: domain of ying.huang@linux.alibaba.com designates 115.124.30.112 as permitted sender) smtp.mailfrom=ying.huang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748248422; a=rsa-sha256; cv=none; b=a0+gRoPKlq8x5qXdOJWDg5bZwBRrj+Lnkks1bGF/mpjXZw4qoNHksVUkNH93qxJ4p0Xela jax5p+0qlHwpJIoWG1kLYvo5zl8uoaFxGTe5AZOSDRvaFkY7Ln5W8eNmEFjDVQcrN62jct dSVWbWUlLmSTM8L4tT9XXUm6VwS3Ddw= DKIM-Signature:v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1748248418; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type; bh=VPwPW0avOtzhWIYNFKRbT9qL1dQ35N08OEEC01EhqzQ=; b=L3acg2qEMfvCV/YEY0Nm2X1rWDSmOx/70IPVQhBJ8FMkO9eohymU1IwRqXAob6tUriA8eSJ3L74Y/XaD53tbVep3oXEbgtF3YzyLjBPegTLa1xsfQVHdOs0UjF4vGJEcFsFjTVDLdiQ89XJTCoU7U5m2n5pvk1YgcfN/0SIS5eo= Received: from DESKTOP-5N7EMDA(mailfrom:ying.huang@linux.alibaba.com fp:SMTPD_---0WbmnakE_1748248415 cluster:ay36) by smtp.aliyun-inc.com; Mon, 26 May 2025 16:33:36 +0800 From: "Huang, Ying" To: Zi Yan Cc: David Hildenbrand , Bharata B Rao , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Jonathan.Cameron@huawei.com, dave.hansen@intel.com, gourry@gourry.net, hannes@cmpxchg.org, mgorman@techsingularity.net, mingo@redhat.com, peterz@infradead.org, raghavendra.kt@amd.com, riel@surriel.com, rientjes@google.com, sj@kernel.org, weixugc@google.com, willy@infradead.org, dave@stgolabs.net, nifan.cxl@gmail.com, joshua.hahnjy@gmail.com, xuezhengchu@huawei.com, yiannis@zptcorp.com, akpm@linux-foundation.org Subject: Re: [RFC PATCH v0 2/2] mm: sched: Batch-migrate misplaced pages In-Reply-To: (Zi Yan's message of "Thu, 22 May 2025 13:30:06 -0400") References: <20250521080238.209678-1-bharata@amd.com> <20250521080238.209678-3-bharata@amd.com> <62cef618-123c-4ffa-b45a-c38b65d2a5a3@redhat.com> <5d6b92d8-251f-463b-adde-724ea25b2d89@redhat.com> <996B013E-4143-4182-959F-356241BE609A@nvidia.com> <382839fc-ea63-421a-8397-72cb35dd8052@redhat.com> Date: Mon, 26 May 2025 16:33:34 +0800 Message-ID: <87wma3bwkh.fsf@DESKTOP-5N7EMDA> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 086BC40002 X-Stat-Signature: yinwnm9kj99kt97a7gsahc3tthm4icmt X-Rspam-User: X-HE-Tag: 1748248420-602799 X-HE-Meta: U2FsdGVkX19Q/BnX65pRhR5ypT+StUtxjBlleQNu0JYUgQONAl3iEpDkLsESXvOELGYOwp2udthOnow/p6kmHbVK9F9psi2X7fN+j6ThOzKWYdErnRJxZnq0OwcfQYR7+u6eUfkttd0sthE6w5+ndaUJ7Hmxz0IiU5cTeoyVzUE37C4WVwIL6480Q26WLOIP0/+s2gZSVRqqxPkFKV+DIrAvIwSuSMWyrEvHQbCx9OH8KBwiBHi9bB7TEMF7K6XkCMf1rQAl4qJztrERfqzp5SkQglLZwhSQ8okRmkhROvHKBAJvjAIEQwsOi2pCw4AFTbcgan+bEzFazEkDc10b2SADciuNYJK/r3qIS0f2T/2sr4q5zpXhhGPLY44icOwVneqe1DfeHyg9BCJRQGy/cxm+jHjgy0v0MiiKwBFHrukJVquHtvcyt/tlg50XzreaOdZGsD0kb6dmTQt+y/j61FJfIdNttMgulvmMNBa30hEvty1gFJesFm0PTR8DIXOV+sSVP95o45FsuDyVNUSpwMC6tG6EL3cRRosI79sdNW4pAei5JzMm6tkcaeP4hEuVZM8DVOF6THDQUZMBAXkxxAB/oJ5MV8cVy6zAMhNe9LTOa1ah0sCHbJ1JED7/vzqp+a0DM9bT+Ebi4Sk1GgzGmjb1b3/XVBoj+bDlE9ZcSSJU1Q+R/0EE79Mqce67wi5edvIQJfe+B8hGRo9g5oHxN9HYZ9aLghkSpscddHntFk+U8dhBDmDtOFJvxjMRNZNloNE+8FD981e2B8ZVnjUqT8P/jdvsNs1prLrGOJM9MZhqtQrT5rfmlGal2/Ddipo8piKOX/rrJ+4PSK8gwR1U1C4mAyiDGFIp0d74vRkhGokJvGcHsPO5OR1cpK5SI/knOZKnZUSJGwGjwPaHr8rG9YDZ03SsVMnT5CmWQKKPcOwNuaTGXEy1VoXVAYD5U7ombnr/y1YPxFn7M6ZdRJj 6bIUjbCF RFawWEXHVRO64FNPZQ8RZ959hlyvwdfTmiAOrlacyWBNjoKgRbr52sQGKHObpL9RsB1KkfigdZyfpa8fDdI7o0yUjNOgM6ABKuZvk4BqXNE0E/Lb6AunNyItTR3FjAP2E9UOgOrcc2YCmtn/mhaSh2aGbzkvGtaE6AV9YL4IFZadzVpJJvIc2b2hc2pEzuJCthBcaVejw5iTupzhdSxKVmJS6svwIyBHkmS3YF97tVZe7Q51s3XSd9E8T/hy8Fn8rd5QIC1j17y7OJONIN00N/X+VKrBmrhgDilEx+y2FKE5IYOoeeU+Mmx0QOw13z8243pHZi4mEy3OCZOMGF5xnRUB+LamIqLMRg/4NJLF27bs02a/h4aZnMTUeRpiLlSm2h1wuE1f6iJXNfSpfxzuED86T3aSEKOngiHhG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Zi Yan writes: > On 22 May 2025, at 13:21, David Hildenbrand wrote: > >> On 22.05.25 18:38, Zi Yan wrote: >>> On 22 May 2025, at 12:26, David Hildenbrand wrote: >>> >>>> On 22.05.25 18:24, Zi Yan wrote: >>>>> On 22 May 2025, at 12:11, David Hildenbrand wrote: >>>>> >>>>>> On 21.05.25 10:02, Bharata B Rao wrote: >>>>>>> Currently the folios identified as misplaced by the NUMA >>>>>>> balancing sub-system are migrated one by one from the NUMA >>>>>>> hint fault handler as and when they are identified as >>>>>>> misplaced. >>>>>>> >>>>>>> Instead of such singe folio migrations, batch them and >>>>>>> migrate them at once. >>>>>>> >>>>>>> Identified misplaced folios are isolated and stored in >>>>>>> a per-task list. A new task_work is queued from task tick >>>>>>> handler to migrate them in batches. Migration is done >>>>>>> periodically or if pending number of isolated foios exceeds >>>>>>> a threshold. >>>>>> >>>>>> That means that these pages are effectively unmovable for other >>>>>> purposes (CMA, compaction, long-term pinning, whatever) until >>>>>> that list was drained. >>>>>> >>>>>> Bad. >>>>> >>>>> Probably we can mark these pages and when others want to migrate the page, >>>>> get_new_page() just looks at the page's target node and get a new page from >>>>> the target node. >>>> >>>> How do you envision that working when CMA needs to migrate this exact page to a different location? >>>> >>>> It cannot isolate it for migration because ... it's already isolated ... so it will give up. >>>> >>>> Marking might not be easy I assume ... >>> >>> I guess you mean we do not have any extra bit to indicate this page is isolated, >>> but it can be migrated. My point is that if this page is going to be migrated >>> due to other reasons, like CMA, compaction, why not migrate it to the target >>> node instead of moving it around within the same node. >> >> I think we'd have to identify that >> >> a) This page is isolate for migration (could be isolated for other >> reasons) >> >> b) The one responsible for the isolation is numa code (could be someone >> else) >> >> c) We're allowed to grab that page from that list (IOW sync against >> others, and especially also against), to essentially "steal" the >> isolated page. > > Right. c) sounds like adding more contention to the candidate list. > I wonder if we can just mark the page as migration candidate (using > a page flag or something else), then migrate it whenever CMA, > compaction, long-term pinning and more look at the page. In addition, > periodically, the migration task would do a PFN scanning and migrate > any migration candidate. I remember Willy did some experiments showing > that PFN scanning is very fast. I think that this could be a second step optimization after the simple implementation has been done. --- Best Regards, Huang, Ying