From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68E93E7717F for ; Tue, 17 Dec 2024 02:35:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CFAA06B007B; Mon, 16 Dec 2024 21:35:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CAA446B0082; Mon, 16 Dec 2024 21:35:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B98F96B0083; Mon, 16 Dec 2024 21:35:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9C8786B007B for ; Mon, 16 Dec 2024 21:35:31 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 4F1E4A033C for ; Tue, 17 Dec 2024 02:35:31 +0000 (UTC) X-FDA: 82902884274.11.1551BE6 Received: from out30-97.freemail.mail.aliyun.com (out30-97.freemail.mail.aliyun.com [115.124.30.97]) by imf23.hostedemail.com (Postfix) with ESMTP id E2E62140004 for ; Tue, 17 Dec 2024 02:35:08 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=hukVWBXR; spf=pass (imf23.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.97 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734402916; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VfhzARcc9uued7slDxaklm2CNSXGlycRCAO6rIZO6hg=; b=3o0CytRUz9pB1AQqm3U/0M7X+wliuB4cdbvIUzsALFWjYYNV/1YK9bXzF96wWSw9ckJYY9 RtlH7Qb3wmG7fHRvh6FnhTzel6txTgWKjFApmTb6PKT8ZDNehTX1wIQCwZiKsttyzNp/KV OyaUIgmSxJa6kzuxDSNlRtTRLm5ZigU= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=hukVWBXR; spf=pass (imf23.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.97 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734402916; a=rsa-sha256; cv=none; b=KenDPS6+Kpz8n8RzzbbQmZm7bWbZIMN2Z9axGtOAX69pumEeL+NI9/M6YmNWAUu8aXFr+2 WN01gaS4KvEtWRHP12PaNiOLa7Ga3OB7PKmOFWSL/fsQ96nISJRJ5BxS5Dep7DzK66ZkIE J0gRZVk+Xa65aro6k9QEMyj0VGRqB5U= DKIM-Signature:v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1734402925; h=Message-ID:Date:MIME-Version:Subject:To:From:Content-Type; bh=VfhzARcc9uued7slDxaklm2CNSXGlycRCAO6rIZO6hg=; b=hukVWBXRAIRSnbqH8EIw3Wo5VEARgL4OMGMpYmfEzvGYt5iGHMqb62+3iMrn+YwbE7wvVQImBbdF/VH7AD8CyzueLuNN2WawY1AzF5fq6naisSBWv/XjfBz3IGnJxFGx5pFGH2DBDjDjUNgaiQvQrq7hBo3UIWySFhr1daxmUjg= Received: from 30.74.144.132(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WLguzw5_1734402924 cluster:ay36) by smtp.aliyun-inc.com; Tue, 17 Dec 2024 10:35:25 +0800 Message-ID: <78586900-a5bc-4377-8fb9-f322f2028310@linux.alibaba.com> Date: Tue, 17 Dec 2024 10:35:24 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH V5] mm, compaction: don't use ALLOC_CMA in long term GUP flow To: yangge1116@126.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org, 21cnbao@gmail.com, david@redhat.com, vbabka@suse.cz, liuzixing@hygon.cn References: <1734350044-12928-1-git-send-email-yangge1116@126.com> From: Baolin Wang In-Reply-To: <1734350044-12928-1-git-send-email-yangge1116@126.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: E2E62140004 X-Stat-Signature: 4xi7ojy3d36dz7oan4koan73dhuaoy64 X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1734402908-324504 X-HE-Meta: U2FsdGVkX1/VZPiAcLuRj0OH3pQ8dUheY/aC2QvZzkxPgc+Hezrvx80L3OFZuDuUvnznzo4jQ+RJxO317K9LycSlgv1JAIvC5AvhanwwQQxZR82CHkJ7BDmuOGSIrOVzzjNoW3RTZsgqZZ64+Uv6hv3vgrd7G8f/fnhrgCHVDMohrjGzKIkmKbA5gPw3Jqz4XTfqfDPM+LXriPaBDxgABj6gBPaTGl0C428BQPg64+Dx61rYCykuBgOvtmyZGxxCuNCToAcDRLfwSbmVogLQHwIXdg/qPCfelyIgrCHSc58CW90ZAsm73hM6DIGWOVqKGBfgKPr5wBL+eKmiu+/shXdoQg6t+s1ZMDiGyyhOisdu4PAcgAqinh2a/ULmLh93wDmR9L7hyui1Pe9pr0dHEiGhC6D58WOd06qQUZ+DP8V/JzcJnL7HRfqELCxCJqKtHpTtC97lO8wobhmkcnzvNvePf8AxhEmgqTwPXcwg3k0bhve/8eKKaBcQ7e+n0lNA4g6NMjrmf1X7sp6/pSCH4BDRtD+qAkRjK+jTwfxM+JCOXNjPoQZBy/n8eXBAxNm6cLo/HY45qvUcwArimklsU8bmDv2hgQfRwqxVs/rZbfNQTdaNY5a8O3uc40DmEj9kGRfxvywwv99dLA68L71UDlnxu9l2S9gAVgenHo6gJ5W2t31SIe3uHbNrbcXoWeTshUYe1pMFAf2agCss6/SjwsuR03JESj+LIk5d2yTXs/ITWRy7/G7UCgFXvvAU9OYMQ3MyCXYjGfoJyGX8/3EEu8Yvrc1ABXMrFZrpIl5YnF/IJPTFzfKezICdLdPn5nU0wfTLTb+UGFy8c4hFqrPfZecL2bq5Mo0ME8bPe2dq9we5OxxNGSXr2g0gXyhNx1A/kEpwVwgh07laitM4VwioSqz6wS5nyREpBwD809QTXwV+I5yryxWMObsVqgrCiDZe0NJYDX7IVD0kIwFDJ4F rfCuqaDt kYUY7NYTXP8kl8nWnS6d7/NzfB0NShgHPwYWDb4FODdtm+HbdRoFcfY1T9FQ6rdKsa43uAzSoiWdlqSqGCWgkrEk6ccXvuTTvSLEKnx/eRaQahwkbjRnvj5CHP7Ds/13KqXDqb8VqOqzE9G0+ToCTKJxuclqr+te6Kuh0gRtrec5QWWk9ucfbfYdRE6DcmzWxrsFQ5nMy6wNCPO+refxNL09nsfFpcOV29qucvkhF1z8sEkXo2CBtZvLEwHpXfSggZcDfn2vvKKfnsVTJxEUM8CoUCiWupQ5biXWbo5em0k2gnUKfZpK/38HWYY3CZprqb+YFA9f0JxyGMCcSLQLH64fe28BG5ka/nEiExZby9tTfYGh/LZVOZYJQyOIhZ3EOWqRRV5661opKrw0oQzLDqAV5RA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.008742, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/12/16 19:54, yangge1116@126.com wrote: > From: yangge > > Since commit 984fdba6a32e ("mm, compaction: use proper alloc_flags > in __compaction_suitable()") allow compaction to proceed when free > pages required for compaction reside in the CMA pageblocks, it's > possible that __compaction_suitable() always returns true, and in > some cases, it's not acceptable. > > There are 4 NUMA nodes on my machine, and each NUMA node has 32GB > of memory. I have configured 16GB of CMA memory on each NUMA node, > and starting a 32GB virtual machine with device passthrough is > extremely slow, taking almost an hour. > > During the start-up of the virtual machine, it will call > pin_user_pages_remote(..., FOLL_LONGTERM, ...) to allocate memory. > Long term GUP cannot allocate memory from CMA area, so a maximum > of 16 GB of no-CMA memory on a NUMA node can be used as virtual > machine memory. Since there is 16G of free CMA memory on the NUMA > node, watermark for order-0 always be met for compaction, so > __compaction_suitable() always returns true, even if the node is > unable to allocate non-CMA memory for the virtual machine. > > For costly allocations, because __compaction_suitable() always > returns true, __alloc_pages_slowpath() can't exit at the appropriate > place, resulting in excessively long virtual machine startup times. > Call trace: > __alloc_pages_slowpath > if (compact_result == COMPACT_SKIPPED || > compact_result == COMPACT_DEFERRED) > goto nopage; // should exit __alloc_pages_slowpath() from here > > In order to quickly fall back to remote node, we should remove > ALLOC_CMA both in __compaction_suitable() and __isolate_free_page() > in long term GUP flow. After this fix, starting a 32GB virtual machine > with device passthrough takes only a few seconds. > > Fixes: 984fdba6a32e ("mm, compaction: use proper alloc_flags in __compaction_suitable()") > Cc: > Signed-off-by: yangge > --- I sent a follow-up fix patch[1] to update the cc->alloc_flags, and with that, looks good to me. Reviewed-by: Baolin Wang [1] https://lore.kernel.org/all/20241217022955.141818-1-baolin.wang@linux.alibaba.com/