From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB12FC4829D for ; Fri, 9 Feb 2024 14:32:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 595A76B00A5; Fri, 9 Feb 2024 09:32:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 544B56B00A6; Fri, 9 Feb 2024 09:32:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3E4C06B00A7; Fri, 9 Feb 2024 09:32:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 2DBBA6B00A5 for ; Fri, 9 Feb 2024 09:32:39 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id F256BA10BF for ; Fri, 9 Feb 2024 14:32:38 +0000 (UTC) X-FDA: 81772506396.29.DA3ABFF Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf09.hostedemail.com (Postfix) with ESMTP id 6011A14003C for ; Fri, 9 Feb 2024 14:32:35 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=dPwpkAYp; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=IoBEP9Rv; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=dPwpkAYp; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=IoBEP9Rv; dmarc=none; spf=pass (imf09.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707489155; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NvaR8GHga17EqoSaRiIPdrwK05K74U/GowrqG7dNIBI=; b=mR2YmAtzpiDDdsJmPR5ztzxHYtGm2xhUxfdv4FpBpgTl/9lXSN6te/oaEo6IOYAJpHWGph OtpHfqT3xw8sXPt0QE4SAfltH1xX2IMiU3/psDny1bvXWz0WjmlvV9rZ0GXhYFYCCW3LYL CUTUt8j6Kx8WaPfLPs6eVHs9PDU6Sjs= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=dPwpkAYp; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=IoBEP9Rv; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=dPwpkAYp; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=IoBEP9Rv; dmarc=none; spf=pass (imf09.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707489155; a=rsa-sha256; cv=none; b=xkzEEZtabbVlxwUJEmEtRdGgzeVSndZU89exvy9Fx7fvkjtz8a5o7VsDLoxYD4i67vNvsb zTnQC1WzjK9DBbGRabOlunhTrjaNzJ/MARBMw97WCeYis2HNz/u28JGVKPnJ2Rc8RobMrz nCVXpjJY0RGZBEo7JlsfCYQIVo+jIu8= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 56884220BC; Fri, 9 Feb 2024 14:32:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1707489153; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NvaR8GHga17EqoSaRiIPdrwK05K74U/GowrqG7dNIBI=; b=dPwpkAYphKOr4kYrjwLDzqi7lhq8dEV2JhcAFDwlgXdwnwvQ48j7cfoplMxNFCWKw2EaGF scDk1jiMNx3qnN9yC7Euf06dZRvd7x5/qEN/dK2icgDH84Hd2LY22O4QsO91peplhYwq7x 5yVazHrHUR7L6muAAVT+UTiCVE8voJ8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1707489153; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NvaR8GHga17EqoSaRiIPdrwK05K74U/GowrqG7dNIBI=; b=IoBEP9RvwoX4kInT8LpmC5UBEx68qAqNXtmZjNcVuszPUOel6y+/FwtViVNkTN7C20Fm47 fMCweZhsaiPw36AQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1707489153; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NvaR8GHga17EqoSaRiIPdrwK05K74U/GowrqG7dNIBI=; b=dPwpkAYphKOr4kYrjwLDzqi7lhq8dEV2JhcAFDwlgXdwnwvQ48j7cfoplMxNFCWKw2EaGF scDk1jiMNx3qnN9yC7Euf06dZRvd7x5/qEN/dK2icgDH84Hd2LY22O4QsO91peplhYwq7x 5yVazHrHUR7L6muAAVT+UTiCVE8voJ8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1707489153; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NvaR8GHga17EqoSaRiIPdrwK05K74U/GowrqG7dNIBI=; b=IoBEP9RvwoX4kInT8LpmC5UBEx68qAqNXtmZjNcVuszPUOel6y+/FwtViVNkTN7C20Fm47 fMCweZhsaiPw36AQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 2C97D1326D; Fri, 9 Feb 2024 14:32:33 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id PRmNCoE3xmVncwAAD6G6ig (envelope-from ); Fri, 09 Feb 2024 14:32:33 +0000 Message-ID: Date: Fri, 9 Feb 2024 15:32:32 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3 1/3] mm/compaction: enable compacting >0 order folios. Content-Language: en-US To: Zi Yan , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: "Huang, Ying" , Ryan Roberts , Andrew Morton , "Matthew Wilcox (Oracle)" , David Hildenbrand , "Yin, Fengwei" , Yu Zhao , "Kirill A . Shutemov" , Johannes Weiner , Baolin Wang , Kemeng Shi , Mel Gorman , Rohan Puri , Mcgrof Chamberlain , Adam Manzanares , "Vishal Moola (Oracle)" References: <20240202161554.565023-1-zi.yan@sent.com> <20240202161554.565023-2-zi.yan@sent.com> From: Vlastimil Babka In-Reply-To: <20240202161554.565023-2-zi.yan@sent.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 6011A14003C X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: p36txrodmj6ticb99i77i4ecmq4ruu8h X-HE-Tag: 1707489155-762284 X-HE-Meta: U2FsdGVkX18VCqAXsValQhzuGMJdE2QdtH1hxj9xhnt8bzLhYVoDZhuMYFFwTthihA1Z+fVXD71crwq5m/rLvcbH4lGXvEWMZ47LdHE+PUtgCFyv8MJAcGH+5sCMgo7iBvDU/GC3jycnorVHXrVl+Y9f0VnrHjk0zP7nsMBKCaMiymUEp4Dm4jgcXt0TceJSD+Ng0maqS04Y3Y9qo+RzuHHw23b4mP8iA4plB+jImom8KZxEG5MmpGmAMp1vJGBWkisGf9/arJ2R8LfdqIMtbSo1D3q8ShdKyUDVGemhuwo5z5eh1md724C9hVlgAstO7ay2pS2BweTGdbiurmpVUD2sOWmS34jHdMDE3M4Qo5bB3QDjbPkG93GihqLPiTyPDvycfgfJuFIydAKyD/6kN//9tuIPOY1ad4u9yxmqVG/U8Z0dphCRIhMGDb5N9AePN/TLjbGjr3EM8GUEfAiC9B0FzDmruZTWZjC1U9hwc8K4reYluY4c1DOUBR+6439fCPf2LsnxLrCGwFnc85V2kAtgeBqeAWoiifHMHxQoJkJzqNLmHvKunlkfeEw55T9fwqLthLNMAJ1m0bzEP9AugHEEyddPh7HpEhGuNt386i0uX8s6zHBq+zND8odDKzuRIz2ul/aFsr6D8X97wx5Q6Em0ZIB1/iTBOLVvQQWqdyvZp7Z8s1N8nljFTGZq0z+JqjkJN6cNIIMHyqTX3H1fDbRtHFfQblxNFYxR2Rvugl+NBMAy7Mogztn2Vw5corBYgYRk4Nuo5J/momCsLvZSdQIL33ZhkxXu7C4SHQ4WI5W/PI5w2c9EmbFque5v54CJ7/cCtt9Ae6ETAssj7pXK3LYL83twQeApfHT5tbOBjLn9RjPwFbKhataEM3AHkwbx+QHjDKjsuuJVBoA7CyPPvoX3CcdDYBolVnyLlHxuYqwFS00IZSlw2l3XbF+MylX3Lin8FO6xWIeS2vk37E9 +T8nNyvt JC6CtNxCsdzKA7fIqqZxMOtgs2OtTJRVAQ7llTB2St6Rm+COl64WTH20FY9C6wnx0BWvNaTvTcBsm8n0GrX7gL3PigK4ztOLmcyo3VkHP8IAtX8sAKV7Nt3evTsz6y7oyUTlEGMcAEJCQRqMqoT3x5WG8vCJpcXokz3KPPoQQc8h9Oq+ysNi3qo+NTKHGceM90ij4doLiSSx9+Reh2Q3D4GU0VVv6dhf9hQFc9srqKdBOdd8Q5NjtXTBPrQWOOW1C1WO5MIMotmAvtsOCGFPg15X5ELoQ7KRPUfK+fSOhRmvDZiv9LPyDIWoUj1weUgwaT/5XikRDSAPMAAidKwpsonaQWd1tv0fvA+OWs1o8Ae1oYPU1FuENINQSXJxSxm+PF3Pfm5GNsEY+Wy2DtUJAxvFcGgRkgSU+Lqt1AbbUpI24HLEmMKyqeCtXgQmB2JiODEm3 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2/2/24 17:15, Zi Yan wrote: > From: Zi Yan > > migrate_pages() supports >0 order folio migration and during compaction, > even if compaction_alloc() cannot provide >0 order free pages, > migrate_pages() can split the source page and try to migrate the base pages > from the split. It can be a baseline and start point for adding support for > compacting >0 order folios. > > Suggested-by: Huang Ying > Signed-off-by: Zi Yan > --- > mm/compaction.c | 43 +++++++++++++++++++++++++++++++++++-------- > 1 file changed, 35 insertions(+), 8 deletions(-) > > diff --git a/mm/compaction.c b/mm/compaction.c > index 4add68d40e8d..e43e898d2c77 100644 > --- a/mm/compaction.c > +++ b/mm/compaction.c > @@ -816,6 +816,21 @@ static bool too_many_isolated(struct compact_control *cc) > return too_many; > } > > +/* > + * 1. if the page order is larger than or equal to target_order (i.e., > + * cc->order and when it is not -1 for global compaction), skip it since > + * target_order already indicates no free page with larger than target_order > + * exists and later migrating it will most likely fail; > + * > + * 2. compacting > pageblock_order pages does not improve memory fragmentation, > + * skip them; > + */ > +static bool skip_isolation_on_order(int order, int target_order) > +{ > + return (target_order != -1 && order >= target_order) || > + order >= pageblock_order; > +} > + > /** > * isolate_migratepages_block() - isolate all migrate-able pages within > * a single pageblock > @@ -1010,7 +1025,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, > /* > * Regardless of being on LRU, compound pages such as THP and > * hugetlbfs are not to be compacted unless we are attempting > - * an allocation much larger than the huge page size (eg CMA). > + * an allocation larger than the compound page size. > * We can potentially save a lot of iterations if we skip them > * at once. The check is racy, but we can consider only valid > * values and the only danger is skipping too much. > @@ -1018,11 +1033,18 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, > if (PageCompound(page) && !cc->alloc_contig) { > const unsigned int order = compound_order(page); > > - if (likely(order <= MAX_PAGE_ORDER)) { > - low_pfn += (1UL << order) - 1; > - nr_scanned += (1UL << order) - 1; > + /* > + * Skip based on page order and compaction target order > + * and skip hugetlbfs pages. > + */ > + if (skip_isolation_on_order(order, cc->order) || > + PageHuge(page)) { Hm I'd try to avoid a new PageHuge() test here. Earlier we have a block that does if (PageHuge(page) && cc->alloc_contig) { ... think I'd rather rewrite it to handle the PageHuge() case completely and just make it skip the 1UL << order pages there for !cc->alloc_config. Even if it means duplicating a bit of the low_pfn and nr_scanned bumping code. Which reminds me the PageHuge() check there is probably still broken ATM: https://lore.kernel.org/all/8fa1c95c-4749-33dd-42ba-243e492ab109@suse.cz/ Even better reason not to add another one. If the huge page materialized since the first check, we should bail out when testing PageLRU later anyway. > + if (order <= MAX_PAGE_ORDER) { > + low_pfn += (1UL << order) - 1; > + nr_scanned += (1UL << order) - 1; > + } > + goto isolate_fail; > } > - goto isolate_fail; > } > > /* > @@ -1165,10 +1187,11 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, > } > > /* > - * folio become large since the non-locked check, > - * and it's on LRU. > + * Check LRU folio order under the lock > */ > - if (unlikely(folio_test_large(folio) && !cc->alloc_contig)) { > + if (unlikely(skip_isolation_on_order(folio_order(folio), > + cc->order) && > + !cc->alloc_contig)) { > low_pfn += folio_nr_pages(folio) - 1; > nr_scanned += folio_nr_pages(folio) - 1; > folio_set_lru(folio); > @@ -1786,6 +1809,10 @@ static struct folio *compaction_alloc(struct folio *src, unsigned long data) > struct compact_control *cc = (struct compact_control *)data; > struct folio *dst; > > + /* this makes migrate_pages() split the source page and retry */ > + if (folio_test_large(src) > 0) > + return NULL; > + > if (list_empty(&cc->freepages)) { > isolate_freepages(cc); >