From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A96EC4828F for ; Fri, 9 Feb 2024 18:43:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9AA426B0072; Fri, 9 Feb 2024 13:43:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9334F6B0074; Fri, 9 Feb 2024 13:43:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 786456B0075; Fri, 9 Feb 2024 13:43:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 630496B0072 for ; Fri, 9 Feb 2024 13:43:29 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 32001141118 for ; Fri, 9 Feb 2024 18:43:29 +0000 (UTC) X-FDA: 81773138538.05.D5E7790 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf27.hostedemail.com (Postfix) with ESMTP id 96AA940009 for ; Fri, 9 Feb 2024 18:43:26 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=oDksP+4a; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=ql57Be3l; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=oDksP+4a; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=ql57Be3l; dmarc=none; spf=pass (imf27.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707504207; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lEYLn4TkDj+5w5M8frhbdwGFmGUrN6QrUzMzWCOgpfU=; b=KYmZQij6zoMHeUU1RGIWDrOAtBWvFLtNz+4Ofo/X0p1k5Yndv9MVcfeiwMXWfyJCaHJs8Z zX7hyBwA5EJnz5rSflUiQ3O0fj7VfIlBjZz4DHYKEsvm3OUpacKslOQDtdK+cFYRTblbjL IKBDYtZQRbrVQsAWNxFnt2DGmEv70Es= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=oDksP+4a; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=ql57Be3l; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=oDksP+4a; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=ql57Be3l; dmarc=none; spf=pass (imf27.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707504207; a=rsa-sha256; cv=none; b=3czn+EMn+/ct0us872eapU8ZbMc7qyMuNAHw6sKhJXJwt8TWFiSjp5uxzRyGXqOrN+dzlp ehHFZ5Cdbxx9l+pDVRzYjM/VnixfDIdN/VnLqT8ZZzKHjMVMQvz2aPtdYDwnTnaeBju0Kw hQMZpMAX2TBNHykF66tD09qCKDIlwLw= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 6C2F71F823; Fri, 9 Feb 2024 18:43:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1707504204; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lEYLn4TkDj+5w5M8frhbdwGFmGUrN6QrUzMzWCOgpfU=; b=oDksP+4a6mK9R7SbcISBVXnlhEr2HRVuI2SVyCZk/mRa/n1u9RW17Na6EZvLNBrO7H8lWu AJHZQSu9Mhpp873ZBYCALox9V2a1AELbDJZV/OqaEcSpKxpgEq0OmOMm2VqdrnfQLKKIom njvOZNL0xhjBnLamyalffZlYF4R2rXI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1707504204; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lEYLn4TkDj+5w5M8frhbdwGFmGUrN6QrUzMzWCOgpfU=; b=ql57Be3lwdHXayCYGQVnolMO/pvNZRDF3YscjV9kIumUUvtynQXIiX0la1ikcpJJyplzY/ zsb6o0nIN3j+7gAQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1707504204; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lEYLn4TkDj+5w5M8frhbdwGFmGUrN6QrUzMzWCOgpfU=; b=oDksP+4a6mK9R7SbcISBVXnlhEr2HRVuI2SVyCZk/mRa/n1u9RW17Na6EZvLNBrO7H8lWu AJHZQSu9Mhpp873ZBYCALox9V2a1AELbDJZV/OqaEcSpKxpgEq0OmOMm2VqdrnfQLKKIom njvOZNL0xhjBnLamyalffZlYF4R2rXI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1707504204; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lEYLn4TkDj+5w5M8frhbdwGFmGUrN6QrUzMzWCOgpfU=; b=ql57Be3lwdHXayCYGQVnolMO/pvNZRDF3YscjV9kIumUUvtynQXIiX0la1ikcpJJyplzY/ zsb6o0nIN3j+7gAQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 3C85013353; Fri, 9 Feb 2024 18:43:24 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id A5voDUxyxmVfPwAAD6G6ig (envelope-from ); Fri, 09 Feb 2024 18:43:24 +0000 Message-ID: <84dfedc4-a0a2-4e02-9be4-2cffc6e9fd06@suse.cz> Date: Fri, 9 Feb 2024 19:43:23 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3 3/3] mm/compaction: optimize >0 order folio compaction with free page split. Content-Language: en-US To: Zi Yan , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: "Huang, Ying" , Ryan Roberts , Andrew Morton , "Matthew Wilcox (Oracle)" , David Hildenbrand , "Yin, Fengwei" , Yu Zhao , "Kirill A . Shutemov" , Johannes Weiner , Baolin Wang , Kemeng Shi , Mel Gorman , Rohan Puri , Mcgrof Chamberlain , Adam Manzanares , "Vishal Moola (Oracle)" References: <20240202161554.565023-1-zi.yan@sent.com> <20240202161554.565023-4-zi.yan@sent.com> From: Vlastimil Babka In-Reply-To: <20240202161554.565023-4-zi.yan@sent.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 96AA940009 X-Stat-Signature: 9c7zxpibp9szqtri1hqnompkyyzguu86 X-HE-Tag: 1707504206-255741 X-HE-Meta: U2FsdGVkX1+aNNFy/NpeaioQomIkfk5pTXJ8WXJEiS+HSAycteHaeDpG7xJoxn3aSPj/PI+HifJmQihF7w7Ua53O8/sAsbXesJtRiUplLRJbxRJ/fhZdcWXNRbMfr4N1dICImcv+F66D9VxzQr37A+Q4yoyN3bdlmMYUYa4+haECFbTj2ASYelbhIFOCgXCCs+krP35hEFmmDON7pMzhdNNiPadNbP9oOZOtprcrZui5ak1s02zGrLZq741qpSinoTOc3Q84BIdLP0vtdlYgauqaKewTazMRWH8oQljIG8gaHv5fMKux6qm282mOWUaRtsUBBQGONw2wsQ3vIjeCGEoz8vcp0KFuCAga0Z0ydnzYBKGg2Fi5dh+7kAHNafnYR79IImw+ID8kkvh6R751PPQFQEkK2NkmIOwHMRkFPhKQSvfae6ZJdNH6kzipkUGHi93SlTIE/yciJ4K3Lg3kvXw9sEkQSo2tVvGUhr2BkHuUxW9qSXI/l+CXycXALtsAKlK0FzrblcBKWEy4CVzJV46TRdNVxKQ8zuL2rN6UyNOoiIDyHV30gxPkvLFqj2SZv3y6Wo1ldw9CUX0vAiTC1axVwozVOtc8NjylmntWw1V30MO9vgP5yTPnQmeEsXqSVxnATDFqFyi3sPdJ3MVFoJGpatKOpfDwYvaRLGhTW9OmfFbcicsxhQd/W3KPl/j5UDMhilK9jB+53PuY8EFdt6zMWS3k94tfno36t4kUg46nF3yUIg9md+6ClkykNVYPlxwjq9pu99TkGsFRfeMClb7O+naxOtuAfqthKsD+HCSUe6oSR1/6AY2EXjfflC0z6p+CZUsJq3qcuV5dKypRmyZHSb0SiodDZcrszjQzJ5z2qyo7xMOSE2KWoCUIW94Lw4dPaNURqhTqzqFmYhELymU2dQnu774y39B+Jkrs6F/28Lyk/aF/oMn64UCjlQLjTkp/0fcz8FRKSGZFupZ cQl9Vgj9 R+6Bd1AOOlJ+8N/5d/q8GE+a/ggeZBRdwwJDwGLUN6xy2iTvOfelKiI4aiaR5GMhLByHb/uSTLIkz0/mFlGjNKCghGftWBveEuHx9kmLuJ8yxtQGK44QENw2m7XEjh+mzOJ4rOvXMjzudtuWQ9SkMuyn/eMjAeptYUUGmLsEm6j2QzSMSN3ddos3VhJ/F5JTAwuVSpRH5cGC6VxSQqkW1hswLFJRfPY+q7/5KehCIUiXNsene3uFeJ1aYPIVHS/sqLZ4AYVPtwfBM14w/udN10NWSWxdUXzjDiAxoDsctw4ccbNNadY1wqDqUPUhy9Efl+nNkcApILMf37iZK8fUoh/xXMtqV0aW3pUayFPq38YG/WaM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2/2/24 17:15, Zi Yan wrote: > From: Zi Yan > > During migration in a memory compaction, free pages are placed in an array > of page lists based on their order. But the desired free page order (i.e., > the order of a source page) might not be always present, thus leading to > migration failures and premature compaction termination. Split a high > order free pages when source migration page has a lower order to increase > migration successful rate. > > Note: merging free pages when a migration fails and a lower order free > page is returned via compaction_free() is possible, but there is too much > work. Since the free pages are not buddy pages, it is hard to identify > these free pages using existing PFN-based page merging algorithm. > > Signed-off-by: Zi Yan > --- > mm/compaction.c | 37 ++++++++++++++++++++++++++++++++++++- > 1 file changed, 36 insertions(+), 1 deletion(-) > > diff --git a/mm/compaction.c b/mm/compaction.c > index 58a4e3fb72ec..fa9993c8a389 100644 > --- a/mm/compaction.c > +++ b/mm/compaction.c > @@ -1832,9 +1832,43 @@ static struct folio *compaction_alloc(struct folio *src, unsigned long data) > struct compact_control *cc = (struct compact_control *)data; > struct folio *dst; > int order = folio_order(src); > + bool has_isolated_pages = false; > > +again: > if (!cc->freepages[order].nr_pages) { > - isolate_freepages(cc); > + int i; > + > + for (i = order + 1; i < NR_PAGE_ORDERS; i++) { You could probably just start with a loop that finds the start_order (and do the isolate_freepages() attempt if there's none) and then handle the rest outside of the loop. No need to separately handle the case where you have the exact order available? > + if (cc->freepages[i].nr_pages) { > + struct page *freepage = > + list_first_entry(&cc->freepages[i].pages, > + struct page, lru); > + > + int start_order = i; > + unsigned long size = 1 << start_order; > + > + list_del(&freepage->lru); > + cc->freepages[i].nr_pages--; > + > + while (start_order > order) { With exact order available this while loop will just be skipped and that's all the difference to it? > + start_order--; > + size >>= 1; > + > + list_add(&freepage[size].lru, > + &cc->freepages[start_order].pages); > + cc->freepages[start_order].nr_pages++; > + set_page_private(&freepage[size], start_order); > + } > + dst = (struct folio *)freepage; > + goto done; > + } > + } > + if (!has_isolated_pages) { > + isolate_freepages(cc); > + has_isolated_pages = true; > + goto again; > + } > + > if (!cc->freepages[order].nr_pages) > return NULL; > } > @@ -1842,6 +1876,7 @@ static struct folio *compaction_alloc(struct folio *src, unsigned long data) > dst = list_first_entry(&cc->freepages[order].pages, struct folio, lru); > cc->freepages[order].nr_pages--; > list_del(&dst->lru); > +done: > post_alloc_hook(&dst->page, order, __GFP_MOVABLE); > if (order) > prep_compound_page(&dst->page, order);