From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BB1CC48297 for ; Fri, 9 Feb 2024 16:37:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D011E6B0072; Fri, 9 Feb 2024 11:37:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CB14D6B0074; Fri, 9 Feb 2024 11:37:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B52206B0075; Fri, 9 Feb 2024 11:37:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id A6A1E6B0072 for ; Fri, 9 Feb 2024 11:37:16 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 497DFC108B for ; Fri, 9 Feb 2024 16:37:16 +0000 (UTC) X-FDA: 81772820472.29.BA7DD11 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf16.hostedemail.com (Postfix) with ESMTP id 5952818000A for ; Fri, 9 Feb 2024 16:37:12 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=tAFh3NcO; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=qYWXlvNi; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=Fp8C1Gx0; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=deVoTIPF; dmarc=none; spf=pass (imf16.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707496632; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CDKKL0U7LaU3VV2//9ga67jQ7AvHjSSoo7VuO9ZKxWU=; b=eJNqVOV8sCB7zT17ax3WZpMFiYiRU1swDLqcYgkN3/Ci05IW1NF50lJvxePaH1v7tDA1ri 5tW5M/0glaUcf7k4IpZxhAXC0tQHEMTdirlKb6EwEI/IqBhzCMafTJwNvk4Mvnr5OcwqkG rHHfi4bxOBJdIuYz0ZIk7x7Ti5DB7Mk= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=tAFh3NcO; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=qYWXlvNi; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=Fp8C1Gx0; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=deVoTIPF; dmarc=none; spf=pass (imf16.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707496632; a=rsa-sha256; cv=none; b=y+yXkEp/Th6EuAPfL467p5IrBehFuxzFwXceFXQcfctuUusln2aemYK9abuwEsnhgLCHrR ZtNB/qhSkQt86XzimSZfhn/Dhn3QDboKMAn8GTq0D1LKORyv6dhHKKqGwfd74zu1Pj2AMN PYPl971SiUw6y+S3XLWMaAxAeEbEnV8= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 7BACA1F812; Fri, 9 Feb 2024 16:37:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1707496629; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CDKKL0U7LaU3VV2//9ga67jQ7AvHjSSoo7VuO9ZKxWU=; b=tAFh3NcO/12GUGYM9ESNwau+NtesuyDpInwivGpGHvicb9IyZgDEbA/uNOL7NWRcBVbT/z zMbr0CstFPBW04eqlM2yCccInC68CmkcC/AWZmz5/0TiM9RU5JJ2J6zWgzXgCt18S9JbzH naPbihBE+Mt2+2JzTeGMr6RQNdlJgOw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1707496629; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CDKKL0U7LaU3VV2//9ga67jQ7AvHjSSoo7VuO9ZKxWU=; b=qYWXlvNiIBSo0MaK4xZGM3pIXHjB1q6XzpCje2LHIkmkGsrTyWe85U8tUi1msB+77CNacQ ZIipfKz7qt6scCBA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1707496628; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CDKKL0U7LaU3VV2//9ga67jQ7AvHjSSoo7VuO9ZKxWU=; b=Fp8C1Gx0nAqZZsH5aaMD7vxMH7wB4Z69bheRK7ZzgiBwHxzhx0PMaEMDRzEhLcdnf78c5z OqiUk93DTM0vTPD+lJBG3H5lM6ESo58UBDIEXN5MXjhzf711uuKAr4kRszRbXrc+DHdl7D eruVmTdEadqusQVJY1B+ANjsqLuqQ0Q= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1707496628; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CDKKL0U7LaU3VV2//9ga67jQ7AvHjSSoo7VuO9ZKxWU=; b=deVoTIPForqnBTr/xCwzq+fOTMCUbiGsBE1EimMxhDN8aP3AUwTr7V6nVZldvgh+uxWlW3 ovzfyLw40WBaCJCQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4CB5013353; Fri, 9 Feb 2024 16:37:08 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id H/BXErRUxmX5GwAAD6G6ig (envelope-from ); Fri, 09 Feb 2024 16:37:08 +0000 Message-ID: <025b7e7c-b17f-47c7-8677-ee36fc6dbc52@suse.cz> Date: Fri, 9 Feb 2024 17:37:08 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3 2/3] mm/compaction: add support for >0 order folio memory compaction. Content-Language: en-US To: Zi Yan , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: "Huang, Ying" , Ryan Roberts , Andrew Morton , "Matthew Wilcox (Oracle)" , David Hildenbrand , "Yin, Fengwei" , Yu Zhao , "Kirill A . Shutemov" , Johannes Weiner , Baolin Wang , Kemeng Shi , Mel Gorman , Rohan Puri , Mcgrof Chamberlain , Adam Manzanares , "Vishal Moola (Oracle)" References: <20240202161554.565023-1-zi.yan@sent.com> <20240202161554.565023-3-zi.yan@sent.com> From: Vlastimil Babka In-Reply-To: <20240202161554.565023-3-zi.yan@sent.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 5952818000A X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 9t7jgqhg6b36ehauggc6ej4kwqt9ech1 X-HE-Tag: 1707496632-473728 X-HE-Meta: U2FsdGVkX1+mK8m/J87adk0zM1zFM+m19yEtopvxjVev67rCHBh27OeY4Ra8Y1XgS4S8x6iE5hRbBjqgaRnZHa4FDHNhmoKLF+hjDyZTmPtDpvCkSjmJMFKzqqZX1E3E+5q8ka8002eLIxcnEmZeoL+1IxUcTrc21T6bXaU1Yt4t2TM+6C13uRIc242dwYVs6IooS07QKn0Hyz4m92HGSWRnIasPhyc4ETDaOkWK4UPJi3/QFS4bBY1M+gkhba9ZpP3Dxhtxc3m2de5kVHDXoE0tUaf0hDn8myB+n3VvGbEPAQOd7RWCBD/qqJk41DngO5plFT96H04P1qrJ/VlOLwBxDepWiT/Tnx2JbGRb/4JSieRpkv15FTSUJEK02Xu91LET9vPOcqjLDiF6ibIjCpBYBtvSSdVzTQKrgIteoteL1ARDrRIYSiKZg2en5gqcfJGaxjjrNAtfGU6L10IAmkefRhteHScicSQ2IYvaIrvisSHXWael1CciSgGFbqSDyjgP4P4WIAdEqHV6nkL0ZOT2IKOkO87y9qTHBucHLJeSbf0PQ7KP3yoCqtfH9It6eTWwsj6YS9K2DxqX8mkxqOIQ4L6d0XqfGo8dbGae0TnzXc8wU8Lcss9bMnenazpYDmcSwK/TitSn2hwfJTGkXscB2CCkLXkYv+4Sr7Ekbv8iRrRw7L1FqIcBQR/aGLutKphf9jli16+nnrXUAjRFVLEOzCZMbLVd7QuflvC08sPIxk+2p7LgJDNIVtPzZZ/6emFB5pMk2W928G0NLJVRgqgBcba843ErhXzcStB4KlyzOxz3EWDK2O0XVtsSGmT7+w6EcWtPJ/y+y1TAC/3m0R8mIxV+5EW800fEb7Fv4L8blghZxC5ANvBOawzRc9AcBxLGQ2SU36AWC8nUOMavX8gDFq3+Y7BBqoqW7YrHBCVIi4SK5J4bFR2bPncXryNfaoue3tcVHNQMcdV+ZKl 62tY5Llg wTKCmus2ubmId9lj+5OF28d5kqluamtrrkYUYJN/0TaHqCcseHEph12Zsub5dmEQ/z4lzP0Yk8w8u1Da8zVusRe1E0fEL67EVhDI9DyhzLgkBiHOfdioFfFtAlLUc468Sz/fv1xwZQ5QoE2p4MV+uHUvkboqCMB3+AVzqexXAS0cUFY71WX9SJKKnIyD5KTxv8JcaKPG3ntQ3W9IIzvFkZ9jOClCAMExwhgvaybp2YlwAg0pd8U5DNDCaDe+8KWBtw8Wkpr+l3KYIJbXc4RuhVc4joDVcc1IUd+/Ij434EpqkoFEmpUufLYUfxR4z3fQS5y/EywjXNoBhAwYGdzvzRFV2bRIZcHoJ3dyrZsMPBap6eDw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2/2/24 17:15, Zi Yan wrote: > From: Zi Yan > > Before last commit, memory compaction only migrates order-0 folios and > skips >0 order folios. Last commit splits all >0 order folios during > compaction. This commit migrates >0 order folios during compaction by > keeping isolated free pages at their original size without splitting them > into order-0 pages and using them directly during migration process. > > What is different from the prior implementation: > 1. All isolated free pages are kept in a NR_PAGE_ORDERS array of page > lists, where each page list stores free pages in the same order. > 2. All free pages are not post_alloc_hook() processed nor buddy pages, > although their orders are stored in first page's private like buddy > pages. > 3. During migration, in new page allocation time (i.e., in > compaction_alloc()), free pages are then processed by post_alloc_hook(). > When migration fails and a new page is returned (i.e., in > compaction_free()), free pages are restored by reversing the > post_alloc_hook() operations using newly added > free_pages_prepare_fpi_none(). > > Step 3 is done for a latter optimization that splitting and/or merging free > pages during compaction becomes easier. > > Note: without splitting free pages, compaction can end prematurely due to > migration will return -ENOMEM even if there is free pages. This happens > when no order-0 free page exist and compaction_alloc() return NULL. > > Signed-off-by: Zi Yan ... > /* > @@ -1835,9 +1857,17 @@ static struct folio *compaction_alloc(struct folio *src, unsigned long data) > static void compaction_free(struct folio *dst, unsigned long data) > { > struct compact_control *cc = (struct compact_control *)data; > + int order = folio_order(dst); > + struct page *page = &dst->page; > + > + folio_set_count(dst, 0); We can't change refcount to 0 like this, after it was already set to 1 and somebody else might have done get_page_unless_zero(). You need to either put_page_testzero() and if it's false, consider the page lost, or leave it refcounted and adjust the code to handle both refcounted and non-refcounted pages on the lists (the first option is simpler and shouldn't be too bad). Perhaps folio_set_count()/set_page_count() should get some comment warning against this kind of mistake. > + free_pages_prepare_fpi_none(page, order); > > - list_add(&dst->lru, &cc->freepages); > - cc->nr_freepages++; > + INIT_LIST_HEAD(&dst->lru); > + > + list_add(&dst->lru, &cc->freepages[order].pages); > + cc->freepages[order].nr_pages++; > + cc->nr_freepages += 1 << order; > } > ... > > extern void free_unref_page(struct page *page, unsigned int order); > @@ -473,6 +475,11 @@ int split_free_page(struct page *free_page, > /* > * in mm/compaction.c > */ > + > +struct page_list { > + struct list_head pages; > + unsigned long nr_pages; I've checked and even with patch 3/3 I don't think you actually need the counter? The only check of the counter I noticed was to check for zero/non-zero, and you could use list_empty() instead. > +}; > /* > * compact_control is used to track pages being migrated and the free pages > * they are being migrated to during memory compaction. The free_pfn starts > @@ -481,7 +488,7 @@ int split_free_page(struct page *free_page, > * completes when free_pfn <= migrate_pfn > */ > struct compact_control { > - struct list_head freepages; /* List of free pages to migrate to */ > + struct page_list freepages[NR_PAGE_ORDERS]; /* List of free pages to migrate to */ > struct list_head migratepages; /* List of pages being migrated */ > unsigned int nr_freepages; /* Number of isolated free pages */ > unsigned int nr_migratepages; /* Number of pages to migrate */ > diff --git a/mm/page_alloc.c b/mm/page_alloc.c > index 5be4cd8f6b5a..c7c135e6d5ee 100644 > --- a/mm/page_alloc.c > +++ b/mm/page_alloc.c > @@ -1179,6 +1179,12 @@ static __always_inline bool free_pages_prepare(struct page *page, > return true; > } > > +__always_inline bool free_pages_prepare_fpi_none(struct page *page, > + unsigned int order) > +{ > + return free_pages_prepare(page, order, FPI_NONE); > +} > + > /* > * Frees a number of pages from the PCP lists > * Assumes all pages on list are in same zone.