From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90BAAC433FE for ; Thu, 19 May 2022 09:45:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0390A6B0072; Thu, 19 May 2022 05:45:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F035F6B0073; Thu, 19 May 2022 05:45:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D7D6B6B0074; Thu, 19 May 2022 05:45:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C16806B0072 for ; Thu, 19 May 2022 05:45:39 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 8F4B520AF5 for ; Thu, 19 May 2022 09:45:39 +0000 (UTC) X-FDA: 79482010398.08.C828575 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf03.hostedemail.com (Postfix) with ESMTP id 4D98820008 for ; Thu, 19 May 2022 09:45:26 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id B04081F7AB; Thu, 19 May 2022 09:45:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1652953535; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iT9kRUwiyxxUSpnNf8Ny1b4Sc9C5VIYIhqK0/GwAUAI=; b=TGyFepsYj68kJ9HulgzE3EwIoy6sDCdr46aZu8LoBwMzdYsjFraX0PBtazePpMl7qlpOIs jGdArDfcNvtFjG+nwAjbY/O9hrMz0sTDx/Zn1TC4D+s6p4we257SKgTM4KrSQzvs2U/ZYK Yspgj/2LYR4LtIAF996fnD/sSBOIFi0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1652953535; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iT9kRUwiyxxUSpnNf8Ny1b4Sc9C5VIYIhqK0/GwAUAI=; b=UnytVQjSWvwo6TxK+6C4QbpQ0d+Ay+lTReNp0NlF/n0Kz22PGQLsX2T9XtmgZC6wrA3loQ F5WWZ6cHh3CdFoDQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 877A713456; Thu, 19 May 2022 09:45:35 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 9ND2H78RhmLFNQAAMHmgww (envelope-from ); Thu, 19 May 2022 09:45:35 +0000 Message-ID: <80c89ad9-17ec-8d64-3774-821451e147c4@suse.cz> Date: Thu, 19 May 2022 11:45:35 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.9.0 Subject: Re: [PATCH 2/6] mm/page_alloc: Use only one PCP list for THP-sized allocations Content-Language: en-US To: Mel Gorman , Andrew Morton Cc: Nicolas Saenz Julienne , Marcelo Tosatti , Michal Hocko , LKML , Linux-MM References: <20220512085043.5234-1-mgorman@techsingularity.net> <20220512085043.5234-3-mgorman@techsingularity.net> From: Vlastimil Babka In-Reply-To: <20220512085043.5234-3-mgorman@techsingularity.net> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 4D98820008 X-Stat-Signature: izubo5o4d1uezqosqydt88q4giejtcq8 X-Rspam-User: Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=TGyFepsY; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=UnytVQjS; spf=pass (imf03.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none X-HE-Tag: 1652953526-5475 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 5/12/22 10:50, Mel Gorman wrote: > The per_cpu_pages is cache-aligned on a standard x86-64 distribution > configuration but a later patch will add a new field which would push > the structure into the next cache line. Use only one list to store > THP-sized pages on the per-cpu list. This assumes that the vast majority > of THP-sized allocations are GFP_MOVABLE but even if it was another type, > it would not contribute to serious fragmentation that potentially causes > a later THP allocation failure. Align per_cpu_pages on the cacheline > boundary to ensure there is no false cache sharing. > > After this patch, the structure sizing is; > > struct per_cpu_pages { > int count; /* 0 4 */ > int high; /* 4 4 */ > int batch; /* 8 4 */ > short int free_factor; /* 12 2 */ > short int expire; /* 14 2 */ > struct list_head lists[13]; /* 16 208 */ > > /* size: 256, cachelines: 4, members: 6 */ > /* padding: 32 */ > } __attribute__((__aligned__(64))); > > Signed-off-by: Mel Gorman > Tested-by: Minchan Kim > Acked-by: Minchan Kim Acked-by: Vlastimil Babka > --- > include/linux/mmzone.h | 11 +++++++---- > mm/page_alloc.c | 4 ++-- > 2 files changed, 9 insertions(+), 6 deletions(-) > > diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h > index 962b14d403e8..abe530748de6 100644 > --- a/include/linux/mmzone.h > +++ b/include/linux/mmzone.h > @@ -358,15 +358,18 @@ enum zone_watermarks { > }; > > /* > - * One per migratetype for each PAGE_ALLOC_COSTLY_ORDER plus one additional > - * for pageblock size for THP if configured. > + * One per migratetype for each PAGE_ALLOC_COSTLY_ORDER. One additional list > + * for THP which will usually be GFP_MOVABLE. Even if it is another type, > + * it should not contribute to serious fragmentation causing THP allocation > + * failures. > */ > #ifdef CONFIG_TRANSPARENT_HUGEPAGE > #define NR_PCP_THP 1 > #else > #define NR_PCP_THP 0 > #endif > -#define NR_PCP_LISTS (MIGRATE_PCPTYPES * (PAGE_ALLOC_COSTLY_ORDER + 1 + NR_PCP_THP)) > +#define NR_LOWORDER_PCP_LISTS (MIGRATE_PCPTYPES * (PAGE_ALLOC_COSTLY_ORDER + 1)) > +#define NR_PCP_LISTS (NR_LOWORDER_PCP_LISTS + NR_PCP_THP) > > /* > * Shift to encode migratetype and order in the same integer, with order > @@ -392,7 +395,7 @@ struct per_cpu_pages { > > /* Lists of pages, one per migrate type stored on the pcp-lists */ > struct list_head lists[NR_PCP_LISTS]; > -}; > +} ____cacheline_aligned_in_smp; > > struct per_cpu_zonestat { > #ifdef CONFIG_SMP > diff --git a/mm/page_alloc.c b/mm/page_alloc.c > index f58f85fdb05f..5851ee88a89c 100644 > --- a/mm/page_alloc.c > +++ b/mm/page_alloc.c > @@ -648,7 +648,7 @@ static inline unsigned int order_to_pindex(int migratetype, int order) > #ifdef CONFIG_TRANSPARENT_HUGEPAGE > if (order > PAGE_ALLOC_COSTLY_ORDER) { > VM_BUG_ON(order != pageblock_order); > - base = PAGE_ALLOC_COSTLY_ORDER + 1; > + return NR_LOWORDER_PCP_LISTS; > } > #else > VM_BUG_ON(order > PAGE_ALLOC_COSTLY_ORDER); > @@ -662,7 +662,7 @@ static inline int pindex_to_order(unsigned int pindex) > int order = pindex / MIGRATE_PCPTYPES; > > #ifdef CONFIG_TRANSPARENT_HUGEPAGE > - if (order > PAGE_ALLOC_COSTLY_ORDER) > + if (pindex == NR_LOWORDER_PCP_LISTS) > order = pageblock_order; > #else > VM_BUG_ON(order > PAGE_ALLOC_COSTLY_ORDER);