From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEC74C77B7D for ; Fri, 5 May 2023 05:52:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1FD746B0075; Fri, 5 May 2023 01:52:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1AD4C6B0078; Fri, 5 May 2023 01:52:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 04E996B007B; Fri, 5 May 2023 01:52:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by kanga.kvack.org (Postfix) with ESMTP id A8A7D6B0075 for ; Fri, 5 May 2023 01:52:13 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1683265933; x=1714801933; h=message-id:date:subject:to:cc:references:from: in-reply-to:content-transfer-encoding:mime-version; bh=MwrqGFl6fzfsbpD23stnEDtm9AdnD6PxIOmdDzeMXW0=; b=B0rM0oQR2v85V5EzI4HzUBPBY0WuvZzxEv3EO4bE071AV6wRqQoOOans y/l5h5GmUKRcZy8O31P6GoEmnB0/10XvSesTJLUDe6YKzvzCqOF6GcF1x FgPxySjF2X8go+1UXqcuCz5tRrwvwoLlN0PldLECfJMtE/KN4+nTB2NEz 7cinpPmz6pBoklVs0mI6cCa1xlH+xw8zLiXgMDqBDuCFpBqpq6EMV0EH+ YISsLzXYML1qkTBUFtgEifcNmbiJhxYKLh6qUmv+jRID/xM+jFftC89Q8 XrxYkYFok0mhxH/iKaL6+v8Vbs4uRxdN0JaiTav98aY8/TQJz2uI/nSUm A==; X-IronPort-AV: E=McAfee;i="6600,9927,10700"; a="414668590" X-IronPort-AV: E=Sophos;i="5.99,250,1677571200"; d="scan'208";a="414668590" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 May 2023 22:52:10 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10700"; a="647669966" X-IronPort-AV: E=Sophos;i="5.99,250,1677571200"; d="scan'208";a="647669966" Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by orsmga003.jf.intel.com with ESMTP; 04 May 2023 22:52:10 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Thu, 4 May 2023 22:52:10 -0700 Received: from fmsmsx601.amr.corp.intel.com (10.18.126.81) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23; Thu, 4 May 2023 22:52:09 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx601.amr.corp.intel.com (10.18.126.81) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.23 via Frontend Transport; Thu, 4 May 2023 22:52:09 -0700 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (104.47.51.44) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.23; Thu, 4 May 2023 22:52:08 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=T7tyME4cll9nU9uIjz86JxRCBuRuTl9AjzWQu2bATFVMS5JR+toJXpFaxTk7kQX9MUrylzySTwd84BVrN2W6rAa4DRTR1wKaZ63GD1BVtGXUqeM+SLXZ5E+WXDKRpRzj1BwyHgfD2TxYuYnYAEeXM5unfKqtBYRREnNHgiINFxEdkNWiNVYuGaFGkjMvwpuIB4EpZHXSUqKpV8E8XSczIu4ppjbvDTCSGeEaDq1B7xRmnk7NlDTLQeUmI68Ac5bdklpCSJyM2Dofcy+Ui6pXfK9W8AqO7A34kbOznPDfLjoxYioILYeCPFbPGr54IbJxvW7txSmR5lQ3Wf7HEX8x4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LoNmbucozgxg1XCywmKyKzvkKKmvOpKsp2bkACRgYao=; b=FTS6KFDP7745GQAD3fJ1qp6f3HaYmQXXAvJV/NJcRM8vZiA4Z+7oNj/4q8riFHw91MfFi660IxOJIgPj1jsXKFatSoItpPcOT05wkdryZ4YDdqCqp8QkcCZTLEWvj5yMvwECzkk2sIfUI7IMH0ZRAxde/5Eb/KDxVw8/hEv3v3VY/orVY7WiI/G1TmH9H9JthWU2ylGJmEN9vQZLq+gWs8AuviSA8kP8Ba/SnkfGkci5txcj5qp3bXY4kPrb/Qc+vucLIZ9V4bXpvxOCOKPSYWTKpO7teRdMERLsnRTc9dGEIg6PZtyGbKX95z5ows8+C8B9jvvENwP/uSB8FTBlZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from CO1PR11MB4820.namprd11.prod.outlook.com (2603:10b6:303:6f::8) by MW4PR11MB5870.namprd11.prod.outlook.com (2603:10b6:303:187::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6363.20; Fri, 5 May 2023 05:52:06 +0000 Received: from CO1PR11MB4820.namprd11.prod.outlook.com ([fe80::d1a0:743b:4659:b72]) by CO1PR11MB4820.namprd11.prod.outlook.com ([fe80::d1a0:743b:4659:b72%7]) with mapi id 15.20.6363.027; Fri, 5 May 2023 05:52:06 +0000 Message-ID: <8d4f938e-4f0a-bb97-3890-910b5838d6f5@intel.com> Date: Fri, 5 May 2023 13:51:57 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Firefox/102.0 Thunderbird/102.10.1 Subject: Re: [PATCH v3 2/2] lru: allow large batched add large folio to lru list Content-Language: en-US To: Matthew Wilcox CC: , , , , , References: <20230429082759.1600796-1-fengwei.yin@intel.com> <20230429082759.1600796-3-fengwei.yin@intel.com> From: "Yin, Fengwei" In-Reply-To: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-ClientProxiedBy: SI2PR01CA0042.apcprd01.prod.exchangelabs.com (2603:1096:4:193::23) To CO1PR11MB4820.namprd11.prod.outlook.com (2603:10b6:303:6f::8) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PR11MB4820:EE_|MW4PR11MB5870:EE_ X-MS-Office365-Filtering-Correlation-Id: 92b4b507-5ef0-4a04-5d28-08db4d2cdb80 X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: UTxgxKL2IDBTv0oqad3k5j2aUNjXvenUoco2i+9Jp4RmdxQhMkR9LoRXPOPae6OfT6RCPvJAbXy9EAi3bRy0c9Izf0WiobrXQnHUk5HbiCOh3+OQDyZKLtDFXI3EQKrHl/UZqeMZJxXd6mUzv7SiBvkf9wqQZHtcyp/YMz/B02EWVBjRswnEY2u3UCzc/DXQLHzMiTTGeG6Yk9ia1ecU8QlwKrC9MG07k9dTHu8gXj7U2fvqywWv8WXGxZFqInFps6ziFvqHRN9ZnOdguQtRkASDkj1OaPu2F8Li8Jh2lAiuDBO/CPoXC1N62cLbhIcZkEkBcbOTgCO+3bqrc0mwi1gxKVLNcnEHjL+NSF5lCR/16OoD2uK1VwPewXP+GRaWK9uqSE6Q1ikxy40lNdASv2tF51o62L6JYF9lKsdHRYP3+fsAbNw8nZff5phjsKma0HgNgxReuKq2hbzpv+cEB0uITuh5DOPn473PsRlX0vNDXqvqRBR172KCrV+aCCswReBO6xYbG8GYvJh0yCtMULCUpSaRmv/qcbpe8nDSs3psWL9/dmDmXG1vCb06wPxjYbFC4un0TlJRQNNsda9swIcWEgiEWYjQSTCTXfFS+C1ryXD1BC6M5xEF5Kf3jAzn+czpD0mx7U+kRgj/KdxQYw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO1PR11MB4820.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(366004)(39860400002)(396003)(136003)(376002)(346002)(451199021)(5660300002)(31696002)(2616005)(86362001)(107886003)(6506007)(53546011)(26005)(6512007)(186003)(82960400001)(38100700002)(83380400001)(41300700001)(8936002)(36756003)(8676002)(4326008)(6916009)(6486002)(316002)(478600001)(6666004)(31686004)(66946007)(66556008)(66476007)(2906002)(43740500002)(45980500001);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SkFWaVVwdDlZR0RTaGhaeUw5UHdqTzd0eTZPd044Z0sxMTdSaTRwaFE5MDBn?= =?utf-8?B?WWhjekNTT09nQ0o5QlNxdkZra2xvRjVqbllIZmFBTjc4YWxKai9nNHhKa2Rn?= =?utf-8?B?dTBnY2N4VGNMVWpkMU9mTjhoQlZNcHBNMkRuSkwwWmd5UkI1eGJvREc2cHFG?= =?utf-8?B?TkV1Unh3cUhMOC9GbTNUaHEyZkhGT1owMlM4cDJDZnBCZEs4TmcwY2RSQ05r?= =?utf-8?B?d2xnaUVDQjgyWWJQMTlWSFJiTTVVY2VPcWxGcE1reDJWL0Y3cWVwL0JpNk00?= =?utf-8?B?T2hvYWQxSldKcGltb2VOeWdvREppNlJHSlJFN0RxR0p6OVlqcEpGU1p2b2lw?= =?utf-8?B?QUtHdVBuTFZNK2pCeHl3Zk0wQzkyTE9acllRM2pmVzBGNDJQdFlhSTFTQXVV?= =?utf-8?B?YkxDQjhlbWtPV055dmpzOE1mR0RqMkl1MFEwYjd5VjQ1Z095SkhRODRFQkdZ?= =?utf-8?B?NFp0d2s1ZmlGeGNlSGhoYm1HYU5XS0ZiZndPaWZJVmh1NlNqY1E5cVNwWlov?= =?utf-8?B?bmdUQTVmSnZDbXFnMzJmSnEwUkYranpJMkpoN1BMZ3hoajNneUd6a1RDZjVS?= =?utf-8?B?Rk5Hb29nNXR3MlI5UXM4dHhvU3YycUpESTJjb0FpbkE4L0YzanJMZEFLNjRu?= =?utf-8?B?NTBqdVhKQVFJcEZ3Q1UrbHhKTkg4cDIrL2JPaHg5a3ZoTGxWZXlCdzRFdGV0?= =?utf-8?B?TjY2N2RDMzYyYVFLZ2tJN2Q0aDRyR2V0cWRmVXc3aUFuQUVTVXN4RVdPR1Bk?= =?utf-8?B?emZWdG1CVkNWOFdPUUZUSWlRRUdSSS9pa3VZRnNPSmdNZS96b05zcE11T1Vm?= =?utf-8?B?aCtMQVZaNzRZNzRhbjYveS9xWFZveHFmRklUMCs5MWpQcWprVysvSWRqc2M0?= =?utf-8?B?b2lMaE5tZE5mUzR3eEpLTnZkQ1VvcHplUTFlN2hhZVJMSFZURkJRK0kzWlkz?= =?utf-8?B?Q1RsWVMxendxVzY4c2dRNmpIM0twSXpEYTRpVnlMSWVCanFnMUVDZlN3ZHdi?= =?utf-8?B?K1BZWUtPWXV1Rk1SMk1EUHVvdm9GaUUxSmZPMGtGV0hqVU8rUHVPeS9lUE9S?= =?utf-8?B?aXVuVTJ3b3E1d1U0dHJINno0VVdEVWdNRzhFYnNudU5hWFZ0cVVzdDFBaWM3?= =?utf-8?B?M0Z6SzlyWitST3U0cTVFZVdoTWpnMjVENU1LT2lOYmZaZ210TnozQTFuQXlj?= =?utf-8?B?UkpvbzBlOFovQ292emYwVFlCTE9oc25acGRJajhqd0hiTDg4S3NMNGgvMitn?= =?utf-8?B?ejU2UlY4WHpaRE8zTE5VbEFoT2FrWE1pazFwai9UaEt2K2NyRDBhZVQvTHJG?= =?utf-8?B?UVkrT1lQcWVCTmluQUNVQXo5M0VtNUxXeHJQL1hqaUt4aStEYlFuNEducS9O?= =?utf-8?B?ZFE5VXNGTFFLeTlMV1hwNDF4RkpydWRzOGhHdnpOS1ZMQVVnL2JUbmNBRXJF?= =?utf-8?B?Vk0wMFArdXlLMjIvbUlDM1JsaFpoc3VVU3U4cXk0WTBwV1dIQ2Q3NSsvUjJk?= =?utf-8?B?SHFqVnJQNlpGZWVTazZ5UWdtd1JiKy9NVTM3cUpIRkR0eWpKZ3dxUzg0ZjhG?= =?utf-8?B?NHR0M0Y4RWJRZDRncm5GZlRmR0NrUTRDWm9DekxZdUJRb09FdCsxUk5WR2pj?= =?utf-8?B?UjZibWV4NUwwYlMvTEt1VW1ZZWJveExrY3lXSmt6em0yaDAvNW1MTUZGV2s4?= =?utf-8?B?QVhyU3o2TTlkeXFtQ2lQeVV5dmluRWpZamFRVEdHazJtZzhyYk9jMnZJWTNB?= =?utf-8?B?QWFEMGFUZHRyK0hKb1VzREVhaTlHcTBmbmN5TFZWdDY2aDFsRjM0dzNzTlpD?= =?utf-8?B?L05MbEVZWm9PRjRJcWhhL20zOUJwMHdaa1ovQktNTjZLand6bkkxVHF0akMy?= =?utf-8?B?bEd0YVYvblRMS1dSayt0RHJtUVhVSjVBbGJYR2pFWmpCcDZVVEpodGo1andS?= =?utf-8?B?eFB0STNSQXFTWkNYSTVKU0FtMjJLbDR1VjdlcVVxQW1RcG82TWpOYmhkdjZy?= =?utf-8?B?NjF2Q2xndTQvNnNLd2JDM3lXczYxZlRrbUVndmx2REtaT2xCZ3l6SG9vQjN4?= =?utf-8?B?N25PZlZvSzJaU3Ivc0xVMU1QelpWNW0wd3FRSk4wUUl0dHEwcGdDeUVISkhx?= =?utf-8?Q?0T8pTgsCJ6XDa/JPsusUAih1P?= X-MS-Exchange-CrossTenant-Network-Message-Id: 92b4b507-5ef0-4a04-5d28-08db4d2cdb80 X-MS-Exchange-CrossTenant-AuthSource: CO1PR11MB4820.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 May 2023 05:52:06.2615 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 4us3HSPl35jKRsuLQqj1B2ZZVHtgIySdrSk806lnAQdEHu0KmweRqftCBgm+xdITAwXGaca+hGhSkhZuQRqpGA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR11MB5870 X-OriginatorOrg: intel.com X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Matthew, On 4/30/2023 6:35 AM, Matthew Wilcox wrote: > On Sat, Apr 29, 2023 at 04:27:59PM +0800, Yin Fengwei wrote: >> @@ -22,6 +23,7 @@ struct address_space; >> struct pagevec { >> unsigned char nr; >> bool percpu_pvec_drained; >> + unsigned short nr_pages; > > I still don't like storing nr_pages in the pagevec/folio_batch. > What about the change like following: diff --git a/mm/swap.c b/mm/swap.c index 57cb01b042f6..5e7e9c0734ab 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -228,8 +228,10 @@ static void folio_batch_move_lru(struct folio_batch *fbatch, move_fn_t move_fn) static void folio_batch_add_and_move(struct folio_batch *fbatch, struct folio *folio, move_fn_t move_fn) { - if (folio_batch_add(fbatch, folio) && !folio_test_large(folio) && - !lru_cache_disabled()) + int nr_pages = folio_nr_pages(folio); + + if (folio_batch_add(fbatch, folio) && !lru_cache_disabled() && + (!folio_test_large(folio) || (nr_pages <= (PAGEVEC_SIZE + 1)))) return; folio_batch_move_lru(fbatch, move_fn); } I did testing about the lru lock contention with different folio size with will-it-scale + deferred queue lock contention mitigated: - If large folio size is 16K (order 2), the lru lock takes 64.31% cpu runtime - If large folio size is 64K (order 4), the lru lock takes 24.24% cpu runtime This is as our expectation: The larger size of folio, the less lru lock contention. It's acceptable to not batched operate on large folio which is large enough. PAGEVEC_SIZE + 1 is chosen here based on following reasons: - acceptable max memory size per batch: 15 x 16 x 4096 = 983040 bytes - the folios with size larger than it will not apply batched operation. But the lru lock contention is not high already. I collected data with lru contention when run will-it-scale.page_fault1: folio with order 2: Without the change: - 64.31% 0.23% page_fault1_pro [kernel.kallsyms] [k] folio_lruvec_lock_irqsave + 64.07% folio_lruvec_lock_irqsave With the change: - 21.55% 0.21% page_fault1_pro [kernel.kallsyms] [k] folio_lruvec_lock_irqsave + 21.34% folio_lruvec_lock_irqsave folio with order 4: Without the change: - 24.24% 0.15% page_fault1_pro [kernel.kallsyms] [k] folio_lruvec_lock_irqsave + 24.09% folio_lruvec_lock_irqsave With the change: - 2.20% 0.09% page_fault1_pro [kernel.kallsyms] [k] folio_lruvec_lock_irqsave + 2.11% folio_lruvec_lock_irqsave folio with order 5: - 8.21% 0.16% page_fault1_pro [kernel.kallsyms] [k] folio_lruvec_lock_irqsave + 8.05% folio_lruvec_lock_irqsave Regards Yin, Fengwei