From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8136C64EC7 for ; Tue, 28 Feb 2023 21:30:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2B9096B0074; Tue, 28 Feb 2023 16:30:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 269CB6B0075; Tue, 28 Feb 2023 16:30:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 12F156B007B; Tue, 28 Feb 2023 16:30:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 0416D6B0074 for ; Tue, 28 Feb 2023 16:30:13 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id B43A2120AE3 for ; Tue, 28 Feb 2023 21:30:12 +0000 (UTC) X-FDA: 80517993864.26.BF73D94 Received: from mail-yw1-f171.google.com (mail-yw1-f171.google.com [209.85.128.171]) by imf06.hostedemail.com (Postfix) with ESMTP id AD8A4180021 for ; Tue, 28 Feb 2023 21:30:10 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Ouuzq1Pc; spf=pass (imf06.hostedemail.com: domain of hughd@google.com designates 209.85.128.171 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677619810; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eA3BqXcGEc46f5liQkJ2TjEWDJOzbNiIQCKLaYI0SXA=; b=hhqGzo/VJ3oLTCakZGREpPcbNJA62RJNZV/krQqqMS+wRFyKOOftrXFqLyhYJFWvAs99HJ zoVj4cylfmGGLNgpU7uwuHo7C6iNGkdymV2Uv3+cl8FHg0gXeOf7Hfjvun8e6bxEV4iDDS a2NlnQOWbZmASh4WdIRN16ESH7DWMcU= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Ouuzq1Pc; spf=pass (imf06.hostedemail.com: domain of hughd@google.com designates 209.85.128.171 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677619810; a=rsa-sha256; cv=none; b=X9whZQ9w8uYyUZqLvRfc2tBivz6MVDs0ykBRjWNWDTNzx5r6vmVBg7H8zUR/cI8uiwV9nM srt5HBy2J7Loq2GMfrBaB131ckFD8dSeV28vI3Ix4GflUKlFyWkJkGMQHseTdP3+FLN0hf OxIr7+J5X8oNy+0GmgIbBpPeznWMTEI= Received: by mail-yw1-f171.google.com with SMTP id 00721157ae682-536be69eadfso312874937b3.1 for ; Tue, 28 Feb 2023 13:30:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1677619809; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=eA3BqXcGEc46f5liQkJ2TjEWDJOzbNiIQCKLaYI0SXA=; b=Ouuzq1PcvpGMrbqUeMOdYicGq81u6UM9+W/BbYRqqFK5MwWQIqkDOPd44hU2juuPkJ 4tNNgAMSiHkrKSgM1sbrB22AN8cpDgL6E3L5lYt9+YHBZQU9iTBmaDC5WVv2lfjYmWBj bRjb/tIHyihDRV+t3qX0CyJS3VDnfppEYs1QXiPsOcG8jmplpdm7R0vpJG08xObquMd9 D0F8DJEvWWprf15YId73yES5PrTNJimpAc0Tu2Jzsyy5tfTBh9WDHOmgIR+PQnBBWjGJ sl3odyGkT0MnbTaRWHArt6Gki1ixDFWgOiqic4jmohVQOZDcQqWN5nyILX6TcpJj1/QX 7TCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677619809; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=eA3BqXcGEc46f5liQkJ2TjEWDJOzbNiIQCKLaYI0SXA=; b=sUdHUIYJS1c9Co3Mbo68x0lYKthY2iA6T2KJv7zI1q9zDabc3o8fFWrU+lQLAqnmAH zluzG1iac5NRbt2Tx4RhrUFfc+flWUmLjC/NTWmq4UVf6hcJ42rET32Epu9BZI+r/2+V SpOpRLFNr2a+R1naNMy8a6zVvGs4UODwDviyrFD4h4E7BxTlJ2h+gFcNHjocKtCJef79 Yq32JPjaHNIw+ufMC94ntR8VNyfoXw4mO9O67F6GIhq053YGUimEasFslx1dAMJYARsm /XD0wWHWujxZN+EqPcoj0/GSZwJUed3+bg+TIVNVDiy9LhHNWPVWBOVtpbLNu/p3Wd7/ jrMg== X-Gm-Message-State: AO0yUKVaVbiHnIblUIw1anLpIs61ns6HSXEpOXInbsUJ9O6LVBhX7NK4 ut01k0nYFAN2Yp+A13HsKVb5JAYeNGfZqHNLlkQ= X-Google-Smtp-Source: AK7set9QLRaN0qvbqFj+yjo3Bh1WlVVO26R8J7dMRJHdm0Bt+ZxyLy3MLUk+OBPK/7bGqF1wQBzB2A== X-Received: by 2002:a05:622a:19a0:b0:3bf:de3a:e717 with SMTP id u32-20020a05622a19a000b003bfde3ae717mr8136570qtc.39.1677619381941; Tue, 28 Feb 2023 13:23:01 -0800 (PST) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id 15-20020ac856ef000000b003bfeb30c24dsm1329481qtu.39.2023.02.28.13.23.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Feb 2023 13:23:01 -0800 (PST) Date: Tue, 28 Feb 2023 13:22:59 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@ripple.attlocal.net To: "Huang, Ying" cc: Hugh Dickins , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, "Xu, Pengfei" , Christoph Hellwig , Stefan Roesch , Tejun Heo , Xin Hao , Zi Yan , Yang Shi , Baolin Wang , Matthew Wilcox , Mike Kravetz Subject: Re: [PATCH 3/3] migrate_pages: try migrate in batch asynchronously firstly In-Reply-To: <87cz5ub5dr.fsf@yhuang6-desk2.ccr.corp.intel.com> Message-ID: <070f71-9af-c29a-30b9-758b5cdf6766@google.com> References: <20230224141145.96814-1-ying.huang@intel.com> <20230224141145.96814-4-ying.huang@intel.com> <87cz5ub5dr.fsf@yhuang6-desk2.ccr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-Rspamd-Queue-Id: AD8A4180021 X-Stat-Signature: yjjra7sxt4mnyopghzxki5nid81grpah X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1677619810-177313 X-HE-Meta: U2FsdGVkX19y5y1cZy6+F+w2jSeKxPUTslPLZn9MCLQ9SprGZofEkhDyHONTabhUkpv6r62AcKZsfRKHlldk/iQrFpjhy4ZzHC61cmCcb9BHoWqnGQpLtMXo2TJqgkG00V/ZrJuGrO42Qysw4LY+IfuptWisp/VAUL/4Z/PdT0SmQYWDZXZSXRW+3khVCAdd04Ty7/ZDT3j39nM+uklMql1ZxzmWN9AUrb+xLkNBk0I2KEQvUpyoy+UbAlF6bDfmZLoysE76SdxWLI9G0ZZgw8Xsqc23uODHjCLAj4P/QazFhMWY88/dXo7OfUlZI9ZGPl67BcMSyDU3CxclpyaS4IXKSv7zSwwOsmbynAudApVDezBJUD5rydUPrSJ0xwDUoWDkATQizR/DqLNjA65p8cs+jNJNTHL8w8GK1dZbu4NLcuDgb94xrtWgRz8yUJF6PCasW/nyLlYCppgdKvu7nGfJFIUrDoTi9uWuM4fBTLn7Cxy7gD7dfochKro+5t4tGOxhp+mejq7Oos/R7B1rKpC/7T45D8VinU3IvXrmtFHjOyP7C6qa1r/+CDwmC2Pk/fyMbyEBc0r34fAJ1O4SAsHAlCu4X/VVh6WRvYh2FzM3nzVR+a25cLVYG8PYNWFSQVkkZeDLWPDdbbRQE1FSExSNE2fmt+m4f9O2+UpS0JUGBDssLomhQYdz/FbFoKewi9ppzTwsHm7CBuZJL46rzDhMBcY8LhOpzcJQ5wGK6MVmN/2vvGQ056hNn++ukdI6Jht5KzVLf9DeEhjd5rz1eUKCvErGL92pYt9rX8lfT20P6MYE5XV1wfGLciBoo0oMTtOrYm59++a0nZrhWK+a8994qPDZHFi17mPXgZYz9bgG0MAzBjsOSoSq4Fas9Kociplsi2SmMsxJD7bQjK1h+kF413QepEAe7Wpp0URQOlOy/ZULpQdVKjC8WuVdgc8vnTMt9wToHHXn9W2P9vA qj6PWIeW iXjOD4Jz5s39T3bVSnNGSq3tCwPbJyq5l9Yqlz5uccZI+it/4CtCcVwsMCToFf2zymqwMEp8gBMg3yKMKrcspZMIRwTq2A8FZ94fivdU9h+X9FicGZxYAMvvJfbdw5uLM9SnK2EhkVjN7ByFEamC0cSEEUVbyXx45xYvctepkr9DoehvZKlRwbVS+t11Pp9yGpVY+QyKLTT/vJT9GeH/bG5D7NjJSHTe9/URApQJDvCy66+DbuYgdEG1kxBoWX+TGH6SeVJwaGypJTGlZuIbBPsoxQCbsVpz2mS6MlkNUxmuIMYnhvgR4Qnj5nSUjj6/KV7u83wUzUmInSA/wd5Gm+PYBkjhLszLY6ERJnx7QNr7DcV949ss5yl8iZraCMmDGNHbLg5AYF5uuD0WYFPbhXi+11blRRqfzwpKB9sjXD5mN4XauP8xiNjLvmK7yHeSSQtXD X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, 28 Feb 2023, Huang, Ying wrote: > Hugh Dickins writes: > > On Fri, 24 Feb 2023, Huang Ying wrote: > >> > >> diff --git a/mm/migrate.c b/mm/migrate.c > >> index 91198b487e49..c17ce5ee8d92 100644 > >> --- a/mm/migrate.c > >> +++ b/mm/migrate.c > >> @@ -1843,6 +1843,51 @@ static int migrate_pages_batch(struct list_head *from, new_page_t get_new_page, > >> return rc; > >> } > >> > >> +static int migrate_pages_sync(struct list_head *from, new_page_t get_new_page, > >> + free_page_t put_new_page, unsigned long private, > >> + enum migrate_mode mode, int reason, struct list_head *ret_folios, > >> + struct list_head *split_folios, struct migrate_pages_stats *stats) > >> +{ > >> + int rc, nr_failed = 0; > >> + LIST_HEAD(folios); > >> + struct migrate_pages_stats astats; > >> + > >> + memset(&astats, 0, sizeof(astats)); > >> + /* Try to migrate in batch with MIGRATE_ASYNC mode firstly */ > >> + rc = migrate_pages_batch(from, get_new_page, put_new_page, private, MIGRATE_ASYNC, > >> + reason, &folios, split_folios, &astats, > >> + NR_MAX_MIGRATE_PAGES_RETRY); > > > > I wonder if that and below would better be NR_MAX_MIGRATE_PAGES_RETRY / 2. > > > > Though I've never got down to adjusting that number (and it's not a job > > to be done in this set of patches), those 10 retries sometimes terrify > > me, from a latency point of view. They can have such different weights: > > in the unmapped case, 10 retries is okay; but when a pinned page is mapped > > into 1000 processes, the thought of all that unmapping and TLB flushing > > and remapping is terrifying. > > > > Since you're retrying below, halve both numbers of retries for now? > > Yes. These are reasonable concerns. > > And in the original implementation, we only wait to lock page and wait > the writeback to complete if pass > 2. This is kind of trying to > migrate asynchronously for 3 times before the real synchronous > migration. So, should we delete the "force" logic (in > migrate_folio_unmap()), and try to migrate asynchronously for 3 times in > batch before migrating synchronously for 7 times one by one? Oh, that's a good idea (but please don't imagine I've thought it through): I hadn't realized the way in which your migrate_pages_sync() addition is kind of duplicating the way that the "force" argument conditions behaviour, It would be very appealing to delete the "force" argument now if you can. But aside from that, you've also made me wonder (again, please remember I don't have a good picture of the new migrate_pages() sequence in my head) whether you have already made a *great* strike against my 10 retries terror. Am I reading it right, that the unmapping is now done on the first try, and the remove_migration_ptes after the last try (all the pages involved having remained locked throughout)? Hugh