From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A5AFC64EC7 for ; Tue, 28 Feb 2023 06:37:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 952C76B0078; Tue, 28 Feb 2023 01:37:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9015E6B007B; Tue, 28 Feb 2023 01:37:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A3356B007D; Tue, 28 Feb 2023 01:37:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 696E66B0078 for ; Tue, 28 Feb 2023 01:37:05 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 2D0D01A11EA for ; Tue, 28 Feb 2023 06:37:05 +0000 (UTC) X-FDA: 80515743210.09.5B605BB Received: from mail-qv1-f48.google.com (mail-qv1-f48.google.com [209.85.219.48]) by imf04.hostedemail.com (Postfix) with ESMTP id 5879240016 for ; Tue, 28 Feb 2023 06:37:03 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=LPQbwZTe; spf=pass (imf04.hostedemail.com: domain of hughd@google.com designates 209.85.219.48 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677566223; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UnSMc2eC7TT8GRaOBiducGgakN8v8m1KKOplmSAdLhE=; b=DlIOdxTiU3FlapYqjH4lnYrMVcuzPa7GDfVlhRvblAlKOUNkSl8XG8ZG7N/M/7N/Ova3gR 8GPdZs3yX3jbdel0KV9UB+R9YK4sOdBk2Hhe04SYdZnD/221Fw1bdX557uk1smuI3Z5zbw 6xAA0L7Q2t2kK5qy97BCnJ1vn8P8lQE= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=LPQbwZTe; spf=pass (imf04.hostedemail.com: domain of hughd@google.com designates 209.85.219.48 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677566223; a=rsa-sha256; cv=none; b=sPF8tV+461cypz2e5o50QkjlVt+YF6GNoXFOQtvW5RmS8JdWsQnZdvRN2j6J0wGaSTK2yY KiF7vovPbUApfVygZxMsif+PRL/+jkAGs5GfeCjWlbRRRkZOsRjK6vIS2eDvASSjMNKquj nOEPR4TgLASXaHHZ4gaSvr4AZ1tjRws= Received: by mail-qv1-f48.google.com with SMTP id f1so6155075qvx.13 for ; Mon, 27 Feb 2023 22:37:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1677566222; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=UnSMc2eC7TT8GRaOBiducGgakN8v8m1KKOplmSAdLhE=; b=LPQbwZTeR4hbSO7sOvb0sBRtbQOloGT3priC/yF1nfWcaDZ7Gn0LcaX1UcK1/7fu4j WTB9sLxP4oT1BSF6P7tVWlsyhC3foajVBh00efjVpBq46ybMNEFhL9gbeYvwbl9DvqEn 9Uj4CSv4cjSh1P30IC5Zv28/RsZQ6p4SYJB0W60Qw/tlmJGTHDriygdgMsvcZPi7RnPO 6dswOJ0YiOpgQ75mdCm940NT53/ctFW02t/CL59u/KypdjdywIXXlHU6JdW2iTNcdq2C cO3xiANcsrVaWArdsRhxz6JxDjNmwQ1XMQt0bHsMokiaalk/MjuLuzAYFX7qkYkqZASd f74Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677566222; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=UnSMc2eC7TT8GRaOBiducGgakN8v8m1KKOplmSAdLhE=; b=hFN5/JwhyQ3V7+MQfb9R499g0bGiVqcUCnpVN1Z7M9WXr3pcq9ZBUD7BrGuvbNot2n 6se59IgV4mG1O8Jilu9mNyOsZqWajb9yvXrXK7ADndN8adOjpIDcvg0QwqXWdnt6z38H YP5EihE/jIwnb28VEkPw+2kU8jMlbPBUkKaiCsLMTos2O7/zJZcEGq07kNkaw9VU8N6L sjZEao07i76bjPKyEIjH0feomF1uTcQrFpHo82DKa2agJm6QZefXT2frGpr8u1IePUoo ReZETdIGrWLK+ySxp6tD4KBl0FsBCpXDqcC6y2h89Vkb9ku1elZ8FO/eGsMy+1eLg+pJ y1XQ== X-Gm-Message-State: AO0yUKUUrIjNm3vyBvtWkaszKXAd4p3shoed63XMMOGWnUNFy+0EgN/a J/WZxmpE2LYQ7By4Sm+ZnjtPVQ== X-Google-Smtp-Source: AK7set+MKUxY0/5FkPLn5n3grygqwxv6M5lR7jdEadEZQK6/eX+4HhqaOK6x7xm0KM9Nu4SpuNVbHg== X-Received: by 2002:a05:6214:2582:b0:56f:c948:411b with SMTP id fq2-20020a056214258200b0056fc948411bmr3683651qvb.29.1677566222383; Mon, 27 Feb 2023 22:37:02 -0800 (PST) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id a190-20020a3798c7000000b00742aff9574asm3483768qke.65.2023.02.27.22.37.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Feb 2023 22:37:01 -0800 (PST) Date: Mon, 27 Feb 2023 22:36:59 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@ripple.attlocal.net To: Huang Ying cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Hugh Dickins , "Xu, Pengfei" , Christoph Hellwig , Stefan Roesch , Tejun Heo , Xin Hao , Zi Yan , Yang Shi , Baolin Wang , Matthew Wilcox , Mike Kravetz Subject: Re: [PATCH 3/3] migrate_pages: try migrate in batch asynchronously firstly In-Reply-To: <20230224141145.96814-4-ying.huang@intel.com> Message-ID: References: <20230224141145.96814-1-ying.huang@intel.com> <20230224141145.96814-4-ying.huang@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 5879240016 X-Stat-Signature: i3b38abojk7c7oqoqhutretiiftyy7nb X-HE-Tag: 1677566223-47161 X-HE-Meta: U2FsdGVkX19lph9XUbkqk4umrX/SKfrd6eII/H+2bxwgSxNnC1Va0NNtC52SgZemDRYFXj932ZjuGnuqyfoaEmRAmnHgz2I1e2VXjjJU2Ns32Y2KB9gjFPPcUq6gXk0bh4Qx7VDEOZURAORaRSrxsMgmdgmpIk2Q0BgEaprGjMp4qu48sMOrzZ/z7xwZaCAK0Vjrc/gmeomkZED0YeKs7pG93D0+5S00FgPxdCuG9sGsDxYLPRQopwJSAu0vbYVwWKctUPLb75fDJ2BXTSWVPoGP+udh04vNK5DoiDadJEhVckZeV+ZWqqEmduRzVvSG1StLW0xEzqEtT9TMRPFcDJDnMJaUCmYUKDxifWY9b0pbe6cnY2kXN/PbauYYu0Cmdc/zJepbDmIRLMxb59A6YOiq2vGvAaVlyro/CFfhoWXJGAapanjAs4hAb0n/bPCeW9pt+iVnX+GYQ/6DjbxVb5Z2bHnkmcQK/RqXPTzuSxKStuWn/KR5PZfbcQ+6u7tap8GqwCurYNHVpASb7ZcIn3wYKIpurljh+aB9X+aj9YfoX2VzzVa1Ld+2yhFcw4AU2j4z2OQfK0QHg+ai9ZGzzmCi6Q8DAukVL4+8pT5kAMSG4s4aZdhkRhF2H11O30prQmIr7ch7rmFVBfdwVccTAFUxt+1XWk2zc39MDIbUEKlWL9RWz+urvEkUydRAIK49Lp74FDli/efW+wKnVTHhVrHwP4Y5RF2eIzKA13FFxtkAjVJEXRKnoOz0BjpuvLoGhKziR7RF4kRXn909fIldPoO4EJKlpcmHRj5xbB61ty5svX6g2/0D9feoOKkNuiuFk5d9Uedw8I0z/y/GG7ZJ+wlFpDgAuNITc79/H3FyQ/ygZGakP7dfF6lpAm1L4kYbqp4EUwwHvA3/Rq2m44u8QAu7cMzeF7fkMSnKsdSvnfTkDd7qtWWTt1sGPM8xQQVflGUqG8Sp9x5mh9m29i1 fS3EJSmf +lBERhBisTwJLnniH8q2th7Pk04sl2Glwhj03IbA8KjjQgDA4Cs7B9ITYbW413j86ZxgdWXnQ/wx13tp0gU2P1ioa2zUhdsnEwOrLQ36A8eetC3NvspkBXHQbOuRsnuff9lce3N1bWO+VYkcnqIV4JEu5bOhA4aeYp5XH84TfL1SVF4cNICU8FzT1tik8/k9Y78J9sTno2WxC3/w/nHtkQzq5q+SPdx6fpw/pPTqZDvfirbb1e0LM0NtI5zaO3rSdYnO7LmmC9opF/5vRf2ryYOvjIoasZYOEPfcU8HP0HFk0CU+mulSY5fXaBO33WVirHBt95suD7D++e0Fj+HBWjgUbUmmU2h98eQCnlXlot47XRghzmPXaj+lkHaHk9y3Ti5gQ6ij9129bqlkV5Sz7JWkKcEE+t8fZDL4ovgppcKDU0C7V97PzG6oWcyfzRcmqNMu6coBKoBkzfmtsu98rb3Je2Ulb5eZTA74qYH5tYxQJ031neXivqzoqQ3nubv/gNkT9w+JoWkvgzX2U3Z+cIhYLGD7qOjudBXsMDSlAs1YEnfUEoz9ROwLfUgA2qUF7Ep2nZeZmvCvMnOGjL29ExZppt963YdS0GBhKFCPlhc0v9LcVUbtIp1N2GnNR0UT+3zXDqk56TSYizzKOeQwUTzLY0rDR3V57l5nK8SErrSHvrG5WhGfGIjTOPA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, 24 Feb 2023, Huang Ying wrote: > When we have locked more than one folios, we cannot wait the lock or > bit (e.g., page lock, buffer head lock, writeback bit) synchronously. > Otherwise deadlock may be triggered. This make it hard to batch the > synchronous migration directly. > > This patch re-enables batching synchronous migration via trying to > migrate in batch asynchronously firstly. And any folios that are > failed to be migrated asynchronously will be migrated synchronously > one by one. > > Test shows that this can restore the TLB flushing batching performance > for synchronous migration effectively. > > Signed-off-by: "Huang, Ying" > Cc: Hugh Dickins I'm not sure whether my 48 hours on two machines counts for a Tested-by: Hugh Dickins or not; but it certainly looks like you've fixed my deadlock. > Cc: "Xu, Pengfei" > Cc: Christoph Hellwig > Cc: Stefan Roesch > Cc: Tejun Heo > Cc: Xin Hao > Cc: Zi Yan > Cc: Yang Shi > Cc: Baolin Wang > Cc: Matthew Wilcox > Cc: Mike Kravetz > --- > mm/migrate.c | 65 ++++++++++++++++++++++++++++++++++++++++++++-------- > 1 file changed, 55 insertions(+), 10 deletions(-) I was initially disappointed, that this was more complicated than I had thought it should be; but came to understand why. My "change the mode to MIGRATE_ASYNC after the first" model would have condemned most of the MIGRATE_SYNC batch of pages to be handled as lightly as MIGRATE_ASYNC: not good enough, you're right be trying harder here. > > diff --git a/mm/migrate.c b/mm/migrate.c > index 91198b487e49..c17ce5ee8d92 100644 > --- a/mm/migrate.c > +++ b/mm/migrate.c > @@ -1843,6 +1843,51 @@ static int migrate_pages_batch(struct list_head *from, new_page_t get_new_page, > return rc; > } > > +static int migrate_pages_sync(struct list_head *from, new_page_t get_new_page, > + free_page_t put_new_page, unsigned long private, > + enum migrate_mode mode, int reason, struct list_head *ret_folios, > + struct list_head *split_folios, struct migrate_pages_stats *stats) > +{ > + int rc, nr_failed = 0; > + LIST_HEAD(folios); > + struct migrate_pages_stats astats; > + > + memset(&astats, 0, sizeof(astats)); > + /* Try to migrate in batch with MIGRATE_ASYNC mode firstly */ > + rc = migrate_pages_batch(from, get_new_page, put_new_page, private, MIGRATE_ASYNC, > + reason, &folios, split_folios, &astats, > + NR_MAX_MIGRATE_PAGES_RETRY); I wonder if that and below would better be NR_MAX_MIGRATE_PAGES_RETRY / 2. Though I've never got down to adjusting that number (and it's not a job to be done in this set of patches), those 10 retries sometimes terrify me, from a latency point of view. They can have such different weights: in the unmapped case, 10 retries is okay; but when a pinned page is mapped into 1000 processes, the thought of all that unmapping and TLB flushing and remapping is terrifying. Since you're retrying below, halve both numbers of retries for now? > + stats->nr_succeeded += astats.nr_succeeded; > + stats->nr_thp_succeeded += astats.nr_thp_succeeded; > + stats->nr_thp_split += astats.nr_thp_split; > + if (rc < 0) { > + stats->nr_failed_pages += astats.nr_failed_pages; > + stats->nr_thp_failed += astats.nr_thp_failed; > + list_splice_tail(&folios, ret_folios); > + return rc; > + } > + stats->nr_thp_failed += astats.nr_thp_split; > + nr_failed += astats.nr_thp_split; > + /* > + * Fall back to migrate all failed folios one by one synchronously. All > + * failed folios except split THPs will be retried, so their failure > + * isn't counted > + */ > + list_splice_tail_init(&folios, from); > + while (!list_empty(from)) { > + list_move(from->next, &folios); > + rc = migrate_pages_batch(&folios, get_new_page, put_new_page, > + private, mode, reason, ret_folios, > + split_folios, stats, NR_MAX_MIGRATE_PAGES_RETRY); NR_MAX_MIGRATE_PAGES_RETRY / 2 ? > + list_splice_tail_init(&folios, ret_folios); > + if (rc < 0) > + return rc; > + nr_failed += rc; > + } > + > + return nr_failed; > +} > + > /* > * migrate_pages - migrate the folios specified in a list, to the free folios > * supplied as the target for the page migration > @@ -1874,7 +1919,7 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page, > enum migrate_mode mode, int reason, unsigned int *ret_succeeded) > { > int rc, rc_gather; > - int nr_pages, batch; > + int nr_pages; > struct folio *folio, *folio2; > LIST_HEAD(folios); > LIST_HEAD(ret_folios); > @@ -1890,10 +1935,6 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page, > if (rc_gather < 0) > goto out; > > - if (mode == MIGRATE_ASYNC) > - batch = NR_MAX_BATCHED_MIGRATION; > - else > - batch = 1; > again: > nr_pages = 0; > list_for_each_entry_safe(folio, folio2, from, lru) { > @@ -1904,16 +1945,20 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page, > } > > nr_pages += folio_nr_pages(folio); > - if (nr_pages >= batch) > + if (nr_pages >= NR_MAX_BATCHED_MIGRATION) > break; > } > - if (nr_pages >= batch) > + if (nr_pages >= NR_MAX_BATCHED_MIGRATION) > list_cut_before(&folios, from, &folio2->lru); > else > list_splice_init(from, &folios); > - rc = migrate_pages_batch(&folios, get_new_page, put_new_page, private, > - mode, reason, &ret_folios, &split_folios, &stats, > - NR_MAX_MIGRATE_PAGES_RETRY); > + if (mode == MIGRATE_ASYNC) > + rc = migrate_pages_batch(&folios, get_new_page, put_new_page, private, > + mode, reason, &ret_folios, &split_folios, &stats, > + NR_MAX_MIGRATE_PAGES_RETRY); > + else > + rc = migrate_pages_sync(&folios, get_new_page, put_new_page, private, > + mode, reason, &ret_folios, &split_folios, &stats); > list_splice_tail_init(&folios, &ret_folios); > if (rc < 0) { > rc_gather = rc; > -- > 2.39.1