From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6F93C433E0 for ; Tue, 2 Feb 2021 18:23:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1771664F68 for ; Tue, 2 Feb 2021 18:23:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1771664F68 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 798C66B0005; Tue, 2 Feb 2021 13:23:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7489F6B0070; Tue, 2 Feb 2021 13:23:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 60EB56B0071; Tue, 2 Feb 2021 13:23:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0082.hostedemail.com [216.40.44.82]) by kanga.kvack.org (Postfix) with ESMTP id 4533D6B0005 for ; Tue, 2 Feb 2021 13:23:08 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 08F39181AEF1A for ; Tue, 2 Feb 2021 18:23:08 +0000 (UTC) X-FDA: 77774149656.19.toe46_0e11df9275cd Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id AD0611AD1B7 for ; Tue, 2 Feb 2021 18:23:07 +0000 (UTC) X-HE-Tag: toe46_0e11df9275cd X-Filterd-Recvd-Size: 12296 Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Tue, 2 Feb 2021 18:23:07 +0000 (UTC) Received: by mail-ed1-f43.google.com with SMTP id y18so6704075edw.13 for ; Tue, 02 Feb 2021 10:23:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=c22PgLZFbY4qGktGzr/nGZ2GejV48wZf+CIJ8/Deskw=; b=dQg2l4G2jTK1mnKF3sFcBvXqkAF1gn/ndYtTfSeGp/TtMpLQ67Az/CMzydUs34TNOn wcPnbzTneNDYCuQ0KXTLPamuGebKbZSVWRAQbT1sGJLq1ahfBmYxaQx6MOEjhX0XcvZT hEMwrspFgxrsIOAhnZfqYYuT3zR5jq2EdmnpSTkb+uN34u786YGRDQbCCji/46qki+0w aG6SGeDoyEKJO+IoniRani9rU5TdEY2YKuC1pnpUmokcoOoxwMZ56v1mJ/ZdlYOmeGfd wNke2YrbjDXcOj87JleGK5ieP24QnVt9YoAFsPNJG/2+ksv8uytOzHZ3VvEGlJ3y7Cx7 +cYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=c22PgLZFbY4qGktGzr/nGZ2GejV48wZf+CIJ8/Deskw=; b=QX9r0j1mxkpmQOvcJmAZoFsdkXzc1kpV1kbf+CIoVSTuZ8c+RPIom9ZUPBJ2+2aQ4o TvKy7EUoiqu9f2y3F0beRofsNy3KqmoGhxrsv7mTjBvoNVjdAS3IBcepHEqaauOUsas/ MXgV2++ahD07Dz91s/up/kyn5pgKxxGlyPs/4YB385VPcEJ1w4Ak2lyGLC3EqhSDOK3T 8Ao9lrZewM+BT3a6WFjktE1sX+Rgbf7rf/1n8xBrIkPZB0RVytd0BkPZmZjrK+4AlAuG g4iCKXRhvNID+n2+u0f0p4ufTQVY6AZSliHvCcQAIp70iCLDh9Wl+Mtae4rG4/ztjsHw es6A== X-Gm-Message-State: AOAM530dCDZ3RvvR/4ka1ceD+3hZ5AGY7I0dS4diI1qbwHM0PJtQ9YSo Vmy3aM/vskAGe0XmXhL6NVOiKIYJd+MxbPgfu2Q= X-Google-Smtp-Source: ABdhPJzMSvtPevXyhMdbLX7xDi3yA7nnJrvoElRq08dU5LagKHZuBbxyan+G9Uq53Lb5GcqvL/+FvioVioB/0PNMDsQ= X-Received: by 2002:aa7:c7c8:: with SMTP id o8mr226408eds.137.1612290185867; Tue, 02 Feb 2021 10:23:05 -0800 (PST) MIME-Version: 1.0 References: <20210126003411.2AC51464@viggo.jf.intel.com> <20210126003427.73DFDD34@viggo.jf.intel.com> In-Reply-To: <20210126003427.73DFDD34@viggo.jf.intel.com> From: Yang Shi Date: Tue, 2 Feb 2021 10:22:54 -0800 Message-ID: Subject: Re: [RFC][PATCH 08/13] mm/migrate: demote pages during reclaim To: Dave Hansen Cc: Linux Kernel Mailing List , Linux MM , Yang Shi , David Rientjes , Huang Ying , Dan Williams , Oscar Salvador Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Jan 25, 2021 at 4:41 PM Dave Hansen wrote: > > > From: Dave Hansen > > This is mostly derived from a patch from Yang Shi: > > https://lore.kernel.org/linux-mm/1560468577-101178-10-git-send-email-yang.shi@linux.alibaba.com/ > > Add code to the reclaim path (shrink_page_list()) to "demote" data > to another NUMA node instead of discarding the data. This always > avoids the cost of I/O needed to read the page back in and sometimes > avoids the writeout cost when the pagee is dirty. > > A second pass through shrink_page_list() will be made if any demotions > fail. This essentally falls back to normal reclaim behavior in the > case that demotions fail. Previous versions of this patch may have > simply failed to reclaim pages which were eligible for demotion but > were unable to be demoted in practice. > > Note: This just adds the start of infratructure for migration. It is > actually disabled next to the FIXME in migrate_demote_page_ok(). > > Signed-off-by: Dave Hansen > Cc: Yang Shi > Cc: David Rientjes > Cc: Huang Ying > Cc: Dan Williams > Cc: osalvador > > -- > > changes from 202010: > * add MR_NUMA_MISPLACED to trace MIGRATE_REASON define > * make migrate_demote_page_ok() static, remove 'sc' arg until > later patch > * remove unnecessary alloc_demote_page() hugetlb warning > * Simplify alloc_demote_page() gfp mask. Depend on > __GFP_NORETRY to make it lightweight instead of fancier > stuff like leaving out __GFP_IO/FS. > * Allocate migration page with alloc_migration_target() > instead of allocating directly. > changes from 20200730: > * Add another pass through shrink_page_list() when demotion > fails. > --- > > b/include/linux/migrate.h | 9 ++++ > b/include/trace/events/migrate.h | 3 - > b/mm/vmscan.c | 81 +++++++++++++++++++++++++++++++++++++++ > 3 files changed, 92 insertions(+), 1 deletion(-) > > diff -puN include/linux/migrate.h~demote-with-migrate_pages include/linux/migrate.h > --- a/include/linux/migrate.h~demote-with-migrate_pages 2021-01-25 16:23:14.591866696 -0800 > +++ b/include/linux/migrate.h 2021-01-25 16:23:14.599866696 -0800 > @@ -27,6 +27,7 @@ enum migrate_reason { > MR_MEMPOLICY_MBIND, > MR_NUMA_MISPLACED, > MR_CONTIG_RANGE, > + MR_DEMOTION, > MR_TYPES > }; > > @@ -196,6 +197,14 @@ struct migrate_vma { > int migrate_vma_setup(struct migrate_vma *args); > void migrate_vma_pages(struct migrate_vma *migrate); > void migrate_vma_finalize(struct migrate_vma *migrate); > +int next_demotion_node(int node); > + > +#else /* CONFIG_MIGRATION disabled: */ > + > +static inline int next_demotion_node(int node) > +{ > + return NUMA_NO_NODE; > +} > > #endif /* CONFIG_MIGRATION */ > > diff -puN include/trace/events/migrate.h~demote-with-migrate_pages include/trace/events/migrate.h > --- a/include/trace/events/migrate.h~demote-with-migrate_pages 2021-01-25 16:23:14.593866696 -0800 > +++ b/include/trace/events/migrate.h 2021-01-25 16:23:14.599866696 -0800 > @@ -20,7 +20,8 @@ > EM( MR_SYSCALL, "syscall_or_cpuset") \ > EM( MR_MEMPOLICY_MBIND, "mempolicy_mbind") \ > EM( MR_NUMA_MISPLACED, "numa_misplaced") \ > - EMe(MR_CONTIG_RANGE, "contig_range") > + EM( MR_CONTIG_RANGE, "contig_range") \ > + EMe(MR_DEMOTION, "demotion") > > /* > * First define the enums in the above macros to be exported to userspace > diff -puN mm/vmscan.c~demote-with-migrate_pages mm/vmscan.c > --- a/mm/vmscan.c~demote-with-migrate_pages 2021-01-25 16:23:14.595866696 -0800 > +++ b/mm/vmscan.c 2021-01-25 16:23:14.601866696 -0800 > @@ -43,6 +43,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -1036,6 +1037,24 @@ static enum page_references page_check_r > return PAGEREF_RECLAIM; > } > > +static bool migrate_demote_page_ok(struct page *page) > +{ > + int next_nid = next_demotion_node(page_to_nid(page)); > + > + VM_BUG_ON_PAGE(!PageLocked(page), page); > + VM_BUG_ON_PAGE(PageHuge(page), page); > + VM_BUG_ON_PAGE(PageLRU(page), page); > + > + if (next_nid == NUMA_NO_NODE) > + return false; > + if (PageTransHuge(page) && !thp_migration_supported()) > + return false; > + > + // FIXME: actually enable this later in the series > + return false; > +} > + > + > /* Check if a page is dirty or under writeback */ > static void page_check_dirty_writeback(struct page *page, > bool *dirty, bool *writeback) > @@ -1066,6 +1085,44 @@ static void page_check_dirty_writeback(s > mapping->a_ops->is_dirty_writeback(page, dirty, writeback); > } > > +static struct page *alloc_demote_page(struct page *page, unsigned long node) > +{ > + struct migration_target_control mtc = { > + /* > + * Fail quickly and quietly. Page will likely > + * just be discarded instead of migrated. > + */ > + .gfp_mask = GFP_HIGHUSER | __GFP_NORETRY | __GFP_NOWARN, > + .nid = node > + }; > + > + return alloc_migration_target(page, (unsigned long)&mtc); Other than the gfp flag question raised by Oscar, I'm wondering how we guarantee the demotion allocation happens on the designated node. In the previous version __GFP_THISNODE is set to guarantee this. In this version you switched to use alloc_migration_target() API but without having nodemask or __GFP_THISNODE. If nodemask is NULL the allocation may fall back to an unexpected node. And GFP_HIGHUSER does respect cpuset, so if the demotion target node is excluded by the cpuset which the task belongs to, the migration would fail. This might be a way to respect cpuset, but it should just work for direct reclaimer. So, is this change really expected? > +} > + > +/* > + * Take pages on @demote_list and attempt to demote them to > + * another node. Pages which are not demoted are left on > + * @demote_pages. > + */ > +static unsigned int demote_page_list(struct list_head *demote_pages, > + struct pglist_data *pgdat, > + struct scan_control *sc) > +{ > + int target_nid = next_demotion_node(pgdat->node_id); > + unsigned int nr_succeeded = 0; > + int err; > + > + if (list_empty(demote_pages)) > + return 0; > + > + /* Demotion ignores all cpuset and mempolicy settings */ > + err = migrate_pages(demote_pages, alloc_demote_page, NULL, > + target_nid, MIGRATE_ASYNC, MR_DEMOTION, > + &nr_succeeded); > + > + return nr_succeeded; > +} > + > /* > * shrink_page_list() returns the number of reclaimed pages > */ > @@ -1078,12 +1135,15 @@ static unsigned int shrink_page_list(str > { > LIST_HEAD(ret_pages); > LIST_HEAD(free_pages); > + LIST_HEAD(demote_pages); > unsigned int nr_reclaimed = 0; > unsigned int pgactivate = 0; > + bool do_demote_pass = true; > > memset(stat, 0, sizeof(*stat)); > cond_resched(); > > +retry: > while (!list_empty(page_list)) { > struct address_space *mapping; > struct page *page; > @@ -1233,6 +1293,16 @@ static unsigned int shrink_page_list(str > } > > /* > + * Before reclaiming the page, try to relocate > + * its contents to another node. > + */ > + if (do_demote_pass && migrate_demote_page_ok(page)) { > + list_add(&page->lru, &demote_pages); > + unlock_page(page); > + continue; > + } > + > + /* > * Anonymous process memory has backing store? > * Try to allocate it some swap space here. > * Lazyfree page could be freed directly > @@ -1479,6 +1549,17 @@ keep: > list_add(&page->lru, &ret_pages); > VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page), page); > } > + /* 'page_list' is always empty here */ > + > + /* Migrate pages selected for demotion */ > + nr_reclaimed += demote_page_list(&demote_pages, pgdat, sc); > + /* Pages that could not be demoted are still in @demote_pages */ > + if (!list_empty(&demote_pages)) { > + /* Pages which failed to demoted go back on on @page_list for retry: */ > + list_splice_init(&demote_pages, page_list); > + do_demote_pass = false; > + goto retry; > + } > > pgactivate = stat->nr_activate[0] + stat->nr_activate[1]; > > _ >