From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62DC0C38A2B for ; Fri, 17 Apr 2020 14:46:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2A52920936 for ; Fri, 17 Apr 2020 14:46:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2A52920936 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BE0748E0026; Fri, 17 Apr 2020 10:46:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BB7C78E0023; Fri, 17 Apr 2020 10:46:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF5C38E0026; Fri, 17 Apr 2020 10:46:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0177.hostedemail.com [216.40.44.177]) by kanga.kvack.org (Postfix) with ESMTP id 974A48E0023 for ; Fri, 17 Apr 2020 10:46:32 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 63AE92485 for ; Fri, 17 Apr 2020 14:46:32 +0000 (UTC) X-FDA: 76717623024.07.songs87_733e60367971b X-HE-Tag: songs87_733e60367971b X-Filterd-Recvd-Size: 3866 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Fri, 17 Apr 2020 14:46:31 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 22DA4AB64; Fri, 17 Apr 2020 14:46:29 +0000 (UTC) Date: Fri, 17 Apr 2020 15:46:26 +0100 From: Mel Gorman To: SeongJae Park Cc: Mel Gorman , Peter Zijlstra , "Huang, Ying" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Ingo Molnar , Rik van Riel , Daniel Jordan , Tejun Heo , Dave Hansen , Tim Chen , Aubrey Li Subject: Re: Re: Re: Re: [RFC] autonuma: Support to scan page table asynchronously Message-ID: <20200417144626.GA3711@suse.de> References: <20200417121629.GA3758@techsingularity.net> <20200417124437.25445-1-sjpark@amazon.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline In-Reply-To: <20200417124437.25445-1-sjpark@amazon.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Apr 17, 2020 at 02:44:37PM +0200, SeongJae Park wrote: > On Fri, 17 Apr 2020 13:16:29 +0100 Mel Gorman wrote: > > > On Fri, Apr 17, 2020 at 12:21:29PM +0200, SeongJae Park wrote: > > > On Fri, 17 Apr 2020 12:04:17 +0200 Peter Zijlstra wrote: > > > > > > > On Fri, Apr 17, 2020 at 09:05:08AM +0200, SeongJae Park wrote: > > > > > I think the main idea of DAMON[1] might be able to applied here. Have you > > > > > considered it? > > > > > > > > > > [1] https://lore.kernel.org/linux-mm/20200406130938.14066-1-sjpark@amazon.com/ > > > > > > > > I've ignored that entire thing after you said the information it > > > > provides was already available through the PMU. > > > > > > Sorry if my answer made you confused. What I wanted to say was that the > > > fundamental access checking mechanism that DAMON depends on is PTE Accessed bit > > > for now, but it could be modified to use PMU or other features instead. > > > > I would not be inclined to lean towards either approach for NUMA > > balancing. Fiddling with the accessed bit can have consequences for page > > aging and residency -- fine for debugging a problem, not to fine for > > normal usage. I would expect the PMU approach would have high overhead > > as well as taking over a PMU counter that userspace debugging may expect > > to be available. > > Good point. But, isn't it ok to use Accessed bit as long as PG_Idle and > PG_Young is adjusted properly? Current DAMON implementation does so, as > idle_page_tracking also does. > PG_idle and PG_young are page flags that are used by idle tracking. The accessed bit I'm thinking of is in the PTE bit and it's the PTE that is changed by page_referenced() during reclaim. So it's not clear how adjusting the page bits help avoid page aging problems during reclaim. Maybe your suggestion was to move NUMA balancing to use the PG_idle and PG_young bits from idle tracking but I'm not sure what that gains us. This may be because I did not look too closely at DAMON as for debugging and analysis, the PMU sounded more suitable. It's not clear to me how looking at the page or pte bit handling of DAMON helps us reduce the scanning cost of numa balancing. There may be some benefit in splitting the address space and rescanning sections that were referenced but it has similar pitfalls to simply tracking the highest/lowest address that trapped a NUMA hinting fault. -- Mel Gorman SUSE Labs