From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86F0BC388F9 for ; Thu, 19 Nov 2020 08:02:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0608C2223F for ; Thu, 19 Nov 2020 08:02:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0608C2223F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 21E816B005C; Thu, 19 Nov 2020 03:02:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1CD7A6B005D; Thu, 19 Nov 2020 03:02:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 06EDC6B0068; Thu, 19 Nov 2020 03:02:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0054.hostedemail.com [216.40.44.54]) by kanga.kvack.org (Postfix) with ESMTP id CA3446B005C for ; Thu, 19 Nov 2020 03:02:57 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 710758249980 for ; Thu, 19 Nov 2020 08:02:57 +0000 (UTC) X-FDA: 77500426794.06.van92_0d0f98727341 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin06.hostedemail.com (Postfix) with ESMTP id 46B7C10049A4B for ; Thu, 19 Nov 2020 08:02:57 +0000 (UTC) X-HE-Tag: van92_0d0f98727341 X-Filterd-Recvd-Size: 5000 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Thu, 19 Nov 2020 08:02:55 +0000 (UTC) IronPort-SDR: hG5tuT8+vlGHXZ/dOJGcOBYQ/zACk+TOFogECByluhU0EWafHH/b/lyykAIF/t8y2vJCApZj8y oDPDZibXp4Uw== X-IronPort-AV: E=McAfee;i="6000,8403,9809"; a="159021618" X-IronPort-AV: E=Sophos;i="5.77,490,1596524400"; d="scan'208";a="159021618" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Nov 2020 00:02:53 -0800 IronPort-SDR: YZIJXDj6Oc005giTLdXw/6wZC7V4AznSc3jfbXisgAILn9ikzw9e1hZAqFL3LjOh1ug57IEMYl OCv9qRKSdiWQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,490,1596524400"; d="scan'208";a="359862232" Received: from yhuang-dev.sh.intel.com (HELO yhuang-dev) ([10.239.159.50]) by fmsmga004.fm.intel.com with ESMTP; 19 Nov 2020 00:02:49 -0800 From: "Huang\, Ying" To: Mel Gorman Cc: Peter Zijlstra , , , Andrew Morton , Ingo Molnar , Rik van Riel , "Johannes Weiner" , "Matthew Wilcox \(Oracle\)" , Dave Hansen , Andi Kleen , Michal Hocko , David Rientjes Subject: Re: [RFC -V5] autonuma: Migrate on fault among multiple bound nodes References: <20201118051952.39097-1-ying.huang@intel.com> <20201118095637.GC3306@suse.de> <878saxvpji.fsf@yhuang-dev.intel.com> <20201119075052.GF3306@suse.de> Date: Thu, 19 Nov 2020 16:02:48 +0800 In-Reply-To: <20201119075052.GF3306@suse.de> (Mel Gorman's message of "Thu, 19 Nov 2020 07:50:52 +0000") Message-ID: <874kllvknr.fsf@yhuang-dev.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Mel Gorman writes: > On Thu, Nov 19, 2020 at 02:17:21PM +0800, Huang, Ying wrote: >> >> Various page placement optimization based on the NUMA balancing can be >> >> done with these flags. As the first step, in this patch, if the >> >> memory of the application is bound to multiple nodes (MPOL_BIND), and >> >> in the hint page fault handler the accessing node are in the policy >> >> nodemask, the page will be tried to be migrated to the accessing node >> >> to reduce the cross-node accessing. >> >> >> > >> > The patch still lacks supporting data. It really should have a basic >> > benchmark of some sort serving as an example of how the policies should >> > be set and a before/after comparison showing the throughput of MPOL_BIND >> > accesses spanning 2 or more nodes is faster when numa balancing is enabled. >> >> Sure. Will add some basic benchmark data and usage example. >> > > Thanks > >> > A man page update should also be added clearly outlining when an >> > application should consider using it with the linux-api people cc'd >> > for review. >> >> Yes. Will Cc linux-api for review and will submit patches to >> manpages.git after the API is finalized. >> > > Add the manpages patch to this series. While it is not merged through > the kernel, it's important for review purposes. > >> > The main limitation is that if this requires application modification, >> > it may never be used. For example, if an application uses openmp places >> > that translates into bind then openmp needs knowledge of the flag. >> > Similar limitations apply to MPI. This feature has a risk that no one >> > uses it. >> >> My plan is to add a new option to `numactl` >> (https://github.com/numactl/numactl/), so users who want to enable NUMA >> balancing within the constrains of NUMA binding can use that. I can >> reach some Openstack and Kubernate developers to check whether it's >> possible to add the support to these software. For other applications, >> Yes, it may take long time for the new flag to be used. >> > > Patch for numactl should also be included to see what it looks like in > practice. Document what happens if the flag does not exist in the > running kernel. > > I know this is awkward, but it's an interface exposed to userspace and > as it is expected that applications will exist that then try run on > older kernels, it needs to be very up-front about what happens on older > kernels. It would not be a complete surprise for openmp and openmpi > packages to be updated on distributions with older kernels (either by > source or via packaging) leading to surprises. Sure. I understand that we should be careful about the user space interface. I will send out a new version together with the man pages and numactl patches with all your comments addressed. Best Regards, Huang, Ying