From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1DACC433EF for ; Fri, 22 Apr 2022 06:24:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 72D666B0072; Fri, 22 Apr 2022 02:24:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6DD636B0073; Fri, 22 Apr 2022 02:24:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 530186B0074; Fri, 22 Apr 2022 02:24:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id 3D1BA6B0072 for ; Fri, 22 Apr 2022 02:24:21 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 05BF82455B for ; Fri, 22 Apr 2022 06:24:21 +0000 (UTC) X-FDA: 79383525522.09.C4E0E92 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by imf12.hostedemail.com (Postfix) with ESMTP id 0DE324001B for ; Fri, 22 Apr 2022 06:24:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1650608660; x=1682144660; h=date:from:to:cc:subject:message-id:references: in-reply-to:mime-version; bh=a3vnLJKYviKPVU7ERlCQTS90byLzAhepdBVAHb+iu0M=; b=jYFXf86eNv9Cg4lFtHH4nGsd4elhAUecqaWlU55zuvZrg4bqiT2wTpU7 2Zob8A8vDyB9DwBh49QOewOHlGRjouc2+rhiSnIdGWBDfh/7GH9HgUdaU qnVWXZ/Vk1ahudv6DoXSkF/hZkmzEWmQW0yUidyyx7EzcwdmbOUI4r9Cm /C1hwilavzpR0VlfaMgI3rOaQYHEf6Pd+cYnjWrn8xwXJo6EoWeZ9LB0t gv74J3HQsGIcNdw+q+H6m7JetfSgc/lPIlB9z8CqYo2jBzyYhFKa2qbOs MQ1/gk2MSC0ILhkxuDbaTxS+zWMcYAOs37m8PiG+c4COGC491sgBw4YDY g==; X-IronPort-AV: E=McAfee;i="6400,9594,10324"; a="351029608" X-IronPort-AV: E=Sophos;i="5.90,280,1643702400"; d="scan'208";a="351029608" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Apr 2022 23:24:19 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,280,1643702400"; d="scan'208";a="562919494" Received: from fmsmsx604.amr.corp.intel.com ([10.18.126.84]) by fmsmga007.fm.intel.com with ESMTP; 21 Apr 2022 23:24:18 -0700 Received: from fmsmsx607.amr.corp.intel.com (10.18.126.87) by fmsmsx604.amr.corp.intel.com (10.18.126.84) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Thu, 21 Apr 2022 23:24:18 -0700 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx607.amr.corp.intel.com (10.18.126.87) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Thu, 21 Apr 2022 23:24:18 -0700 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (104.47.70.106) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Thu, 21 Apr 2022 23:24:18 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TrxkcVY45EvzfxPoPVOYtCSOOJTtOPElEKBn+ri5ZeuH9/fJiWUc+ALBZzNm3x9H4nhRqTrwZD/3OlSBunPnoL2wWwLhWuE07fikF5qazBPPJZcddPSfQt9wxK5UwdRnt5PoMxFEDrmj5xNVqHktVBLURVawh7R9Ta3IAFiFTimmXSH2pKliu67KoVXqbKZWHrmuNvKI+EzCZDosIDDWA2HPNPeG3ibYuNRdCeqRg99eTtpzvjG8/igx+JH2Lw+QRvKDOoHiT6xMHXts0bx4eT3W9Gt5ovtbm082DbMvlwwUeXlChU0b39alUHNHk92zY8SvOWeItLV0UZxX1+WYQw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pJT2ZVCo+0CtX6v5bpMJpjbIESB9q7GexBpSg44kMec=; b=BSI3RCXHhvEjfsrZGlMELANsMSMGAFeTWsImUX37xtA1sWpl/SgNYpgE60G4pBfB39YUEBlGuHl3tOdxq+/mLOlMgmzqFbbX9NcwhHjo5fYVWz4BURWSJtNJVmtX+1DVPm9c0A4Z4qQLJ6qM0XoYLusaozGhvWNaJoZgQyjgWUz3bIBFvfsXqo2DZAaoY+CTALLTlmXMCMHtXQhVwoxW+9zsjr74PrLv75QeXrUu3D4nsihUYA3o9FEyrDHFv8lA/nU2EMrNlbTQaj3wYt3Vd/hI+ffMAawnhWnzp7l9r/71DgbPHek5t63o6++N5lkNwVjS1v9DuRd9v4bNtF3rmg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from BYAPR11MB3062.namprd11.prod.outlook.com (2603:10b6:a03:92::18) by PH0PR11MB4855.namprd11.prod.outlook.com (2603:10b6:510:41::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.13; Fri, 22 Apr 2022 06:24:14 +0000 Received: from BYAPR11MB3062.namprd11.prod.outlook.com ([fe80::30f5:ac84:8cae:c977]) by BYAPR11MB3062.namprd11.prod.outlook.com ([fe80::30f5:ac84:8cae:c977%6]) with mapi id 15.20.5164.026; Fri, 22 Apr 2022 06:24:14 +0000 Date: Fri, 22 Apr 2022 14:24:05 +0800 From: Aaron Lu To: "ying.huang@intel.com" CC: Yang Shi , Michal Hocko , "Andrew Morton" , Linux MM , "Linux Kernel Mailing List" Subject: Re: [PATCH] mm: swap: determine swap device by using page nid Message-ID: References: <20220407020953.475626-1-shy828301@gmail.com> <6f7210be7353d1c01dc9f872b2692b83f87f5452.camel@intel.com> Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <6f7210be7353d1c01dc9f872b2692b83f87f5452.camel@intel.com> X-ClientProxiedBy: HK2PR0401CA0011.apcprd04.prod.outlook.com (2603:1096:202:2::21) To BYAPR11MB3062.namprd11.prod.outlook.com (2603:10b6:a03:92::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f931b2a6-3caa-490e-3606-08da2428b880 X-MS-TrafficTypeDiagnostic: PH0PR11MB4855:EE_ X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0JJQxSmH5R9t+F8QB2HpSsrmFHlgfCRnonAUf85BTn2q3MgINxtUTpb/8zWl2+tvFczg60m9pqdxfH0RaOV6zwdJjlU4+PJOPiitrBpXClYNMWG0AdZLx3Ti8YjVsjHOaeCQpJih2IvnMmkGExCi4wxvDRWnPeiHWE3NCHRSF6Fl68ze2WChGeQdjTu6eK9xsiRVAOwwdT72IrxcRjdKV+t8vyEz6RhS5J8v+ksrXKJR7K0lkcaW6KGwEMmTbnsbRdaWMBVz0fu3lE4OPzKIwbg2v6/YutZaW4lsrM/CIEMkPvO54L1YFrQcrhU+Mfp1RzXzjPE58hDzVY8Hehnr6aMuuj5bg8bpEZ4GvMYWFao4EBxNES461PiUt5yNkoFhMOhfLPaf4+dLohbXethH4XjWjAqA4wFtj+AsYytF3CBopqSdBUt0GFFLSzjTJn5chjd0IDFHkmabD14AamEoOHTB2vsfPYbX7tTrNSbd6tVRWJSO6uHUw288+/T35wTrzAmlcabGiAbtgEaF9x3QuTqNzAU7Ph+HhqlJ9uzhLs3lXX/6g+7GgtEManVDww4GP7MhTyhbXBW/SoLw0it+ypbSHn1xLcciyyN0+Zr4eh0x0eyQXB5vR1o1TKGyqx+d58rzVSjsoF29l9Z5TBFFVa7vWmC7iy4aJk+Yw40Ftz8frs9tQgWsgp7xnFa8hOKlSWrUCn3puoNSyfYq9kecDg2xENvtytOVFkkfgj8AZxQ= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR11MB3062.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(7916004)(366004)(66556008)(38100700002)(33716001)(5660300002)(66476007)(66946007)(8676002)(6862004)(316002)(4326008)(6666004)(8936002)(2906002)(186003)(966005)(6486002)(6636002)(6506007)(86362001)(508600001)(83380400001)(44832011)(26005)(9686003)(6512007)(82960400001)(54906003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?3aHfeClzLR347pvXlFfceg0Gd9Uk3QlrF05qC1Y3Pv/cA4UnoXqwwPrsvam8?= =?us-ascii?Q?vUTg1QQlP8zaNfpOioeX2Vu+z1bblCeSdRb4bMMexZDDoSpOBRwUTo64UVYU?= =?us-ascii?Q?t/2pwNLyAp9MCiE+dx3TQ2CZzbDA4sYzbAKMXfF73FHOaIm6yDSNqPHmHGvR?= =?us-ascii?Q?dudKFWrV/MspuuTx1M2z4YgCo9ItunwVfuA9Ey2G4+N9NbXtyfbrZS3X81qd?= =?us-ascii?Q?CpH1kHUVtQwaU+uGLNZhzbtOyD99D5DTOCQX+wny+9NUiuaFhXGCdNo506VA?= =?us-ascii?Q?mbuGYrz0Pmch05c1r7XNPrjrwKDoGmk+pAuLZOcRJAm5fv0MwS6iMARuEeut?= =?us-ascii?Q?QxERt/pplx9b1VDWf2XWuQ+GlNRro+UONpuSqkj0cnT+Jqh2pJse0e3eZ5Al?= =?us-ascii?Q?jLo0bdpxyoQ9QDoowwh29n0g6pDJGeMm0Sc5IC6EvI7h5Q6ozxaxEi3C9Nml?= =?us-ascii?Q?fu+UwnATU8uwWTE5N1k5+yVsrfVojkpUXSmEOx7p7KRtGQiSVxJqg+0IIjok?= =?us-ascii?Q?+ivn9H49jk8wSVCjtMgKh03+qio553AZWAGqwfIf66qz5539t63OsARSdrOf?= =?us-ascii?Q?BjY0RYQ3rdx6qqUN4RRrbnJX0sY+O53kwr6iNgzTigMe2uoTjn7lWP5JIkp9?= =?us-ascii?Q?x0s+b21xxOW01+YweT0wBKxhORU1ufiRz9xTO4fx9YnSSXPx27rZJgwnaQve?= =?us-ascii?Q?gze9SKnTJsedaXIAU3fioZQv1Ziga6aDxoYPn4r9BEAQM0NzOagW/vxvMrfS?= =?us-ascii?Q?VQhz00npC/5ZaVKtN5iD/VI8fhnYCbLnvlYKiRue+/9dZ6t92ZFzm9uLZEZR?= =?us-ascii?Q?nnyaC7ugs3XMz7k5+xw2WHXL6Ri2zqUoWKGzjM8s7hjtMjnw+EzXgaS6e/7w?= =?us-ascii?Q?Zg7OQhkDR1IE+5HduRdvRsTvlj+ui38zKI4/Dv3JsAGyVHdBYI8lDAKz3k4y?= =?us-ascii?Q?Rh543sxsKJjVMeMK7ysYWqNNkqWd8zvttsT4/3IWoLmoa2en7ara5zCPOc/a?= =?us-ascii?Q?4Pu7p7CWqNuc96QhKgGMgN07mHdtiNyodsVnqqsBNZjzvn3iRkNsLYVlCKFC?= =?us-ascii?Q?vOuJ99sBhyUsYjlGf+nv5zACOtzit5Htu43X+hyGsz9ez0nISV1GE3YyQZvg?= =?us-ascii?Q?xPBtWQYx0hVNy18+PnL+kB2WbVfqVO2lcp7eOX2y8yN7Br6D2RqvxvpRxxg4?= =?us-ascii?Q?IL9hi6h4xn1oSY79Svi+c4H+9CxpLZNL2AZ7xB3K7ahl25As5OWqqcmFL1Er?= =?us-ascii?Q?i5PfBxqJ54LtoJEipBLJXmEkT+qI6Mlt55xCWm6pKcqkZx89PGvvOGfRmonq?= =?us-ascii?Q?ldath/PqMQCZdAfIs7eaepahW0+mN9aqEK5yUfpU/od7wMEhsZ1DAyqOa2KM?= =?us-ascii?Q?D9OPzSv68AkVBOXnS0EYqb0xbgnktX1rT8MnH/0po8JAfWS8v7o2OtygD1WG?= =?us-ascii?Q?KoxGrZQGZ5ZbRpUU9jOQulq5aui0M/G5sVKWsgJleLWB1SAHGBFZd/f2fhg9?= =?us-ascii?Q?hgTJsbaY8mQ1YxyvWViS5JRU14VIUQ8jlF7UfQxDLlHql+n1/LzzHHFz6DFo?= =?us-ascii?Q?vHnXTzjI+sgPoVUktgU7xqAD6XuJTrsVfoDe9ILdeR1lowr8xv9Ic0Auda3+?= =?us-ascii?Q?bb1ek4qe11+mgAGGLmtIkgATIwS0YwSFridylhGG128CGgGUC0uMhq/TSFri?= =?us-ascii?Q?7Re80R2h0k2gQd9XLteqJUrNaxBPTTBjlmdYNT9LVZZJNEoCCcI2leHN91lk?= =?us-ascii?Q?HNNEQ28VZw=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: f931b2a6-3caa-490e-3606-08da2428b880 X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3062.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2022 06:24:14.2798 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: yfqDuCDL3gfS4JpWySXKXTQyVMG2oLgVadF73IkYwDltlOO/LaJVQcxi/fR/aNBhvVFWMsohdwJARIUZu7yf2g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB4855 X-OriginatorOrg: intel.com X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 0DE324001B X-Stat-Signature: z64n3syhzqi3r8st1e5qqk8yr3ayuroh Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=jYFXf86e; spf=none (imf12.hostedemail.com: domain of aaron.lu@intel.com has no SPF policy when checking 192.55.52.43) smtp.mailfrom=aaron.lu@intel.com; dmarc=pass (policy=none) header.from=intel.com X-Rspam-User: X-HE-Tag: 1650608655-451425 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Apr 21, 2022 at 04:34:09PM +0800, ying.huang@intel.com wrote: > On Thu, 2022-04-21 at 16:17 +0800, Aaron Lu wrote: > > On Thu, Apr 21, 2022 at 03:49:21PM +0800, ying.huang@intel.com wrote: ... ... > > > For swap-in latency, we can use pmbench, which can output latency > > > information. > > > > > > > OK, I'll give pmbench a run, thanks for the suggestion. > > Better to construct a senario with more swapin than swapout. For > example, start a memory eater, then kill it later. What about vm-scalability/case-swapin? https://git.kernel.org/pub/scm/linux/kernel/git/wfg/vm-scalability.git/tree/case-swapin I think you are pretty familiar with it but still: 1) it starts $nr_task processes and each mmaps $size/$nr_task area and then consumes the memory, after this, it waits for a signal; 2) start another process to consume $size memory to push the memory in step 1) to swap device; 3) kick processes in step 1) to start accessing their memory, thus trigger swapins. The metric of this testcase is the swapin throughput. I plan to restrict the cgroup's limit to $size. Considering there is only one NVMe drive attached to node 0, I will run the test as described before: 1) bind processes to run on node 0, allocate on node 1 to test the performance when reclaimer's node id is the same as swap device's. 2) bind processes to run on node 1, allocate on node 0 to test the performance when page's node id is the same as swap device's. Ying and Yang, Let me know what you think about the case used and the way the test is conducted.