From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8CC91E6895E for ; Thu, 31 Oct 2024 08:43:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F09116B008C; Thu, 31 Oct 2024 04:43:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EBA1B6B0092; Thu, 31 Oct 2024 04:43:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D7FF26B0093; Thu, 31 Oct 2024 04:43:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id B9DAF6B008C for ; Thu, 31 Oct 2024 04:43:41 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id EBEB3161324 for ; Thu, 31 Oct 2024 08:43:40 +0000 (UTC) X-FDA: 82733258748.29.8CCC549 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) by imf15.hostedemail.com (Postfix) with ESMTP id 5E83EA0036 for ; Thu, 31 Oct 2024 08:43:10 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=fzIyfkdR; spf=pass (imf15.hostedemail.com: domain of ying.huang@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1730364058; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gqD+sZBhhkxPhnEFtkY3FZmSoaszE7umW3pu9/Vo7wg=; b=bc9RhwCbwEExMbrcVViuNqcPFvFNt8WqFDnTG+AXbX/tcdt9cSk3IuIX0WomZE95BQh0XM zj5oqQvRgwtRP3AbWAcx2B/NCX4ApzNgGdcNd9UAhB7MlhCWbf4/YvnvTsoLaRyunuWCOt GGkOpNF4UT6RuNh9A7CxuVSr4FUwMjU= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=fzIyfkdR; spf=pass (imf15.hostedemail.com: domain of ying.huang@intel.com designates 198.175.65.20 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1730364058; a=rsa-sha256; cv=none; b=dKOfqGUg1mROZkTr3reEoBiGVfsSnPBLAk3yBZanZW38l5MQsiCXC2mz4nSAAV6XC9xnXA W2C3BHBQ6PuZ9GyPN5/bzYIt/XIgygsdl8nnqJwvQMAdg0E94CzoY+fHRGVk631zinbhbM 7yqX2X9JWdAvs082v9gmfBwnyZFDmjk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1730364218; x=1761900218; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version; bh=KTUO/7zAI8tma9a81CkuviKFc2jVZIAkxCQibDW7ejk=; b=fzIyfkdR1tNqL+z7rsy/oe6I5g4MFxDrZY+ldEMr/2pul2PKApWtsQ0G 1RVumirVD8N6OPa5OUJxS9X1+XY67fqJmEZrkZlX1AueN1MbNbpQJu5Hb MpV0F9h/GWYOWOEFFJaTtzsaqVA0soEfooq4K+wTv6w9JdCF33AkIn88K WvOVnhvL0GJbQyydD5bZ6Ccl6gQ3DKFMwcwuig5jWZc0DZgcdSkfPkLy3 noFB2VsrV/Gnmgb7MkZaeQZAfTL4aJjVj9xMGRVOy/UXvjoADA5p/c0vo RyiakyC5vrl4RTMCmVHMtvHCnHdutrt4SlnjV55icWCpkxuevxIwcNieQ A==; X-CSE-ConnectionGUID: lgGKhgeKQIu4xH6mcfKcrg== X-CSE-MsgGUID: 7y4pVlLaTbuXWwd1NkmNhA== X-IronPort-AV: E=McAfee;i="6700,10204,11222"; a="29854897" X-IronPort-AV: E=Sophos;i="6.11,199,1725346800"; d="scan'208";a="29854897" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2024 01:43:27 -0700 X-CSE-ConnectionGUID: btejkn8WQY6gyaxdIKBqSw== X-CSE-MsgGUID: xoxDCbL5Qv+8ZVNiFtOD2g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,247,1725346800"; d="scan'208";a="82217996" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Oct 2024 01:43:25 -0700 From: "Huang, Ying" To: Kefeng Wang Cc: David Hildenbrand , Andrew Morton , Matthew Wilcox , "Muchun Song" , , Zi Yan Subject: Re: [PATCH v2 1/2] mm: use aligned address in clear_gigantic_page() In-Reply-To: <64f1c69d-3706-41c5-a29f-929413e3dfa2@huawei.com> (Kefeng Wang's message of "Wed, 30 Oct 2024 13:05:33 +0800") References: <20241026054307.3896926-1-wangkefeng.wang@huawei.com> <54f5f3ee-8442-4c49-ab4e-c46e8db73576@huawei.com> <4219a788-52ad-4d80-82e6-35a64c980d50@redhat.com> <127d4a00-29cc-4b45-aa96-eea4e0adaed2@huawei.com> <9b06805b-4f4f-4b37-861f-681e3ab9d470@huawei.com> <113d3cb9-0391-48ab-9389-f2fd1773ab73@redhat.com> <878qu6wgcm.fsf@yhuang6-desk2.ccr.corp.intel.com> <87sese9sy9.fsf@yhuang6-desk2.ccr.corp.intel.com> <64f1c69d-3706-41c5-a29f-929413e3dfa2@huawei.com> Date: Thu, 31 Oct 2024 16:39:53 +0800 Message-ID: <87v7x88y3q.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 5E83EA0036 X-Stat-Signature: gkeuj3c4fpwzrcb1dxmftjgc7dycekqf X-Rspam-User: X-HE-Tag: 1730364190-309729 X-HE-Meta: U2FsdGVkX18g7qIvQby9e5lXayPX3s3LM/rzBwZ4IjRah7HAnGGtYbCX4BeNAGlB+2PWd09AfaUbeS/2hfWaCfgUc/9bC18S7I0SeFIvn3bRWhh+3Tdm5DJwObF6Mc2U303lpXI09+OlMCIoiIlpIqXyJlfTrrBojXK1OfnbOx/uIeWRmViPVhcppOV1zIdroxUNpkl8gMvWkZldR/eqg4+Kbvsp5NE2EYdJ8Q+KmHQ1776dYbaxp7uzVoaH/VFSG8f6S+QrY/lgHQuXhNww3oECFFiylgAcHCHBXa3GDoLO+l09aQnbZsxOiXcro3WwQInQqJLi/tVMEu7Attl1nfj6k8v2BuAgRV1HRUhrRUCjAeiag6Cpu6L6iRxRJAZO2MTpyP3pGF0W8kqujj8kALBklkQJsH+5nnfoWx6Y9JN0LEVFDN/nQvcP0kId9e0vhJke6mGhIyENMTi16RC1FGklwubE3QdOGuNZ7ZhI+wSKvLsJw/REfluNw8+jurOts193PQFsR2P9/BHwt0qg2JFgsg70NW8QFFIL1F73COpP5iwCXef6RphOe7a66QXmqWY9dzjY9JfuUuO5hxg8gi7pXKFWFEVvGiEqkLvmaZmEgCpAgFFwRaYj6ZzFfROT4R61rKbBzOUSydzhSFkAZmMa6eHJV7PJHO0CLtVsvklwfVLlKyEKpS63xQ7tgaP17VEEuAnjcjnJfrP0Efyi0Ge3vf5PK7d6Hxtd9K3+wCDyNb/qjdyMjyfbDoqHPFwpnRR0xp+xc+lu2kZBla9GccVg2/z7DUaOpRN027onbtqNRwBVnLek97w3HHW7dxE+x3eE+T29hZ6aTiWxKf3NI0z/IcJf5UIRNlAqJJ9C5DH5C8GnEwKGFdsyzoBobwfkcwZQQ1QqJwPGo3mlvQ/eMvV2nvs/zwJG0XW+DDl3PXXIdz+jLIdf2AKVXK2c63v8ePS4xVROd/vy4qj/hdr qaGArTxf 2h1M3VAmu5V9ig7b7rFKoSxvUWFUHGb9Hrhxq3iUvjtbAPtcsoIlyvcTSQfB1Rcyo1kuY/6yIq/0IBtVGThdg+7C/6WE3tgaV/96tOJEXxF3J4ha0pZIyBnXgAFB7HbGnjQobqRrPxQpLMKQQ3j4GfVbXKwn4f7hTy+BePTcZ021t1RPEjyqxa97PjF0VNuHEAnZ6HQky7v74E1xJ0HTDEjBNM3TVrOiQYJy5LXmtkbMWNsDCGpglhd62Dk07eiHbbqjb X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Kefeng Wang writes: [snip] > >>> 1) Will test some rand test to check the different of performance as >>> David suggested. >>> >>> 2) Hope the LKP to run more tests since it is very useful(more test >>> set and different machines) >> I'm starting to use LKP to test. > > Greet. I have run some tests with LKP to test. Firstly, there's almost no measurable difference between clearing pages from start to end or from end to start on Intel server CPU. I guess that there's some similar optimization for both direction. For multiple processes (same as logical CPU number) vm-scalability/anon-w-seq test case, the benchmark score increases about 22.4%. For multiple processes vm-scalability/anon-w-rand test case, no measurable difference for benchmark score. So, the optimization helps sequential workload mainly. In summary, on x86, process_huge_page() will not introduce any regression. And it helps some workload. However, on ARM64, it does introduce some regression for clearing pages from end to start. That needs to be addressed. I guess that the regression can be resolved via using more clearing from start to end (but not all). For example, can you take a look at the patch below? Which uses the similar framework as before, but clear each small trunk (mpage) from start to end. You can adjust MPAGE_NRPAGES to check when the regression can be restored. WARNING: the patch is only build tested. Best Regards, Huang, Ying -----------------------------------8<---------------------------------------- >From 406bcd1603987fdd7130d2df6f7d4aee4cc6b978 Mon Sep 17 00:00:00 2001 From: Huang Ying Date: Thu, 31 Oct 2024 11:13:57 +0800 Subject: [PATCH] mpage clear --- mm/memory.c | 70 ++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 67 insertions(+), 3 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 3ccee51adfbb..1fdc548c4275 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -6769,6 +6769,68 @@ static inline int process_huge_page( return 0; } +#define MPAGE_NRPAGES (1<<4) +#define MPAGE_SIZE (PAGE_SIZE * MPAGE_NRPAGES) +static inline int clear_huge_page( + unsigned long addr_hint, unsigned int nr_pages, + int (*process_subpage)(unsigned long addr, int idx, void *arg), + void *arg) +{ + int i, n, base, l, ret; + unsigned long addr = addr_hint & + ~(((unsigned long)nr_pages << PAGE_SHIFT) - 1); + unsigned long nr_mpages = ((unsigned long)nr_pages << PAGE_SHIFT) / MPAGE_SIZE; + + /* Process target subpage last to keep its cache lines hot */ + might_sleep(); + n = (addr_hint - addr) / MPAGE_SIZE; + if (2 * n <= nr_mpages) { + /* If target subpage in first half of huge page */ + base = 0; + l = n; + /* Process subpages at the end of huge page */ + for (i = nr_mpages - 1; i >= 2 * n; i--) { + cond_resched(); + ret = process_subpage(addr + i * MPAGE_SIZE, + i * MPAGE_NRPAGES, arg); + if (ret) + return ret; + } + } else { + /* If target subpage in second half of huge page */ + base = nr_mpages - 2 * (nr_mpages - n); + l = nr_mpages - n; + /* Process subpages at the begin of huge page */ + for (i = 0; i < base; i++) { + cond_resched(); + ret = process_subpage(addr + i * MPAGE_SIZE, + i * MPAGE_NRPAGES, arg); + if (ret) + return ret; + } + } + /* + * Process remaining subpages in left-right-left-right pattern + * towards the target subpage + */ + for (i = 0; i < l; i++) { + int left_idx = base + i; + int right_idx = base + 2 * l - 1 - i; + + cond_resched(); + ret = process_subpage(addr + left_idx * MPAGE_SIZE, + left_idx * MPAGE_NRPAGES, arg); + if (ret) + return ret; + cond_resched(); + ret = process_subpage(addr + right_idx * MPAGE_SIZE, + right_idx * MPAGE_NRPAGES, arg); + if (ret) + return ret; + } + return 0; +} + static void clear_gigantic_page(struct folio *folio, unsigned long addr, unsigned int nr_pages) { @@ -6784,8 +6846,10 @@ static void clear_gigantic_page(struct folio *folio, unsigned long addr, static int clear_subpage(unsigned long addr, int idx, void *arg) { struct folio *folio = arg; + int i; - clear_user_highpage(folio_page(folio, idx), addr); + for (i = 0; i < MPAGE_NRPAGES; i++) + clear_user_highpage(folio_page(folio, idx + i), addr + i * PAGE_SIZE); return 0; } @@ -6798,10 +6862,10 @@ void folio_zero_user(struct folio *folio, unsigned long addr_hint) { unsigned int nr_pages = folio_nr_pages(folio); - if (unlikely(nr_pages > MAX_ORDER_NR_PAGES)) + if (unlikely(nr_pages != HPAGE_PMD_NR)) clear_gigantic_page(folio, addr_hint, nr_pages); else - process_huge_page(addr_hint, nr_pages, clear_subpage, folio); + clear_huge_page(addr_hint, nr_pages, clear_subpage, folio); } static int copy_user_gigantic_page(struct folio *dst, struct folio *src, -- 2.39.2