From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E243C46CD2 for ; Tue, 30 Jan 2024 06:31:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 860296B007E; Tue, 30 Jan 2024 01:31:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 80FB16B0080; Tue, 30 Jan 2024 01:31:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6D73E6B0081; Tue, 30 Jan 2024 01:31:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5FAE66B007E for ; Tue, 30 Jan 2024 01:31:05 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id EFCC3A0A88 for ; Tue, 30 Jan 2024 06:31:04 +0000 (UTC) X-FDA: 81735004848.27.6304B20 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) by imf04.hostedemail.com (Postfix) with ESMTP id D3D104000B for ; Tue, 30 Jan 2024 06:31:01 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=l+UXZcQt; spf=pass (imf04.hostedemail.com: domain of ying.huang@intel.com designates 192.198.163.8 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706596263; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=y1kDFhsM+dWg/tnAW4zNl6AIQDdI2xbKe7Z1IvCoNvM=; b=uYDmU8YYgaVa8oTvzXMO+mq4HUyT8+IW1FCcGJHImQoseOHG8UVtcApCuTkjss4+fE6kBj jrtNV6Mms4WHx6mCl4VYP+s7jbDsIT9IuJ6CRGzSw7Q/DxywGDyhdp3+SEDSMOCL+DHBtm L4ulDhpSO+19PpCK9OutvvlCQAkIecQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706596263; a=rsa-sha256; cv=none; b=VStb2Qwz20D2RLqvl/84OC03MQ2Q6Tl2ususEy1TBp/jlFZrwCrTViw9g3V2Fka+df2q1I E2kQvdqTaoTTeblv8dbeDHJR2/SYxJcG9fFr4mKDUAnovTiHR9dz+F0z4QJGzfHq0Bkg30 RT+td1hmFNyojgCyoGH9eSlFq1WzSvo= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=l+UXZcQt; spf=pass (imf04.hostedemail.com: domain of ying.huang@intel.com designates 192.198.163.8 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706596262; x=1738132262; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version; bh=uwEkKH2wJSHeUP8JhuJ79HOvJUStRfChWk3uYCaa9Fc=; b=l+UXZcQtMKn8Vz4fY0R2Wy83/OOxUpuSYBqk/ev2A29SgPKDSig4rO99 3Nz7SzCNETQyu/Qw9O8bV/hlOFOhXUi0I1QbCURhLKUVU7zNKvRG12GSw FmXdI7cGkXcr7yrJllPnfIGIV2o7/VHBFpvpsJcxfs7+2yGETuVjhVj7I 3wwRnh3qMELOd1DX6MGhst1LrH1PsBqQSD/l4fvmE/tEsMYZBgGzPjreU imxBwkxESzbQ553eSz4xjF6PgAWfAyMPlX1Cd8VRV6UKZSSvmyyxv9NSM S6qjzt6LDh17/805v6bQ4hrbLx9XP511UD2OYUOMykT/XTTFRaeWb5GHk A==; X-IronPort-AV: E=McAfee;i="6600,9927,10968"; a="16716563" X-IronPort-AV: E=Sophos;i="6.05,707,1701158400"; d="scan'208";a="16716563" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jan 2024 22:31:00 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10968"; a="911336949" X-IronPort-AV: E=Sophos;i="6.05,707,1701158400"; d="scan'208";a="911336949" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jan 2024 22:30:56 -0800 From: "Huang, Ying" To: Kairui Song Cc: linux-mm@kvack.org, Kairui Song , Andrew Morton , Chris Li , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , Yosry Ahmed , David Hildenbrand , linux-kernel@vger.kernel.org Subject: Re: [PATCH v3 4/7] mm/swap: introduce swapin_entry for unified readahead policy In-Reply-To: <20240129175423.1987-5-ryncsn@gmail.com> (Kairui Song's message of "Tue, 30 Jan 2024 01:54:19 +0800") References: <20240129175423.1987-1-ryncsn@gmail.com> <20240129175423.1987-5-ryncsn@gmail.com> Date: Tue, 30 Jan 2024 14:29:00 +0800 Message-ID: <87o7d3gwr7.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspamd-Queue-Id: D3D104000B X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: kn7imwjuaeiisxntycu1rsqx9i6z4ajt X-HE-Tag: 1706596261-911867 X-HE-Meta: U2FsdGVkX19Gx/UxbdemU177wO8w4Sad/3QQajZ4khQzM4dvDzlLHUQr+axcPzxYG+GbYdqVN+Rl8S9tsHZAjR2ItT7YVZEgkL7VhtWV18vc8faqPwt08IYdPa/QdkBxkBN5NlP0h8u1oKj0Zrqt4iQAxVLj5gJUT6it9csMAbk3LqEo7163XI/NiHCeMEnG7vVJ1OzQpcM3IhwZAdTocSTiMeP9YXqSV6h8O5fWnsf1dyPgLbI7Kgw1lOxIsYlcZgj4BK6xifGw58OeeGfaOKa+cP69CFlJZSvxGgUsmCqt/LTnorvK0gqVdufVhI6+cHBFXMezI2uRwQ0ermsSJ7art8fAfvhX4qHq7mxwXVViva5XCxTv1cYbsIJeKoSDo1wh6khz62fgysm8oZ9ujSDoRd9o/bWkY563RdoeVOENqlhszN62EqyMHfXnt14rFnrAv5b7fES8FZLnz1zpAJ2sXW07CWHGfuomAY6SIH87JVVRkF9uAutc9NYXz3kGQw1hE5xeEHY7zmtKaxipRaf2bokMHVNso3NPagMaGleJHtsGNPjnjNnqbk7hUqJXoVWK5N2FuLVXHv9nAEgao8Zy4BEyjFWXcjrEMIIKKEvFwZ+5wFXYZEfNFl6xyBf4xFE9QEEvsRN+lpy6fb10PbgEj1mCujLUlID+zOZJFAfZrb+ClFEkt2NWZjvTDhewkz131/eXy32eEfWXKlJ1uaKcVD2X88JjOQDEn99+NvJThG3nvyjLEdxWwrj/qG0APmOjBgWnXmmmju5uUK1ufWqxvnpYxdzhbxGb/p/QldN31MEbqStJcNyUeLJ1LNwCO7jq6fqTUrn/I23EEA95DglUKJAvnupRdsxylIEbWhkzUs1umRD39Sh1DM0ZSPRfKLMYC0JPBtbpzEUiwQjzAFc96qaJ6781xxohUaHgK/n2jmOOVXUN5qwVWCoHsjQ9cjIXpso4emzee5Ku5Bw kaHLsLsn Y056coJ0tjcb1hj3yehiRH3eHmRsyb97mSJiBZrrL/dv6LDlg5rA4kpvbY0pExnQau6G6RSMobv5p6+OiLhmqasPFU0WH7F9yLtkzP8yT7at+R7QpSWUHgIf7s1/uCN8RYMoU61rSZ/h/4nwfeo+rU62pgJ3BI82kq6r2Wri4yT+2PrkLFXbCd3wzdBHKNArVksgCnEl9rk5DfbWZqoovtpddwYwFSiARDSFeZiniu4Ax4wuuF+f8n+RZQbARfyFtFEVu50O/5Rh0+chp1WLnapTMflXlkhe0hzP3iz9onGBK734NnmPRoojvPWr10Z984GnniPINuv9kbzU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Kairui Song writes: > From: Kairui Song > > Introduce swapin_entry which merges swapin_readahead and swapin_direct > making it the main entry for swapin pages, and use a unified swapin > readahead policy. > > This commit makes swapoff make use of this new helper and skip readahead > for SYNCHRONOUS_IO device since it's not helpful here. Now swapping > off a 10G ZRAM (lzo-rle) after same workload is faster since readahead > is skipped and overhead is reduced. > > Before: > time swapoff /dev/zram0 > real 0m12.337s > user 0m0.001s > sys 0m12.329s > > After: > time swapoff /dev/zram0 > real 0m9.728s > user 0m0.001s > sys 0m9.719s > > Signed-off-by: Kairui Song > --- > mm/memory.c | 18 +++--------------- > mm/swap.h | 16 ++++------------ > mm/swap_state.c | 40 ++++++++++++++++++++++++---------------- > mm/swapfile.c | 7 ++----- > 4 files changed, 33 insertions(+), 48 deletions(-) > > diff --git a/mm/memory.c b/mm/memory.c > index 81dc9d467f4e..8711f8a07039 100644 > --- a/mm/memory.c > +++ b/mm/memory.c > @@ -3864,20 +3864,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) > swapcache = folio; > > if (!folio) { > - if (data_race(si->flags & SWP_SYNCHRONOUS_IO) && > - __swap_count(entry) == 1) { > - /* skip swapcache and readahead */ > - folio = swapin_direct(entry, GFP_HIGHUSER_MOVABLE, vmf); > - if (folio) > - page = &folio->page; > - } else { > - page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, > - vmf); > - if (page) > - folio = page_folio(page); > - swapcache = folio; > - } > - > + folio = swapin_entry(entry, GFP_HIGHUSER_MOVABLE, > + vmf, &swapcache); > if (!folio) { > /* > * Back out if somebody else faulted in this pte > @@ -3890,11 +3878,11 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) > ret = VM_FAULT_OOM; > goto unlock; > } > - Change by accident? > /* Had to read the page from swap area: Major fault */ > ret = VM_FAULT_MAJOR; > count_vm_event(PGMAJFAULT); > count_memcg_event_mm(vma->vm_mm, PGMAJFAULT); > + page = folio_file_page(folio, swp_offset(entry)); Better to move this line just after checking !folio. This make it a little easier to associate page to folio. > } else if (PageHWPoison(page)) { > /* > * hwpoisoned dirty swapcache pages are kept for killing > diff --git a/mm/swap.h b/mm/swap.h > index 83eab7b67e77..8f8185d3865c 100644 > --- a/mm/swap.h > +++ b/mm/swap.h > @@ -54,10 +54,8 @@ struct folio *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_flags, > bool skip_if_exists); > struct folio *swap_cluster_readahead(swp_entry_t entry, gfp_t flag, > struct mempolicy *mpol, pgoff_t ilx); > -struct page *swapin_readahead(swp_entry_t entry, gfp_t flag, > - struct vm_fault *vmf); > -struct folio *swapin_direct(swp_entry_t entry, gfp_t flag, > - struct vm_fault *vmf); > +struct folio *swapin_entry(swp_entry_t entry, gfp_t flag, > + struct vm_fault *vmf, struct folio **swapcached); > > static inline unsigned int folio_swap_flags(struct folio *folio) > { > @@ -88,14 +86,8 @@ static inline struct folio *swap_cluster_readahead(swp_entry_t entry, > return NULL; > } > > -struct folio *swapin_direct(swp_entry_t entry, gfp_t flag, > - struct vm_fault *vmf) > -{ > - return NULL; > -} > - > -static inline struct page *swapin_readahead(swp_entry_t swp, gfp_t gfp_mask, > - struct vm_fault *vmf) > +static inline struct folio *swapin_entry(swp_entry_t swp, gfp_t gfp_mask, > + struct vm_fault *vmf, struct folio **swapcached) > { > return NULL; > } > diff --git a/mm/swap_state.c b/mm/swap_state.c > index a450d09fc0db..5e06b2e140d4 100644 > --- a/mm/swap_state.c > +++ b/mm/swap_state.c > @@ -870,8 +870,8 @@ static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask, > * Returns the struct folio for entry and addr after the swap entry is read > * in. > */ > -struct folio *swapin_direct(swp_entry_t entry, gfp_t gfp_mask, > - struct vm_fault *vmf) > +static struct folio *swapin_direct(swp_entry_t entry, gfp_t gfp_mask, > + struct vm_fault *vmf) > { > struct vm_area_struct *vma = vmf->vma; > struct folio *folio; > @@ -908,33 +908,41 @@ struct folio *swapin_direct(swp_entry_t entry, gfp_t gfp_mask, > } > > /** > - * swapin_readahead - swap in pages in hope we need them soon > + * swapin_entry - swap in a folio from swap entry > * @entry: swap entry of this memory > * @gfp_mask: memory allocation flags > * @vmf: fault information > + * @swapcache: set to the swapcache folio if swapcache is used > * > * Returns the struct page for entry and addr, after queueing swapin. > * > - * It's a main entry function for swap readahead. By the configuration, > + * It's the main entry function for swap in. By the configuration, > * it will read ahead blocks by cluster-based(ie, physical disk based) > - * or vma-based(ie, virtual address based on faulty address) readahead. > + * or vma-based(ie, virtual address based on faulty address) readahead, > + * or skip the readahead(ie, ramdisk based swap device). > */ > -struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, > - struct vm_fault *vmf) > +struct folio *swapin_entry(swp_entry_t entry, gfp_t gfp_mask, > + struct vm_fault *vmf, struct folio **swapcache) > { > struct mempolicy *mpol; > - pgoff_t ilx; > struct folio *folio; > + pgoff_t ilx; ditto. Otherwise, looks good to me. Thanks! Reviewed-by: "Huang, Ying" > > - mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); > - folio = swap_use_vma_readahead() ? > - swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf) : > - swap_cluster_readahead(entry, gfp_mask, mpol, ilx); > - mpol_cond_put(mpol); > + if (data_race(swp_swap_info(entry)->flags & SWP_SYNCHRONOUS_IO) && > + __swap_count(entry) == 1) { > + folio = swapin_direct(entry, gfp_mask, vmf); > + } else { > + mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); > + if (swap_use_vma_readahead()) > + folio = swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf); > + else > + folio = swap_cluster_readahead(entry, gfp_mask, mpol, ilx); > + mpol_cond_put(mpol); > + if (swapcache) > + *swapcache = folio; > + } > > - if (!folio) > - return NULL; > - return folio_file_page(folio, swp_offset(entry)); > + return folio; > } > > #ifdef CONFIG_SYSFS > diff --git a/mm/swapfile.c b/mm/swapfile.c > index 606d95b56304..1cf7e72e19e3 100644 > --- a/mm/swapfile.c > +++ b/mm/swapfile.c > @@ -1867,7 +1867,6 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, > > folio = swap_cache_get_folio(entry, vma, addr); > if (!folio) { > - struct page *page; > struct vm_fault vmf = { > .vma = vma, > .address = addr, > @@ -1875,10 +1874,8 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, > .pmd = pmd, > }; > > - page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, > - &vmf); > - if (page) > - folio = page_folio(page); > + folio = swapin_entry(entry, GFP_HIGHUSER_MOVABLE, > + &vmf, NULL); > } > if (!folio) { > /*