From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA410C3DA6E for ; Fri, 5 Jan 2024 07:30:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1885A6B02EF; Fri, 5 Jan 2024 02:30:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 137BC6B02F0; Fri, 5 Jan 2024 02:30:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F406E6B02F1; Fri, 5 Jan 2024 02:30:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id DDD836B02EF for ; Fri, 5 Jan 2024 02:30:23 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id B14E516045C for ; Fri, 5 Jan 2024 07:30:23 +0000 (UTC) X-FDA: 81644434326.12.CED92B7 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.20]) by imf05.hostedemail.com (Postfix) with ESMTP id AD19A10001C for ; Fri, 5 Jan 2024 07:30:21 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=jtAwhy66; spf=pass (imf05.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704439822; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KGASav5ZgUWYeuhGwZg5RFbrkk4/VRro4/0IIlCszFQ=; b=oAATDmE6DwqP9CjE8Bp/XfpiddZE844i+tYfzOF0BIfCmLgket+T/gyL1917B8aT2i+SKK dPkIpmXUxT4tO/GFlTthNDN0MJaSLzfYW/iu+Dgafv9GmwHNIRCzv7n2bv8w4f9PMP1S1x +WX6MvF4ClLCMcLz83sFE1vu21qgWQA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704439822; a=rsa-sha256; cv=none; b=E+3gvRVtYo0wRHPV/n/WQAF4RAN4qTi61xa2WKvIwu/LXXhbO2nBA+SctJCkL7hWAW5CwM 8mw0E+gO0rllsAbU7GVw4m5Mn5h4xtQXXuRTF1+Rpr3IBSShth9QG+v/PrSWEL9oG1wpG0 Mp8B469BDjE5MwxecfTXBUzsQ0B7ZeY= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=jtAwhy66; spf=pass (imf05.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1704439821; x=1735975821; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version; bh=miZsVC0cwSQ3wmtUqCj/8APGGPCSDLSbzew/wWh9PYo=; b=jtAwhy665vvrGetuAm6rcvFST5iJRs2qCJ7sw5X8YHsJW5KO3l73XYWf ohb9SIqi3Ud1hWZIpvB1Yef6DDrXy4yn/7C9520CyudieJHbl1wdhUXp3 TNsOM9ksjjvIC/VY9fxZZY2zN4BQEmFaujzK/+JVapvcKiOMS9+gAN37+ UDc4/tQ/J1V37OTrQ6zfur5yD6W2/+YwWXcYk1y+t0PeebCKBGMj2QI4O VkniVxhd7nzBMZH4vGIQmNwnYeX1pR5Ud8vovZWwJwi8ydvHSRvBQo6l2 mQbmmLnCYHdiTLY1LQCCT7OVBKLkCNkfbgLNh61v8KHuFdl+AxDj5J5KO g==; X-IronPort-AV: E=McAfee;i="6600,9927,10943"; a="387901239" X-IronPort-AV: E=Sophos;i="6.04,333,1695711600"; d="scan'208";a="387901239" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jan 2024 23:30:19 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.04,333,1695711600"; d="scan'208";a="29060591" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jan 2024 23:30:16 -0800 From: "Huang, Ying" To: Kairui Song Cc: linux-mm@kvack.org, Kairui Song , Andrew Morton , Chris Li , Hugh Dickins , Johannes Weiner , Matthew Wilcox , Michal Hocko , Yosry Ahmed , David Hildenbrand , linux-kernel@vger.kernel.org Subject: Re: [PATCH v2 5/9] mm/swap: introduce swapin_entry for unified readahead policy In-Reply-To: <20240102175338.62012-6-ryncsn@gmail.com> (Kairui Song's message of "Wed, 3 Jan 2024 01:53:34 +0800") References: <20240102175338.62012-1-ryncsn@gmail.com> <20240102175338.62012-6-ryncsn@gmail.com> Date: Fri, 05 Jan 2024 15:28:18 +0800 Message-ID: <8734vcb5nx.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspamd-Queue-Id: AD19A10001C X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: oarkcxcp3xnijmqya1cf5thjw1iep7cd X-HE-Tag: 1704439821-783546 X-HE-Meta: U2FsdGVkX19tztv7YkhiaBbAjGzuiAwRbu1p55bl9qlHb1GhTmI8mYF24Hux2ThxdIIibsXo/jMHkpWJS/xpmJzNdKw9TxwEUeyxeC3OBu9NKe61rmHJEWLZw9txYFkY8bd5pNXNMBqBYTqURUs0EMOD0astNp3Eu0FMhlA47ht96439eG5blxPKP1HT74/t3xF2R6CYTk/uJ3k+5TkutT3Leo/Al/y0owgjl26rfXKePJweqPUyYD8NFZ+XSZP/h876On3HMHZHBAhiM0yQ9b3gUdFTpkW2t9Ga5WqTeKyEKtD9YpyaExNiuDGbQ5o/azWjEK2aYy1DmDmh1sLjv477QONx24msz+vvQjwQ3DnYPrH3iiJERbFlNmFJhMKSJcDz95K8H5olN/QCT5RqoAqSVT+uhg2caTlGjnY9QBCTS4DpiSrS/FbZUg/Y4Pi2P1L+dARHhsKOPqTGJh0rUwsEa2woj6prkjsIDImMqbTGlTGFi1ZRFLXAmpxg0kJwFZtVOQ4iiaIzuuLTAf/8IvAarUVjBjFfzV2mNQugTgtPYBSq3NANy0VF06NEVDZlqk0zlQCiV8OzXOP+QsRGFhVavRfXip+w8oH4dpGS+pBVu60/SgCLt/8jit/Un+oln+1ZmRCLRgy9/YtGKs+8Q9R8YYJn47MbjtmW7jI20YY5852YNkDcVE1cgMgtGvxRbOdlV76FRky/EU0iAQ3pzqMEFiRjDnL5Q9RdHOSYQta81quMcDLZIZ3HCMUfzbi0RChU3ysv3OqZKv72X/Y6JehAiQHFz86ZebOVekWZXVqzlFMIt4omGAk3b/L0XmWmm1wqEqOxbn1xHsUYQJomj8G9SNM70a4f2OhHCy35+2iM9ztl1kt+E4pBP5ZIREbk6INC4qZ81pHmQqQLH1cqhw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Kairui Song writes: > From: Kairui Song > > Introduce swapin_entry which merges swapin_readahead and swapin_direct > making it the main entry for swapin pages, and use a unified swapin > policy. > > This commit makes swapoff make use of this new helper and now swapping > off a 10G ZRAM (lzo-rle) is faster since readahead is skipped. > > Before: > time swapoff /dev/zram0 > real 0m12.337s > user 0m0.001s > sys 0m12.329s > > After: > time swapoff /dev/zram0 > real 0m9.728s > user 0m0.001s > sys 0m9.719s > > Signed-off-by: Kairui Song > --- > mm/memory.c | 21 +++++++-------------- > mm/swap.h | 16 ++++------------ > mm/swap_state.c | 49 +++++++++++++++++++++++++++++++++---------------- > mm/swapfile.c | 7 ++----- > 4 files changed, 46 insertions(+), 47 deletions(-) > > diff --git a/mm/memory.c b/mm/memory.c > index 0165c8cad489..b56254a875f8 100644 > --- a/mm/memory.c > +++ b/mm/memory.c > @@ -3801,6 +3801,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) > rmap_t rmap_flags = RMAP_NONE; > bool exclusive = false; > swp_entry_t entry; > + bool swapcached; > pte_t pte; > vm_fault_t ret = 0; > > @@ -3864,21 +3865,13 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) > swapcache = folio; > > if (!folio) { > - if (data_race(si->flags & SWP_SYNCHRONOUS_IO) && > - __swap_count(entry) == 1) { > - /* skip swapcache and readahead */ > - folio = swapin_direct(entry, GFP_HIGHUSER_MOVABLE, vmf); > - if (folio) > - page = &folio->page; > + folio = swapin_entry(entry, GFP_HIGHUSER_MOVABLE, > + vmf, &swapcached); > + if (folio) { > + page = folio_file_page(folio, swp_offset(entry)); > + if (swapcached) > + swapcache = folio; > } else { > - page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, > - vmf); > - if (page) > - folio = page_folio(page); > - swapcache = folio; > - } > - > - if (!folio) { > /* > * Back out if somebody else faulted in this pte > * while we released the pte lock. > diff --git a/mm/swap.h b/mm/swap.h > index 83eab7b67e77..502a2801f817 100644 > --- a/mm/swap.h > +++ b/mm/swap.h > @@ -54,10 +54,8 @@ struct folio *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_flags, > bool skip_if_exists); > struct folio *swap_cluster_readahead(swp_entry_t entry, gfp_t flag, > struct mempolicy *mpol, pgoff_t ilx); > -struct page *swapin_readahead(swp_entry_t entry, gfp_t flag, > - struct vm_fault *vmf); > -struct folio *swapin_direct(swp_entry_t entry, gfp_t flag, > - struct vm_fault *vmf); > +struct folio *swapin_entry(swp_entry_t entry, gfp_t flag, > + struct vm_fault *vmf, bool *swapcached); > > static inline unsigned int folio_swap_flags(struct folio *folio) > { > @@ -88,14 +86,8 @@ static inline struct folio *swap_cluster_readahead(swp_entry_t entry, > return NULL; > } > > -struct folio *swapin_direct(swp_entry_t entry, gfp_t flag, > - struct vm_fault *vmf) > -{ > - return NULL; > -} > - > -static inline struct page *swapin_readahead(swp_entry_t swp, gfp_t gfp_mask, > - struct vm_fault *vmf) > +static inline struct folio *swapin_entry(swp_entry_t swp, gfp_t gfp_mask, > + struct vm_fault *vmf, bool *swapcached) > { > return NULL; > } > diff --git a/mm/swap_state.c b/mm/swap_state.c > index d39c5369da21..66ff187aa5d3 100644 > --- a/mm/swap_state.c > +++ b/mm/swap_state.c > @@ -316,6 +316,11 @@ void free_pages_and_swap_cache(struct encoded_page **pages, int nr) > release_pages(pages, nr); > } > > +static inline bool swap_use_no_readahead(struct swap_info_struct *si, swp_entry_t entry) > +{ > + return data_race(si->flags & SWP_SYNCHRONOUS_IO) && __swap_count(entry) == 1; > +} > + It appears that there's only one caller of the function in the same file? Why add a function? > static inline bool swap_use_vma_readahead(void) > { > return READ_ONCE(enable_vma_readahead) && !atomic_read(&nr_rotate_swap); > @@ -870,8 +875,8 @@ static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask, > * Returns the struct folio for entry and addr after the swap entry is read > * in. > */ > -struct folio *swapin_direct(swp_entry_t entry, gfp_t gfp_mask, > - struct vm_fault *vmf) > +static struct folio *swapin_direct(swp_entry_t entry, gfp_t gfp_mask, > + struct vm_fault *vmf) > { > struct vm_area_struct *vma = vmf->vma; > struct folio *folio; > @@ -908,33 +913,45 @@ struct folio *swapin_direct(swp_entry_t entry, gfp_t gfp_mask, > } > > /** > - * swapin_readahead - swap in pages in hope we need them soon > + * swapin_entry - swap in a page from swap entry > * @entry: swap entry of this memory > * @gfp_mask: memory allocation flags > * @vmf: fault information > + * @swapcached: pointer to a bool used as indicator if the > + * page is swapped in through swapcache. > * > * Returns the struct page for entry and addr, after queueing swapin. > * > - * It's a main entry function for swap readahead. By the configuration, > + * It's a main entry function for swap in. By the configuration, > * it will read ahead blocks by cluster-based(ie, physical disk based) > - * or vma-based(ie, virtual address based on faulty address) readahead. > + * or vma-based(ie, virtual address based on faulty address) readahead, > + * or skip the readahead (ie, ramdisk based swap device). > */ > -struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask, > - struct vm_fault *vmf) > +struct folio *swapin_entry(swp_entry_t entry, gfp_t gfp_mask, > + struct vm_fault *vmf, bool *swapcached) May be better to use struct folio *swapin_entry(swp_entry_t entry, gfp_t gfp_mask, struct vm_fault *vmf, struct folio **swapcache) In this way, we can reduce the number of source lines in the caller. > { > struct mempolicy *mpol; > - pgoff_t ilx; > struct folio *folio; > + pgoff_t ilx; > + bool cached; > > - mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); > - folio = swap_use_vma_readahead() ? > - swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf) : > - swap_cluster_readahead(entry, gfp_mask, mpol, ilx); > - mpol_cond_put(mpol); > + if (swap_use_no_readahead(swp_swap_info(entry), entry)) { > + folio = swapin_direct(entry, gfp_mask, vmf); > + cached = false; > + } else { > + mpol = get_vma_policy(vmf->vma, vmf->address, 0, &ilx); > + if (swap_use_vma_readahead()) > + folio = swap_vma_readahead(entry, gfp_mask, mpol, ilx, vmf); > + else > + folio = swap_cluster_readahead(entry, gfp_mask, mpol, ilx); > + mpol_cond_put(mpol); > + cached = true; > + } > > - if (!folio) > - return NULL; > - return folio_file_page(folio, swp_offset(entry)); > + if (swapcached) > + *swapcached = cached; > + > + return folio; > } > > #ifdef CONFIG_SYSFS > diff --git a/mm/swapfile.c b/mm/swapfile.c > index f7271504aa0a..ce4e6c10dce7 100644 > --- a/mm/swapfile.c > +++ b/mm/swapfile.c > @@ -1866,7 +1866,6 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, > > folio = swap_cache_get_folio(entry, vma, addr); > if (!folio) { > - struct page *page; > struct vm_fault vmf = { > .vma = vma, > .address = addr, > @@ -1874,10 +1873,8 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, > .pmd = pmd, > }; > > - page = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, > - &vmf); > - if (page) > - folio = page_folio(page); > + folio = swapin_entry(entry, GFP_HIGHUSER_MOVABLE, > + &vmf, NULL); > } > if (!folio) { > /* -- Best Regards, Huang, Ying