From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3064AC3DA7F for ; Thu, 1 Aug 2024 01:13:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4EA1B6B0092; Wed, 31 Jul 2024 21:13:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 498EF6B009D; Wed, 31 Jul 2024 21:13:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2ECD26B00A0; Wed, 31 Jul 2024 21:13:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 053056B0092 for ; Wed, 31 Jul 2024 21:13:22 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 64A08C07CC for ; Thu, 1 Aug 2024 01:13:22 +0000 (UTC) X-FDA: 82401903444.19.BC40611 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) by imf20.hostedemail.com (Postfix) with ESMTP id 052F21C002B for ; Thu, 1 Aug 2024 01:13:18 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="cAU/C/o8"; spf=pass (imf20.hostedemail.com: domain of ying.huang@intel.com designates 198.175.65.10 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722474772; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+ZUv8j6rkxwNmcjzwbvbHeEdfbgQtAJtqOtqCA1GdyY=; b=7FTUf9Z+WQjgUnjL2grvYtaDf3mSiM5ML7a2FOQ1ONAehFNlNcb+lmFzl8QNIdzUHNs7+5 peTb+23PBASMS24EjFEsfYhrqCNzSyZ0U4XSd3spw02afiJNEuMdjMYNlN/s1BoySfBhHY 3BwpWpX1gfitVdjeIBH37RQ4H5/3uHQ= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="cAU/C/o8"; spf=pass (imf20.hostedemail.com: domain of ying.huang@intel.com designates 198.175.65.10 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722474772; a=rsa-sha256; cv=none; b=G5kjUmRKv7Ltjo5186ODc2w/80vj+70RiKzaGYzzLIxFzUUnqbakBd26QRFHyxqOcUEB4w TURqye6g0KW6h5qzViF3QyzGbX8IsJUl06UgML6nJLMaglhGfV0Hh7wrM4z9fGiVKxlntu pIlehFVJpLaANrpWYTRxIrKdM1tkLpc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1722474799; x=1754010799; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version:content-transfer-encoding; bh=Q/8dpL+pqe2FWAWn7wdFvFIwNaNCt3HG4rvKtOaO0UQ=; b=cAU/C/o8VMeMrX1DsqHt4FCL8n8X/MFnDWlUMX/1gE3+woXm2GJveLDV CnsGi78RrGmGqj32XOxRdRrFqgGGXkVZqe7H5sld2GGYnPF0SX3MpKLbT nldo55JE8gZf9/yEDz5A0qnuj20NuQCR0v60Q8dqdQhFuZWWRhzliNLPy sp4MhsUkuh+BpE5w3m7KAJ6URehM8Ja05PHe7/1TRVydJA1QPCU34JJLE 3geSEpFgk8PWGI3OBMyZJAAKaMy8b6xs87KxMllgSjZblPue83PMsADDW 2t/WcnWiwUyc5P+soYYp1dLtl/pzjBsGxolDKiR3Pizrj4EYc18ZRy3VR Q==; X-CSE-ConnectionGUID: yFK6hAY1Rh2OjOUb0yWuug== X-CSE-MsgGUID: 4FvBqun/Rxm1BfNdkZTjAg== X-IronPort-AV: E=McAfee;i="6700,10204,11150"; a="37866065" X-IronPort-AV: E=Sophos;i="6.09,253,1716274800"; d="scan'208";a="37866065" Received: from orviesa010.jf.intel.com ([10.64.159.150]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2024 18:13:17 -0700 X-CSE-ConnectionGUID: 2c7BeppsTH6pR8cYE5exbA== X-CSE-MsgGUID: eHxBcuXKQqeBT6eZ28YpCg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,253,1716274800"; d="scan'208";a="54775380" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orviesa010-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jul 2024 18:13:11 -0700 From: "Huang, Ying" To: Barry Song <21cnbao@gmail.com> Cc: akpm@linux-foundation.org, linux-mm@kvack.org, baolin.wang@linux.alibaba.com, chrisl@kernel.org, david@redhat.com, hannes@cmpxchg.org, hughd@google.com, kaleshsingh@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, mhocko@suse.com, minchan@kernel.org, nphamcs@gmail.com, ryan.roberts@arm.com, senozhatsky@chromium.org, shakeel.butt@linux.dev, shy828301@gmail.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, xiang@kernel.org, yosryahmed@google.com Subject: Re: [PATCH 1/1] mm: swap: add nr argument in swapcache_prepare and swapcache_clear to support large folios In-Reply-To: (Barry Song's message of "Wed, 31 Jul 2024 18:23:11 +0800") References: <20240730071339.107447-1-21cnbao@gmail.com> <20240730071339.107447-2-21cnbao@gmail.com> <874j86oubf.fsf@yhuang6-desk2.ccr.corp.intel.com> <87zfpynf2r.fsf@yhuang6-desk2.ccr.corp.intel.com> Date: Thu, 01 Aug 2024 09:09:37 +0800 Message-ID: <871q39kq0e.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 052F21C002B X-Stat-Signature: ytmp93uch3n63178sy1g8gbgsgk475et X-HE-Tag: 1722474798-629932 X-HE-Meta: U2FsdGVkX1+AKWswTKtlPPrZdOXxztSudKyKNnArCDpG4KUZV1fWLyI86ln9V8fDhA+OPP6N/pr7NGMZx6aWnS8qiM69q0+jizcG5Tkc9s7vzc/WxjXo3Ni0eOWSlddsvFPYhTAlmGT9yyU4EyFOwfMfaArCkGVNfS6pb8abHud+kwXQSjOVfauZbqPXYzDYrsTENzT82N2B2MQ56Qgg3RiyWUh52kIx7d/QkXfT3pR7Zzk81L3fjwuyZE+7ZsEYISHoH7rBkosFidy60bNhy7JS29KGoooywgqu/6J4CwESdtb+oQw/aECJXwtVsBLMmZpSgsGunyRuu/jsD59OurLfhsj+tx3jkiBf2UKq0TOA9CcxyM84uDRosT697e3+aJzTSGDQq7gwzZJNre7m6ogMASsvkimVBnpA/Z7O2tUNBHMNNf+JWiuQ+rDtEeT167hvpe6wNem+Q0VRMYsHR480vq3OnsiIoBvqV16E/3nymUxFBJe8Jo9qNOxAHYa80FLAozEiUoJgLv6xpQq7G+WcZ2pSQrr5lrvy11tGo0nbfky1nAK9goSgFC0SEOuAAgGLIWiG1bZbz3udGfP0frn3crIHjXfcfLdGBWXFjzKMAtrsDDjMXvnQH5m+JU9Ef3s6Y4jPHwYOXNRbps64y+lAC2mw7EZ0achFJ24MheabAsujL5hY0ICjcXmr+DbMZ5QsGrqapoCqd0W/mFu305m9r094FxnZ4v3/0YJAyC+fIt0mx3aNTLSdm8lz7MBQtMit8WsbxcqaJtSuajyrFJd+Gbp/e2QFUbXFz0dUqrKcL+g90pt+S3mBB6LkI6kSy7BaE64c73wyzDZHcqVtenecMT9Mnmrc6t9llxFbcxtE1Ow0j+YBewzbnzBgkiE70hyOkSLcd6Kq0UGuNAgN5o4NwHBrN8mbqkpT53DMtmsrtFvOVElVCYnJxm//fcnnwpmMSDDpIR1by1rYedi eaQ3ecQ9 T3OWzRUO5LVUF3aWsWM04FVjPM6gC1+d6wBypWpDOoer8c5ZfUTVsCwG3QcpllUfEP7HWNU9iYov4RRbbhIcrxWCk/okXtxvURN8Qze6n4PcqxIoGd1KNYntk8bamSJ+zaSOsovDOCcrnLdGWBigFk4l4R1V34JBjoOzlEUE9/KfAVdZJdLhyswEdEPai/9xmE7Xsl1acFkBv6qakDJYnCZaqUBVLC58hyGvZWp2SF0kefJ0NXXTMBQXZQheDE+POBEBlGVTPiaRzTDZEP2GUCrqPizNuo0cjC80mGbqg7PdZt4WnOBkEji7KYVMGXT8VTDjUm3Qy45c6v0FqR+ZyFHwKPEqlN9dH2jfQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Barry Song <21cnbao@gmail.com> writes: > On Wed, Jul 31, 2024 at 4:28=E2=80=AFPM Huang, Ying wrote: >> >> Barry Song <21cnbao@gmail.com> writes: >> >> > On Wed, Jul 31, 2024 at 4:14=E2=80=AFPM Huang, Ying wrote: >> >> >> >> Hi, Barry, >> >> >> >> Barry Song <21cnbao@gmail.com> writes: >> >> >> >> > From: Barry Song >> >> > >> >> > Right now, swapcache_prepare() and swapcache_clear() supports one e= ntry >> >> > only, to support large folios, we need to handle multiple swap entr= ies. >> >> > >> >> > To optimize stack usage, we iterate twice in __swap_duplicate(): the >> >> > first time to verify that all entries are valid, and the second time >> >> > to apply the modifications to the entries. >> >> > >> >> > Currently, we're using nr=3D1 for the existing users. >> >> > >> >> > Reviewed-by: Baolin Wang >> >> > Signed-off-by: Barry Song >> >> > --- >> >> > include/linux/swap.h | 4 +- >> >> > mm/memory.c | 6 +-- >> >> > mm/swap.h | 5 ++- >> >> > mm/swap_state.c | 2 +- >> >> > mm/swapfile.c | 101 +++++++++++++++++++++++++--------------= ---- >> >> > 5 files changed, 68 insertions(+), 50 deletions(-) >> >> > >> >> > diff --git a/include/linux/swap.h b/include/linux/swap.h >> >> > index ba7ea95d1c57..5b920fa2315b 100644 >> >> > --- a/include/linux/swap.h >> >> > +++ b/include/linux/swap.h >> >> > @@ -480,7 +480,7 @@ extern int get_swap_pages(int n, swp_entry_t sw= p_entries[], int order); >> >> > extern int add_swap_count_continuation(swp_entry_t, gfp_t); >> >> > extern void swap_shmem_alloc(swp_entry_t); >> >> > extern int swap_duplicate(swp_entry_t); >> >> > -extern int swapcache_prepare(swp_entry_t); >> >> > +extern int swapcache_prepare(swp_entry_t entry, int nr); >> >> > extern void swap_free_nr(swp_entry_t entry, int nr_pages); >> >> > extern void swapcache_free_entries(swp_entry_t *entries, int n); >> >> > extern void free_swap_and_cache_nr(swp_entry_t entry, int nr); >> >> > @@ -554,7 +554,7 @@ static inline int swap_duplicate(swp_entry_t sw= p) >> >> > return 0; >> >> > } >> >> > >> >> > -static inline int swapcache_prepare(swp_entry_t swp) >> >> > +static inline int swapcache_prepare(swp_entry_t swp, int nr) >> >> > { >> >> > return 0; >> >> > } >> >> > diff --git a/mm/memory.c b/mm/memory.c >> >> > index 833d2cad6eb2..b8675617a5e3 100644 >> >> > --- a/mm/memory.c >> >> > +++ b/mm/memory.c >> >> > @@ -4081,7 +4081,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) >> >> > * reusing the same entry. It's undetectable = as >> >> > * pte_same() returns true due to entry reuse. >> >> > */ >> >> > - if (swapcache_prepare(entry)) { >> >> > + if (swapcache_prepare(entry, 1)) { >> >> > /* Relax a bit to prevent rapid repea= ted page faults */ >> >> > schedule_timeout_uninterruptible(1); >> >> > goto out; >> >> > @@ -4387,7 +4387,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) >> >> > out: >> >> > /* Clear the swap cache pin for direct swapin after PTL unloc= k */ >> >> > if (need_clear_cache) >> >> > - swapcache_clear(si, entry); >> >> > + swapcache_clear(si, entry, 1); >> >> > if (si) >> >> > put_swap_device(si); >> >> > return ret; >> >> > @@ -4403,7 +4403,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) >> >> > folio_put(swapcache); >> >> > } >> >> > if (need_clear_cache) >> >> > - swapcache_clear(si, entry); >> >> > + swapcache_clear(si, entry, 1); >> >> > if (si) >> >> > put_swap_device(si); >> >> > return ret; >> >> > diff --git a/mm/swap.h b/mm/swap.h >> >> > index baa1fa946b34..7c6330561d84 100644 >> >> > --- a/mm/swap.h >> >> > +++ b/mm/swap.h >> >> > @@ -59,7 +59,7 @@ void __delete_from_swap_cache(struct folio *folio, >> >> > void delete_from_swap_cache(struct folio *folio); >> >> > void clear_shadow_from_swap_cache(int type, unsigned long begin, >> >> > unsigned long end); >> >> > -void swapcache_clear(struct swap_info_struct *si, swp_entry_t entr= y); >> >> > +void swapcache_clear(struct swap_info_struct *si, swp_entry_t entr= y, int nr); >> >> > struct folio *swap_cache_get_folio(swp_entry_t entry, >> >> > struct vm_area_struct *vma, unsigned long addr); >> >> > struct folio *filemap_get_incore_folio(struct address_space *mappi= ng, >> >> > @@ -120,7 +120,7 @@ static inline int swap_writepage(struct page *p= , struct writeback_control *wbc) >> >> > return 0; >> >> > } >> >> > >> >> > -static inline void swapcache_clear(struct swap_info_struct *si, sw= p_entry_t entry) >> >> > +static inline void swapcache_clear(struct swap_info_struct *si, sw= p_entry_t entry, int nr) >> >> > { >> >> > } >> >> > >> >> > @@ -172,4 +172,5 @@ static inline unsigned int folio_swap_flags(str= uct folio *folio) >> >> > return 0; >> >> > } >> >> > #endif /* CONFIG_SWAP */ >> >> > + >> >> >> >> NITPICK: Is it necessary to add a blank line here? But I don't think= a >> >> new version is necessary if this is the only change needed. >> > >> > No need to add a blank line; it was probably a mistake I made in Vim. >> > >> >> >> >> > #endif /* _MM_SWAP_H */ >> >> > diff --git a/mm/swap_state.c b/mm/swap_state.c >> >> > index a1726e49a5eb..b06f2a054f5a 100644 >> >> > --- a/mm/swap_state.c >> >> > +++ b/mm/swap_state.c >> >> > @@ -477,7 +477,7 @@ struct folio *__read_swap_cache_async(swp_entry= _t entry, gfp_t gfp_mask, >> >> > /* >> >> > * Swap entry may have been freed since our caller ob= served it. >> >> > */ >> >> > - err =3D swapcache_prepare(entry); >> >> > + err =3D swapcache_prepare(entry, 1); >> >> > if (!err) >> >> > break; >> >> > >> >> > diff --git a/mm/swapfile.c b/mm/swapfile.c >> >> > index 5f73a8553371..757d38a86f56 100644 >> >> > --- a/mm/swapfile.c >> >> > +++ b/mm/swapfile.c >> >> > @@ -3363,7 +3363,7 @@ void si_swapinfo(struct sysinfo *val) >> >> > } >> >> > >> >> > /* >> >> > - * Verify that a swap entry is valid and increment its swap map co= unt. >> >> > + * Verify that nr swap entries are valid and increment their swap = map counts. >> >> > * >> >> > * Returns error code in following case. >> >> > * - success -> 0 >> >> > @@ -3373,60 +3373,77 @@ void si_swapinfo(struct sysinfo *val) >> >> > * - swap-cache reference is requested but the entry is not used. = -> ENOENT >> >> > * - swap-mapped reference requested but needs continued swap coun= t. -> ENOMEM >> >> > */ >> >> > -static int __swap_duplicate(swp_entry_t entry, unsigned char usage) >> >> > +static int __swap_duplicate(swp_entry_t entry, unsigned char usage= , int nr) >> >> > { >> >> > struct swap_info_struct *p; >> >> > struct swap_cluster_info *ci; >> >> > unsigned long offset; >> >> > unsigned char count; >> >> > unsigned char has_cache; >> >> > - int err; >> >> > + int err, i; >> >> > >> >> > p =3D swp_swap_info(entry); >> >> > >> >> > offset =3D swp_offset(entry); >> >> > + VM_WARN_ON(nr > SWAPFILE_CLUSTER - offset % SWAPFILE_CLUSTER); >> >> > ci =3D lock_cluster_or_swap_info(p, offset); >> >> > >> >> > - count =3D p->swap_map[offset]; >> >> > + err =3D 0; >> >> > + for (i =3D 0; i < nr; i++) { >> >> > + count =3D p->swap_map[offset + i]; >> >> > >> >> > - /* >> >> > - * swapin_readahead() doesn't check if a swap entry is valid,= so the >> >> > - * swap entry could be SWAP_MAP_BAD. Check here with lock hel= d. >> >> > - */ >> >> > - if (unlikely(swap_count(count) =3D=3D SWAP_MAP_BAD)) { >> >> > - err =3D -ENOENT; >> >> > - goto unlock_out; >> >> > - } >> >> > + /* >> >> > + * swapin_readahead() doesn't check if a swap entry i= s valid, so the >> >> > + * swap entry could be SWAP_MAP_BAD. Check here with = lock held. >> >> > + */ >> >> > + if (unlikely(swap_count(count) =3D=3D SWAP_MAP_BAD)) { >> >> > + err =3D -ENOENT; >> >> > + goto unlock_out; >> >> > + } >> >> > >> >> > - has_cache =3D count & SWAP_HAS_CACHE; >> >> > - count &=3D ~SWAP_HAS_CACHE; >> >> > - err =3D 0; >> >> > + has_cache =3D count & SWAP_HAS_CACHE; >> >> > + count &=3D ~SWAP_HAS_CACHE; >> >> > >> >> > - if (usage =3D=3D SWAP_HAS_CACHE) { >> >> > + if (usage =3D=3D SWAP_HAS_CACHE) { >> >> > + /* set SWAP_HAS_CACHE if there is no cache an= d entry is used */ >> >> > + if (!has_cache && count) >> >> > + continue; >> >> > + else if (has_cache) /* someone el= se added cache */ >> >> > + err =3D -EEXIST; >> >> > + else /* no users r= emaining */ >> >> > + err =3D -ENOENT; >> >> > >> >> > - /* set SWAP_HAS_CACHE if there is no cache and entry = is used */ >> >> > - if (!has_cache && count) >> >> > - has_cache =3D SWAP_HAS_CACHE; >> >> > - else if (has_cache) /* someone else added= cache */ >> >> > - err =3D -EEXIST; >> >> > - else /* no users remaining= */ >> >> > - err =3D -ENOENT; >> >> > + } else if (count || has_cache) { >> >> > >> >> > - } else if (count || has_cache) { >> >> > + if ((count & ~COUNT_CONTINUED) < SWAP_MAP_MAX) >> >> > + continue; >> >> > + else if ((count & ~COUNT_CONTINUED) > SWAP_MA= P_MAX) >> >> > + err =3D -EINVAL; >> >> > + else if (swap_count_continued(p, offset + i, = count)) >> >> > + continue; >> >> >> >> IIUC, this will make the change to swap map directly instead of >> >> verification. If the verification failed for some entry later, the >> >> count will be wrong? Or I missed something? >> > >> > To avoid using a bitmap or a larger stack, we actually verify during >> > the first iteration. >> > This ensures that by the second iteration, we can safely commit the >> > modification. >> > >> > I actually put some words in the changelog :-) >> > >> > To optimize stack usage, we iterate twice in __swap_duplicate(): the >> > first time to verify that all entries are valid, and the second time >> > to apply the modifications to the entries. >> >> Yes, I have seen it and I think that it is a good strategy. >> >> But, IIUC, swap_count_continued() will change the higher bits of the >> swap_map instead of verifying. Or, my understanding is wrong? >> > > Ying, your understanding is 100% correct. but the code also has nothing > broken. we didn't extend swap_duplicate() to have argument nr, > so all users which can set usage=3D1 will definitely have nr=3D1. > > int swap_duplicate(swp_entry_t entry) > { > int err =3D 0; > > while (!err && __swap_duplicate(entry, 1, 1) =3D=3D -ENOMEM) > err =3D add_swap_count_continuation(entry, GFP_ATOMIC); > return err; > } I understand that we don't have requirements to support "usage =3D=3D 1 && nr > 1" case for __swap_duplicate() at least for now. > Maybe I can add a VM_WARN_ON to warn those people who might > want to extend swap_duplicate()? in that case, things could be quite > tricky. > > --- a/mm/swapfile.c > +++ b/mm/swapfile.c > @@ -3386,6 +3386,7 @@ static int __swap_duplicate(swp_entry_t entry, > unsigned char usage, int nr) > > offset =3D swp_offset(entry); > VM_WARN_ON(nr > SWAPFILE_CLUSTER - offset % SWAPFILE_CLUSTER); > + VM_WARN_ON(usage =3D=3D 1 && nr > 1); > ci =3D lock_cluster_or_swap_info(p, offset); > > err =3D 0; Please add this. And, I think that we need to make it explicit in patch description and comments to avoid potential confusing. And, because it's hard to implement the verify and change strategy if "usage =3D=3D 1". Can we only use that strategy for "usage =3D=3D SWAP_HAS_CACHE"? -- Best Regards, Huang, Ying >> >> >> >> > + else >> >> > + err =3D -ENOMEM; >> >> > + } else >> >> > + err =3D -ENOENT; /* unused s= wap entry */ >> >> > >> >> > - if ((count & ~COUNT_CONTINUED) < SWAP_MAP_MAX) >> >> > + if (err) >> >> > + goto unlock_out; >> >> > + } >> >> > + >> >> > + for (i =3D 0; i < nr; i++) { >> >> > + count =3D p->swap_map[offset + i]; >> >> > + has_cache =3D count & SWAP_HAS_CACHE; >> >> > + count &=3D ~SWAP_HAS_CACHE; >> >> > + >> >> > + if (usage =3D=3D SWAP_HAS_CACHE) >> >> > + has_cache =3D SWAP_HAS_CACHE; >> >> > + else if ((count & ~COUNT_CONTINUED) < SWAP_MAP_MAX) >> >> > count +=3D usage; >> >> > - else if ((count & ~COUNT_CONTINUED) > SWAP_MAP_MAX) >> >> > - err =3D -EINVAL; >> >> > - else if (swap_count_continued(p, offset, count)) >> >> > - count =3D COUNT_CONTINUED; >> >> > else >> >> > - err =3D -ENOMEM; >> >> > - } else >> >> > - err =3D -ENOENT; /* unused swap entr= y */ >> >> > + count =3D COUNT_CONTINUED; >> >> > >> >> > - if (!err) >> >> > - WRITE_ONCE(p->swap_map[offset], count | has_cache); >> >> > + WRITE_ONCE(p->swap_map[offset + i], count | has_cache= ); >> >> > + } >> >> > >> >> > unlock_out: >> >> > unlock_cluster_or_swap_info(p, ci); >> >> > @@ -3439,7 +3456,7 @@ static int __swap_duplicate(swp_entry_t entry= , unsigned char usage) >> >> > */ >> >> > void swap_shmem_alloc(swp_entry_t entry) >> >> > { >> >> > - __swap_duplicate(entry, SWAP_MAP_SHMEM); >> >> > + __swap_duplicate(entry, SWAP_MAP_SHMEM, 1); >> >> > } >> >> > >> >> > /* >> >> > @@ -3453,29 +3470,29 @@ int swap_duplicate(swp_entry_t entry) >> >> > { >> >> > int err =3D 0; >> >> > >> >> > - while (!err && __swap_duplicate(entry, 1) =3D=3D -ENOMEM) >> >> > + while (!err && __swap_duplicate(entry, 1, 1) =3D=3D -ENOMEM) >> >> > err =3D add_swap_count_continuation(entry, GFP_ATOMIC= ); >> >> > return err; >> >> > } >> >> > >> >> > /* >> >> > - * @entry: swap entry for which we allocate swap cache. >> >> > + * @entry: first swap entry from which we allocate nr swap cache. >> >> > * >> >> > - * Called when allocating swap cache for existing swap entry, >> >> > + * Called when allocating swap cache for existing swap entries, >> >> > * This can return error codes. Returns 0 at success. >> >> > * -EEXIST means there is a swap cache. >> >> > * Note: return code is different from swap_duplicate(). >> >> > */ >> >> > -int swapcache_prepare(swp_entry_t entry) >> >> > +int swapcache_prepare(swp_entry_t entry, int nr) >> >> > { >> >> > - return __swap_duplicate(entry, SWAP_HAS_CACHE); >> >> > + return __swap_duplicate(entry, SWAP_HAS_CACHE, nr); >> >> > } >> >> > >> >> > -void swapcache_clear(struct swap_info_struct *si, swp_entry_t entr= y) >> >> > +void swapcache_clear(struct swap_info_struct *si, swp_entry_t entr= y, int nr) >> >> > { >> >> > unsigned long offset =3D swp_offset(entry); >> >> > >> >> > - cluster_swap_free_nr(si, offset, 1, SWAP_HAS_CACHE); >> >> > + cluster_swap_free_nr(si, offset, nr, SWAP_HAS_CACHE); >> >> > } >> >> > >> >> > struct swap_info_struct *swp_swap_info(swp_entry_t entry) >> >> >> >> -- >> >> Best Regards, >> >> Huang, Ying >> > >> > Thanks >> > Barry