From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3978EB64DA for ; Tue, 18 Jul 2023 10:03:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 211C16B0071; Tue, 18 Jul 2023 06:03:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1C22A8D0002; Tue, 18 Jul 2023 06:03:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B17C8D0001; Tue, 18 Jul 2023 06:03:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id EF4996B0071 for ; Tue, 18 Jul 2023 06:03:03 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id BFEB11602C1 for ; Tue, 18 Jul 2023 10:03:03 +0000 (UTC) X-FDA: 81024294246.06.09E85EC Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf18.hostedemail.com (Postfix) with ESMTP id C93C41C0028 for ; Tue, 18 Jul 2023 10:03:01 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689674582; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aRIQ7OVKgfbastfxdPweHEo6F1mpakvLoTUMSjyTlN0=; b=DaQvg83XsnLLUABeOfyxVvjIBmzhi4UvN7Ie8Z0hwgNzCUSEKQLbxjsXfwKe6cQMMxJ9Eq mTjCW+YEYHT+LOQxnXIUrni4yPCEDClCgzxMMvLHL+1c7FcZeA+quCHpZK3Mp+L/bZQw5d EMEhIgXZOQw4wfWYeKBgpzUZFfsjOHY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689674582; a=rsa-sha256; cv=none; b=iWammFv5XxzVQwbMB7f43IC+Nk/eS1kk+snVFqoADfJiTdxA0YEXNP37u8T3WmR+4buSBH sUnGrgk0EDGQ/cKbXFdiQPQFwvN5ePMgSAby3z6AjiQJkwXLVFMXOPuZzaoJPOwe6ndqqt v4qVhN5ecZMekLDS5t8B3TgFABDc2TY= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 11EF32F4; Tue, 18 Jul 2023 03:03:44 -0700 (PDT) Received: from [10.1.34.52] (C02Z41KALVDN.cambridge.arm.com [10.1.34.52]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 306493F67D; Tue, 18 Jul 2023 03:02:59 -0700 (PDT) Message-ID: <1aada499-4bb3-668c-10d0-06e0845efca1@arm.com> Date: Tue, 18 Jul 2023 11:02:57 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: [PATCH v1 2/3] mm: Implement folio_remove_rmap_range() To: "Huang, Ying" Cc: Andrew Morton , Matthew Wilcox , Yin Fengwei , David Hildenbrand , Yu Zhao , Yang Shi , Zi Yan , linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20230717143110.260162-1-ryan.roberts@arm.com> <20230717143110.260162-3-ryan.roberts@arm.com> <87zg3tbsn0.fsf@yhuang6-desk2.ccr.corp.intel.com> From: Ryan Roberts In-Reply-To: <87zg3tbsn0.fsf@yhuang6-desk2.ccr.corp.intel.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: C93C41C0028 X-Rspam-User: X-Stat-Signature: nq8zcijwe3c7sxwbkmg5qz3w6h471wk7 X-Rspamd-Server: rspam03 X-HE-Tag: 1689674581-459887 X-HE-Meta: U2FsdGVkX1+ut4Nj+kgX5lmf/vXo0c8WWxsCI7pW67FN6zvckDpQeAoPHel1HH2woE34QetEP6vuCsEf1mDcxUWcfe5X8+wUdJY/dbaaJGUG+4oywJbcRkUmEpL0w23t8y49mmoBQYsuBoyoOLra3zyqrPML1QRlzgsqWbfYcjDgrrd/ciicD9/G4eSib6IF7XQHIG8gTBnTxVTbkWcxqJebz8dYG/8Z48Q/4OHCcjoGYPZx3NJnroY3+idQTBp9XOXU9xxsH7m8LWPQEEnEY+ldmfuadUVRyy26L9fDl2YrVme4qnCuvZe19dlML8c1eOJHNlVXhsXSpGhgpS8MDibwBjaFqHYf1SZeO+lqDsd7Lgu/1NA/wjol+vydIhQfwuE0rtvY65v1gRlRWuu6Lbh7Pd/OHqstyb2Uy8IIUfG0EUXRf4jBO6Co/ucs/MPHypvF8SW49qz1Vk7mLv0pnxms/9MSQMBcrdyiycBRiRfSnUf2yOJeZg2fdMm8xlk18JayATJAFaN/67T5FDljxs0DfUbkhMPUN4ZTQOasd2kLvjuOmLWIpSlJtplQG7twra0HGgj0P2HSiYUkeYZfmV9d+BefD9mnHALbHWrJi32OMQKf6sepik3DmAwvmSLZopv7mQHf/w3aEpQir/Z9LciDH5ayxXy/32HZEjwdtufuTtyZWVMrj48rYk1tZwp4ynB9OECqarmZXfi8U2Lzx8b2H3/glFQj8QyH2h1WUvHTAO3NSSnNuSd1SMSXfCQZpUUbmHVsb0bzO75xUDvqE/dSmNLSh0Prhk+tPG0gQlIfQ6WqRVvdOvPPGPG8oP6yQsiHH/eB/hEPcbMTbxJtfgd5SHxnYl6YOvGI6XQyQy7ZRkgR1zy0Fif+H5WOOOkpB3t3DOvLQGiVsIXjdLl5yHsxF9X63Ja834y1fx0vPttcC5Jo9x8b3D7fo2Sqg72sDDyOGzSvSETtAGpkgEd oDQwn8Tw 0v4sLEZ9UDKJMbohhBolWZEihqu0Qc4JZ/7jC/npodGycAq+notcK3B0qEVZ+NGqMsBeSYH6/cR8X4R/KsUsfZyszUFznaARrV1sy3d/gURo7sNGjsZrDByyilr+E4ygreF0j/D6dAxKJEGVg5V3wq0+xt4k7bsJRLliwiq05za9jp8rO6qAyaqij7eMkFNDWQsrxAw6w2SvQrKHYhLWdLyjgFQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 18/07/2023 08:12, Huang, Ying wrote: > Ryan Roberts writes: > >> Like page_remove_rmap() but batch-removes the rmap for a range of pages >> belonging to a folio. This can provide a small speedup due to less >> manipuation of the various counters. But more crucially, if removing the >> rmap for all pages of a folio in a batch, there is no need to >> (spuriously) add it to the deferred split list, which saves significant >> cost when there is contention for the split queue lock. >> >> All contained pages are accounted using the order-0 folio (or base page) >> scheme. >> >> Signed-off-by: Ryan Roberts >> --- >> include/linux/rmap.h | 2 ++ >> mm/rmap.c | 65 ++++++++++++++++++++++++++++++++++++++++++++ >> 2 files changed, 67 insertions(+) >> >> diff --git a/include/linux/rmap.h b/include/linux/rmap.h >> index b87d01660412..f578975c12c0 100644 >> --- a/include/linux/rmap.h >> +++ b/include/linux/rmap.h >> @@ -200,6 +200,8 @@ void page_add_file_rmap(struct page *, struct vm_area_struct *, >> bool compound); >> void page_remove_rmap(struct page *, struct vm_area_struct *, >> bool compound); >> +void folio_remove_rmap_range(struct folio *folio, struct page *page, >> + int nr, struct vm_area_struct *vma); >> >> void hugepage_add_anon_rmap(struct page *, struct vm_area_struct *, >> unsigned long address, rmap_t flags); >> diff --git a/mm/rmap.c b/mm/rmap.c >> index 2baf57d65c23..1da05aca2bb1 100644 >> --- a/mm/rmap.c >> +++ b/mm/rmap.c >> @@ -1359,6 +1359,71 @@ void page_add_file_rmap(struct page *page, struct vm_area_struct *vma, >> mlock_vma_folio(folio, vma, compound); >> } >> >> +/* >> + * folio_remove_rmap_range - take down pte mappings from a range of pages >> + * belonging to a folio. All pages are accounted as small pages. >> + * @folio: folio that all pages belong to >> + * @page: first page in range to remove mapping from >> + * @nr: number of pages in range to remove mapping from >> + * @vma: the vm area from which the mapping is removed >> + * >> + * The caller needs to hold the pte lock. >> + */ >> +void folio_remove_rmap_range(struct folio *folio, struct page *page, >> + int nr, struct vm_area_struct *vma) > > Can we call folio_remove_ramp_range() in page_remove_rmap() if > !compound? This can give us some opportunities to reduce code > duplication? I considered that, but if felt like the savings were pretty small so my opinion was that it was cleaner not to do this. This is the best I came up with. Perhaps you can see further improvements? void page_remove_rmap(struct page *page, struct vm_area_struct *vma, bool compound) { struct folio *folio = page_folio(page); atomic_t *mapped = &folio->_nr_pages_mapped; int nr = 0, nr_pmdmapped = 0; bool last; enum node_stat_item idx; VM_BUG_ON_PAGE(compound && !PageHead(page), page); /* Hugetlb pages are not counted in NR_*MAPPED */ if (unlikely(folio_test_hugetlb(folio))) { /* hugetlb pages are always mapped with pmds */ atomic_dec(&folio->_entire_mapcount); return; } /* Is page being unmapped by PTE? Is this its last map to be removed? */ if (likely(!compound)) { folio_remove_rmap_range(folio, page, 1, vma); return; } else if (folio_test_pmd_mappable(folio)) { /* That test is redundant: it's for safety or to optimize out */ last = atomic_add_negative(-1, &folio->_entire_mapcount); if (last) { nr = atomic_sub_return_relaxed(COMPOUND_MAPPED, mapped); if (likely(nr < COMPOUND_MAPPED)) { nr_pmdmapped = folio_nr_pages(folio); nr = nr_pmdmapped - (nr & FOLIO_PAGES_MAPPED); /* Raced ahead of another remove and an add? */ if (unlikely(nr < 0)) nr = 0; } else { /* An add of COMPOUND_MAPPED raced ahead */ nr = 0; } } } if (nr_pmdmapped) { if (folio_test_anon(folio)) idx = NR_ANON_THPS; else if (folio_test_swapbacked(folio)) idx = NR_SHMEM_PMDMAPPED; else idx = NR_FILE_PMDMAPPED; __lruvec_stat_mod_folio(folio, idx, -nr_pmdmapped); } if (nr) { idx = folio_test_anon(folio) ? NR_ANON_MAPPED : NR_FILE_MAPPED; __lruvec_stat_mod_folio(folio, idx, -nr); /* * Queue anon THP for deferred split if at least one * page of the folio is unmapped and at least one page * is still mapped. */ if (folio_test_anon(folio) && nr < nr_pmdmapped) deferred_split_folio(folio); } /* * It would be tidy to reset folio_test_anon mapping when fully * unmapped, but that might overwrite a racing page_add_anon_rmap * which increments mapcount after us but sets mapping before us: * so leave the reset to free_pages_prepare, and remember that * it's only reliable while mapped. */ munlock_vma_folio(folio, vma, compound); } > > Best Regards, > Huang, Ying >