From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3290FC282DE for ; Wed, 5 Mar 2025 18:34:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D408328000E; Wed, 5 Mar 2025 13:32:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CC86E280008; Wed, 5 Mar 2025 13:32:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B684A28000E; Wed, 5 Mar 2025 13:32:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 95339280008 for ; Wed, 5 Mar 2025 13:32:36 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 76CFDC096D for ; Wed, 5 Mar 2025 18:16:35 +0000 (UTC) X-FDA: 83188302750.20.C735080 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf14.hostedemail.com (Postfix) with ESMTP id 9DEE510000F for ; Wed, 5 Mar 2025 18:16:33 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="r8L/NUdf"; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf14.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741198593; a=rsa-sha256; cv=none; b=GWpF7douyHz7afg9UgrWJCDYMuVX9vpGOZmY6RVFK8WOu234vQgQ4RQS0TLVMvijo5BJ5B PUSw60Xj0pXpYrwxJpRaOlaMluLT6vwZYc6qoaHLedWxwUzNIMVZMfDBSipmQW3uzNNZdh j6YUMFvsfdx30AeImx9boXuvSnnW7Vs= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="r8L/NUdf"; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf14.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741198593; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gVPO4Ea1pSCy3YBrrhBI2WcK2UBR56XVCrzmOynZcVk=; b=s+oLbuwhLZd7Ijz3QsbpduI2xuOuZ2ywSZQePJbIGWkOpDXXtQjBj04NbKhcOzN8/SQdnE So7M+P4KVyhAl5wM6dIUaa0HxEetFZZnFuhZ8++Wg5EqdtjllxyTyNfrQVqjL5x1GYIknV zUI+qKXgEhgHMit8v2ACd5mdi7c/QkA= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 156825C69C6; Wed, 5 Mar 2025 18:14:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6302EC4CED1; Wed, 5 Mar 2025 18:16:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741198592; bh=VbOLwUfPS96J+Dn4D6VO7VbIh87bUFOgKgh9tH90XtA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=r8L/NUdfSFARyAX4ToIaUViXGLmoyIt6P5pSeStwqeYkAQ5qsal4uIh28/nnnWeWx BCNP3KLDIJahIo1NSGAj+CfpF7d8ZolvmgY0DtPbCg2fS8a5Jr4bjfaHQLGTS06Bq5 K8SKd17LjNI6OuV8m+WGpk3/duWYX2ANVJpKET0autEmll2LvI72K+80s7Mqi+3zR4 JsjMvJHCTEHS3ViA/E1bI3BUEz9eoCp1fO7kFQxq+HD581fSbgb9kQFooJJabZRMqH mgGf2F2R3P624GAUf9a+ZalbXy3HjYOkzBrTDBfiIOHw1woUG9LbBrJygjfOzltkJX Xd305rSU3wOQQ== From: SeongJae Park To: Cc: SeongJae Park , "Liam R. Howlett" , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Shakeel Butt , Vlastimil Babka , kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 11/16] mm/madvise: let madvise_free_single_vma() caller batches tlb flushes Date: Wed, 5 Mar 2025 10:16:06 -0800 Message-Id: <20250305181611.54484-12-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250305181611.54484-1-sj@kernel.org> References: <20250305181611.54484-1-sj@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 9DEE510000F X-Stat-Signature: j97hcpsm4ub8rr8e873yapuk1u4q4s5w X-HE-Tag: 1741198593-318324 X-HE-Meta: U2FsdGVkX1+VfPsRgbmXFR8pQzrKX9pxhPvWRHv4wNL8RcBfQK2NajSZa5cTkyP6MErry9NXgdCcAPKFHu1OTeSnHSEikm9bPqtFVvMT4XN1smLDPUY1kNPKfueKa2I7osohFY106zDVllov0nkOomivq+kTK3QLVTC4AzcfFnE0ag2CFg6Y7OHiT1GhHDptQiHtex9yE+hWEPC7UUOSXeNcXwQb4b951WmUP02HiO2Wfxk1YNCZG9YfIj15HalCiPQNqosYC3qdrdrOGhJzTkmsF7Ld5bqg/gP1T7LVM3hPcvXmiDSygqrOLuZBE+ijnCaKhaeQ9Q4dJq4m3/5oMiOe+2ng9JZfm8j0OPH2sDI0TFrztvXBESET0hQDypcWx3IpAhV8cS1st9WGX6rlmtPc7N4X0i9b3UWNlz0JxtBCPY23+c0VJXofcE+qsP4AFV0kHWNwpSpFNRzq4xQj+S2YK7vXRTPik4VHFkLv/CYh3hZ/dvJupYOHn6PB2CypdTCzJrgPFJdLcPsYahYFuSGCGKoKvhkbR8GcemRLyxQPVxHnxP4Yvr4Qu+qC8PibP+jsxg6wIRB3WELdKsNmNDZ5tmEW9rSHIuT/PkXPCI4UDoX1YBItbo+bkZkIPiPttlsHQ8NtNEXRcwTp75ed/ilB9QVpUa74gbcQKUGePghNpMjzWLAvQPSd3IJDLkY4nAwpGA7OaT2AV9TNN7r2mV44uDk8a49g7adLhh5PI/UyS6C0ofOboPZKKmR0Jv3eNTcNkVh90AThcPpNnrBB0AWO4PWvYN2PCp/1Vjp9Lwc6M/if96QXVtVbiMV0J8/4eZcQNBx22gFd6fXtYWq/Y4/NNJsQvFfYQH3QFEqSBtECLz3VwLg1xlrfwwpIAgjIiKjBVWNnQMJacIm/qvcR2XInpIpTl/L/D9bz/DBwjTgEn+VPGYWQX7sarv4nvIJqXXf50lq7Ae7/I+uQW9l 1f1s3CQi 4bzI4G0Uzv/a1XJ3LlOLpu+wI4BTOeuXJ8RoNZJs8z1cnC+ql6+JLgUMxbGGwa7YRj2UgiLE1MBBuy0Y8fcM/s9pebJR60jPjQvQ0eef5LwPARUCpYWIBA6EbRfg5O4fFEfpniTyAYcLvlqk/cHeAY02dSEGMTI65M/Ld6vCR6b2BS3ZZdBPjVp5saKuRyOQNEDrQF8uK2FAQUj3xUiSe0O4xJMqcLB4igp+ATZED8yxE0scBH/nahIGbfe64gtOxTQl8Ge0b6pbVqyy2f1SxSafV9A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Update madvise_free_single_vma() function so that the caller can pass an mmu_gather object that should be initialized and will be finished outside, for batched tlb flushes. Also modify madvise_free_single_vma() internal code to support such usage while keeping support of olde usage that the mmu_gather object is not passed. Signed-off-by: SeongJae Park --- mm/madvise.c | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 22da6699613c..767c5d21ee75 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -794,12 +794,19 @@ static const struct mm_walk_ops madvise_free_walk_ops = { .walk_lock = PGWALK_RDLOCK, }; -static int madvise_free_single_vma(struct vm_area_struct *vma, - unsigned long start_addr, unsigned long end_addr) +static int madvise_free_single_vma( + struct mmu_gather *caller_tlb, struct vm_area_struct *vma, + unsigned long start_addr, unsigned long end_addr) { struct mm_struct *mm = vma->vm_mm; struct mmu_notifier_range range; - struct mmu_gather tlb; + struct mmu_gather self_tlb; + struct mmu_gather *tlb; + + if (caller_tlb) + tlb = caller_tlb; + else + tlb = &self_tlb; /* MADV_FREE works for only anon vma at the moment */ if (!vma_is_anonymous(vma)) @@ -815,16 +822,18 @@ static int madvise_free_single_vma(struct vm_area_struct *vma, range.start, range.end); lru_add_drain(); - tlb_gather_mmu(&tlb, mm); + if (!caller_tlb) + tlb_gather_mmu(tlb, mm); update_hiwater_rss(mm); mmu_notifier_invalidate_range_start(&range); - tlb_start_vma(&tlb, vma); + tlb_start_vma(tlb, vma); walk_page_range(vma->vm_mm, range.start, range.end, - &madvise_free_walk_ops, &tlb); - tlb_end_vma(&tlb, vma); + &madvise_free_walk_ops, tlb); + tlb_end_vma(tlb, vma); mmu_notifier_invalidate_range_end(&range); - tlb_finish_mmu(&tlb); + if (!caller_tlb) + tlb_finish_mmu(tlb); return 0; } @@ -957,7 +966,7 @@ static long madvise_dontneed_free(struct vm_area_struct *vma, if (behavior == MADV_DONTNEED || behavior == MADV_DONTNEED_LOCKED) return madvise_dontneed_single_vma(NULL, vma, start, end); else if (behavior == MADV_FREE) - return madvise_free_single_vma(vma, start, end); + return madvise_free_single_vma(NULL, vma, start, end); else return -EINVAL; } -- 2.39.5