From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69AC6C5B543 for ; Tue, 10 Jun 2025 03:50:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7D6BF6B007B; Mon, 9 Jun 2025 23:50:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7AE2C6B0089; Mon, 9 Jun 2025 23:50:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6EC856B008A; Mon, 9 Jun 2025 23:50:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 50CE56B007B for ; Mon, 9 Jun 2025 23:50:58 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C741DBE45D for ; Tue, 10 Jun 2025 03:50:57 +0000 (UTC) X-FDA: 83538114954.18.6B6DEDB Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf09.hostedemail.com (Postfix) with ESMTP id DD56F140004 for ; Tue, 10 Jun 2025 03:50:55 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; spf=pass (imf09.hostedemail.com: domain of dev.jain@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=dev.jain@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1749527456; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references; bh=I2kafO90dek3WklMCvFfObb6TvxXxb4Rkv2h5Sb8rfk=; b=Hc7oGQ6vRydd56PAvXcrMqR14SeHpF12sdCD7vHg+HZpGnSJy6UH8eTI7jnaLRQdslOWOg CNpcyXapshVN09/tfy4cdMtG5m22u7xuprKsVAJl7iMMVzYmpsouFES1+UG6+K8pJKNGGX fGULV+Ei2KteZv190bKQxBDjCf29J/E= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; spf=pass (imf09.hostedemail.com: domain of dev.jain@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=dev.jain@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1749527456; a=rsa-sha256; cv=none; b=DgGcJCPMy3K8iLqdSC77Io83dRrMdTT+UXesKml+CPbfIT30bizjujEKSo28fgh46hCoBW 3OsV7jaEoilL84KhTa8CnHKFs2tnTi4kmXnOBeUiqyVXcWbS+m5ztJuIdvZnDNksQ80JfS G6PvLvWhOyhhi7+dXKvw7emZtq3Mqj0= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9466F14BF; Mon, 9 Jun 2025 20:50:35 -0700 (PDT) Received: from MacBook-Pro.blr.arm.com (MacBook-Pro.blr.arm.com [10.164.18.48]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id F15773F59E; Mon, 9 Jun 2025 20:50:47 -0700 (PDT) From: Dev Jain To: akpm@linux-foundation.org Cc: Liam.Howlett@oracle.com, lorenzo.stoakes@oracle.com, vbabka@suse.cz, jannh@google.com, pfalcato@suse.de, linux-mm@kvack.org, linux-kernel@vger.kernel.org, david@redhat.com, peterx@redhat.com, ryan.roberts@arm.com, mingo@kernel.org, libang.li@antgroup.com, maobibo@loongson.cn, zhengqi.arch@bytedance.com, baohua@kernel.org, anshuman.khandual@arm.com, willy@infradead.org, ioworker0@gmail.com, yang@os.amperecomputing.com, baolin.wang@linux.alibaba.com, ziy@nvidia.com, hughd@google.com, Dev Jain Subject: [PATCH v4 0/2] Optimize mremap() for large folios Date: Tue, 10 Jun 2025 09:20:41 +0530 Message-Id: <20250610035043.75448-1-dev.jain@arm.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: DD56F140004 X-Stat-Signature: 9rf37ty6eupfsatiwdgmm4mjebyeprrw X-Rspam-User: X-HE-Tag: 1749527455-852774 X-HE-Meta: U2FsdGVkX18g9Op7hO8vvscmNLo9v6D3gkgi3ygka2RdtCSzoaudJLyxIwZdSo21hubBZCyLXl8LbqB1u18biagGI/RGCzMePLPjsLq3/BvCIw6LS5VBW3bsSq9uEeFePlZs+Mf/0bp5jqMdbIuFFqWQXpRd/AqWmpu5yBYy1mMgHalD/84i/YDJE/e0NYjr66hk7kYpHybaje7TIECvKaIKgAevs1QagrngPm6Pjen/AT1zx0O3h7jJJTQynXeUR32pZeUHx7v/fu59b3ey00qQ8ZYC3Op+ySZ1XdAihrlLhikpdPyVUHa6V5+8d2Aj/6b0Ch5dyRc62zYCWGEc0sTmH1ezt7NhhbgSPgNrUQK7jMd5sfc4LmaQxL98zpAFd8XyluxBnGk4PGtdGGUVpZIs7cmMsgCJKptUh+TfqRr5Xnzrlvx2dS5IDbe33Jlr7kjqXHWPBp3uZ97gwXKcs3X/7nKyQILWcwQfQxfTm3fOg1nwdLdV4j65usX66pFbon3g/0s4V9JYZk2nr2Pa+HYHwsZuz5RqcZgtidDZLnA8CIjRcsrWZpZUe+AH9RaGLPJrQsPTLCYyYWsYpCDuKEjgQ8z6H4d8n7hWv8LYoeMIhKlDZKR98dXFjhrc/4+ulurNXk2FcHFex7dOxGH86ZSv9LggUAY6D2msrIGv60yBH56+csz9MIEP/SXz+EDL29DImbs9Gfz13VtYvFZ8MoEP01IDiHHHA0E9sd9s1zEAptNem15hxfAEglpHoexzAr8zHCFPgBM9AaaShHZk//VPapi12iMi7GzvFasxUtNr5TihzRR3HfUWJ8IPJZ1e2faTavTHM368P0uo+luE5xUNIj9xNPUKe3IKzxk++uFaBKi2Xtg9BvGLVOjjSqWPNtzJ3N0BvyJC8QQnGKYfpJlew4GpJ6ovqcBhkeaJqN0pK4uPMzWeCNd76el3hNWXDgO5wG8FFjsTQe1Zx1x XNMoWMQC szAdkkqphzNZF7RQLgQ0E5Q15sp4eCeRKE/rq6y9bpf9+7REG3H1FJpAL9d3FXvQ81TXk4I8Mr/k0Oov+wNBDFLrSw/MyLewaAse1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently move_ptes() iterates through ptes one by one. If the underlying folio mapped by the ptes is large, we can process those ptes in a batch using folio_pte_batch(), thus clearing and setting the PTEs in one go. For arm64 specifically, this results in a 16x reduction in the number of ptep_get() calls (since on a contig block, ptep_get() on arm64 will iterate through all 16 entries to collect a/d bits), and we also elide extra TLBIs through get_and_clear_full_ptes, replacing ptep_get_and_clear. Mapping 1M of memory with 64K folios, memsetting it, remapping it to src + 1M, and munmapping it 10,000 times, the average execution time reduces from 1.9 to 1.2 seconds, giving a 37% performance optimization, on Apple M3 (arm64). No regression is observed for small folios. The patchset is based on mm-unstable (6ebffe676fcf). Test program for reference: #define _GNU_SOURCE #include #include #include #include #include #include #define SIZE (1UL << 20) // 1M int main(void) { void *new_addr, *addr; for (int i = 0; i < 10000; ++i) { addr = mmap((void *)(1UL << 30), SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); if (addr == MAP_FAILED) { perror("mmap"); return 1; } memset(addr, 0xAA, SIZE); new_addr = mremap(addr, SIZE, SIZE, MREMAP_MAYMOVE | MREMAP_FIXED, addr + SIZE); if (new_addr != (addr + SIZE)) { perror("mremap"); return 1; } munmap(new_addr, SIZE); } } v3->v4: - Remove comment above mremap_folio_pte_batch, improve patch description differentiating between folio splitting and pagetable splitting v2->v3: - Refactor mremap_folio_pte_batch, drop maybe_contiguous_pte_pfns, fix indentation (Lorenzo), fix cover letter description (512K -> 1M) v1->v2: - Expand patch descriptions, move pte declarations to a new line, reduce indentation in patch 2 by introducing mremap_folio_pte_batch(), fix loop iteration (Lorenzo) - Merge patch 2 and 3 (Anshuman, Lorenzo) - Fix maybe_contiguous_pte_pfns (Willy) Dev Jain (2): mm: Call pointers to ptes as ptep mm: Optimize mremap() by PTE batching mm/mremap.c | 58 ++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 42 insertions(+), 16 deletions(-) -- 2.30.2