From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB75AC021B1 for ; Thu, 20 Feb 2025 12:48:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4B3A42802E1; Thu, 20 Feb 2025 07:48:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 463762802DA; Thu, 20 Feb 2025 07:48:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 32B6E2802E1; Thu, 20 Feb 2025 07:48:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 1541F2802DA for ; Thu, 20 Feb 2025 07:48:58 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 8894750CE6 for ; Thu, 20 Feb 2025 12:48:57 +0000 (UTC) X-FDA: 83140302714.27.D7A4922 Received: from mail-yw1-f177.google.com (mail-yw1-f177.google.com [209.85.128.177]) by imf01.hostedemail.com (Postfix) with ESMTP id C7BBA40008 for ; Thu, 20 Feb 2025 12:48:55 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=davidfrank-ch.20230601.gappssmtp.com header.s=20230601 header.b=EjIT9ZkL; spf=none (imf01.hostedemail.com: domain of david@davidfrank.ch has no SPF policy when checking 209.85.128.177) smtp.mailfrom=david@davidfrank.ch; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740055735; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=nKxSyhj0r8L4FxdqwgDIGIROdko5JdS3mVV6x3uzZL0=; b=P7Z65yk908wa05XaSUp6nrNOS4d6ogmUpIqmBi6zppb2S6WW6ZpEqIn4uO1M/CA0YCQLc+ cu5J67ON/5hcXI2DP0I3z5IirOizXWUYknIfdekoP+q61wm8Tac6qQMkL0fn1n3RxopRc/ IbCkOCjuU59OAPw3O0C1CUrZumrritA= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=davidfrank-ch.20230601.gappssmtp.com header.s=20230601 header.b=EjIT9ZkL; spf=none (imf01.hostedemail.com: domain of david@davidfrank.ch has no SPF policy when checking 209.85.128.177) smtp.mailfrom=david@davidfrank.ch; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740055735; a=rsa-sha256; cv=none; b=ylQvaPKkbc9aihmFeMq9NmDKxlpYfPWRqO+JhKSRSASChhngQPxd58uPmtbns1blzOsg4L GfUsXxN496ZjyEpc6y0RbJvF49eJaEAbAsdeEIoIQloftM4+iI+HzGdgp5yRV2KrOl+ApZ ekWcCSNKXoM5s6t+62IZoZ/mjym7/r4= Received: by mail-yw1-f177.google.com with SMTP id 00721157ae682-6efe4324f96so7778797b3.1 for ; Thu, 20 Feb 2025 04:48:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=davidfrank-ch.20230601.gappssmtp.com; s=20230601; t=1740055734; x=1740660534; darn=kvack.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=nKxSyhj0r8L4FxdqwgDIGIROdko5JdS3mVV6x3uzZL0=; b=EjIT9ZkLXcPsUYe4Aoir9R4h4BLGBnpXcG3RzwIc2vmgqPVpDfvufQ01UldeeQDPmw /bWKfKfeCkRp2NH6LaJNlFsQWsDzRJn1mCzp9PqKGGZEX9Zom8gHW4W9ZZNwZeSzfHpc 9pwaNAdxf6joRBezXQAeRQaZcTc6tVxuShAV9R14Hl3VjHF44CBD5vfgIS+aCsQUvuvm abbKKk6WfSLOL7IsqPEIEtj8lzTSTxluLRZjEqy9PLVnKKkcCwhocOs/JgB9ym412EiF vKXrK3f1enwFrTVDJ3DOSKLQ+3I3VElX0sFBtSbFrMHmIPHiHteEIAn2Gn8Qkbh+TNnR D8Nw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740055734; x=1740660534; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=nKxSyhj0r8L4FxdqwgDIGIROdko5JdS3mVV6x3uzZL0=; b=MI9cmmAqMXIFOHfublNT3OpHRl2p0D+fmM771DWBrr/pAow09BFSzoVJzgsisKIfY7 r26sJvrMtCF1QH8KzoSCSN9PaVN+oEgJwQXTlWu5aeWHg7mU5KAGKLeyfTyLlrkVhNb+ eSWSFU21Sp3LfBOaVQDVLc+aQvAHA7foeH2yN9JPLD8MBI2s0nqZV6V3NeuhDxipX8Ct aqSbEeZsjKxk65zkjvY7HNqKSPkB8ghd0DgxzdyPDmOhDwlx9vs/wTWRAz/EflBreFWz 6jf+DkSLNm9cMklNO+36ROj1HuZTpGwXt1VJEYvqdZzl1KBK/cincMADSt4oaRLATPTO pVug== X-Gm-Message-State: AOJu0YxlY84Z1/tHg/pW9IlsKTg+JScVH9jCDckf+T5IQs/jl5LzKJ7Y /Ya2l3hJicNxOY+fN7ei3qguAcb9qruF+fzsm2Qz1a7+0tT11WzPbucSos/hW1ywK5l94U+OLLl ILrZLzPW6e3uaThAsGxcIG4g5QYKUXYmzBn6Rr1Ik7/qoui1FwqVeSA== X-Gm-Gg: ASbGnct/W7LEUsyNkhzPo+fXSiHGmDcXE3cEQHTy3KqOunBwm3vGg8klxeVBJhLw1fD 0RSJITDR723JwsWMIMS9F8nmpVewiRWN5aBO7sD2LF3bmA3qdtOwgxvQ00Egdhq7uC3DobeWPNA == X-Google-Smtp-Source: AGHT+IF6sUhg1H60bkwB0j9K+QiZP9UeP8lDnzI6ksqOJ8re/sVGcfRrNu3l/MFgaYNwGJuLYWq4mrxuyRD/VEOq35M= X-Received: by 2002:a05:690c:6809:b0:6f9:a3c6:b2dc with SMTP id 00721157ae682-6fbbb5f412fmr19353557b3.2.1740055734700; Thu, 20 Feb 2025 04:48:54 -0800 (PST) MIME-Version: 1.0 From: David Frank Date: Thu, 20 Feb 2025 13:48:18 +0100 X-Gm-Features: AWEUYZmQ8373J4A9x9Xcisreel_DSOg7hz5xJHPEonORDnqod_vzotma0fdIevA Message-ID: Subject: Efficient mapping of sparse file holes to zero-pages To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: C7BBA40008 X-Stat-Signature: 6rsoeqtbuihup4sqprn839xgr7rg4jxq X-HE-Tag: 1740055735-437085 X-HE-Meta: U2FsdGVkX18TywkAbdIdISwvcPQ+9EpUVIiDa+mu6BLfx36WORycyjhc7ESxyR3pgRS9xqgENSpqCSyXZyc3F/aVtNf3ozWdDDFgJmap9vamXoQGzD5055EAtnV4gUzGnq04ktXFJQFzR2bEOGforEykj5RsDRby4r6qE44W+QbrfTwT5o7SgY9o/cyw+fZgqgyGb8kQzOm0hgSc5VlGl1eC6daxuUJruNWO5gp6XvR6WxPCj3kx7JGeSf6iU8tuh99MKTHm41G7kOI3fxf62erG/T1T/VB0sp4HW6zpcritulWk6203nu+paISckG2qEwGkA2BLnPw5AqbaGZDCs/UHDVv+aNJHtoK7OQ5CKoUFYrZbS5k1LLPerT9ZcPvJsEuYMaaoMRE9rIh1Y4u4NwhFjsw5uCdGtHvrEkicxf+2LTn8KAV/1uhoulD8QB4um5jIwQzwE2HWnW+iNMGu4CfWzqz+LVgk2WdOpTfUar+uvRA48DkwhmemcmORky0u4LAlCWlLtqKkg2ehDDmU0QmC0FuzXQEpr/0bQaXGQJgTtTXqalW5EB+X3JnjSjvt7dAO9H9Axmcfk2udMxcPp4oWMFpOE+/Zbfjv6RxKk718ZCK+DliXQONXJ/aqqVPEaOoP7QBQgksejXrbmNcqJuJ4OjMduF+tehu0f0EA+d92spUCbnAI0NWADyCLUmp+JpraA7BrfFSBenY76z8bHaubBGPH8Qy2y/Hzh5nbUJKtxtbixlsIv7tJC1LTFSvmAB8DmPteX4RyK9EFwOLSg93LSHUy8j7usmQotAm38JTXrOFETKFXOMZXjfvazcnD4xvCZQ2lqyJl5XMbNoan2dpU7VyslcUiwa/R2HhGZSmTs+4zeThtdzE7Qcqc++v5sRsmEvuI5Egx1Gtk/Kj4gKv1x59fVMLyLcap3rWzZy19YHmDNa6onYGcJq/VigIro6qH+1Bgnm54Pjkx5l+ 4TogY6Xs asa+MjkNPE5JbkxFhbYneK5yWK9bhNh54vhIvk5rML4rCK8785KdEzj4BGCfJlhlAqO09bpo4sslywrwjGsrPq+mzpzXxY/DOwRfYCnglUIgb6mpZidU/Vndk24vNa3cLuERCg+KASJPuZibenOKbPM6cvDDADCKk6KZingNqgwHxtmKVYM6mJbvIxYCCYOKdrdCdXizxif37+p89JXtunGe6s/U0w/qbd4h2IDfSTQWNH64USHEuEN7ld6fZj+gc/FVeiR4aV1ORGP8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.063032, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi all, I'd like to efficiently mmap a large sparse file (ext4), 95% of which is holes. I was unsatisfied with the performance and after profiling, I found that most of the time is spent in filemap_add_folio and filemap_alloc_folio - much more than in my algorithm: - 97.87% filemap_fault - 97.57% do_sync_mmap_readahead - page_cache_ra_order - 97.28% page_cache_ra_unbounded - 40.80% filemap_add_folio + 21.93% __filemap_add_folio + 8.88% folio_add_lru + 7.56% workingset_refault + 28.73% filemap_alloc_folio + 22.34% read_pages + 3.29% xa_load As a workaround, I started using lseek and SEEK_HOLE+SEEK_DATA and changed the algorithm to use a static array filled with zeros instead of reading from the holes. This works ~30x faster, however, it introduces substantial complexity in the implementation. I was wondering if mapping holes to zero pages with COW in the kernel is being considered. I found [a related thread][1] from early 2022 which mentions mapping to zero pages for shared memory objects. There seemed to be some concerns about the complexity, I wonder if it's different for (even just private/readonly) mmap. [1]: https://lore.kernel.org/lkml/4b1885b8-eb95-c50-2965-11e7c8efbf36@google.com/T/ Thanks, David