From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBAEEC54E65 for ; Thu, 22 May 2025 09:03:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7AAFC6B0088; Thu, 22 May 2025 05:03:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 782F36B008A; Thu, 22 May 2025 05:03:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6C0116B008C; Thu, 22 May 2025 05:03:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 503AF6B0088 for ; Thu, 22 May 2025 05:03:17 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7942F80B8A for ; Thu, 22 May 2025 09:03:16 +0000 (UTC) X-FDA: 83469954792.29.82A3117 Received: from mout-p-101.mailbox.org (mout-p-101.mailbox.org [80.241.56.151]) by imf04.hostedemail.com (Postfix) with ESMTP id D5BD940003 for ; Thu, 22 May 2025 09:03:14 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; spf=pass (imf04.hostedemail.com: domain of kernel@pankajraghav.com designates 80.241.56.151 as permitted sender) smtp.mailfrom=kernel@pankajraghav.com; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=samsung.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1747904595; a=rsa-sha256; cv=none; b=W4a6Tuv3+VpubNnBgIFx82nyDsj8g4zy/JD0LLN4SmyljhfPBcfCPHta9YXhB9XvYJMu0N zWr4WVn0zBPPA4nyQJP35gaMxt5AglLhbLJdJiAbOYDS4q6hcV6X7c9rPpyaE3HqpVCd5t ssPAMg+H/5Lw4+Oa90N62Sco3unyMq8= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; spf=pass (imf04.hostedemail.com: domain of kernel@pankajraghav.com designates 80.241.56.151 as permitted sender) smtp.mailfrom=kernel@pankajraghav.com; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=samsung.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1747904595; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rVvOoSFrJ0YG0+xn36+G/pwA+3Gp7lVXpESnUvCMzBM=; b=eNVzhBSbdi5ljjOIAIooyMMHjQOEdwrXO/Lb0Rp88kb6nbz8r2pOWtWNRCHNT4zfXEonuT TxMp1Gi5PO/6TS8HpKBwRDEeqQcg0GrZ6xMLjbL6RAggGL7HOkWKgacUr4z6BtXJbOKO6l uZGBuGYwAqdIWp2lpxSTkvezya7JnoU= Received: from smtp202.mailbox.org (smtp202.mailbox.org [10.196.197.202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-p-101.mailbox.org (Postfix) with ESMTPS id 4b32Pq2VNhz9t4X; Thu, 22 May 2025 11:03:11 +0200 (CEST) From: Pankaj Raghav To: Suren Baghdasaryan , Vlastimil Babka , Ryan Roberts , Mike Rapoport , Michal Hocko , Thomas Gleixner , Nico Pache , Dev Jain , Baolin Wang , Borislav Petkov , Ingo Molnar , "H . Peter Anvin" , Zi Yan , Dave Hansen , David Hildenbrand , Lorenzo Stoakes , Andrew Morton , "Liam R . Howlett" , Jens Axboe Cc: linux-block@vger.kernel.org, linux-fsdevel@vger.kernel.org, "Darrick J . Wong" , gost.dev@samsung.com, kernel@pankajraghav.com, hch@lst.de, linux-kernel@vger.kernel.org, linux-mm@kvack.org, willy@infradead.org, x86@kernel.org, mcgrof@kernel.org, Pankaj Raghav Subject: [RFC v2 2/2] block: use mm_huge_zero_folio in __blkdev_issue_zero_pages() Date: Thu, 22 May 2025 11:02:43 +0200 Message-ID: <20250522090243.758943-3-p.raghav@samsung.com> In-Reply-To: <20250522090243.758943-1-p.raghav@samsung.com> References: <20250522090243.758943-1-p.raghav@samsung.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: D5BD940003 X-Stat-Signature: ffxoibcfeg15zkkn79knidyanz9kzfsj X-Rspam-User: X-HE-Tag: 1747904594-186584 X-HE-Meta: U2FsdGVkX1/yEHXYgUwq1RAJifU/qye5E+MJzdx3JDpSkLq6Lli9teawnc7+oHHeFRKAztHwez3N3aLwNX3EN5kxbDCQXrXEoUW8pgccBAWv05Cu1WGiMymbnhmBteezaETkGHZ0wScJaG1jwRVyphjeS4LZt63V2y8z4KS/RNfCyR6+CTsOehjaGRJYfc/xhcMyWvMHgQd14BzDY7L7jMwWQgnm/yNJygF/6KU+BEj0KQdBAkD+Q7nGNu0Ot/t8Q+MWAZ4zCwjesIw5JqCnIBYOf+U1X3fXGtfSd+hNni0ZtsyJsvEGooJajYq9c7UQget9cW7pbgd6JshsDuGgPBPCsfaqvGrWfftbwIhOceY76vV6uK+tPkp92sZkY+YaVMIbqXfYDob06EujhxCsibNYOBLnLSkaIauFTHXw7tWn2PvOP4WAjT00IZp06gtSd45iU1zUEp8h/diqdgUOoFBB+kkruUO+UEhrTGk/3G8QuIX3wskDc9bKl0xXlth6lI0ah9alzdh4Wt3HErZPIoj1UxAUWP3yGaszCmYcNRaWvQw6eyJlkcjmEOk8x9Iub2Iqi0nOnufmo8h3rGbM645b2hqIA4c8ATh30KSFablqh64b9jXnLKUjIZ2AlO+WVTwLUg3LyD+rGv71rTH6FQEcvfekUiRTe8+KE8KgS1nAcIW6f/fshhTYVcje7w+13shvFa/dS0eio5mvdN+j+98ajzpg0PjswLPJIDd1dIlFWwKtI81h4T8tPUdunSsGCBwXNPRXaPymoSD6aDl5pEd/AblTGtRbPM+zZnGXknOUQ8pzlfWgh09fu98kt7yujA+9TtyII0p3/6k90zHx1nSLgRbg7bpSXDzsrz/6q2Cc5EfqDkrpHBX5Ld6xw+c9uVcAeemt/F6MuRhhCTz20ZwDAUOBOtzR6jDaHqqfTBCEhdM/ScHwnWiNl1gHSVxwuymO8/nBbnb8W/upBbR Y/FUog4D lNpY3KvquoTh39Y+/J/EGe3LYJQYax2wmBIkaZz0CJ/7v6nv7lTH2qWyU2NpMmsYlZdDECB5iNhoQp9GrjXa8zG5S37utvluV12EPglYrHD0dMrhSQfNp303n2RMg4AKOe2mxyYMpgbhLseDqtCdVmIb0CZXRy3jxtTT/iXWxSJI2toakMGfhU8bH0e2BlUOZejIRSSAtyYbZ0+oQl6VBY7NnIAsYusjXvhC4Cc/jhUHNbM5tiqmjkwJ/zXq7hoBMCWoxyrYM5jj4OPNZLejjSrM7LHgJwTYgSDnJ7N2wAG2J+OmGoUQmw4Gn2hBR8KsLb1efN7+eujYR1oG3NAI0zA/Luu1YYntU7Ry1drSOitrWNl1U5gVakCDeZP6Ano84b5Qd+OiWyhxhsDwZZHx345kA5GGe9PFFaIT+mq2Y1M+S1TI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use mm_huge_zero_folio in __blkdev_issue_zero_pages(). Fallback to ZERO_PAGE if mm_huge_zero_folio is not available. On systems that allocates mm_huge_zero_folio, we will end up sending larger bvecs instead of multiple small ones. Noticed a 4% increase in performance on a commercial NVMe SSD which does not support OP_WRITE_ZEROES. The device's MDTS was 128K. The performance gains might be bigger if the device supports bigger MDTS. Signed-off-by: Pankaj Raghav --- block/blk-lib.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/block/blk-lib.c b/block/blk-lib.c index 4c9f20a689f7..221389412359 100644 --- a/block/blk-lib.c +++ b/block/blk-lib.c @@ -196,6 +196,12 @@ static void __blkdev_issue_zero_pages(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp_mask, struct bio **biop, unsigned int flags) { + struct folio *zero_folio; + + zero_folio = mm_get_huge_zero_folio(NULL); + if (!zero_folio) + zero_folio = page_folio(ZERO_PAGE(0)); + while (nr_sects) { unsigned int nr_vecs = __blkdev_sectors_to_bio_pages(nr_sects); struct bio *bio; @@ -208,11 +214,12 @@ static void __blkdev_issue_zero_pages(struct block_device *bdev, break; do { - unsigned int len, added; + unsigned int len, added = 0; - len = min_t(sector_t, - PAGE_SIZE, nr_sects << SECTOR_SHIFT); - added = bio_add_page(bio, ZERO_PAGE(0), len, 0); + len = min_t(sector_t, folio_size(zero_folio), + nr_sects << SECTOR_SHIFT); + if (bio_add_folio(bio, zero_folio, len, 0)) + added = len; if (added < len) break; nr_sects -= added >> SECTOR_SHIFT; -- 2.47.2