From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6EF49C5B552 for ; Fri, 30 May 2025 10:40:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E6B6A6B00CF; Fri, 30 May 2025 06:40:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E1BA46B00F7; Fri, 30 May 2025 06:40:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D10D86B00CF; Fri, 30 May 2025 06:40:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id AB4E86B00CF for ; Fri, 30 May 2025 06:40:58 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 5CD691A0285 for ; Fri, 30 May 2025 10:40:58 +0000 (UTC) X-FDA: 83499231396.30.0B811BD Received: from mta21.hihonor.com (mta21.hihonor.com [81.70.160.142]) by imf03.hostedemail.com (Postfix) with ESMTP id 1A13620005 for ; Fri, 30 May 2025 10:40:55 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of tao.wangtao@honor.com designates 81.70.160.142 as permitted sender) smtp.mailfrom=tao.wangtao@honor.com; dmarc=pass (policy=none) header.from=honor.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748601656; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hQKY+JTY6P+0g5GVM398JA0PTI3A6X+NXBjmoR8Cc+o=; b=AuSip1KFp7Qew9dFobVnG8FOxoQKQmUOrsBiV+gRhhcnSI9P9UyK4mK97gKefaawoMpvWJ lmEeknv2yGJGhd4kYn6PRpjI07U5FhdH23vCE1w5jOU58XvCHjaBtGoV8t0gNo5rlBdSWM v1dMJEFlNoA8dBG74mlgdSWRg3kmLxQ= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of tao.wangtao@honor.com designates 81.70.160.142 as permitted sender) smtp.mailfrom=tao.wangtao@honor.com; dmarc=pass (policy=none) header.from=honor.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748601656; a=rsa-sha256; cv=none; b=yxtOF9QloHPLyXh7Nil1qLHIoAaFOrJcoSFEh8cZ6l5a2lkT7WaAsb6i4Bo++3V3eet7oy L/0e/ihxTMEvx2eeUaw1HG9G2PpUQaCFPntPZMYWUMNUjD+bhp3cgik47e6Q7QpbwzPGrm tgubYjFSvwkE/317lD1VAGCZ6cXjI94= Received: from w011.hihonor.com (unknown [10.68.20.122]) by mta21.hihonor.com (SkyGuard) with ESMTPS id 4b808Y0D6pzYl7rs; Fri, 30 May 2025 18:38:53 +0800 (CST) Received: from a010.hihonor.com (10.68.16.52) by w011.hihonor.com (10.68.20.122) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Fri, 30 May 2025 18:40:52 +0800 Received: from localhost.localdomain (10.144.18.117) by a010.hihonor.com (10.68.16.52) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Fri, 30 May 2025 18:40:52 +0800 From: wangtao To: , , , , , , , , CC: , , , , , , , , , , , , , , , , wangtao Subject: [PATCH v3 3/4] udmabuf: Implement udmabuf rw_file callback Date: Fri, 30 May 2025 18:39:40 +0800 Message-ID: <20250530103941.11092-4-tao.wangtao@honor.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250530103941.11092-1-tao.wangtao@honor.com> References: <20250530103941.11092-1-tao.wangtao@honor.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.144.18.117] X-ClientProxiedBy: w011.hihonor.com (10.68.20.122) To a010.hihonor.com (10.68.16.52) X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 1A13620005 X-Stat-Signature: ho7w8kybp1imstpbe96cdrke3azo17zx X-Rspam-User: X-HE-Tag: 1748601655-566053 X-HE-Meta: U2FsdGVkX1+9E0HP02mQqhe0dmdLvioxJUg+sA858jxlZClCvQHt9RBHIE8+XDKVdL98DcjpvPtu7Wyr2zPQ7/Upswub2YxErrixIYG4KeOvcP0BZNiLqOMJjT1h38zBS/v+wvMv+1Ahh+pQd8FBBvedlsYsfCtRAaqeF1J9gPc2x8XzbRllesjAORPIG+JGdCjRLXsWuNp0moaNooW9Z6QC9vkTvNIFHgebsDO5yw3mzZfUi9kE7BGs8mB+YzwVYVE5vU7GTeBHX6lhmXZkeM4C6D9qBHrmEI+0FZyjHK1kCQiWWYfeam9sxg1GWrLEQcbERb6N8pg49QV8SwJhzA5Avybcu6xhEWtMEyK1hLDNHIDDSWAcR2+02Eyuak8uR4M3E6TL3AVJvwxK0eKyan0j5tquR/DKv/nXP/YdrH6Ckh1T6A96wJvJ+jeZrZvi1OEuSkvY7yaqukFmWSX4ovddVRC7nz/JiKB4CNayl8JqlPPZmcMCe0AVmtrChi6IVekKJGMPaVM40OboY+9oylK2uDmmPMikU7bfy04dASly9/5ifb8faVWOb/hoeEjkO4cwMzhXtG9Y4cR1Pj3t30xnDQObBonSnPsAu71OtPkBd4OlYT0q6viZDTnAmV57D1kaThIILZ7RP+IjkaO8/T/AESF2DnEhJMYOYGms4iXsj3mbZFPBosMWJ7Gi8na0vjQ3kesIR746aoPBoezJWxULKHKonYlmBOYj7p3daC6GTQGzqdA7AenEFDsm8qZh6CdsR6FwSoVCKNlAyhwHY10CJAszclKzE5GMe4+j5W/AHvzF/KUhtMI6Zm3q5xJ1bmCx2pE6iRe8sXuyHlXEcxRY1tH2PzWUXHDo5Lnhxy9TXxmsknNP6dpQHjA63xNcCy/mikjz5+eQ8sE2iWjUHehGhEleIxkcvrbB0r4kfNAobj4tVMUkr2FLqzMkmmt6bMI/plZiYd715ULRIEd deTL7Inn 3DNQZ0yDaO/qOvBCZBENA1Tc5HVHYi7Jb+96qHlAP6qiWdJfb48wun0odTYCMFBvKw6Z9Kt8NfdzmF8n3Ub7DsfapIf2EORfjfiIK1OLO8Lw6Aha8jCDB7YpvEi27XAjiOrIjHKu1PtrVPZBMKBp3HC/F7Gfar1nRve4Z6JL8hTq1mcwV1KvgNOhTfN6xs81P5BCR+YsHLr748IJrLCukKjvSNO/4ZaaEuQjoKduTvSvhZie32QterO0KzZCvR3chj5EN X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Construct bio_vec from folios, then call the other file's r/w callbacks for IO operations. Test data shows direct I/O copy_file_range improves performance by over 50% vs direct I/O mmap&read (2557 vs 1534). Test data: | 32x32MB Read 1024MB |Creat-ms|Close-ms| I/O-ms|I/O-MB/s| I/O% |-------------------------|--------|--------|--------|--------|----- | 1)Beg udmabuf buffer R/W| 580 | 323 | 1238 | 867 | 100% | 2) dmabuf buffer R/W| 48 | 5 | 1149 | 934 | 107% | 3) udma+memfd buffer R/W| 597 | 340 | 2157 | 497 | 57% | 4) udma+memfd direct R/W| 573 | 340 | 700 | 1534 | 176% | 5) u+mfd buffer sendfile| 577 | 340 | 1204 | 891 | 102% | 6) u+mfd direct sendfile| 567 | 339 | 2272 | 472 | 54% | 7) u+mfd buffer splice| 570 | 337 | 1114 | 964 | 111% | 8) u+mfd direct splice| 564 | 335 | 793 | 1355 | 156% | 9) udmabuf buffer c_f_r| 577 | 323 | 1059 | 1014 | 116% |10) udmabuf direct c_f_r| 582 | 325 | 420 | 2557 | 294% |11)End udmabuf buffer R/W| 586 | 323 | 1188 | 903 | 104% Signed-off-by: wangtao --- drivers/dma-buf/udmabuf.c | 59 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c index e74e36a8ecda..573275a51674 100644 --- a/drivers/dma-buf/udmabuf.c +++ b/drivers/dma-buf/udmabuf.c @@ -284,6 +284,64 @@ static int end_cpu_udmabuf(struct dma_buf *buf, return 0; } +static ssize_t udmabuf_rw_file(struct dma_buf *dmabuf, loff_t my_pos, + struct file *other, loff_t pos, + size_t count, bool is_write) +{ + struct udmabuf *ubuf = dmabuf->priv; + loff_t my_end = my_pos + count, bv_beg, bv_end = 0; + pgoff_t pg_idx = my_pos / PAGE_SIZE; + pgoff_t pg_end = DIV_ROUND_UP(my_end, PAGE_SIZE); + size_t i, bv_off, bv_len, bv_num, bv_idx = 0, bv_total = 0; + struct bio_vec *bvec; + struct kiocb kiocb; + struct iov_iter iter; + unsigned int direction = is_write ? ITER_SOURCE : ITER_DEST; + ssize_t ret = 0, rw_total = 0; + struct folio *folio; + + bv_num = min_t(size_t, pg_end - pg_idx + 1, 1024); + bvec = kvcalloc(bv_num, sizeof(*bvec), GFP_KERNEL); + if (!bvec) + return -ENOMEM; + + init_sync_kiocb(&kiocb, other); + kiocb.ki_pos = pos; + + for (i = 0; i < ubuf->nr_pinned && my_pos < my_end; i++) { + folio = ubuf->pinned_folios[i]; + bv_beg = bv_end; + bv_end += folio_size(folio); + if (bv_end <= my_pos) + continue; + + bv_len = min(bv_end, my_end) - my_pos; + bv_off = my_pos - bv_beg; + my_pos += bv_len; + bv_total += bv_len; + bvec_set_page(&bvec[bv_idx], &folio->page, bv_len, bv_off); + if (++bv_idx < bv_num && my_pos < my_end) + continue; + + /* start R/W if bvec is full or count reaches zero. */ + iov_iter_bvec(&iter, direction, bvec, bv_idx, bv_total); + if (is_write) + ret = other->f_op->write_iter(&kiocb, &iter); + else + ret = other->f_op->read_iter(&kiocb, &iter); + if (ret <= 0) + break; + rw_total += ret; + if (ret < bv_total || fatal_signal_pending(current)) + break; + + bv_idx = bv_total = 0; + } + kvfree(bvec); + + return rw_total > 0 ? rw_total : ret; +} + static const struct dma_buf_ops udmabuf_ops = { .cache_sgt_mapping = true, .map_dma_buf = map_udmabuf, @@ -294,6 +352,7 @@ static const struct dma_buf_ops udmabuf_ops = { .vunmap = vunmap_udmabuf, .begin_cpu_access = begin_cpu_udmabuf, .end_cpu_access = end_cpu_udmabuf, + .rw_file = udmabuf_rw_file, }; #define SEALS_WANTED (F_SEAL_SHRINK) -- 2.17.1