From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC7A5C6FD1F for ; Wed, 22 Mar 2023 18:57:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 76C226B0078; Wed, 22 Mar 2023 14:57:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6F7836B007B; Wed, 22 Mar 2023 14:57:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 56F256B007D; Wed, 22 Mar 2023 14:57:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 48A066B0078 for ; Wed, 22 Mar 2023 14:57:16 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id C82EC120402 for ; Wed, 22 Mar 2023 18:57:15 +0000 (UTC) X-FDA: 80597442030.26.79CBE0C Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) by imf23.hostedemail.com (Postfix) with ESMTP id CDDF114000E for ; Wed, 22 Mar 2023 18:57:13 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=pSMg5UEB; spf=pass (imf23.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.42 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679511433; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ziv2ovcgc/3NOuBG0+YWYfMhKDq2ZBtAkLCMMX3QtWE=; b=cmVdn1X75FCtN2rtQjhfE+AyUov5xnRstGJk62CdeweRqtMu/WnSf0m1g9Crwp0oPAAXUQ o60l1MMQZHSNaCj+DIV458eq7uxacRrwsL0NeJU90FWcdn2YaUQIDPZd48vGNrKjolLTRy 8NZzQ+Av6wFjB0QJb6phFmbvF55TTkE= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=pSMg5UEB; spf=pass (imf23.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.42 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679511433; a=rsa-sha256; cv=none; b=PFQO4F6PKH9axIs4b6yECiLn2QnDEUXPGtdaCIQSYElrvKpHgXIGohPHm8X4sB3HBr0Zg2 yJudWZiqWmnPbs9bbrilFlGM+H6VfgjGpRZyrre02+NrMKc19RfE6bTnhRNk5OKe76Oqba dKsEwkhgnBtHXVXKFNQrzRQCZL8ahCQ= Received: by mail-wm1-f42.google.com with SMTP id v20-20020a05600c471400b003ed8826253aso1011047wmo.0 for ; Wed, 22 Mar 2023 11:57:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679511432; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ziv2ovcgc/3NOuBG0+YWYfMhKDq2ZBtAkLCMMX3QtWE=; b=pSMg5UEBvQYaeaphxsZGSMu90NwfzNxBS853TRITWW30wbesMhqO1tZGT+kb4EgOvf u5JiSwpHERnz4s0V0LYNAYZ47ItfIYo5L0N6vhezRC7f7jPvNZSIP89HlkXwl1EYgdl4 yU53nBJsGttjxnq94XhWM3OZccecYFq4rReA4dFVP2bRHG47H/fknqADtiM1EsKzU5sH CNQQN+puNLYcVyJ+gS6g0ylCPWw/pb0qao085ebZ7ugsE3WO4bzvQAdsbpFtqv/tRkc6 gbp/3Myl+5qGfXYWHj/0cyV0ZoHXxIXk9va4erc8oy8C2kJroyaroczPNG89Pz+bgEv5 6jEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679511432; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ziv2ovcgc/3NOuBG0+YWYfMhKDq2ZBtAkLCMMX3QtWE=; b=olnKtm1HQe6owqbmXdtfuw26+AtK1QY1P6fimKEWeXI0RmD32AQwUjC/7zMVAaLAo3 3T1ZBmsQSDZzShdZ0J+d6u/Q2oeXr8BnBoc5e85GFzU/VTehYRtgOBK6jFxrRhyBXeN7 zBobQ3CWx1uwnA3iydQ+4SOS4kTiiYSMvG7LErvp1C6xgHnHUktWpo2fu4nuOvNwXwQa qqYQfUt4W25bSt8Wsvg0Ijh/IVfkUGRRfhdumGCtG186+LUznVdbJYaQQMmgb1vHJ01Q 261n1JPEqP+8a8lbZPluFJ0YLxW8S0uVz4BRpIiZZED8C1R/Cn0Rze/9yq64WC2fmH7v 5Wqw== X-Gm-Message-State: AO0yUKXL9WJQb37zcrVjX8YunjdCUTcaUlpybw7hPLsfTk6R1XsKJIOD xIx5QlRvnG+1vnMsOmTQl8e3AZ5t/Qk= X-Google-Smtp-Source: AK7set8bE42fH1R1oGv50QRT3x9gS/sS2xnJ60wHoLpsl3YDcPHlPhDOLkQG/4Fu97XTiqchgJ3K8A== X-Received: by 2002:a05:600c:3503:b0:3ee:598:7a9b with SMTP id h3-20020a05600c350300b003ee05987a9bmr457743wmq.11.1679511432121; Wed, 22 Mar 2023 11:57:12 -0700 (PDT) Received: from lucifer.home ([2a00:23c5:dc8c:8701:1663:9a35:5a7b:1d76]) by smtp.googlemail.com with ESMTPSA id j21-20020a05600c42d500b003ee581f37a9sm3181241wme.10.2023.03.22.11.57.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Mar 2023 11:57:11 -0700 (PDT) From: Lorenzo Stoakes To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Andrew Morton Cc: Baoquan He , Uladzislau Rezki , Matthew Wilcox , David Hildenbrand , Liu Shixin , Jiri Olsa , Jens Axboe , Alexander Viro , Lorenzo Stoakes Subject: [PATCH v7 3/4] iov_iter: add copy_page_to_iter_nofault() Date: Wed, 22 Mar 2023 18:57:03 +0000 Message-Id: <19734729defb0f498a76bdec1bef3ac48a3af3e8.1679511146.git.lstoakes@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: CDDF114000E X-Stat-Signature: kgp56zn5793oom5gtzi5zj7r4niiyf5q X-HE-Tag: 1679511433-578011 X-HE-Meta: U2FsdGVkX186ibmakVmI5zlk1GbTfTyB7NWl6GwHC1VIenZVnafgT2Z4EgQySidfjq2kYr5EPsLXG96lKRACGJLQYWNHVKRNNsUw4sIAkPmy6i2siZipeUrK/+EubQR4sr44FA2eSBP5FwfsyP8Qe5ow+xKj2aW29y7M6dXv0jl/8R/ENupCvf9AKp4bRWMGuW/AuucKHBBiQ6ZZWYfsEY070fsYI+Ba3kh5Jvz6njG0qMwDhMvPxUp2T26IlNjkT2fe77ZOcISmNkyKaUDZvIhBOITlIym3aeCfzQjy4VkOt0XkxAKOURmSKVEwx2f35G11oI3DJLV7fEkUQsT/BajB/qrjUvvUFjx9NHZxhuqxpERuLOmx8mvTiASla+tG4ZKppedx+6C2rmp19uXrbP8R2IPdxbbBYoDqE0U+p9OK7/EbB5GWC5MVE9epTQ7/Ni4HLWfyZICECfe2Yu3PCJQM1mwwg9JBu8dTUW0NDw1FS4xHahT6zKl/l++WPQHdQm6woGhAMfwin76+Cx0rT8Rg1Nf1bx35lKgGYGOxGdtsY/xQF4T7RH8URj4HHooOKZtyZF++2ywyl9+4HlNO/Rra5sG7+irOF7ndyQrrrb1f2XrSAVM4IZn2XvDVU7FNGuIHaK+nq6mfZyukSn71Teht45Z12PbYyY8aCyalTmVnJxYOHOGKio6igshSqAXGHD4W8bv/Rd1hjpipZtfbykIL4i56H5xFuMdMaTDCbO2b2HOpVxfllfTuJkT4E5NXIEezfijUVktenECeWNUoKtY6dPD22PtZ76jhaJkjn4SPiU0cG9BoTGx/cDs8XivGY7TPrA8T1YloHlQqZ6S9WO2cP2Wp1YZibO+PPljUKvSKLfuLh4KQtU2sjCV07GvjIGr6ZL06CjwgSZkgt73DRYdnRKJCloQUF8gsYN6g9PlAmhjkEgWqwPWSfKKOJhMP/C+v2pVXJOgHEwkwzUg J94wolsn /CmaG/pOkzg4I8Bdk2Ta6Sa6PCeSPv9I8+bUdj8mIsUepGmoHHs4gStfeUGutSAceHfu059ZeEGkq3VqEylsjCCceAkaJvr++/ReDZ8rhp5mCZgYMo36MAJ4RmIBTubryrHrq/3/OUPPR6Ue3uLfoQR9iP+z+O5djs0YwZi8cQQxrm4nF04Ak5schc3D2YgGbuS6T5FdYIRtPpeO6ZkLrM3t7XCqXTi9ytk9Kibcc2GU17gdP0h8ESz9lOZ1/oLsABIrEX1kGUkcZW6x9hPWqJvC4w9F8wC9OMQ6a9yaeNJzouXgsp9WOixdR1jNxD7gDxDESIr2rBeWihC98LuA2pOhjVv8EqPvYn/9P00QMt5tr97yf5Srplv3S21Zv9MiUDWJFVq1pK3fQ7bRa+PWdBId1XqKVnndkOI8Ca9ki4Ojc0LzycrBcEE7S+1lW6iYqpyQzuLXOhvrXBHka9PIZnXkhVhdzSwBaX3+huvLKbo1Ut/0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Provide a means to copy a page to user space from an iterator, aborting if a page fault would occur. This supports compound pages, but may be passed a tail page with an offset extending further into the compound page, so we cannot pass a folio. This allows for this function to be called from atomic context and _try_ to user pages if they are faulted in, aborting if not. The function does not use _copy_to_iter() in order to not specify might_fault(), this is similar to copy_page_from_iter_atomic(). This is being added in order that an iteratable form of vread() can be implemented while holding spinlocks. Signed-off-by: Lorenzo Stoakes --- include/linux/uio.h | 2 ++ lib/iov_iter.c | 48 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 50 insertions(+) diff --git a/include/linux/uio.h b/include/linux/uio.h index 27e3fd942960..29eb18bb6feb 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -173,6 +173,8 @@ static inline size_t copy_folio_to_iter(struct folio *folio, size_t offset, { return copy_page_to_iter(&folio->page, offset, bytes, i); } +size_t copy_page_to_iter_nofault(struct page *page, unsigned offset, + size_t bytes, struct iov_iter *i); static __always_inline __must_check size_t copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i) diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 274014e4eafe..34dd6bdf2fba 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -172,6 +172,18 @@ static int copyout(void __user *to, const void *from, size_t n) return n; } +static int copyout_nofault(void __user *to, const void *from, size_t n) +{ + long res; + + if (should_fail_usercopy()) + return n; + + res = copy_to_user_nofault(to, from, n); + + return res < 0 ? n : res; +} + static int copyin(void *to, const void __user *from, size_t n) { size_t res = n; @@ -734,6 +746,42 @@ size_t copy_page_to_iter(struct page *page, size_t offset, size_t bytes, } EXPORT_SYMBOL(copy_page_to_iter); +size_t copy_page_to_iter_nofault(struct page *page, unsigned offset, size_t bytes, + struct iov_iter *i) +{ + size_t res = 0; + + if (!page_copy_sane(page, offset, bytes)) + return 0; + if (WARN_ON_ONCE(i->data_source)) + return 0; + if (unlikely(iov_iter_is_pipe(i))) + return copy_page_to_iter_pipe(page, offset, bytes, i); + page += offset / PAGE_SIZE; // first subpage + offset %= PAGE_SIZE; + while (1) { + void *kaddr = kmap_local_page(page); + size_t n = min(bytes, (size_t)PAGE_SIZE - offset); + + iterate_and_advance(i, n, base, len, off, + copyout_nofault(base, kaddr + offset + off, len), + memcpy(base, kaddr + offset + off, len) + ) + kunmap_local(kaddr); + res += n; + bytes -= n; + if (!bytes || !n) + break; + offset += n; + if (offset == PAGE_SIZE) { + page++; + offset = 0; + } + } + return res; +} +EXPORT_SYMBOL(copy_page_to_iter_nofault); + size_t copy_page_from_iter(struct page *page, size_t offset, size_t bytes, struct iov_iter *i) { -- 2.39.2