From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E709C6FD1C for ; Wed, 22 Mar 2023 14:55:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C90096B0078; Wed, 22 Mar 2023 10:55:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C41C06B007B; Wed, 22 Mar 2023 10:55:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F76F6B007D; Wed, 22 Mar 2023 10:55:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 8EF346B0078 for ; Wed, 22 Mar 2023 10:55:40 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 5BC241A02E6 for ; Wed, 22 Mar 2023 14:55:40 +0000 (UTC) X-FDA: 80596833240.04.D6AD405 Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) by imf09.hostedemail.com (Postfix) with ESMTP id 732FC140002 for ; Wed, 22 Mar 2023 14:55:38 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=OAV5bOCA; spf=pass (imf09.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.44 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679496938; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ziv2ovcgc/3NOuBG0+YWYfMhKDq2ZBtAkLCMMX3QtWE=; b=LcmyXdVUEuWTRjKq5c7FFpbOjmXk7/6DHidM0+tpUcKBJdlpjejWE5Eciy2usBEHHzt9rh 2j2HJ1fogDpqoGplgVOrG5ORK2zdsjMHbsJzQZSL8Ge7lhq92gvTfegulXc9Sqfch7ZZK4 AZ6k0DbzSmQOw+A5IaFlILn8BnSSIEs= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=OAV5bOCA; spf=pass (imf09.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.44 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679496938; a=rsa-sha256; cv=none; b=bJV8J8vEH79lvAHlbPQ4LodXm6BSKnbCeQ1Z0h4j6HX7e9O8wAxDTMZaR+ihSRYoG1QhME ma7wD0wJIohmoVSVVeauw5hnIAm5wHbZ1Kr7J2uF7Sh2s7qy4HsN0kpevw35mpjpJJF8uM 9F4B3I+vZFsCRSSqF6ChozS6TV6yRiE= Received: by mail-wm1-f44.google.com with SMTP id az3-20020a05600c600300b003ed2920d585so13210311wmb.2 for ; Wed, 22 Mar 2023 07:55:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679496937; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ziv2ovcgc/3NOuBG0+YWYfMhKDq2ZBtAkLCMMX3QtWE=; b=OAV5bOCAXt/brQqvvLh3Ler2AJ9GHlTfIBj1zt+bsAVk1mAGgHj8zeXzy1CFUIp59c g0j4bfCzrA7HOPI24EkKttjDSOoekv+lp+Ux319tiQlEiMRhczQCWjFgd+nzWyQKlYOT 4G+WxPHSgXXiBWEEVZp2ZsT7q1mu6rQC7V+PJJFavEAzMrHcGzFz/V0d2Ten7uIdDb2o Rh+kJe2IOaU5+vJojOU8XbVJBIgq4JPoY7Lw2AfMgj7MaAlx776xF93KeEnSwUv6Plzp l1moQKfShHAfPpJjJMHywcku9UjCVzyket6xtX2PJvJCiWX1uH1VH3plioVFy68wB7v2 TPhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679496937; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ziv2ovcgc/3NOuBG0+YWYfMhKDq2ZBtAkLCMMX3QtWE=; b=pPC8ld92eRSvLy/rMsJgLHoxm0l2wYhrd0n94rUjYPmdGHmk1tPczsdiM8PwRROkMF tNmnblXoOrA/l+jfSADgdeZenC+aqqE/FVj2CFBzXtYmLsrykqa+nVXGeYzpzJmEmpCF PVfW4zwDTs3bTV/1fEK2X8w4O6F0+r8+v2ikSJ/dmhsmbq1NUwykEor2ZNTnPoVzT3Jl BkrgQNUoHp4xopBMBsq9m+c978oE/tG34SPVXCWPxOQwuniJMo78+72+P+ocVWbzHo0s pGMQWvf7p5O92sUzYR9vHcyR04rWT+fMmFNQJglSELTyMBCaco91c57JRSJZqPkegLBg ouqQ== X-Gm-Message-State: AO0yUKVdg3SfFiQmAtXPZg0w8yCswlD8FKlY3jxmwz6QQZAfL88NxkF7 +UnwaGAi5xlDOgnqwsZE8gISZ9K17VI= X-Google-Smtp-Source: AK7set9doYzaEsFJwhlnO7ROmhc0QBK17nVojzEcg2JEds9E8AVQb6TMCOYkJXMHKA13gZcDAvM6xw== X-Received: by 2002:a05:600c:2048:b0:3ed:cf2a:3fe8 with SMTP id p8-20020a05600c204800b003edcf2a3fe8mr5815815wmg.8.1679496936720; Wed, 22 Mar 2023 07:55:36 -0700 (PDT) Received: from lucifer.home ([2a00:23c5:dc8c:8701:1663:9a35:5a7b:1d76]) by smtp.googlemail.com with ESMTPSA id n23-20020a7bcbd7000000b003ed243222adsm16812246wmi.42.2023.03.22.07.55.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Mar 2023 07:55:35 -0700 (PDT) From: Lorenzo Stoakes To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Andrew Morton Cc: Baoquan He , Uladzislau Rezki , Matthew Wilcox , David Hildenbrand , Liu Shixin , Jiri Olsa , Jens Axboe , Alexander Viro , Lorenzo Stoakes Subject: [PATCH v6 3/4] iov_iter: add copy_page_to_iter_nofault() Date: Wed, 22 Mar 2023 14:55:27 +0000 Message-Id: <19734729defb0f498a76bdec1bef3ac48a3af3e8.1679496827.git.lstoakes@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: rupfacwzuig7atq3yptgfy8yqgimzdwh X-Rspamd-Queue-Id: 732FC140002 X-HE-Tag: 1679496938-506360 X-HE-Meta: U2FsdGVkX19dC/NiGC+90WnW6vWPhJZCQGttWvV98Q61oYKo7sFbA60NPAH9GlI7WmSTrV6FgIyqkSuSNjYbQtjGfEwCy1lcoQ+g3UP9D7SQyQes+dUWw39Hd9MNA30W38+kCY94hnKNB6CSE0TeLO7zgxlkvIMeCl+cGgyFp7M9MfNPzBuwkJoHn7KrDvlrNxeSItUT31Kk3us0V5jSvDRmpm4JQS7ACxg+ivfcrt4lCzRc1cm8OaWGChnySmB0TZmj1uQ4KEq98J5pVSDp39+KMiHjwLFS8wIUYCrsBcoyKLGIgKLP3phPGluwvnIC92dL2m2XgnQMLw48CNQSrSefoz3jSHHJ415vOwBep+XTzQIZrEvMWA8+RRxpVmBhkuasZwgY9jHa+f8mCZPHbNCLRjdFkB2BAomGB0Vhovq10kRjOjbzlOceoJem2PZY+X4C3G/EXjsgJ1sD+KsYNtwvAt73Cho3SKL0PpjOCF8XnVRf1lyRH6eJk2s8aZHRLCuQVK89C3Ikm54ZkT+nwQjeYjB4Mx9j3K4RkmzawgLcZ7AoSNsByY5LZbwURzmUQ/37mVcpeRDmYduUYXqtJSizblSvxCs3Zc1khdlAxsHbMexeFjFAmwo6KTQH0U7Pp6AIKNMD6KoF5l/D5OoyRZosp42bYuz34KUqJD+Q6F1WlUjaj7QE8RDmCovCFuWFYPOiloT9aOLAGgK4gCfoiLOSFwbUPodm7yXUack8EpJRt/LaPtOQ6Z+ajJfxDoU1Ft/XPheBlgYY4Ku+5ZaIR3nPZK72IAqcSwKDjgd4nnV0rI1OtiF5O/VpV5a9LG0KbxCeuuAdsmCoJV2e6RLM2LFZeATOD7/5RAWpAwAjlkY7znHoQ3YUllfMB0vTb6AE2ILcT2cbR73MaPqY6s5paERlDcfknecCo/u3jnxxVURYeHyOpySWim/6C3vRGlsFGWwCQQojjo3Q/rl/Enh nYNKh4YX 8kMRIbCDUWOnODPnAPwBtCMpZsv/H9bNewFilvKf27RBaTuE7zLep5VA2KSxvbp/OOgxS98MjvSUiiEQgQy6PvUSh4BWMVJmpLwiTMbAkpI2MURd4SxfozOMxYFVlARgyMH1sEtFKeUjthmgiyDtkCZZPy0RHQNvFjqDvUJ8zcoNNNu76AX5U8saXF6w/H/+PfqNUoK8n3D85sDRd2nVHYnVDTnohr3OJwKrQiST/boMEC79ho/QkWol9RQX2hGmoo515OMZy82gcP2VA0nw9O8b9HzBEIwEcDc3ZcC99e0jFcIKeJ/WWnjy6568/+26G70AIxrZGTrd4g+xitH9BICzmmxwgiV92PDpImYHi3S/cMb3QijdfTaV8NupLu6gqIJtLsQQTghZgPKomraTMYnfXhReHausEWRrqLsi9nQw/7y7x6E5vli45T3gr80Z4VIqUHTIZwBrSkczyzgtaSbw6n8DAaHjq3zjHY3gD74VtB5A= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Provide a means to copy a page to user space from an iterator, aborting if a page fault would occur. This supports compound pages, but may be passed a tail page with an offset extending further into the compound page, so we cannot pass a folio. This allows for this function to be called from atomic context and _try_ to user pages if they are faulted in, aborting if not. The function does not use _copy_to_iter() in order to not specify might_fault(), this is similar to copy_page_from_iter_atomic(). This is being added in order that an iteratable form of vread() can be implemented while holding spinlocks. Signed-off-by: Lorenzo Stoakes --- include/linux/uio.h | 2 ++ lib/iov_iter.c | 48 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 50 insertions(+) diff --git a/include/linux/uio.h b/include/linux/uio.h index 27e3fd942960..29eb18bb6feb 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -173,6 +173,8 @@ static inline size_t copy_folio_to_iter(struct folio *folio, size_t offset, { return copy_page_to_iter(&folio->page, offset, bytes, i); } +size_t copy_page_to_iter_nofault(struct page *page, unsigned offset, + size_t bytes, struct iov_iter *i); static __always_inline __must_check size_t copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i) diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 274014e4eafe..34dd6bdf2fba 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -172,6 +172,18 @@ static int copyout(void __user *to, const void *from, size_t n) return n; } +static int copyout_nofault(void __user *to, const void *from, size_t n) +{ + long res; + + if (should_fail_usercopy()) + return n; + + res = copy_to_user_nofault(to, from, n); + + return res < 0 ? n : res; +} + static int copyin(void *to, const void __user *from, size_t n) { size_t res = n; @@ -734,6 +746,42 @@ size_t copy_page_to_iter(struct page *page, size_t offset, size_t bytes, } EXPORT_SYMBOL(copy_page_to_iter); +size_t copy_page_to_iter_nofault(struct page *page, unsigned offset, size_t bytes, + struct iov_iter *i) +{ + size_t res = 0; + + if (!page_copy_sane(page, offset, bytes)) + return 0; + if (WARN_ON_ONCE(i->data_source)) + return 0; + if (unlikely(iov_iter_is_pipe(i))) + return copy_page_to_iter_pipe(page, offset, bytes, i); + page += offset / PAGE_SIZE; // first subpage + offset %= PAGE_SIZE; + while (1) { + void *kaddr = kmap_local_page(page); + size_t n = min(bytes, (size_t)PAGE_SIZE - offset); + + iterate_and_advance(i, n, base, len, off, + copyout_nofault(base, kaddr + offset + off, len), + memcpy(base, kaddr + offset + off, len) + ) + kunmap_local(kaddr); + res += n; + bytes -= n; + if (!bytes || !n) + break; + offset += n; + if (offset == PAGE_SIZE) { + page++; + offset = 0; + } + } + return res; +} +EXPORT_SYMBOL(copy_page_to_iter_nofault); + size_t copy_page_from_iter(struct page *page, size_t offset, size_t bytes, struct iov_iter *i) { -- 2.39.2