From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1FE1DC433F5 for ; Fri, 8 Apr 2022 02:44:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 68DE56B0071; Thu, 7 Apr 2022 22:44:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 617876B0072; Thu, 7 Apr 2022 22:44:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4412D8D0001; Thu, 7 Apr 2022 22:44:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0172.hostedemail.com [216.40.44.172]) by kanga.kvack.org (Postfix) with ESMTP id 2DACA6B0071 for ; Thu, 7 Apr 2022 22:44:03 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id D4064AA4B0 for ; Fri, 8 Apr 2022 02:44:02 +0000 (UTC) X-FDA: 79332167124.28.9A48D1F Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by imf18.hostedemail.com (Postfix) with ESMTP id 81EFB1C0002 for ; Fri, 8 Apr 2022 02:44:01 +0000 (UTC) Received: from kwepemi100003.china.huawei.com (unknown [172.30.72.56]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4KZMsY1yX8zBrgq; Fri, 8 Apr 2022 10:39:45 +0800 (CST) Received: from kwepemm600017.china.huawei.com (7.193.23.234) by kwepemi100003.china.huawei.com (7.221.188.122) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Fri, 8 Apr 2022 10:43:57 +0800 Received: from [10.174.179.234] (10.174.179.234) by kwepemm600017.china.huawei.com (7.193.23.234) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.21; Fri, 8 Apr 2022 10:43:56 +0800 Message-ID: Date: Fri, 8 Apr 2022 10:43:55 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.6.1 Subject: Re: [RFC PATCH -next V2 7/7] arm64: add pagecache reading to machine check safe To: Robin Murphy , Mark Rutland , CC: Andrew Morton , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Catalin Marinas , Will Deacon , Alexander Viro , , "H. Peter Anvin" , , , , Kefeng Wang References: <20220406091311.3354723-1-tongtiangen@huawei.com> <20220406091311.3354723-8-tongtiangen@huawei.com> From: Tong Tiangen In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed X-Originating-IP: [10.174.179.234] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To kwepemm600017.china.huawei.com (7.193.23.234) X-CFilter-Loop: Reflected X-Stat-Signature: ibi8x51jx4z9nrktjaz199emahh5hj3k X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 81EFB1C0002 Authentication-Results: imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of tongtiangen@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=tongtiangen@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com X-Rspam-User: X-HE-Tag: 1649385841-181364 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: =E5=9C=A8 2022/4/7 23:53, Robin Murphy =E5=86=99=E9=81=93: > On 2022-04-07 15:56, Tong Tiangen wrote: >> >> >> =E5=9C=A8 2022/4/6 19:27, Mark Rutland =E5=86=99=E9=81=93: >>> On Wed, Apr 06, 2022 at 09:13:11AM +0000, Tong Tiangen wrote: >>>> When user process reading file, the data is cached in pagecache and >>>> the data belongs to the user process, When machine check error is >>>> encountered during pagecache reading, killing the user process and >>>> isolate the user page with hardware memory errors is a more reasonab= le >>>> choice than kernel panic. >>>> >>>> The __arch_copy_mc_to_user() in copy_to_user_mc.S is largely borrows >>>> from __arch_copy_to_user() in copy_to_user.S and the main difference >>>> is __arch_copy_mc_to_user() add the extable entry to support machine >>>> check safe. >>> >>> As with prior patches, *why* is the distinction necessary? >>> >>> This patch adds a bunch of conditional logic, but *structurally* it=20 >>> doesn't >>> alter the handling to be substantially different for the MC and=20 >>> non-MC cases. >>> >>> This seems like pointless duplication that just makes it harder to=20 >>> maintain >>> this code. >>> >>> Thanks, >>> Mark. >> >> Agreed, The implementation here looks a little ugly and harder to=20 >> maintain. >> >> The purpose of my doing this is not all copy_to_user can be recovered. >> >> A memory error is consumed when reading pagecache using copy_to_user. >> I think in this scenario, only the process is affected because it=20 >> can't read >> pagecache data correctly. Just kill the process and don't need the who= le >> kernel panic. >> >> So I need two different copy_to_user implementation, one is existing=20 >> __arch_copy_to_user, >> this function will panic when consuming memory errors. The other one=20 >> is this new helper >> __arch_copy_mc_to_user, this interface is used when reading pagecache.= =20 >> It can recover from >> consume memory error. >=20 > OK, but do we really need two almost-identical implementations of every= =20 > function where the only difference is how the exception table entries=20 > are annotated? Could the exception handler itself just figure out who=20 > owns the page where the fault occurred and decide what action to take a= s=20 > appropriate? >=20 > Robin. >=20 Thank you, Robin. I added this call path in this patchset: do_sea() -> fixup_exception(),=20 the purpose is to provide a chance for sea fault to fixup (refer patch=20 3/7). If fixup successful, panic can be avoided. Otherwise, panic can be=20 eliminated according to the original logic. fixup_exception() will set regs->pc and jump to regs->pc when the=20 context recovery of an exception occurs. If mc-safe-fixup added to __arch_copy_to_user(), in *non pagecache=20 reading* scenario encount memory error when call __arch_copy_to_user() ,=20 do_sea() -> fixup_exception() will not return fail and will miss the=20 panic logic in do_sea(). So I add new helper __arch_copy_mc_to_user() and add mc-safe-fixup to=20 this helper, which can be used in the required scenarios. At present,=20 there is only one pagecache reading scenario, other scenarios need to be=20 developed. This is my current confusion. Of course, I will think about the solution=20 to solve the duplicate code problem. Thanks, Tong. >> >> In future, if find a scenario use copy_to_user can also be recovered,=20 >> we can also use this mc >> safe helper instead it. >> >> Thanks, >> Tong. >> >>> >>>> In _copy_page_to_iter(), machine check safe only considered ITER_IOV= EC >>>> which is used by pagecache reading. >>>> >>>> Signed-off-by: Tong Tiangen >>>> --- >>>> =C2=A0 arch/arm64/include/asm/uaccess.h | 15 ++++++ >>>> =C2=A0 arch/arm64/lib/Makefile=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 |=C2=A0 2 +- >>>> =C2=A0 arch/arm64/lib/copy_to_user_mc.S | 78 +++++++++++++++++++++++= ++++++ >>>> =C2=A0 include/linux/uio.h=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 |=C2=A0 9 +++- >>>> =C2=A0 lib/iov_iter.c=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 | 85=20 >>>> +++++++++++++++++++++++++------- >>>> =C2=A0 5 files changed, 170 insertions(+), 19 deletions(-) >>>> =C2=A0 create mode 100644 arch/arm64/lib/copy_to_user_mc.S >>>> >>>> diff --git a/arch/arm64/include/asm/uaccess.h=20 >>>> b/arch/arm64/include/asm/uaccess.h >>>> index 24b662407fbd..f0d5e811165a 100644 >>>> --- a/arch/arm64/include/asm/uaccess.h >>>> +++ b/arch/arm64/include/asm/uaccess.h >>>> @@ -448,6 +448,21 @@ extern long strncpy_from_user(char *dest, const= =20 >>>> char __user *src, long count); >>>> =C2=A0 extern __must_check long strnlen_user(const char __user *str,= long=20 >>>> n); >>>> +#ifdef CONFIG_ARCH_HAS_COPY_MC >>>> +extern unsigned long __must_check __arch_copy_mc_to_user(void=20 >>>> __user *to, >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 const void *from, unsigned long n); >>>> +static inline unsigned long __must_check >>>> +copy_mc_to_user(void __user *to, const void *from, unsigned long n) >>>> +{ >>>> +=C2=A0=C2=A0=C2=A0 uaccess_ttbr0_enable(); >>>> +=C2=A0=C2=A0=C2=A0 n =3D __arch_copy_mc_to_user(__uaccess_mask_ptr(= to), from, n); >>>> +=C2=A0=C2=A0=C2=A0 uaccess_ttbr0_disable(); >>>> + >>>> +=C2=A0=C2=A0=C2=A0 return n; >>>> +} >>>> +#define copy_mc_to_user copy_mc_to_user >>>> +#endif >>>> + >>>> =C2=A0 #ifdef CONFIG_ARCH_HAS_UACCESS_FLUSHCACHE >>>> =C2=A0 struct page; >>>> =C2=A0 void memcpy_page_flushcache(char *to, struct page *page, size= _t=20 >>>> offset, size_t len); >>>> diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile >>>> index 29c578414b12..9b3571227fb4 100644 >>>> --- a/arch/arm64/lib/Makefile >>>> +++ b/arch/arm64/lib/Makefile >>>> @@ -23,4 +23,4 @@ obj-$(CONFIG_ARM64_MTE) +=3D mte.o >>>> =C2=A0 obj-$(CONFIG_KASAN_SW_TAGS) +=3D kasan_sw_tags.o >>>> -obj-$(CONFIG_ARCH_HAS_CPY_MC) +=3D copy_page_mc.o >>>> +obj-$(CONFIG_ARCH_HAS_COPY_MC) +=3D copy_page_mc.o copy_to_user_mc.= o >>>> diff --git a/arch/arm64/lib/copy_to_user_mc.S=20 >>>> b/arch/arm64/lib/copy_to_user_mc.S >>>> new file mode 100644 >>>> index 000000000000..9d228ff15446 >>>> --- /dev/null >>>> +++ b/arch/arm64/lib/copy_to_user_mc.S >>>> @@ -0,0 +1,78 @@ >>>> +/* SPDX-License-Identifier: GPL-2.0-only */ >>>> +/* >>>> + * Copyright (C) 2012 ARM Ltd. >>>> + */ >>>> + >>>> +#include >>>> + >>>> +#include >>>> +#include >>>> +#include >>>> + >>>> +/* >>>> + * Copy to user space from a kernel buffer (alignment handled by=20 >>>> the hardware) >>>> + * >>>> + * Parameters: >>>> + *=C2=A0=C2=A0=C2=A0 x0 - to >>>> + *=C2=A0=C2=A0=C2=A0 x1 - from >>>> + *=C2=A0=C2=A0=C2=A0 x2 - n >>>> + * Returns: >>>> + *=C2=A0=C2=A0=C2=A0 x0 - bytes not copied >>>> + */ >>>> +=C2=A0=C2=A0=C2=A0 .macro ldrb1 reg, ptr, val >>>> +=C2=A0=C2=A0=C2=A0 1000: ldrb=C2=A0 \reg, [\ptr], \val; >>>> +=C2=A0=C2=A0=C2=A0 _asm_extable_mc 1000b, 9998f; >>>> +=C2=A0=C2=A0=C2=A0 .endm >>>> + >>>> +=C2=A0=C2=A0=C2=A0 .macro strb1 reg, ptr, val >>>> +=C2=A0=C2=A0=C2=A0 user_ldst_mc 9998f, sttrb, \reg, \ptr, \val >>>> +=C2=A0=C2=A0=C2=A0 .endm >>>> + >>>> +=C2=A0=C2=A0=C2=A0 .macro ldrh1 reg, ptr, val >>>> +=C2=A0=C2=A0=C2=A0 1001: ldrh=C2=A0 \reg, [\ptr], \val; >>>> +=C2=A0=C2=A0=C2=A0 _asm_extable_mc 1001b, 9998f; >>>> +=C2=A0=C2=A0=C2=A0 .endm >>>> + >>>> +=C2=A0=C2=A0=C2=A0 .macro strh1 reg, ptr, val >>>> +=C2=A0=C2=A0=C2=A0 user_ldst_mc 9997f, sttrh, \reg, \ptr, \val >>>> +=C2=A0=C2=A0=C2=A0 .endm >>>> + >>>> +=C2=A0=C2=A0=C2=A0 .macro ldr1 reg, ptr, val >>>> +=C2=A0=C2=A0=C2=A0 1002: ldr \reg, [\ptr], \val; >>>> +=C2=A0=C2=A0=C2=A0 _asm_extable_mc 1002b, 9998f; >>>> +=C2=A0=C2=A0=C2=A0 .endm >>>> + >>>> +=C2=A0=C2=A0=C2=A0 .macro str1 reg, ptr, val >>>> +=C2=A0=C2=A0=C2=A0 user_ldst_mc 9997f, sttr, \reg, \ptr, \val >>>> +=C2=A0=C2=A0=C2=A0 .endm >>>> + >>>> +=C2=A0=C2=A0=C2=A0 .macro ldp1 reg1, reg2, ptr, val >>>> +=C2=A0=C2=A0=C2=A0 1003: ldp \reg1, \reg2, [\ptr], \val; >>>> +=C2=A0=C2=A0=C2=A0 _asm_extable_mc 1003b, 9998f; >>>> +=C2=A0=C2=A0=C2=A0 .endm >>>> + >>>> +=C2=A0=C2=A0=C2=A0 .macro stp1 reg1, reg2, ptr, val >>>> +=C2=A0=C2=A0=C2=A0 user_stp 9997f, \reg1, \reg2, \ptr, \val >>>> +=C2=A0=C2=A0=C2=A0 .endm >>>> + >>>> +end=C2=A0=C2=A0=C2=A0 .req=C2=A0=C2=A0=C2=A0 x5 >>>> +srcin=C2=A0=C2=A0=C2=A0 .req=C2=A0=C2=A0=C2=A0 x15 >>>> +SYM_FUNC_START(__arch_copy_mc_to_user) >>>> +=C2=A0=C2=A0=C2=A0 add=C2=A0=C2=A0=C2=A0 end, x0, x2 >>>> +=C2=A0=C2=A0=C2=A0 mov=C2=A0=C2=A0=C2=A0 srcin, x1 >>>> +#include "copy_template.S" >>>> +=C2=A0=C2=A0=C2=A0 mov=C2=A0=C2=A0=C2=A0 x0, #0 >>>> +=C2=A0=C2=A0=C2=A0 ret >>>> + >>>> +=C2=A0=C2=A0=C2=A0 // Exception fixups >>>> +9997:=C2=A0=C2=A0=C2=A0 cbz=C2=A0=C2=A0=C2=A0 x0, 9998f=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 // Check machine c= heck exception >>>> +=C2=A0=C2=A0=C2=A0 cmp=C2=A0=C2=A0=C2=A0 dst, dstin >>>> +=C2=A0=C2=A0=C2=A0 b.ne=C2=A0=C2=A0=C2=A0 9998f >>>> +=C2=A0=C2=A0=C2=A0 // Before being absolutely sure we couldn't copy= anything, try=20 >>>> harder >>>> +=C2=A0=C2=A0=C2=A0 ldrb=C2=A0=C2=A0=C2=A0 tmp1w, [srcin] >>>> +USER(9998f, sttrb tmp1w, [dst]) >>>> +=C2=A0=C2=A0=C2=A0 add=C2=A0=C2=A0=C2=A0 dst, dst, #1 >>>> +9998:=C2=A0=C2=A0=C2=A0 sub=C2=A0=C2=A0=C2=A0 x0, end, dst=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 // bytes not co= pied >>>> +=C2=A0=C2=A0=C2=A0 ret >>>> +SYM_FUNC_END(__arch_copy_mc_to_user) >>>> +EXPORT_SYMBOL(__arch_copy_mc_to_user) >>>> diff --git a/include/linux/uio.h b/include/linux/uio.h >>>> index 739285fe5a2f..539d9ee9b032 100644 >>>> --- a/include/linux/uio.h >>>> +++ b/include/linux/uio.h >>>> @@ -147,10 +147,17 @@ size_t _copy_to_iter(const void *addr, size_t=20 >>>> bytes, struct iov_iter *i); >>>> =C2=A0 size_t _copy_from_iter(void *addr, size_t bytes, struct iov_i= ter *i); >>>> =C2=A0 size_t _copy_from_iter_nocache(void *addr, size_t bytes, stru= ct=20 >>>> iov_iter *i); >>>> +#ifdef CONFIG_ARCH_HAS_COPY_MC >>>> +size_t copy_mc_page_to_iter(struct page *page, size_t offset,=20 >>>> size_t bytes, >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 struct iov_iter *i); >>>> +#else >>>> +#define copy_mc_page_to_iter copy_page_to_iter >>>> +#endif >>>> + >>>> =C2=A0 static inline size_t copy_folio_to_iter(struct folio *folio,=20 >>>> size_t offset, >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 size_t bytes,= struct iov_iter *i) >>>> =C2=A0 { >>>> -=C2=A0=C2=A0=C2=A0 return copy_page_to_iter(&folio->page, offset, b= ytes, i); >>>> +=C2=A0=C2=A0=C2=A0 return copy_mc_page_to_iter(&folio->page, offset= , bytes, i); >>>> =C2=A0 } >>>> =C2=A0 static __always_inline __must_check >>>> diff --git a/lib/iov_iter.c b/lib/iov_iter.c >>>> index 6dd5330f7a99..2c5f3bb6391d 100644 >>>> --- a/lib/iov_iter.c >>>> +++ b/lib/iov_iter.c >>>> @@ -157,6 +157,19 @@ static int copyout(void __user *to, const void=20 >>>> *from, size_t n) >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return n; >>>> =C2=A0 } >>>> +#ifdef CONFIG_ARCH_HAS_COPY_MC >>>> +static int copyout_mc(void __user *to, const void *from, size_t n) >>>> +{ >>>> +=C2=A0=C2=A0=C2=A0 if (access_ok(to, n)) { >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 instrument_copy_to_user(= to, from, n); >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 n =3D copy_mc_to_user((_= _force void *) to, from, n); >>>> +=C2=A0=C2=A0=C2=A0 } >>>> +=C2=A0=C2=A0=C2=A0 return n; >>>> +} >>>> +#else >>>> +#define copyout_mc copyout >>>> +#endif >>>> + >>>> =C2=A0 static int copyin(void *to, const void __user *from, size_t n= ) >>>> =C2=A0 { >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (should_fail_usercopy()) >>>> @@ -169,7 +182,7 @@ static int copyin(void *to, const void __user=20 >>>> *from, size_t n) >>>> =C2=A0 } >>>> =C2=A0 static size_t copy_page_to_iter_iovec(struct page *page, size= _t=20 >>>> offset, size_t bytes, >>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 struct iov_iter *i) >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 struct iov_iter *i, bool mc_safe) >>>> =C2=A0 { >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 size_t skip, copy, left, wanted; >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 const struct iovec *iov; >>>> @@ -194,7 +207,10 @@ static size_t copy_page_to_iter_iovec(struct=20 >>>> page *page, size_t offset, size_t b >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 from =3D kadd= r + offset; >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* first chun= k, usually the only one */ >>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 left =3D copyout(buf, fr= om, copy); >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (mc_safe) >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = left =3D copyout_mc(buf, from, copy); >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 else >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = left =3D copyout(buf, from, copy); >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 copy -=3D lef= t; >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 skip +=3D cop= y; >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 from +=3D cop= y; >>>> @@ -204,7 +220,10 @@ static size_t copy_page_to_iter_iovec(struct=20 >>>> page *page, size_t offset, size_t b >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 iov++; >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 buf =3D iov->iov_base; >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 copy =3D min(bytes, iov->iov_len); >>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = left =3D copyout(buf, from, copy); >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = if (mc_safe) >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 left =3D copyout_mc(buf, from, copy); >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = else >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 left =3D copyout(buf, from, copy); >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 copy -=3D left; >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 skip =3D copy; >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 from +=3D copy; >>>> @@ -223,7 +242,10 @@ static size_t copy_page_to_iter_iovec(struct=20 >>>> page *page, size_t offset, size_t b >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 kaddr =3D kmap(page); >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 from =3D kaddr + offset; >>>> -=C2=A0=C2=A0=C2=A0 left =3D copyout(buf, from, copy); >>>> +=C2=A0=C2=A0=C2=A0 if (mc_safe) >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 left =3D copyout_mc(buf,= from, copy); >>>> +=C2=A0=C2=A0=C2=A0 else >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 left =3D copyout(buf, fr= om, copy); >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 copy -=3D left; >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 skip +=3D copy; >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 from +=3D copy; >>>> @@ -232,7 +254,10 @@ static size_t copy_page_to_iter_iovec(struct=20 >>>> page *page, size_t offset, size_t b >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 iov++; >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 buf =3D iov->= iov_base; >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 copy =3D min(= bytes, iov->iov_len); >>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 left =3D copyout(buf, fr= om, copy); >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (mc_safe) >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = left =3D copyout_mc(buf, from, copy); >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 else >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = left =3D copyout(buf, from, copy); >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 copy -=3D lef= t; >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 skip =3D copy= ; >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 from +=3D cop= y; >>>> @@ -674,15 +699,6 @@ size_t _copy_to_iter(const void *addr, size_t=20 >>>> bytes, struct iov_iter *i) >>>> =C2=A0 EXPORT_SYMBOL(_copy_to_iter); >>>> =C2=A0 #ifdef CONFIG_ARCH_HAS_COPY_MC >>>> -static int copyout_mc(void __user *to, const void *from, size_t n) >>>> -{ >>>> -=C2=A0=C2=A0=C2=A0 if (access_ok(to, n)) { >>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 instrument_copy_to_user(= to, from, n); >>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 n =3D copy_mc_to_user((_= _force void *) to, from, n); >>>> -=C2=A0=C2=A0=C2=A0 } >>>> -=C2=A0=C2=A0=C2=A0 return n; >>>> -} >>>> - >>>> =C2=A0 static size_t copy_mc_pipe_to_iter(const void *addr, size_t b= ytes, >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 struct iov_iter *i) >>>> =C2=A0 { >>>> @@ -846,10 +862,10 @@ static inline bool page_copy_sane(struct page=20 >>>> *page, size_t offset, size_t n) >>>> =C2=A0 } >>>> =C2=A0 static size_t __copy_page_to_iter(struct page *page, size_t=20 >>>> offset, size_t bytes, >>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 struct iov_iter *i) >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 struct iov_iter *i, bool mc_safe) >>>> =C2=A0 { >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (likely(iter_is_iovec(i))) >>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return copy_page_to_iter= _iovec(page, offset, bytes, i); >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return copy_page_to_iter= _iovec(page, offset, bytes, i,=20 >>>> mc_safe); >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (iov_iter_is_bvec(i) || iov_iter_i= s_kvec(i) ||=20 >>>> iov_iter_is_xarray(i)) { >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 void *kaddr =3D= kmap_local_page(page); >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 size_t wanted= =3D _copy_to_iter(kaddr + offset, bytes, i); >>>> @@ -878,7 +894,7 @@ size_t copy_page_to_iter(struct page *page,=20 >>>> size_t offset, size_t bytes, >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 offset %=3D PAGE_SIZE; >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 while (1) { >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 size_t n =3D = __copy_page_to_iter(page, offset, >>>> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 min(bytes, (size_t)PAGE_SIZE - offset), i); >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 min(bytes, (size_t)PAGE_SIZE - offset), i, false); >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 res +=3D n; >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 bytes -=3D n; >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (!bytes ||= !n) >>>> @@ -893,6 +909,41 @@ size_t copy_page_to_iter(struct page *page,=20 >>>> size_t offset, size_t bytes, >>>> =C2=A0 } >>>> =C2=A0 EXPORT_SYMBOL(copy_page_to_iter); >>>> +#ifdef CONFIG_ARCH_HAS_COPY_MC >>>> +/** >>>> + * copy_mc_page_to_iter - copy page to iter with source memory=20 >>>> error exception handling. >>>> + * >>>> + * The filemap_read deploys this for pagecache reading and the main= =20 >>>> differences between >>>> + * this and typical copy_page_to_iter() is call __copy_page_to_iter= =20 >>>> with mc_safe true. >>>> + * >>>> + * Return: number of bytes copied (may be %0) >>>> + */ >>>> +size_t copy_mc_page_to_iter(struct page *page, size_t offset,=20 >>>> size_t bytes, >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 struct iov_iter *i) >>>> +{ >>>> +=C2=A0=C2=A0=C2=A0 size_t res =3D 0; >>>> + >>>> +=C2=A0=C2=A0=C2=A0 if (unlikely(!page_copy_sane(page, offset, bytes= ))) >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return 0; >>>> +=C2=A0=C2=A0=C2=A0 page +=3D offset / PAGE_SIZE; // first subpage >>>> +=C2=A0=C2=A0=C2=A0 offset %=3D PAGE_SIZE; >>>> +=C2=A0=C2=A0=C2=A0 while (1) { >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 size_t n =3D __copy_page= _to_iter(page, offset, >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 min(bytes, (size_t)PAGE_SIZE - offset), i, true); >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 res +=3D n; >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 bytes -=3D n; >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (!bytes || !n) >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = break; >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 offset +=3D n; >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (offset =3D=3D PAGE_S= IZE) { >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = page++; >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = offset =3D 0; >>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 } >>>> +=C2=A0=C2=A0=C2=A0 } >>>> +=C2=A0=C2=A0=C2=A0 return res; >>>> +} >>>> +#endif >>>> + >>>> =C2=A0 size_t copy_page_from_iter(struct page *page, size_t offset,=20 >>>> size_t bytes, >>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 struct iov_iter *i) >>>> =C2=A0 { >>>> --=20 >>>> 2.18.0.huawei.25 >>>> >>>> >>>> _______________________________________________ >>>> linux-arm-kernel mailing list >>>> linux-arm-kernel@lists.infradead.org >>>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel >>> . >> >> _______________________________________________ >> linux-arm-kernel mailing list >> linux-arm-kernel@lists.infradead.org >> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel > .