From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31247C4332F for ; Wed, 13 Dec 2023 23:37:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8A3FB8D007A; Wed, 13 Dec 2023 18:37:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 827B76B05A3; Wed, 13 Dec 2023 18:37:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 604A48D007A; Wed, 13 Dec 2023 18:37:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 481116B05A1 for ; Wed, 13 Dec 2023 18:37:13 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E822212098A for ; Wed, 13 Dec 2023 23:37:12 +0000 (UTC) X-FDA: 81563408304.26.D03C432 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf19.hostedemail.com (Postfix) with ESMTP id 903701A0007 for ; Wed, 13 Dec 2023 23:37:10 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=eFWo7b1K; spf=pass (imf19.hostedemail.com: domain of iii@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=iii@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702510630; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=s1bHT1l6xxSUQY/AfEWlqSevErmthDammvxOjEN6IQ8=; b=FwWp3yMxHK1oNHDkiQ87QCQ5nQxnvINM9RzBqUCuHYtiW7XkLz8xFi1EfASmRP7GgYEm6B 0Co9aHkyDUzGC40zBegCabsE+Z24c/DQF7FAM0iaOxfi1X0VMVNIVNj4DcKSR0ct3a8jA9 FKBFgz1g0VBiKxy0qptufiHtZPYYPOI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702510630; a=rsa-sha256; cv=none; b=H5b6MGuPNn3YyqVu4T/wwckcOobe5Ml3ksrUz+HHtqRnxnV1rKBrKaTyJsBSjdDQu5oTWm Q6vvZ5i++q6JB1oSnPesMXnjSSskamP0cDGzaQSuII9fITdwet0m1Lu9ZzFdIv8/oUG/4C zgrHaPnMpw7FXasEhIJ2SFLMJbCfxFw= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=eFWo7b1K; spf=pass (imf19.hostedemail.com: domain of iii@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=iii@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com Received: from pps.filterd (m0353727.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3BDMF2rG005788; Wed, 13 Dec 2023 23:37:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=s1bHT1l6xxSUQY/AfEWlqSevErmthDammvxOjEN6IQ8=; b=eFWo7b1KUzgXdx5vqlqVeNhRqK1tdQtALSHa1s1EbwwsgkcZij7QozGfmStBSAV94GRu jzmiBjcUlzpWgkcg9/nOfNUChmxIigh8lxVkRSdoaGSUmvUj+CPK02spTrU9LA4htChY gPUIEFg2R+arbDmu9xRfuv/n8UQIdqj2aSyjG5PuW23Jv0esiDW56Zhk7Dmj+sTCJZ3M eY93I7w9qcuXfO5IxuCYgKRyotH6swgzOgx3ZoFZCVS2NdCh5L4JvdvUzR2WXYVekROl BlbcOV9sTXFsAv4o/uqr6Y5VxOt7OeuH+BrBUro9QCUlZ8SjUlqFsXotZLW6mkVTsdbB rg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3uybw52s8h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Dec 2023 23:37:03 +0000 Received: from m0353727.ppops.net (m0353727.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BDLQCdm020018; Wed, 13 Dec 2023 23:37:02 GMT Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3uybw52s81-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Dec 2023 23:37:02 +0000 Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3BDGTUb8014824; Wed, 13 Dec 2023 23:37:01 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 3uw42kg1y1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Dec 2023 23:37:01 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 3BDNawNT27263502 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 13 Dec 2023 23:36:58 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E56E420040; Wed, 13 Dec 2023 23:36:57 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7283920043; Wed, 13 Dec 2023 23:36:56 +0000 (GMT) Received: from heavy.boeblingen.de.ibm.com (unknown [9.171.70.156]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 13 Dec 2023 23:36:56 +0000 (GMT) From: Ilya Leoshkevich To: Alexander Gordeev , Alexander Potapenko , Andrew Morton , Christoph Lameter , David Rientjes , Heiko Carstens , Joonsoo Kim , Marco Elver , Masami Hiramatsu , Pekka Enberg , Steven Rostedt , Vasily Gorbik , Vlastimil Babka Cc: Christian Borntraeger , Dmitry Vyukov , Hyeonggon Yoo <42.hyeyoo@gmail.com>, kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org, Mark Rutland , Roman Gushchin , Sven Schnelle , Ilya Leoshkevich Subject: [PATCH v3 31/34] s390/uaccess: Add KMSAN support to put_user() and get_user() Date: Thu, 14 Dec 2023 00:24:51 +0100 Message-ID: <20231213233605.661251-32-iii@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20231213233605.661251-1-iii@linux.ibm.com> References: <20231213233605.661251-1-iii@linux.ibm.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-GUID: GuXSY6mkyFJdvPjUGGnInPnY89ZZCBBh X-Proofpoint-ORIG-GUID: lCgzKW-fN2NdnghC9Ytb979L4h5Hs2ko X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-13_14,2023-12-13_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 bulkscore=0 phishscore=0 spamscore=0 suspectscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000 definitions=main-2312130167 X-Rspamd-Queue-Id: 903701A0007 X-Rspam-User: X-Stat-Signature: dmzjf7bs17j4obzhtbdk3ncpxpkka7fe X-Rspamd-Server: rspam03 X-HE-Tag: 1702510630-489612 X-HE-Meta: U2FsdGVkX1+P5oN+Lo1SLzytrb1cFZ5YCf5rvFTPLjeYCDr7FA33QKEuZczLW2AOUHjogCjJPia0zoGI0BhusjGgmpE6UwjFUt/edI8+f+Gf/yjV0GRsPu52mrUMeAyKQ334n6DSp7eAGPz9nFLcCVxS2tOMs/v98s1L5QrrirnLvQAJJ8VlzHH2pLc8zE1pxGoMG+4yTZpJNiuX2oOdIM399jFx5b5MsgaT5CRLYXl8nKHQrFNvbFBu+v0iGgc2EAdqTP8rgxwL20Cluzko0eaJhjLU0+mrVl42bdj0vmfBn8Fjin6of8IXvXkXJUnPxTcONG2AIzcI/Y79MZHeqsHFGvLZDpvALz6AS/jyjB+ZnP7V4C1zrg4dZsxrhjDOk9r+zmZy+7eP/RsIRlN4SAkGbRtt3BB1OJt8IK2Nbc/ShFMoCR49aZXq4MLNayStTImYmOl1rqaKtYZaOBazw34yOBPn9KqIYBFu/cFSffUsxvyNyCE1lvHDyRp0hrSL9dyQ3rDu2l6uGOSBeFl5zKaulfv6aSTqV6nH1sDIzRcaV/oMiFc4Cqzn/lzJida3wLVyrYjCP9aTbuLonl4NKHaRPV1wJYU3QiLofZm9oSdFZ4KT9uPRGRRpkYJ3fI5vw5Qq8PvnK1y+1d7FP7cnI6c1K2fStA5suFjFrMj5c1rJtciumzx7rnRnuiXVaKFK94mBQcrnAmJWgWZxdjD3oACui2jwryQOyHwJ/60gBY6nxzTJtpURnRZfeKXSaE81m2MeXjB6BMrULHGXsdxnBKWnnptT2juMydstMrgfI/ioiP3mffxphDS497vosQW6htEAy00zJGnkPWj7BvoL/sEZy8WALILbOOJAdjlTpoGz91bB5+IIBEOZXc7NfXhsZnD4WFcf5kBYEtW91s9hQ5N2/BrUmgGob91dtjz/ztjSOqRVCtMbpSiQP2Bya6nhEb7cp8C0F/QYcWraqYJ M/wZfqXW qBaakYjy+zHUfSnSQQgWqsPwXvziYWYWeDIkN43xb4T3aulhc4YAg2wl2XA+twXtVVYAGuow/qB9p799mOo1/2+ZTvGx9mCGUXi1+zeGbXVL66+R78MgUaqbMxU0fKkfWjQmbIVY15DVHJFNZu/wqdw2uegvFHcol4R2gxrOD7+TfDYWessv5qxVNOu5DFcGKAckiej9h+fiEyOucxui1pgFrf+41uo2iFOtWD5QfDho0fhZlNH/HB1o7gy715SsVwTPg6uswSlsaE3Q= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: put_user() uses inline assembly with precise constraints, so Clang is in principle capable of instrumenting it automatically. Unfortunately, one of the constraints contains a dereferenced user pointer, and Clang does not currently distinguish user and kernel pointers. Therefore KMSAN attempts to access shadow for user pointers, which is not a right thing to do. An obvious fix to add __no_sanitize_memory to __put_user_fn() does not work, since it's __always_inline. And __always_inline cannot be removed due to the __put_user_bad() trick. A different obvious fix of using the "a" instead of the "+Q" constraint degrades the code quality, which is very important here, since it's a hot path. Instead, repurpose the __put_user_asm() macro to define __put_user_{char,short,int,long}_noinstr() functions and mark them with __no_sanitize_memory. For the non-KMSAN builds make them __always_inline in order to keep the generated code quality. Also define __put_user_{char,short,int,long}() functions, which call the aforementioned ones and which *are* instrumented, because they call KMSAN hooks, which may be implemented as macros. The same applies to get_user() as well. Acked-by: Heiko Carstens Signed-off-by: Ilya Leoshkevich --- arch/s390/include/asm/uaccess.h | 111 +++++++++++++++++++++++--------- 1 file changed, 79 insertions(+), 32 deletions(-) diff --git a/arch/s390/include/asm/uaccess.h b/arch/s390/include/asm/uaccess.h index 81ae8a98e7ec..c3c26dd1fc04 100644 --- a/arch/s390/include/asm/uaccess.h +++ b/arch/s390/include/asm/uaccess.h @@ -78,13 +78,24 @@ union oac { int __noreturn __put_user_bad(void); -#define __put_user_asm(to, from, size) \ -({ \ +#ifdef CONFIG_KMSAN +#define GET_PUT_USER_NOINSTR_ATTRIBUTES \ + noinline __maybe_unused __no_sanitize_memory +#else +#define GET_PUT_USER_NOINSTR_ATTRIBUTES __always_inline +#endif + +#define DEFINE_PUT_USER(type) \ +static GET_PUT_USER_NOINSTR_ATTRIBUTES int \ +__put_user_##type##_noinstr(unsigned type __user *to, \ + unsigned type *from, \ + unsigned long size) \ +{ \ union oac __oac_spec = { \ .oac1.as = PSW_BITS_AS_SECONDARY, \ .oac1.a = 1, \ }; \ - int __rc; \ + int rc; \ \ asm volatile( \ " lr 0,%[spec]\n" \ @@ -93,12 +104,28 @@ int __noreturn __put_user_bad(void); "2:\n" \ EX_TABLE_UA_STORE(0b, 2b, %[rc]) \ EX_TABLE_UA_STORE(1b, 2b, %[rc]) \ - : [rc] "=&d" (__rc), [_to] "+Q" (*(to)) \ + : [rc] "=&d" (rc), [_to] "+Q" (*(to)) \ : [_size] "d" (size), [_from] "Q" (*(from)), \ [spec] "d" (__oac_spec.val) \ : "cc", "0"); \ - __rc; \ -}) + return rc; \ +} \ + \ +static __always_inline int \ +__put_user_##type(unsigned type __user *to, unsigned type *from, \ + unsigned long size) \ +{ \ + int rc; \ + \ + rc = __put_user_##type##_noinstr(to, from, size); \ + instrument_put_user(*from, to, size); \ + return rc; \ +} + +DEFINE_PUT_USER(char); +DEFINE_PUT_USER(short); +DEFINE_PUT_USER(int); +DEFINE_PUT_USER(long); static __always_inline int __put_user_fn(void *x, void __user *ptr, unsigned long size) { @@ -106,24 +133,24 @@ static __always_inline int __put_user_fn(void *x, void __user *ptr, unsigned lon switch (size) { case 1: - rc = __put_user_asm((unsigned char __user *)ptr, - (unsigned char *)x, - size); + rc = __put_user_char((unsigned char __user *)ptr, + (unsigned char *)x, + size); break; case 2: - rc = __put_user_asm((unsigned short __user *)ptr, - (unsigned short *)x, - size); + rc = __put_user_short((unsigned short __user *)ptr, + (unsigned short *)x, + size); break; case 4: - rc = __put_user_asm((unsigned int __user *)ptr, + rc = __put_user_int((unsigned int __user *)ptr, (unsigned int *)x, size); break; case 8: - rc = __put_user_asm((unsigned long __user *)ptr, - (unsigned long *)x, - size); + rc = __put_user_long((unsigned long __user *)ptr, + (unsigned long *)x, + size); break; default: __put_user_bad(); @@ -134,13 +161,17 @@ static __always_inline int __put_user_fn(void *x, void __user *ptr, unsigned lon int __noreturn __get_user_bad(void); -#define __get_user_asm(to, from, size) \ -({ \ +#define DEFINE_GET_USER(type) \ +static GET_PUT_USER_NOINSTR_ATTRIBUTES int \ +__get_user_##type##_noinstr(unsigned type *to, \ + unsigned type __user *from, \ + unsigned long size) \ +{ \ union oac __oac_spec = { \ .oac2.as = PSW_BITS_AS_SECONDARY, \ .oac2.a = 1, \ }; \ - int __rc; \ + int rc; \ \ asm volatile( \ " lr 0,%[spec]\n" \ @@ -149,13 +180,29 @@ int __noreturn __get_user_bad(void); "2:\n" \ EX_TABLE_UA_LOAD_MEM(0b, 2b, %[rc], %[_to], %[_ksize]) \ EX_TABLE_UA_LOAD_MEM(1b, 2b, %[rc], %[_to], %[_ksize]) \ - : [rc] "=&d" (__rc), "=Q" (*(to)) \ + : [rc] "=&d" (rc), "=Q" (*(to)) \ : [_size] "d" (size), [_from] "Q" (*(from)), \ [spec] "d" (__oac_spec.val), [_to] "a" (to), \ [_ksize] "K" (size) \ : "cc", "0"); \ - __rc; \ -}) + return rc; \ +} \ + \ +static __always_inline int \ +__get_user_##type(unsigned type *to, unsigned type __user *from, \ + unsigned long size) \ +{ \ + int rc; \ + \ + rc = __get_user_##type##_noinstr(to, from, size); \ + instrument_get_user(*to); \ + return rc; \ +} + +DEFINE_GET_USER(char); +DEFINE_GET_USER(short); +DEFINE_GET_USER(int); +DEFINE_GET_USER(long); static __always_inline int __get_user_fn(void *x, const void __user *ptr, unsigned long size) { @@ -163,24 +210,24 @@ static __always_inline int __get_user_fn(void *x, const void __user *ptr, unsign switch (size) { case 1: - rc = __get_user_asm((unsigned char *)x, - (unsigned char __user *)ptr, - size); + rc = __get_user_char((unsigned char *)x, + (unsigned char __user *)ptr, + size); break; case 2: - rc = __get_user_asm((unsigned short *)x, - (unsigned short __user *)ptr, - size); + rc = __get_user_short((unsigned short *)x, + (unsigned short __user *)ptr, + size); break; case 4: - rc = __get_user_asm((unsigned int *)x, + rc = __get_user_int((unsigned int *)x, (unsigned int __user *)ptr, size); break; case 8: - rc = __get_user_asm((unsigned long *)x, - (unsigned long __user *)ptr, - size); + rc = __get_user_long((unsigned long *)x, + (unsigned long __user *)ptr, + size); break; default: __get_user_bad(); -- 2.43.0