From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CDB78C77B61 for ; Mon, 24 Apr 2023 21:11:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 38B9F6B0072; Mon, 24 Apr 2023 17:11:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 33BB56B0074; Mon, 24 Apr 2023 17:11:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1DCAE6B0075; Mon, 24 Apr 2023 17:11:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 0BB296B0072 for ; Mon, 24 Apr 2023 17:11:28 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id BB938A036C for ; Mon, 24 Apr 2023 21:11:27 +0000 (UTC) X-FDA: 80717530614.20.C897F08 Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) by imf03.hostedemail.com (Postfix) with ESMTP id F031F2000A for ; Mon, 24 Apr 2023 21:11:24 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=Aw81Vs8x; spf=pass (imf03.hostedemail.com: domain of yosryahmed@google.com designates 209.85.208.49 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1682370685; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=en+fDziQkermLiJrmAgywyXMopFaq5a4hDQGR7+I+b8=; b=SGbiWVnNKHKDw/Zlrv+oTVdgrVLqecLooriSbyPlO2rbFJ5A09J/yoETcGSdHSCuiGa7Z6 /GFMgMG78uVjLNJVFfdJLPz9Cr360jhLb70yqgCwFL4UAoiz/K1ABX7zslCSGaxb+UWr0E NWFMmUZ+NXjNTJ9K5bOgHmFTzRpN1qA= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=Aw81Vs8x; spf=pass (imf03.hostedemail.com: domain of yosryahmed@google.com designates 209.85.208.49 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1682370685; a=rsa-sha256; cv=none; b=SW/KnFyId2nBF73tJhELslsiU4BcJToQ5n7F2TfGAPn8N9JBsWAwnz74EvUVsdNGKZgDdc G6JOHyHIzOeEMmlnOr9dD3hS5u5Qbwhdk1uRLi5KvAp1jv2Yx9kbk8jlgXWlLHoBCZs/RH AWkKYONPFxopukkxUmFShgzm1/nHJjQ= Received: by mail-ed1-f49.google.com with SMTP id 4fb4d7f45d1cf-504efe702d5so7310308a12.3 for ; Mon, 24 Apr 2023 14:11:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1682370683; x=1684962683; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=en+fDziQkermLiJrmAgywyXMopFaq5a4hDQGR7+I+b8=; b=Aw81Vs8xODS4N4yCsb8UY7WKISjZiAzaU98K9+SV5cfJefzYEAt3L4hCrVYmdCq/o/ 6bnbBCEbpVZH1vsqiaP73oHrhnlPgrrq/z853nomhm90GxBnJxr0OzOWRmT82qo9YPgP UBSA98oT6OmEMyOtGbpmowMX/nHDv+ZjTo/aZa7V6RwGKP/K/6cK4SIs0NBMkNjAgSuB TZvjbGYmKaslVKhbjIa5i93pXSfOitzj+zUjttAj746d23+K2zf9i8k2eeyGSa/K/tqP 2oixeMajbWGaty4xylGDe/PSUuISU6POOH3xNGZE8b65SLJ9Q7jHaz3YvP07awx3hwbV mNGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682370683; x=1684962683; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=en+fDziQkermLiJrmAgywyXMopFaq5a4hDQGR7+I+b8=; b=D75ZTYhkbfSHXCACDwCyfKn+f6Z92l8xEn0JxpgnKw9jFELsk/FbJm+M3MX2WBmhUs oChFdESEWcTbrEgay7AbEPGSkpDG9eXzsfabAE2KR6JXW6wDLqOu29Cmzs1W88O25IoG FOmH3u1LGJJwkVUhlLVoBT56ZkOIDcOgZHAdHGN/TIf4LY28NIhg46GCTHidaRHV+sFv dcvA8SH/naLN9YxU/YyykVUC4JqO/giIs6cVurSCard34j975BY8umEQuDWxz7xAKppV XkC+fGxqoltNZxFLcCA8hbZV+LvqoWzrvmZvKnF5sD7RmR2hDBbPi/ZVlKj/c7vS93UD Bnug== X-Gm-Message-State: AAQBX9dsSoisSmIxyVzvA3QMTuiWhouTRgfuYgYvwLHK2yN67AaxYhwC S+EH2LyFdRBPnkD1HS2c+eVm5TFFrLNZwCGchGJ64A== X-Google-Smtp-Source: AKy350btuw+h0HUnuPC3BgQ5YYh1ckdwk/2bD/XmlcTKvMakQJHu7GokVy0MEpO69cL0WMloaf2bhOWtA07RE7AgOso= X-Received: by 2002:a05:6402:603:b0:506:c22e:cbcf with SMTP id n3-20020a056402060300b00506c22ecbcfmr12424759edv.36.1682370683209; Mon, 24 Apr 2023 14:11:23 -0700 (PDT) MIME-Version: 1.0 References: <00000000000058b63f05f9d98811@google.com> <20230421174054.3434533-1-roman.gushchin@linux.dev> In-Reply-To: From: Yosry Ahmed Date: Mon, 24 Apr 2023 14:10:45 -0700 Message-ID: Subject: Re: [PATCH] mm: kmem: fix a NULL pointer dereference in obj_stock_flush_required() To: Shakeel Butt Cc: Dmitry Vyukov , Roman Gushchin , Muchun Song , Linux Memory Management List , Andrew Morton , Johannes Weiner , Michal Hocko , linux-kernel@vger.kernel.org, syzbot+774c29891415ab0fd29d@syzkaller.appspotmail.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: F031F2000A X-Stat-Signature: 7tezuxay1f8m5zofybrpfx8xum3cupby X-Rspam-User: X-HE-Tag: 1682370684-501023 X-HE-Meta: U2FsdGVkX1+jj6OFOrnam5h/J+2BKlY4vfH/jSH9VyjHXH0qrjmavWpkIr83bOMj4bxIqLa490puY3n/gmC8ajGlpIi/VNdgVAMWr09KGF/wWbGjHogExNhT/Ro2uYX6A3pVv3757kQgIdX2SbSx73/qgGH12htqmf1aSf9GiJL7kgP4pbFwKToEh0k3zobE7JP+ejl/56/EYhi8te3O5gYSFvShODkR5eqxGCiz65jEoyBiiJ8067c27/1jGLR4t5OFZlYQ19nwjzbb/AupXN8w0ITkqqOBxNRLpCkKS3XITtsW5FmY17WIceEOb1zN61jeGLperaSlPzmJKnRH/FZM8JPlJeOkIyEAnWja527Eo5G5z9EktNz7v6XtMSPi4n/MrwEF24uN5dHfWhkSzeayAtQFtkcqwHJQf8NW74b4mQk534EAmQ8rUETXCfGg0eGUVDKA76tp0hegtbhbs1J9bhi6cF3rVsw8xCVHcccdIluAp0zu4N34gWc7fIWcnnXe9JbeJhoY8WFyzsOSPk2BoyI++TOyVfOXP2aiCvNBqjP94gcLMHR0webWWssstQr8rBk6bv/t0mB00ZyVGPQe+C0Kt4Oz1yL6DrCBwMMsikwsUiBU7ysNXjB7+G4EfGCNAAyB1PB7vAV8ivIqI/d/flemHSl+9r7FAaCpXYcU6UxNOCNKn2q0aFPmGs3QAlM/I12cjxRKH9W6QxLRZ/CVWsPkPMILhAfnaE0cBnGFzFS6jz4hyCsAGmZVqXnRCt2jZzcbOjd0QuDoyj3qbSW+Au1sMRHB1dXio+KaRReQH9Aa7ArckkAtCayUQEI6GLNZZ130MUQkTxjiQCWrIPx5JgDMDIcCwrMObe+pdbtkVf/VKGlUP6DwF5LXgqVKkl5tazDwGH5TLNGQyZLvrlNydShXgw2mhUfJF6EPlEy0mywZfMAPHlz1gJ7kcBeg8Jr8ewUMk9nsyGQBbDb F4pxmOQ8 A2gvH/KOdngxwHGLFDbTqKZboAbiOMuavInn+5z1JQcG92UxE+yDgBhkaUogUWsoBdk4JvEyQ/c31Ipc8JRPIJ35IvmPORhXoLRNViWRn9TOAEMObPbHPE0yES47Z8uWW9Yrrd17b7x/DgOKFzLuLOZcQEu2HwNC/Z3ilenEZmLOpFM6U9U9FNG9BdKqF24w+GmxSCf++pUjqO4oBoTjoZ+nsUh8U+oNc7jqsU47sEGA+EPWm+1HyWK7TiZidVP99BDpWvo7+nJWYj4JZrACr5Ox2CmemqJU5PbsCEg+DxMrizaYdHvKY/BDy76A3IEuxNQG/0MXSIy37BiurMcDTX4drTlUTbSWNsYVKHQUdZq+SY/zB6e7lNbFGh0D04L2rscZZTSrlnTle0tiMThXJMztw1HQcwfICimUj7DQzRa51SbcdDTTDSz7Pwx6WfS5RVEdZiEuAt7gw6cC/m/QJez+U+M23nm24uf64sDrbj93ZlaQXGeI20XncvinNElefNYdEt/hhS/8pkumuzfETOpAmBmLVf0meUDK+u3U3Q1RwPkZRAHKSZX3ZBGjJLcbD0dSbAq1RzjWVri5pqMZ0WPU/vw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Apr 24, 2023 at 10:10=E2=80=AFAM Shakeel Butt = wrote: > > On Mon, Apr 24, 2023 at 2:13=E2=80=AFAM Yosry Ahmed wrote: > > > > On Sun, Apr 23, 2023 at 11:51=E2=80=AFPM Dmitry Vyukov wrote: > > > > > > On Sun, 23 Apr 2023 at 04:26, Muchun Song wro= te: > > > > > On Apr 22, 2023, at 01:40, Roman Gushchin wrote: > > > > > > > > > > KCSAN found an issue in obj_stock_flush_required(): > > > > > stock->cached_objcg can be reset between the check and dereferenc= e: > > > > > > > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > > BUG: KCSAN: data-race in drain_all_stock / drain_obj_stock > > > > > > > > > > write to 0xffff888237c2a2f8 of 8 bytes by task 19625 on cpu 0: > > > > > drain_obj_stock+0x408/0x4e0 mm/memcontrol.c:3306 > > > > > refill_obj_stock+0x9c/0x1e0 mm/memcontrol.c:3340 > > > > > obj_cgroup_uncharge+0xe/0x10 mm/memcontrol.c:3408 > > > > > memcg_slab_free_hook mm/slab.h:587 [inline] > > > > > __cache_free mm/slab.c:3373 [inline] > > > > > __do_kmem_cache_free mm/slab.c:3577 [inline] > > > > > kmem_cache_free+0x105/0x280 mm/slab.c:3602 > > > > > __d_free fs/dcache.c:298 [inline] > > > > > dentry_free fs/dcache.c:375 [inline] > > > > > __dentry_kill+0x422/0x4a0 fs/dcache.c:621 > > > > > dentry_kill+0x8d/0x1e0 > > > > > dput+0x118/0x1f0 fs/dcache.c:913 > > > > > __fput+0x3bf/0x570 fs/file_table.c:329 > > > > > ____fput+0x15/0x20 fs/file_table.c:349 > > > > > task_work_run+0x123/0x160 kernel/task_work.c:179 > > > > > resume_user_mode_work include/linux/resume_user_mode.h:49 [inline= ] > > > > > exit_to_user_mode_loop+0xcf/0xe0 kernel/entry/common.c:171 > > > > > exit_to_user_mode_prepare+0x6a/0xa0 kernel/entry/common.c:203 > > > > > __syscall_exit_to_user_mode_work kernel/entry/common.c:285 [inlin= e] > > > > > syscall_exit_to_user_mode+0x26/0x140 kernel/entry/common.c:296 > > > > > do_syscall_64+0x4d/0xc0 arch/x86/entry/common.c:86 > > > > > entry_SYSCALL_64_after_hwframe+0x63/0xcd > > > > > > > > > > read to 0xffff888237c2a2f8 of 8 bytes by task 19632 on cpu 1: > > > > > obj_stock_flush_required mm/memcontrol.c:3319 [inline] > > > > > drain_all_stock+0x174/0x2a0 mm/memcontrol.c:2361 > > > > > try_charge_memcg+0x6d0/0xd10 mm/memcontrol.c:2703 > > > > > try_charge mm/memcontrol.c:2837 [inline] > > > > > mem_cgroup_charge_skmem+0x51/0x140 mm/memcontrol.c:7290 > > > > > sock_reserve_memory+0xb1/0x390 net/core/sock.c:1025 > > > > > sk_setsockopt+0x800/0x1e70 net/core/sock.c:1525 > > > > > udp_lib_setsockopt+0x99/0x6c0 net/ipv4/udp.c:2692 > > > > > udp_setsockopt+0x73/0xa0 net/ipv4/udp.c:2817 > > > > > sock_common_setsockopt+0x61/0x70 net/core/sock.c:3668 > > > > > __sys_setsockopt+0x1c3/0x230 net/socket.c:2271 > > > > > __do_sys_setsockopt net/socket.c:2282 [inline] > > > > > __se_sys_setsockopt net/socket.c:2279 [inline] > > > > > __x64_sys_setsockopt+0x66/0x80 net/socket.c:2279 > > > > > do_syscall_x64 arch/x86/entry/common.c:50 [inline] > > > > > do_syscall_64+0x41/0xc0 arch/x86/entry/common.c:80 > > > > > entry_SYSCALL_64_after_hwframe+0x63/0xcd > > > > > > > > > > value changed: 0xffff8881382d52c0 -> 0xffff888138893740 > > > > > > > > > > Reported by Kernel Concurrency Sanitizer on: > > > > > CPU: 1 PID: 19632 Comm: syz-executor.0 Not tainted 6.3.0-rc2-syzk= aller-00387-g534293368afa #0 > > > > > Hardware name: Google Google Compute Engine/Google Compute Engine= , BIOS Google 03/02/2023 > > > > > > > > > > Fix it by reading the cached_objcg with READ_ONCE(). > > > > > > > > > > Fixes: bf4f059954dc ("mm: memcg/slab: obj_cgroup API") > > > > > Reported-by: syzbot+774c29891415ab0fd29d@syzkaller.appspotmail.co= m > > > > > Reported-by: Dmitry Vyukov > > > > > Link: https://lore.kernel.org/linux-mm/CACT4Y+ZfucZhM60YPphWiCLJr= 6+SGFhT+jjm8k1P-a_8Kkxsjg@mail.gmail.com/T/#t > > > > > Signed-off-by: Roman Gushchin > > > > > > > > Acked-by: Muchun Song > > > > > > > > Thanks. > > > > > > This improves things, but strictly speaking the write side also needs > > > WRITE_ONCE. Ordering is always a game of two. It's not possible to > > > order things on one side, if the other side messes up the ordering. > > > > > > > It looks like most other accesses use memcg_stock.stock_lock for > > synchronization. Based on the output of obj_stock_flush_required() > > we call drain_local_stock(), which acquires that lock as well. Should > > we refactor the code to extend the lock section to cover both > > obj_stock_flush_required() and drain_local_stock()? > > > > IIUC this may unify the synchronization handling and > > READ_ONCE/WRITE_ONCE may no longer be needed. This should also avoid > > any inaccuracies (e.g. unnecessary flushes) that may happen if the > > cached objcg changes between obj_stock_flush_required() and > > drain_local_stock(). > > > > Did I miss anything here? > > Yes, drain_local_stock only works on local cpu and > obj_stock_flush_required can touch the stock of all the cpus. Oh right, for other cpus this is infeasible as we schedule the work to be done later. > > The patch is good but I agree with Dmitry that we should add the > WRITE_ONCE as well. Agreed. I guess WRITE_ONCE will be needed in multiple places that update the cached objcg. Thanks.