From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B268EEAA58 for ; Thu, 14 Sep 2023 16:56:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 543266B0292; Thu, 14 Sep 2023 12:56:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4F3296B0295; Thu, 14 Sep 2023 12:56:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3E2186B0296; Thu, 14 Sep 2023 12:56:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 2B0F76B0292 for ; Thu, 14 Sep 2023 12:56:57 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id EDC65120566 for ; Thu, 14 Sep 2023 16:56:56 +0000 (UTC) X-FDA: 81235807632.08.FFB4D59 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf21.hostedemail.com (Postfix) with ESMTP id EA8131C0014 for ; Thu, 14 Sep 2023 16:56:53 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=Rheo62hL; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf21.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.29 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694710614; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9dYbcygFs+B71xYNYqVHVDk/ZWOqRCfnzVK6KuhBKJI=; b=6viM7OeXEnlGmg5BU0FZD7NiN0cK+cAC/whIhx5jSdGXLg3ud+vpIVPzOQDONNtotOfHD7 ukt0t/FGKv1SNoJIBjfkDdC0JRAO+RDsJ+4krK8ROLFi/0JUjxr7j+6E+5TnovHCPRbNl2 x5p6f8hEljYzLJ0Thk9XoPkVNKR/gws= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=Rheo62hL; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf21.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.29 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694710614; a=rsa-sha256; cv=none; b=yZmSCF+UDyZrEDGtL3VGQd37ZXvBaQawUhuFLkklLB8Ev/NVrsQ7DHjYM1uoBUstK7vGL4 Clo6qovok6T2XO/v5fa5XeQNFcl/y/Jq33TvFQfqJmBviTCuUmuwUub8Tx5fg1dKDVD1vJ O3k7PDwlF/pUypKdYD+1LtVlJbHRdcE= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id BD4181F74C; Thu, 14 Sep 2023 16:56:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1694710611; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=9dYbcygFs+B71xYNYqVHVDk/ZWOqRCfnzVK6KuhBKJI=; b=Rheo62hLVc0dxeDLyvQ5sjmbAvZLXFpqZnskMAwq8m+e5BP+OY4EoCEVJQ3IQ5hGcOetul D0eG7FEfYFe0Wa2I6io9FWd3rsnfPJJoNzKXLRNFdTRj1+/QGWbH1pHxqrvxRCLxw7i1D/ 26Omj6jLkgHAsDtjC0MhGXWkYenKals= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 9DFEF139DB; Thu, 14 Sep 2023 16:56:51 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id eTH/IlM7A2XnPgAAMHmgww (envelope-from ); Thu, 14 Sep 2023 16:56:51 +0000 Date: Thu, 14 Sep 2023 18:56:50 +0200 From: Michal Hocko To: Johannes Weiner Cc: Andrew Morton , Shakeel Butt , Breno Leitao , Josef Bacik , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH] mm: memcontrol: fix GFP_NOFS recursion in memory.high enforcement Message-ID: References: <20230914152139.100822-1-hannes@cmpxchg.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230914152139.100822-1-hannes@cmpxchg.org> X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: EA8131C0014 X-Stat-Signature: op1ogmsg3mxqhzdm1ag5htahb35gtj6u X-HE-Tag: 1694710613-943028 X-HE-Meta: U2FsdGVkX18INbl/avzV/M1z9PVQRFU7LiEw6ZJ9FdLhk1qjeGw/FuODb8rgJ/lr6l3U2sB5L7L4s+7CZWziyj4I5I4eQC8jOsuKcr4I2GVPnvF1tdr+NCJp+X17ShILye63om0C2lhk45277nvD/b/bd0kcyTL2jCBG8RsDCKmY8PDl+FQ9r2E0LT6AuGkZs7dVMxvLCVd34R20mfTeyaohfl/CBGMfBdxPdgZ8rTZzPhpH0HPJ2KJFVIVz8pGb444H2SNmYzw0KaMmPbeJpECYebkbhW8SFzMkhJahnyBVV2pytbsOXbU5Ntd/ulFj85+TmDLrQRSYnanTqTsUOM/7NAL1M9PSxDFGjOf8Ag07JnxddRPdan2Ssebf7QDOdi1MHgU19J1LFkJTWiMoHHh6HNmt/k7vlrkQ8haD962Eos5oPmPxH1tSD9mTXp6cEmHmon9RxKh4hqB3Dc0//GgGlvYQa4c3mUlMnuBw/jLJlOY0NMPuxeJXyIaBvYko+7aiJsfTwNBPEgAg0qjLJ6t0jslcdCJ0jMGBgriEYDrEsFdHY8eW3rBz+HvKzA8z1WfJ7frWZrPf02xnPShaRIZSULsTZ3RPbJHk9Cec4vUvhuRwlQTY/iMLAwkGGyXTp+6+lGha4LaJFbkF4DAMeDLoXtwx6lgXYEB/S6JkQYc1MKFJhzOeTXabPzOKePBPGvdWD1dtG1oMX2o2L+1RPP1oEdvVWQfQMuyAy98G6FwRP13vzjSvd4nkZGAb48/2G6bmG91u+5zc9fsGw36DBe3/h4ikxgLmPgRgKwg77uG/QsOtiFtDJMHXPUSZ/AvTV+hW+Gl6RE7lnY+5GSt3EAOT4pRKiZFSUtzvVFYwIwIuqP3Z9Rfh5gg/nzBl+MzUHaK5swYj9oxsshtati66lozsyB7VnZqt0uqZ6x/qUAKJZ/3hZ831Jx2UeP4ya3c+P9XbVXBAt7VjVVKt6O7 I9fk7Zia Q7MyE0dV5bTP6QSLjTdTLtSKYfoLsfj+p6E38wUH8aK/LA5fRohWu5sS3mLMnRYyk9Urbi+dyikW+Z2vZy/j9YkFvSCL9xEZaUbPoIq2KYAXp5ThbnzbsXW16KXyDQgLkq3Tu1Wvck+Z8mvkgk/RypOsbwV+yCLd6q3l7bZxn/F0w+p3ML4vLajXzMeh5+LnAsp2D6+vCDg3BpdGboy2ubKDbJmhkcNU3j+Tv28et7Tb1b4rmW8clCTxx6lbt+Ko5/zCH8pgn7C6p0gfuKd8FHqTEKkDvNtRNO1IONZowsvZBOGtUNWrJ46xW2Yl/1kXls1NWeFeeQDXyUJsONkCVJ7xi32OYHa0Ut/JiYepDyHQfRmJNziTkD61YVEFZJBbexCdFn/rhVFyHexIQR9Kw/qaGXrm+GTXIYBYSJVlqBB2nEMuf8rJ+uXdAWALCGGkHRRU8XPYsCrXYco2NCZcgMZkeoz4dCFhbvRIgUYLSdPK2DFM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu 14-09-23 11:21:39, Johannes Weiner wrote: > Breno and Josef report a deadlock scenario from cgroup reclaim > re-entering the filesystem: > > [ 361.546690] ====================================================== > [ 361.559210] WARNING: possible circular locking dependency detected > [ 361.571703] 6.5.0-0_fbk700_debug_rc0_kbuilder_13159_gbf787a128001 #1 Tainted: G S E > [ 361.589704] ------------------------------------------------------ > [ 361.602277] find/9315 is trying to acquire lock: > [ 361.611625] ffff88837ba140c0 (&delayed_node->mutex){+.+.}-{4:4}, at: __btrfs_release_delayed_node+0x68/0x4f0 > [ 361.631437] > [ 361.631437] but task is already holding lock: > [ 361.643243] ffff8881765b8678 (btrfs-tree-01){++++}-{4:4}, at: btrfs_tree_read_lock+0x1e/0x40 > > [ 362.904457] mutex_lock_nested+0x1c/0x30 > [ 362.912414] __btrfs_release_delayed_node+0x68/0x4f0 > [ 362.922460] btrfs_evict_inode+0x301/0x770 > [ 362.982726] evict+0x17c/0x380 > [ 362.988944] prune_icache_sb+0x100/0x1d0 > [ 363.005559] super_cache_scan+0x1f8/0x260 > [ 363.013695] do_shrink_slab+0x2a2/0x540 > [ 363.021489] shrink_slab_memcg+0x237/0x3d0 > [ 363.050606] shrink_slab+0xa7/0x240 > [ 363.083382] shrink_node_memcgs+0x262/0x3b0 > [ 363.091870] shrink_node+0x1a4/0x720 > [ 363.099150] shrink_zones+0x1f6/0x5d0 > [ 363.148798] do_try_to_free_pages+0x19b/0x5e0 > [ 363.157633] try_to_free_mem_cgroup_pages+0x266/0x370 > [ 363.190575] reclaim_high+0x16f/0x1f0 > [ 363.208409] mem_cgroup_handle_over_high+0x10b/0x270 > [ 363.246678] try_charge_memcg+0xaf2/0xc70 > [ 363.304151] charge_memcg+0xf0/0x350 > [ 363.320070] __mem_cgroup_charge+0x28/0x40 > [ 363.328371] __filemap_add_folio+0x870/0xd50 > [ 363.371303] filemap_add_folio+0xdd/0x310 > [ 363.399696] __filemap_get_folio+0x2fc/0x7d0 > [ 363.419086] pagecache_get_page+0xe/0x30 > [ 363.427048] alloc_extent_buffer+0x1cd/0x6a0 > [ 363.435704] read_tree_block+0x43/0xc0 > [ 363.443316] read_block_for_search+0x361/0x510 > [ 363.466690] btrfs_search_slot+0xc8c/0x1520 > > This is caused by the mem_cgroup_handle_over_high() not respecting the > gfp_mask of the allocation context. We used to only call this function > on resume to userspace, where no locks were held. But c9afe31ec443 > ("memcg: synchronously enforce memory.high for large overcharges") > added a call from the allocation context without considering the gfp. > > Reported-by: Breno Leitao > Reported-by: Josef Bacik > Fixes: c9afe31ec443 ("memcg: synchronously enforce memory.high for large overcharges") > Cc: stable@vger.kernel.org # 5.17+ > Signed-off-by: Johannes Weiner Acked-by: Michal Hocko Thanks! -- Michal Hocko SUSE Labs