From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 390E5C00A94 for ; Fri, 12 Apr 2024 08:47:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 858E56B007B; Fri, 12 Apr 2024 04:47:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 809056B0082; Fri, 12 Apr 2024 04:47:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6AA5D6B0083; Fri, 12 Apr 2024 04:47:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 4D5716B007B for ; Fri, 12 Apr 2024 04:47:40 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 1102C40DB9 for ; Fri, 12 Apr 2024 08:47:40 +0000 (UTC) X-FDA: 82000251480.24.0E6D561 Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) by imf08.hostedemail.com (Postfix) with ESMTP id 2FDFF160014 for ; Fri, 12 Apr 2024 08:47:37 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=D6BvSzXO; spf=pass (imf08.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.216.54 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712911658; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=w6ysC/lfoTTvwlV3mc3WbVSv5d9bBjero909B69z+ZQ=; b=F9QHpEmJ9XLbgSbUTTNp1W4OwM/QMQoS0lOM2L2IMLTNn6gCSbEiu8iDi0BtVU/KJDS/s4 ZaT6OpleVqdOBt8TNUfuIFvbVqyZFDgH1vOzGOqJNRc9JaiTj8BuqUTui/maqQynAv8Kht yWtjoQ5lhnKiH0gRdy5LyYZ862lYW04= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712911658; a=rsa-sha256; cv=none; b=GR5KJ5WQS1bi5Dbu7sDeIYkOL1LJdMroZIzIiMqcOII/UFETBV1URO3LjR1ATe0a9S5W0y gn+DTk495gKs1/NOQ5dNnk0iiZ+Gg+Q4pjZntbUmRmDygNsUwAoKvbnrKuPq15Au/FOmPQ kgiYGK0g2JTvc1GpN9yDey383IOHI6A= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=D6BvSzXO; spf=pass (imf08.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.216.54 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pj1-f54.google.com with SMTP id 98e67ed59e1d1-2a2f2cd61cdso177965a91.1 for ; Fri, 12 Apr 2024 01:47:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1712911656; x=1713516456; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=w6ysC/lfoTTvwlV3mc3WbVSv5d9bBjero909B69z+ZQ=; b=D6BvSzXOn4ZWUIHFG+AsnLtwkRDig3CXOhNgsuyJ7BCTqRVk7GVBaUt5M3kRzVVEFF QKUw/YXNqM7FgZGgbrcgvDk4m01/sVF+VCmjQZV1WCD+/QFIf6wFm7FP/mzAhKJ0lAXg DNZ3oP4HneMm/WZljn7uvZeNkyKwNubkqNb5IazYnvDI94Z2GjfYY4qdQbUPI+HWm/8c clC5+hJQ3muDWj0nfFUwni9a9y6HKToawgGfTGs+0VUz/Df4j5/RUPxTXo1PetOQZTQU +coarAgaW5bBUysHWTMidEYtowwnNJpkZ+AKq0tVOY5k1WRalc7JHA8FgNmor/26serF hVQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712911656; x=1713516456; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=w6ysC/lfoTTvwlV3mc3WbVSv5d9bBjero909B69z+ZQ=; b=o8lgN4o2yXfBpHF5CjVZOoDQOK5LOGGPD6YeCmdRT+izQHFRP+9ppWwA1eJAvuapMN q+N3hkbqqwlPUBwxxOLn+KnZN8vjTXjlf2N9Js3dmBCRHQMju8OIaM1CxcuACGuKjFst jB4TCxUbNwXVAAMKJYZ2Av6bIiNzPtuYxwJe+1s5D1Ta0O3FnwwZw0SxJqM3LynX5eFf JfUlsd4OOxLAdVtjyaUj8xQAkf6Dc6sXeBaE3G+ZmwwyRGQgj4mhS5DZvQgx/AajzZu2 2M6WfSKj+L8DaXEfHv6FwpzLnNKsILz4zBmqfGEhJt4TpLMr6+Yzzdq3nWjKpj5KZgrX vC8Q== X-Forwarded-Encrypted: i=1; AJvYcCX+tFdbvIbQ2YwNVARHE9S8ur12IWQga3WqN5/37uTQ6ifb+g+bBqaeu9ejSJJdy6LywpXbDmT5W+99oVf5eMsdmdg= X-Gm-Message-State: AOJu0Yx66rTlnAL3JIv53OrEcZjrcpETklIWcHGuq1kgF8BSoBCYjw80 GM2yY3tcHcdkZmUCFpfCPsX/trVmOyiqwA/jsaPS5zHvC7eIckqCewVX97Ntag4= X-Google-Smtp-Source: AGHT+IGXHCR7DkV2Qhp9EnIAwrmEKcmf9qAOdfy4XzgtY1646I6SP5pvyUHVAUz3Lr/p3J69gLG3ow== X-Received: by 2002:a17:90a:db0f:b0:2a6:8e66:d585 with SMTP id g15-20020a17090adb0f00b002a68e66d585mr1849884pjv.4.1712911655796; Fri, 12 Apr 2024 01:47:35 -0700 (PDT) Received: from [10.84.154.17] ([203.208.167.149]) by smtp.gmail.com with ESMTPSA id b8-20020a17090aa58800b002a574ab7f5esm2284129pjq.53.2024.04.12.01.47.32 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 12 Apr 2024 01:47:35 -0700 (PDT) Message-ID: Date: Fri, 12 Apr 2024 16:47:30 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] mm/shrinker: add SHRINKER_NO_DIRECT_RECLAIM Content-Language: en-US To: lipeifeng@oppo.com Cc: akpm@linux-foundation.org, david@fromorbit.com, roman.gushchin@linux.dev, muchun.song@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20240412080706.744-1-lipeifeng@oppo.com> From: Qi Zheng In-Reply-To: <20240412080706.744-1-lipeifeng@oppo.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Stat-Signature: drbfhpqbubr86x66hq48rtxo4fcgkqzf X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 2FDFF160014 X-Rspam-User: X-HE-Tag: 1712911657-665885 X-HE-Meta: U2FsdGVkX19WTTD0U0WTF9lVxOSrWq+Sgax8b305PQUjmLV4GSblu0d4tpO5PyyNEN66ETYEKM5xv6pGW+fOOGM0f3wN6j76R3qiiBljAEgA8Vr54ZWlN9XiSo3YR1MWGQuTLffQGeqiQOl60olsFr6MaTl0HUw8O1QRoO4HvZJ/ijk3YkcXyhhG9svzXzNldtTR5P0bD6ZFaSPyOxbX4EmH2u/hubl0eoflXbS2fmvx6KelKU9wqX/nLSnz3l8tIof5uThSpeidzzj7RN5fDV40/InV73wd2DEF0EyvyaAgzGlt0mx5nfwmGg5xtWLh63Y+h1X1iEu6Y63lBGyH6P2zG73vzml0w0K9gw9CzUBr9U+QcD5xZqv/zpAt388cASClOvD/CTE8Lyuq2nHJer9Ah2YE7t93M9HLcwDItB9tsitIQ+ZckmuLtxmf6Xq7fSAfvOOcdcrj+fc96l9t927nr7aUOX6ynkYmaZG8H28wDvljcSPZnhTfkd4RSLip5zAcEvhUc5zF849OIocyqqHXpdHarDgaD+0phsqz0XIEPtUOIwev3N2HEhVyzZUN8qEAmx7j1UUQ6QKhWD+OwWsvMY8rarikOtvYZySonTp8716ufXAlpj6xRaOlU6RA6Uq695teVixndnqyYdBcF/q2NSSRidZWQ8NfdVQTrN62z/Bn37LYrvx826VsOQ8bnRxGj1L2vuOJAjJMXE4ibz7kYgC16VtxzPD2XlsE5FL25t2/nyqFeL2CxiYcftAJG2miJBOkOdtA0tuAJaLSflXSsXbDcHNbkHjm4yzcjMJnVIEpsAbD9GrXwvq5tLQpP7YVD5GKhvuGTHL4vWmuNxe45PiOBhjI+aXn+/Mm4xZ/srVhjk9kdIiJAzLh84LhXn/yEZD98q4VzmaxUjxaNRpTfTUmyUq/fx5O/qUmCwAPOVah2R+5wF67ktFvy3eX1bRLzPu+lXeYrbT7ws/ v8RLkgIc NdPDMblqGLGmZNZk8/KMR0Jht8uF4XYlbxoPnmG2mi4r/srLG/UPXYUgJ8IvEmvwqxuqJciNO9ZuExalkhZcgXcmzByMn2Jwd5vahsRpFA+3doL91e17M24FgOaD2+2TamnDQteMw7IAfadEZI5bWAkmpzC04EBGw6/G99EOGd23OTIvZkLs2RCRBDLnH7aP7eA3ocaDPReEKhF9FE91s4mAvkHaBz09Z78C0OGeEL7hokiND7LP9s9B5YejBQSbtmbkOhVdNKREMRnOw80GltWXWZlpPL0Oul82A1ncPOIo0rRUsTumKqF5A1yp/XNALTNVXGsWUI10sIXWYgCov85XkhLI6ZXN8aKcfjhglrkSDMdl1jvyyRjmtjEgMNNUUFcUrXM4QJgFjCxfHXVhOCEIylTzu+6z+o2Gqu9NFRntTX7n/PnUOrWKBQVPUA1rK/xDSp9AgesJ3S5zcQjYite8rCpKpkjhTs5UFe2aoU4KwEI9i5nACKyRDfhtjqROdLmrO X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Peifeng, On 2024/4/12 16:07, lipeifeng@oppo.com wrote: > From: Peifeng Li > > In the case of insufficient memory, threads will be in direct_reclaim to > reclaim memory, direct_reclaim will call shrink_slab to run sequentially > each shrinker callback. If there is a lock-contention in the shrinker > callback,such as spinlock,mutex_lock and so on, threads may be likely to > be stuck in direct_reclaim for a long time, even if the memfree has reached > the high watermarks of the zone, resulting in poor performance of threads. > > Example 1: shrinker callback may wait for spinlock > static unsigned long mb_cache_shrink(struct mb_cache *cache, > unsigned long nr_to_scan) > { > struct mb_cache_entry *entry; > unsigned long shrunk = 0; > > spin_lock(&cache->c_list_lock); > while (nr_to_scan-- && !list_empty(&cache->c_list)) { > entry = list_first_entry(&cache->c_list, > struct mb_cache_entry, e_list); > if (test_bit(MBE_REFERENCED_B, &entry->e_flags) || > atomic_cmpxchg(&entry->e_refcnt, 1, 0) != 1) { > clear_bit(MBE_REFERENCED_B, &entry->e_flags); > list_move_tail(&entry->e_list, &cache->c_list); > continue; > } > list_del_init(&entry->e_list); > cache->c_entry_count--; > spin_unlock(&cache->c_list_lock); > __mb_cache_entry_free(cache, entry); > shrunk++; > cond_resched(); > spin_lock(&cache->c_list_lock); > } > spin_unlock(&cache->c_list_lock); > > return shrunk; > } > Example 2: shrinker callback may wait for mutex lock > static > unsigned long kbase_mem_evictable_reclaim_scan_objects(struct shrinker *s, > struct shrink_control *sc) > { > struct kbase_context *kctx; > struct kbase_mem_phy_alloc *alloc; > struct kbase_mem_phy_alloc *tmp; > unsigned long freed = 0; > > kctx = container_of(s, struct kbase_context, reclaim); > > // MTK add to prevent false alarm > lockdep_off(); > > mutex_lock(&kctx->jit_evict_lock); > > list_for_each_entry_safe(alloc, tmp, &kctx->evict_list, evict_node) { > int err; > > err = kbase_mem_shrink_gpu_mapping(kctx, alloc->reg, > 0, alloc->nents); > if (err != 0) { > freed = -1; > goto out_unlock; > } > > alloc->evicted = alloc->nents; > > kbase_free_phy_pages_helper(alloc, alloc->evicted); > freed += alloc->evicted; > list_del_init(&alloc->evict_node); > > kbase_jit_backing_lost(alloc->reg); > > if (freed > sc->nr_to_scan) > break; > } > out_unlock: > mutex_unlock(&kctx->jit_evict_lock); > > // MTK add to prevent false alarm > lockdep_on(); > > return freed; > } > > In mobile-phone,threads are likely to be stuck in shrinker callback during > direct_reclaim, with example like the following: > <...>-2806 [004] ..... 866458.339840: mm_shrink_slab_start: > dynamic_mem_shrink_scan+0x0/0xb8 ... priority 2 > <...>-2806 [004] ..... 866459.339933: mm_shrink_slab_end: > dynamic_mem_shrink_scan+0x0/0xb8 ... > > For the above reason, the patch introduces SHRINKER_NO_DIRECT_RECLAIM that > allows driver to set shrinker callback not to be called in direct_reclaim > unless sc->priority is 0. Hmm, this is just a workaround, no shrinker will want to set this flag. If a shrinker has a lock contention problem, then it needs to be fixed. Perhaps executing do_shrink_slab() asynchronously may be a more fundamental solution, but this may result in untimely reclamation. > > The reason why sc->priority=0 allows shrinker callback to be called in > direct_reclaim is for two reasons: > 1.Always call all shrinker callback in drop_slab that priority is 0. > 2.sc->priority is 0 during direct_reclaim, allow direct_reclaim to call > shrinker callback, to reclaim memory timely. > > Note: > 1.Register_shrinker_prepared() default not to set This API is no longer included in the latest upstream code. Please submit a patch based on the latest code. Thanks, Qi > SHRINKER_NO_DIRECT_RECLAIM, to maintain the current behavior of the code. > 2.Logic of kswapd and drop_slab to call shrinker callback isn't affected. > > Signed-off-by: Peifeng Li > --- > include/linux/shrinker.h | 5 +++++ > mm/shrinker.c | 36 ++++++++++++++++++++++++++++++++++-- > 2 files changed, 39 insertions(+), 2 deletions(-) > > diff --git a/include/linux/shrinker.h b/include/linux/shrinker.h > index 1a00be90d93a..2d5a8b3a720b 100644 > --- a/include/linux/shrinker.h > +++ b/include/linux/shrinker.h > @@ -130,6 +130,11 @@ struct shrinker { > * non-MEMCG_AWARE shrinker should not have this flag set. > */ > #define SHRINKER_NONSLAB BIT(4) > +/* > + * Can shrinker callback be called in direct_relcaim unless > + * sc->priority is 0? > + */ > +#define SHRINKER_NO_DIRECT_RECLAIM BIT(5) > > __printf(2, 3) > struct shrinker *shrinker_alloc(unsigned int flags, const char *fmt, ...); > diff --git a/mm/shrinker.c b/mm/shrinker.c > index dc5d2a6fcfc4..3ac50da72494 100644 > --- a/mm/shrinker.c > +++ b/mm/shrinker.c > @@ -544,7 +544,23 @@ static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid, > if (!memcg_kmem_online() && > !(shrinker->flags & SHRINKER_NONSLAB)) > continue; > - > + /* > + * SHRINKER_NO_DIRECT_RECLAIM, mean that shrinker callback > + * should not be called in direct_reclaim unless priority > + * is 0. > + */ > + if ((shrinker->flags & SHRINKER_NO_DIRECT_RECLAIM) && > + !current_is_kswapd()) { > + /* > + * 1.Always call shrinker callback in drop_slab that > + * priority is 0. > + * 2.sc->priority is 0 during direct_reclaim, allow > + * direct_reclaim to call shrinker callback, to reclaim > + * memory timely. > + */ > + if (priority) > + continue; > + } > ret = do_shrink_slab(&sc, shrinker, priority); > if (ret == SHRINK_EMPTY) { > clear_bit(offset, unit->map); > @@ -658,7 +674,23 @@ unsigned long shrink_slab(gfp_t gfp_mask, int nid, struct mem_cgroup *memcg, > continue; > > rcu_read_unlock(); > - > + /* > + * SHRINKER_NO_DIRECT_RECLAIM, mean that shrinker callback > + * should not be called in direct_reclaim unless priority > + * is 0. > + */ > + if ((shrinker->flags & SHRINKER_NO_DIRECT_RECLAIM) && > + !current_is_kswapd()) { > + /* > + * 1.Always call shrinker callback in drop_slab that > + * priority is 0. > + * 2.sc->priority is 0 during direct_reclaim, allow > + * direct_reclaim to call shrinker callback, to reclaim > + * memory timely. > + */ > + if (priority) > + continue; > + } > ret = do_shrink_slab(&sc, shrinker, priority); > if (ret == SHRINK_EMPTY) > ret = 0;