From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C6FFCDB465 for ; Thu, 12 Oct 2023 02:52:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE1AA8D00F8; Wed, 11 Oct 2023 22:52:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E919D8D0002; Wed, 11 Oct 2023 22:52:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D5A4C8D00F8; Wed, 11 Oct 2023 22:52:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id C5B7D8D0002 for ; Wed, 11 Oct 2023 22:52:08 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 9FBA91A04C1 for ; Thu, 12 Oct 2023 02:52:08 +0000 (UTC) X-FDA: 81335285136.15.D117EEE Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by imf20.hostedemail.com (Postfix) with ESMTP id CAF6A1C000D for ; Thu, 12 Oct 2023 02:52:05 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=cnW4YVH3; spf=pass (imf20.hostedemail.com: domain of zhangpeng.00@bytedance.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=zhangpeng.00@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697079126; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sgvsnkzJDg833cgyFUp1z86vPxjfwxf/muQwdEgBqiQ=; b=IPyofsgwueZBnCs5tKgrz5qyQAJNUMXxLHYnGRVJzB1tOZBDhjoboKQ0UsCNLeYPQaYlQD 2RLnzffgG5Tj50HZAcdb3bDb1Q0J3lbvpSduDDXqLuElC4iv1zenK94HzpaXzVoJ4zHKb2 yF9vboWAo8uXdkKy0trCms6TmP1FJjc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697079126; a=rsa-sha256; cv=none; b=MJVp1XhbUDE5nov7hqbMOIeyB3xZ8mcvosj1udPycdvfeyiCvqp3wWK/wtc2yaFKkHBNIK t+Z8rA9RoyUF6rPOGOsF2zRyIgkIWEBzhQK37Oyu9Pwn0XmNA+JB8UQrd1/mVUv/EckNtP 3PGUUUnDHvx9oG1q1bCGHIKyhGdwQDU= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=cnW4YVH3; spf=pass (imf20.hostedemail.com: domain of zhangpeng.00@bytedance.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=zhangpeng.00@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-1c894e4573bso4173075ad.0 for ; Wed, 11 Oct 2023 19:52:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1697079124; x=1697683924; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=sgvsnkzJDg833cgyFUp1z86vPxjfwxf/muQwdEgBqiQ=; b=cnW4YVH3ROXKQ+AQ/2N+pTyqwqk04GOx6zslA8V7eOsVLYTskBUxNyOWyieMKf5/H3 LLQbZXW+FX2zaa/pc2jkukiqNCmyP2HJETHHYytMoIHoVpBFTlrMVfexsIWK36WBpm1r GUOEfGYM/noSy9NQ6VIWPAJAKYieFQFNZZy8jlwA+KM2ofQd51ikdVkQsaggMKBn4h8q xLJ3DNj3ns1DiJGG5/tkMp7cZqsfa9gpAN8aGAGtq7K+Th3RmnsD9BQOTAoNBiKKCmUb yquUeHd5TAL6J1Sj8as1vDtg31tgBWmNmnWeMZqXyXU5jFII4h9sv0Qwx1I9F1wvP/VW uF+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697079124; x=1697683924; h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=sgvsnkzJDg833cgyFUp1z86vPxjfwxf/muQwdEgBqiQ=; b=hcbmUY8fhtzM2DorRPlmqaaVjpWVb6KsLxYi1sW+ayvcVJJEOsCtynCxA9CKTVJT/R w7CJUhBKk2wCEsN+vK1tdEUc5xT8IcBOQheG9I9O52W2rQrDQGAYG2u3XuwTdjcG6Wj+ JwQ1bBoa63BSJdNV60deILj/xdUirc9/KcK3TbpxwJtWz0efdX+EQIviDcY5/qMtGC6G W6R9lQq2LHiRHLf4iceeY07t+kG43oOMSzul2DsGjw9Ac7xosIjBJF9gdxmjyLJQmt2Z 78/PovkcNp3UiRvDtpABEwjtL7kb8Ku3MNcly8M9ukq74+BBQ+VmI4KqM+AYYFgbUKMo 6d6Q== X-Gm-Message-State: AOJu0YwX3N01XSklqhgoKzN3T3vCmRKe/0Z9WevFjz2nfnipC8DhgBLC sl/IWUicep/pAcmqs7ObiQbRfg== X-Google-Smtp-Source: AGHT+IE3KCzPp1Sgb0WmF/euQZXp9MPXqgBd3ibxjoqdrYsQ/KvMgeyxAyVFoIqIRbHcd59EcyjcRw== X-Received: by 2002:a17:902:c952:b0:1c0:d17a:bfef with SMTP id i18-20020a170902c95200b001c0d17abfefmr23975582pla.30.1697079124416; Wed, 11 Oct 2023 19:52:04 -0700 (PDT) Received: from [10.84.144.218] ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id ji20-20020a170903325400b001bbc8d65de0sm635992plb.67.2023.10.11.19.52.01 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 11 Oct 2023 19:52:04 -0700 (PDT) Message-ID: <14679c1d-463c-4dfb-8ccb-8b58c9e36c36@bytedance.com> Date: Thu, 12 Oct 2023 10:51:58 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] maple_tree: Add GFP_KERNEL to allocations in mas_expected_entries() To: "Liam R. Howlett" Cc: maple-tree@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org, jason.sim@samsung.com, Andrew Morton References: <20231010141737.1592866-1-Liam.Howlett@oracle.com> From: Peng Zhang In-Reply-To: <20231010141737.1592866-1-Liam.Howlett@oracle.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Stat-Signature: wpwmg6aqjaj84dxxii71ut7mgo87emtf X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: CAF6A1C000D X-Rspam-User: X-HE-Tag: 1697079125-885210 X-HE-Meta: U2FsdGVkX19TMYQ3sjrwDc7g0XCN5JR5rfHOo18DYYyMV0gdzAZzX6NNdzSdYGso+at/4qlueOy0a0092y78LDDnhmnIuy5m1jYSZeqmxPUjIDmAvSGqy75vrHoExWvFa+Ga72H93IDEmmO5ABGHhYF3vnISEy8FPfjUUIzEQXyrKfYUW52DfqMfans7jd3HZm9CYvpz9u0Q4x4thD7d+OBOzu551pSolZP1197kaKoFuz0TcI0S4qpgB5WwFKsRXvaOOZCMURlsVxQfU07oqHggmu1ZjZl4yKigb69I8u0KMWjfw2VsMk20fIdAKqXuqo2N8d62tROSFxYbk1UYu+KqI1QJe1TqTiwpAtFVFJtgRamyq4c0cgJr2rWsmGkpwGghwn/HnKq3MtFvzQLcSdWH9MbP9E9d0AVj80ViYFnnbeQZ9OJ97saAUjCIogIZahGHIohEJFHruSo3eILIS+NRPqMpU4nfg0m/q7aVPGDpNK4+d+qMCEDao45Lk7fVA9B+9DzqGRdN+wtBxVNyc4u8eFI6O6DxcRs3Dz8OBmrlBmUBFNNBX8oz6rKky5e1qylL5o/4Vn0Tm+mzZuuB0kA04rSs04HDV4SLlTuCRQXc4xIE1rULEKudl3g0YAkwfotHHpqtmmy30w9da4u60Urf4r1HdCVbQ8al2XpMvw91LnKNXVOYMYn1A1VEOJRlP0DrQQbv8O2v2FEsDt89SnBDXz1zFzUKscBIR2cKpJvZXPN/Mu14IRjk9MGvh/3b0tMkb85ljiNwL8voJhyQy9MGrDqqVFe8bRM71uj/9NYaVjEFsUsDOyRpDDFMuUy+ujBB17q+RhL2YsAr4c7BbH0WUmIhckcW+wdOZxjUip4QCMktUzV1AH3lqyeaIRF/f82HJPyjhKLNbgOSJwNjA3WiluoQz3S3/T93od+ZE+rNQIa1N9lYJoWHBZsY1Ud3SAr2dh5jkQXqE1oTE9O lWNevArb pMlmboAbdo+q00odeypFZN1hDMka5AFSoVvQlvWfdfxdja9g6618JU7Qbb+DulJ5ry1Hh6HsOH62QLd3yIKnjww2VuPtVQRSqtHAx9AAUTdq237vU/K+UT9xDmL8HPT8QrkR5emsOPxgUjLCd4bsmrmzV0caZAF0yA80jyfi5mSTRbnWdZJvcjg4KYsqWMaIfJVCIY60vtxJ6PRv9h16ql1QMkOtskmBSLrhvm5HnaAdjYRalEi3E+fKJGs8fOqbpy4HzaU+TD5zM7ri8PF+9SjKCyyC1NS1KGD1aLp4B0aJCUELCRjvnBkzO2FhZrLtGzOaUUh9CER1llAh8ZXCszIgpIusjUVts/3iHHx6d9C4msmj2Roy39ZYSK7THTVLiGOfIP14kfU452i+KnrNCCbHy+bv9Rbb3gro+FF2g4MKfKREkX/Bmrk1Q+mfPMTzN8idYtR5wuKzUKuUm39AWIyMeMkq+IAb05ls64ufjNo/zDU7eaZGY6EnhzQfdfZQldPPx8+nLRtN4lsrycRFzi9I81sgDvhszg3hKYqEVbsLAD8rzkJ5U1JaxyRB45ZgUZ3rl1sPYCYaT+qM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: 在 2023/10/10 22:17, Liam R. Howlett 写道: > Users complained about OOM errors during fork without triggering > compaction. This can be fixed by modifying the flags used in > mas_expected_entries() so that the compaction will be triggered in low > memory situations. Since mas_expected_entries() is only used during > fork, the extra argument does not need to be passed through. > > Additionally, the testing in the maple tree fork testing needed to be > altered to use the correct locking type so that allocations would not > trigger sleeping and thus failures in the testing. The additional > locking change requires rwsem support additions to the tools/ directory > through the use of pthreads pthread_rwlock_t. With this change > test_maple_tree works in userspace, as a module, and in-kernel. > > Users may notice that the system gave up early on attempting to start > new processes instead of attempting to reclaim memory. You still haven't updated check_forking() and bench_forking(). It will still produce the same error as [1]. [1] https://lore.kernel.org/oe-lkp/202309242123.7ebe65b5-oliver.sang@intel.com/ > > Link: https://lkml.kernel.org/r/20230915093243epcms1p46fa00bbac1ab7b7dca94acb66c44c456@epcms1p4 > Fixes: 54a611b60590 ("Maple Tree: add new data structure") > Cc: > Cc: jason.sim@samsung.com > Signed-off-by: Liam R. Howlett > --- > lib/maple_tree.c | 2 +- > lib/test_maple_tree.c | 13 ++++++++---- > tools/include/linux/rwsem.h | 40 +++++++++++++++++++++++++++++++++++++ > 3 files changed, 50 insertions(+), 5 deletions(-) > create mode 100644 tools/include/linux/rwsem.h > > diff --git a/lib/maple_tree.c b/lib/maple_tree.c > index 0e00a84e8e8f..bb24d84a4922 100644 > --- a/lib/maple_tree.c > +++ b/lib/maple_tree.c > @@ -5627,7 +5627,7 @@ int mas_expected_entries(struct ma_state *mas, unsigned long nr_entries) > /* Internal nodes */ > nr_nodes += DIV_ROUND_UP(nr_nodes, nonleaf_cap); > /* Add working room for split (2 nodes) + new parents */ > - mas_node_count(mas, nr_nodes + 3); > + mas_node_count_gfp(mas, nr_nodes + 3, GFP_KERNEL); > > /* Detect if allocations run out */ > mas->mas_flags |= MA_STATE_PREALLOC; > diff --git a/lib/test_maple_tree.c b/lib/test_maple_tree.c > index 06959165e2f9..204743bc161c 100644 > --- a/lib/test_maple_tree.c > +++ b/lib/test_maple_tree.c > @@ -9,6 +9,7 @@ > > #include > #include > +#include > > #define MTREE_ALLOC_MAX 0x2000000000000Ul > #define CONFIG_MAPLE_SEARCH > @@ -2616,6 +2617,10 @@ static noinline void __init check_dup_gaps(struct maple_tree *mt, > void *tmp; > MA_STATE(mas, mt, 0, 0); > MA_STATE(newmas, &newmt, 0, 0); > + struct rw_semaphore newmt_lock; > + > + init_rwsem(&newmt_lock); > + mt_set_external_lock(&newmt, &newmt_lock); > > if (!zero_start) > i = 1; > @@ -2625,9 +2630,9 @@ static noinline void __init check_dup_gaps(struct maple_tree *mt, > mtree_store_range(mt, i*10, (i+1)*10 - gap, > xa_mk_value(i), GFP_KERNEL); > > - mt_init_flags(&newmt, MT_FLAGS_ALLOC_RANGE); > + mt_init_flags(&newmt, MT_FLAGS_ALLOC_RANGE | MT_FLAGS_LOCK_EXTERN); > mt_set_non_kernel(99999); > - mas_lock(&newmas); > + down_write(&newmt_lock); > ret = mas_expected_entries(&newmas, nr_entries); > mt_set_non_kernel(0); > MT_BUG_ON(mt, ret != 0); > @@ -2640,9 +2645,9 @@ static noinline void __init check_dup_gaps(struct maple_tree *mt, > } > rcu_read_unlock(); > mas_destroy(&newmas); > - mas_unlock(&newmas); > > - mtree_destroy(&newmt); > + __mt_destroy(&newmt); > + up_write(&newmt_lock); > } > > /* Duplicate many sizes of trees. Mainly to test expected entry values */ > diff --git a/tools/include/linux/rwsem.h b/tools/include/linux/rwsem.h > new file mode 100644 > index 000000000000..83971b3cbfce > --- /dev/null > +++ b/tools/include/linux/rwsem.h > @@ -0,0 +1,40 @@ > +/* SPDX-License-Identifier: GPL-2.0+ */ > +#ifndef _TOOLS__RWSEM_H > +#define _TOOLS__RWSEM_H > + > +#include > + > +struct rw_semaphore { > + pthread_rwlock_t lock; > +}; > + > +static inline int init_rwsem(struct rw_semaphore *sem) > +{ > + return pthread_rwlock_init(&sem->lock, NULL); > +} > + > +static inline int exit_rwsem(struct rw_semaphore *sem) > +{ > + return pthread_rwlock_destroy(&sem->lock); > +} > + > +static inline int down_read(struct rw_semaphore *sem) > +{ > + return pthread_rwlock_rdlock(&sem->lock); > +} > + > +static inline int up_read(struct rw_semaphore *sem) > +{ > + return pthread_rwlock_unlock(&sem->lock); > +} > + > +static inline int down_write(struct rw_semaphore *sem) > +{ > + return pthread_rwlock_wrlock(&sem->lock); > +} > + > +static inline int up_write(struct rw_semaphore *sem) > +{ > + return pthread_rwlock_unlock(&sem->lock); > +} > +#endif /* _TOOLS_RWSEM_H */