From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1593ACDB465 for ; Mon, 16 Oct 2023 03:40:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8B3216B0195; Sun, 15 Oct 2023 23:40:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 83C456B0196; Sun, 15 Oct 2023 23:40:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6DC896B0197; Sun, 15 Oct 2023 23:40:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 58ED86B0195 for ; Sun, 15 Oct 2023 23:40:38 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 97825A0A6C for ; Mon, 16 Oct 2023 03:40:37 +0000 (UTC) X-FDA: 81349922514.15.60559BD Received: from mail-yw1-f174.google.com (mail-yw1-f174.google.com [209.85.128.174]) by imf15.hostedemail.com (Postfix) with ESMTP id F27C2A0005 for ; Mon, 16 Oct 2023 03:40:34 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=g4lQEzP4; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf15.hostedemail.com: domain of zhangpeng.00@bytedance.com designates 209.85.128.174 as permitted sender) smtp.mailfrom=zhangpeng.00@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697427635; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BI2/fjvxux9kOhr+MN6nR47zrqV+MlgEJ7nQc3CGiEU=; b=Co2QQIDNEui9NEk59p2VGOjhAKy4Bc4zqw/r3UqH6jXaNeup4h2EevofV1njv8QYXzq0yN l/b7lVlvbBl9I3+eQeuDFzPegpiJW35+8dKu6aPXWmeJzjM7xgOvKyDlmZ3WSWNVeAESZ9 QwDTstpx9E9Xc3BS+/LZmneV5kQwTYI= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=g4lQEzP4; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf15.hostedemail.com: domain of zhangpeng.00@bytedance.com designates 209.85.128.174 as permitted sender) smtp.mailfrom=zhangpeng.00@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697427635; a=rsa-sha256; cv=none; b=q+t8t4yX5nmgsJo/hQuNSetYxNBcDHhvKxCpCUG1P8AKLuSDUQbgytjif3AnaxZAlPL+t1 DeHJgoVftcPueXYGJr4tjPu6QyTOgrkSin/fNW+AFS5LoQLe9ZBzUMstn+32ZVHf9+swzU HMSSs23r8yBreN237qik8isfwJrbARE= Received: by mail-yw1-f174.google.com with SMTP id 00721157ae682-5a81ab75f21so33561017b3.2 for ; Sun, 15 Oct 2023 20:40:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1697427634; x=1698032434; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=BI2/fjvxux9kOhr+MN6nR47zrqV+MlgEJ7nQc3CGiEU=; b=g4lQEzP4+3f8DI47H4/8XQycjaBLU6iMo50W+mWW3CHnOKXGPhBFj9ZqF41HX2S8/B qZE8kTEGY1us9ULLqNA9WeWqB+fSQNA350+3fdk/lZpjU1HszGVd8rEOgScXWLDNvJ46 SIbA4AksEaUFbrcKlQQcOxzjFzZijEhsb8g7vOmhrsreUvE8InDz0GayxXqMsiwjVyMA yTU0VQZRuqC+UDoSgdbLVoGKci/FRnaH6a1kvBy0MLwC3w6ue7RPaYq4LjsTsfE8yFQ2 qxgGtIcW/wJ9wYLLWUPKQer01JxTefy7t/Cd8m3aE0j7UFsx+auQJhRAiMy5+KOFSgZo ZmDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697427634; x=1698032434; h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=BI2/fjvxux9kOhr+MN6nR47zrqV+MlgEJ7nQc3CGiEU=; b=rU5f6eSLTqmyT1dUyVSGFtsG2wuTFhiocbLoGajaanc88HBh8vqnrhAB/lYvOSlZqg Jh7Gf9jDssaS/lkOsAs6JjNTJEH5oSbK0UO58Tm+fxHW4AazcPPjBSShlG86y6YglqXs 4boR82XIWUDSGVKdOpmYJ5f9BUzQuD+FvCPMsVqk2dHXAPjfmyc2jUxMfa0IW6iDjrky 51VyXguUxC3V/HbJ8P9jWjFNLBwyKylD1yt0FkzoF+soekesrUoZPNgaed+nT+apaxUH OjduOTBTWitA4Rxh78jVY0RFzVciU0QF69q4IVW0lIn5p6vBa4nkHn+XmLNqdlV88kb9 ofxw== X-Gm-Message-State: AOJu0YyevHgKondv+9wNVgWcgt5eOysygyDdfE8AugVU+fcDy8f+EMiE 4KizOu+x3onhdEhK8grOBd5HqA== X-Google-Smtp-Source: AGHT+IHros4bTzExUZqJrCKsN+Ddzh3apHzeOi8pHztUz/zBms1TVFaa5iq2H9se67ANKhFZ0rn2OA== X-Received: by 2002:a0d:d64b:0:b0:5a8:60ad:39a4 with SMTP id y72-20020a0dd64b000000b005a860ad39a4mr3655444ywd.3.1697427633775; Sun, 15 Oct 2023 20:40:33 -0700 (PDT) Received: from [10.255.187.14] ([139.177.225.232]) by smtp.gmail.com with ESMTPSA id 66-20020a630045000000b005ab7b055573sm4753882pga.79.2023.10.15.20.40.27 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 15 Oct 2023 20:40:33 -0700 (PDT) Message-ID: Date: Mon, 16 Oct 2023 11:40:24 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v5 00/10] Introduce __mt_dup() to improve the performance of fork() To: Liam.Howlett@oracle.com, corbet@lwn.net, akpm@linux-foundation.org, willy@infradead.org, brauner@kernel.org, surenb@google.com, michael.christie@oracle.com, mjguzik@gmail.com, mathieu.desnoyers@efficios.com, npiggin@gmail.com, peterz@infradead.org, oliver.sang@intel.com, mst@redhat.com Cc: maple-tree@lists.infradead.org, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Peng Zhang References: <20231016032226.59199-1-zhangpeng.00@bytedance.com> From: Peng Zhang In-Reply-To: <20231016032226.59199-1-zhangpeng.00@bytedance.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: F27C2A0005 X-Stat-Signature: d64hgwihh4a8bhnego4aju654e3gx358 X-Rspam-User: X-HE-Tag: 1697427634-234491 X-HE-Meta: U2FsdGVkX1+LGYwkxH4ARp20Tf3xvj1xSCJMDuElVRus58/Jr0DANIZHFH4T5VvizvJFGV1ys/XsOSDHU3dXgCPCaBVfee4ulOggbrhWEhLcEkn8W2cxQfro9MfpbqF/4fgbQ7mct9AKjMYsi6wt9ev5I4xUVbZ/CEFvVEGItXMdj5B1SgCSvIhGzl45YovtxsX8jaZye9DKvqO4i+hk+kr8NJ/eVfVyP3ElSqqgYP7riKYb1l5cS3EkwYbCP5k2cWzfg3KL5s88Pdj+NLVcqlqTqt7BoBVzY0UtHZSOz7+DBZLUqpCvjOaQTmr682nBhCXBBFCgqAOpXrsK6Q4yD7etnIjFO0m5Hlb+gTeHwNfEK55nNu0l0yRo6VtcopBLMGJHuMWoTy8JGM6VtwwEgV9PYorByUIyNm7h8z9mzodj3R8oB0Zn95oamtOHJzAhUmDrtIJQEVHkE7geJ/dwZ5zDfhnQ485G9c1LbMPi/MKy9FkIjCegltl9QUIN45HFZVzOsi9xrmBS8jlSoHfXNzST/Pf7UpkmIiJtEebZJjt26FdTMB8+zCkzUZbcfldkYUhhA4SmZVvYOoh8h2ejMbI58ilvz+rifJ8x0dYgY4PdhK8fv4tr9d53GU7D6eZ3bSVd3qYsNUjVokyN+rE9cMyryBviGr0UNgZ4QgEaN6BdfHWSre40cVQC8ZK2b7BEHNxVkAjAkPjiGEQBU37WvN3Rq8jEFytPk7eLIAHh6BADXyWi6CDVLr3MiKXY3WubvIv9EkRkGZFhaWojYMAFNBCwyJjQF8as+bZSIEECg7TRBzsFH/OQXDgLhGol9SYgXM8hESLdq9evlJpD/PRXva/APRsTlUPUcaUlRM08mGEaL7ZjXEvL5X0ucBR++gcmV7UqaIPW1fCEoAfGCERoRDvzimSocBxRQmlrr4g2psQhB0CEXdugf+tItCeC+fM6cBXiT8Wep+E8BHNU2Gd L7pKBCXZ 3ntb+2i42hfCCvuHARHkB+RBVRZfcLi1lrrsDAD6HAXi18QnxBRgmclypp9PYJEWJYNfUcB/AQbVzBCM50zJmeiAIhEcDCXLdIUuTLdLGnRobNlZRSlA0RekFeFsV+HHcxZgJT7OFMnJUjGfri7cYUQLAmL6lgl7M/u4hHV/n1ALd4Qjg0aAXGBMPXGn/tkCLG7HxEmOO9w0DcHygGTA7klMSzQaaXtVUgFA+RlkjerJHadRPjX73svI9XmF3zKoH+VQ31OMEUtOY7OBCehXVcXkad1J9XajvnGTVUML51YuT3ePghKjdY4VscxkOpRqmJDLq9mdF7DVWSL3ghalBRL0/cIiiNVVJJR+HuYrzmhAddXGlGghXm6+qT1IKEJGUFABwPDU8cPyEoGgTJuxJZ7VwwUBcOAflNC9Ekaetpks8V16RaXSfcsq71UclnfcBxXqmsC8BHUeE51VdvWnfc8dgkRbFI1xtKH4JfiS8PoEJjFCzrlrK0ReU32bY47d6p4CD40RBEcszdq+O8VaG7hVEBd/xQRA6XyVHPL+V1eRC6fLaf9LZfhfPWKNK27pForlaCChrjosqF5RyQbnIWZZxJhleFLVN8o/sbtfvpBN6UNs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: 在 2023/10/16 11:22, Peng Zhang 写道: > Hi all, > > This series introduces __mt_dup() to improve the performance of fork(). During > the duplication process of mmap, all VMAs are traversed and inserted one by one > into the new maple tree, causing the maple tree to be rebalanced multiple times. > Balancing the maple tree is a costly operation. To duplicate VMAs more > efficiently, mtree_dup() and __mt_dup() are introduced for the maple tree. They > can efficiently duplicate a maple tree. > > Here are some algorithmic details about {mtree,__mt}_dup(). We perform a DFS > pre-order traversal of all nodes in the source maple tree. During this process, > we fully copy the nodes from the source tree to the new tree. This involves > memory allocation, and when encountering a new node, if it is a non-leaf node, > all its child nodes are allocated at once. > > Some previous discussions can be referred to as [1]. For a more detailed > analysis of the algorithm, please refer to the logs for patch [3/10] and patch > [10/10] > > There is a "spawn" in byte-unixbench[2], which can be used to test the > performance of fork(). I modified it slightly to make it work with > different number of VMAs. > > Below are the test results. The first row shows the number of VMAs. > The second and third rows show the number of fork() calls per ten seconds, > corresponding to next-20231006 and the this patchset, respectively. The > test results were obtained with CPU binding to avoid scheduler load > balancing that could cause unstable results. There are still some > fluctuations in the test results, but at least they are better than the > original performance. > > 21 121 221 421 821 1621 3221 6421 12821 25621 51221 > 112100 76261 54227 34035 20195 11112 6017 3161 1606 802 393 > 114558 83067 65008 45824 28751 16072 8922 4747 2436 1233 599 > 2.19% 8.92% 19.88% 34.64% 42.37% 44.64% 48.28% 50.17% 51.68% 53.74% 52.42% > > Thanks for Liam's review. > > Changes since v4: > - Change the handling method for the failure of dup_mmap(). Handle it in > exit_mmap(). > - Update check_forking() and bench_forking(). > - Add the corresponding copyright statement. > I apologize for forgetting to include all the links while editing the cover letter. Here they are: [1] https://lore.kernel.org/lkml/463899aa-6cbd-f08e-0aca-077b0e4e4475@bytedance.com/ [2] https://github.com/kdlucas/byte-unixbench/tree/master v1: https://lore.kernel.org/lkml/20230726080916.17454-1-zhangpeng.00@bytedance.com/ v2: https://lore.kernel.org/lkml/20230830125654.21257-1-zhangpeng.00@bytedance.com/ v3: https://lore.kernel.org/lkml/20230925035617.84767-1-zhangpeng.00@bytedance.com/ v4: https://lore.kernel.org/lkml/20231009090320.64565-1-zhangpeng.00@bytedance.com/ > Peng Zhang (10): > maple_tree: Add mt_free_one() and mt_attr() helpers > maple_tree: Introduce {mtree,mas}_lock_nested() > maple_tree: Introduce interfaces __mt_dup() and mtree_dup() > radix tree test suite: Align kmem_cache_alloc_bulk() with kernel > behavior. > maple_tree: Add test for mtree_dup() > maple_tree: Update the documentation of maple tree > maple_tree: Skip other tests when BENCH is enabled > maple_tree: Update check_forking() and bench_forking() > maple_tree: Preserve the tree attributes when destroying maple tree > fork: Use __mt_dup() to duplicate maple tree in dup_mmap() > > Documentation/core-api/maple_tree.rst | 4 + > include/linux/maple_tree.h | 7 + > kernel/fork.c | 39 ++- > lib/maple_tree.c | 304 ++++++++++++++++++++- > lib/test_maple_tree.c | 123 +++++---- > mm/memory.c | 7 +- > mm/mmap.c | 9 +- > tools/include/linux/rwsem.h | 4 + > tools/include/linux/spinlock.h | 1 + > tools/testing/radix-tree/linux.c | 45 +++- > tools/testing/radix-tree/maple.c | 363 ++++++++++++++++++++++++++ > 11 files changed, 815 insertions(+), 91 deletions(-) >