From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13163C4345F for ; Fri, 12 Apr 2024 13:53:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 945116B0082; Fri, 12 Apr 2024 09:53:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8F5116B0093; Fri, 12 Apr 2024 09:53:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 795E16B0099; Fri, 12 Apr 2024 09:53:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 5C91A6B0082 for ; Fri, 12 Apr 2024 09:53:39 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 1C34B40EA8 for ; Fri, 12 Apr 2024 13:53:39 +0000 (UTC) X-FDA: 82001022558.15.018B407 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf21.hostedemail.com (Postfix) with ESMTP id 982671C0005 for ; Fri, 12 Apr 2024 13:53:35 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="CBD/MbzD"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=7AuqAnHB; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="CBD/MbzD"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=7AuqAnHB; dmarc=none; spf=pass (imf21.hostedemail.com: domain of jack@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=jack@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712930016; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jCLBp0r4GxrmZerl3rC8Ggqs0l1ATGPD2aQmbUqPAUc=; b=3qcM8cKhgUEuBn81iUVq7BHhPVg8JrP/NddLxvHNKpAUJe0PkSfWMc6jrUQMZm+ML/7Ida ZJtexvTb9rEB7h1TlLSAEq/O4aUzGlFzinopaUTl/9vhNYCs27Lam6YCmFAqrJ4lsl2YMi D7vNrTfLOxBoDVobygqrW5fqDqk+j1s= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="CBD/MbzD"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=7AuqAnHB; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="CBD/MbzD"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=7AuqAnHB; dmarc=none; spf=pass (imf21.hostedemail.com: domain of jack@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=jack@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712930016; a=rsa-sha256; cv=none; b=wC6SHDBkkrX/zkanHqZ0yo5mv5KF+b4qsWTz8ic750kUTB5Ci32OzoTBqbDG1Rk7XOKlhu vqAhINHLWMRkK/oU7PeLeKFgxXU1/EOOzm0N6j8uycT67xTIXlDA/f6YFSDsnR+jlL5Noo JR6QR7tkSaZ4H1pCag1LureXMTZJCkA= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id F337E3830F; Fri, 12 Apr 2024 13:53:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1712930014; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=jCLBp0r4GxrmZerl3rC8Ggqs0l1ATGPD2aQmbUqPAUc=; b=CBD/MbzDWjQxuey1zwBm6rnFSe32HR4hrhhkHaHodFYalLO6TlPXNHri8G0bE2G4xVeeTb jW+ooGE7kZG4iLYvVCHW5upttpBzAQ1/kyHcmVt0vthqpdbEbyXN5iKG4oHwieR0twYkwI hll/Dy6z5NTe5Y/PB3XLuypUfR7posI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1712930014; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=jCLBp0r4GxrmZerl3rC8Ggqs0l1ATGPD2aQmbUqPAUc=; b=7AuqAnHBi6HzN+IXZm6EqRvPOTn+ifE0iMDwxISYQ9A9DjH0F8CdrTTvga+MgtlnopAZAq ssNK/HhvvJjJrJDw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1712930014; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=jCLBp0r4GxrmZerl3rC8Ggqs0l1ATGPD2aQmbUqPAUc=; b=CBD/MbzDWjQxuey1zwBm6rnFSe32HR4hrhhkHaHodFYalLO6TlPXNHri8G0bE2G4xVeeTb jW+ooGE7kZG4iLYvVCHW5upttpBzAQ1/kyHcmVt0vthqpdbEbyXN5iKG4oHwieR0twYkwI hll/Dy6z5NTe5Y/PB3XLuypUfR7posI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1712930014; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=jCLBp0r4GxrmZerl3rC8Ggqs0l1ATGPD2aQmbUqPAUc=; b=7AuqAnHBi6HzN+IXZm6EqRvPOTn+ifE0iMDwxISYQ9A9DjH0F8CdrTTvga+MgtlnopAZAq ssNK/HhvvJjJrJDw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id D978413942; Fri, 12 Apr 2024 13:53:33 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id Tr4MNd08GWaFZgAAD6G6ig (envelope-from ); Fri, 12 Apr 2024 13:53:33 +0000 Received: by quack3.suse.cz (Postfix, from userid 1000) id 5758CA071E; Fri, 12 Apr 2024 15:53:33 +0200 (CEST) Date: Fri, 12 Apr 2024 15:53:33 +0200 From: Jan Kara To: Peng Zhang Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, dennisszhou@gmail.com, shakeelb@google.com, jack@suse.cz, surenb@google.com, kent.overstreet@linux.dev, mhocko@suse.cz, vbabka@suse.cz, yuzhao@google.com, yu.ma@intel.com, wangkefeng.wang@huawei.com, sunnanyong@huawei.com Subject: Re: [RFC PATCH 0/3] mm: convert mm's rss stats into lazy_percpu_counter Message-ID: <20240412135333.btd6e7wfprg4cmx2@quack3> References: <20240412092441.3112481-1-zhangpeng362@huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240412092441.3112481-1-zhangpeng362@huawei.com> X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 982671C0005 X-Stat-Signature: ujjrojhk9qntdaquuzueru8ym895j153 X-Rspam-User: X-HE-Tag: 1712930015-399230 X-HE-Meta: U2FsdGVkX1+gCGQ5o9Iho0S5jMpQy8ULqXmJy+3ro1sz0xKl1wUp213dzFMNiEnVZAOdq/bFYIFjsKNiSam9UF6gctyaFbwcWMMr7enWRLtzA1x6RHaZkcV/hnmlr9lEM/9+bl31U9NNchnX3Dqjb7OEnZOoiTc9u3RpTUuU0xfUuj8dAxySTFRMOZ1Du+LXhOtZfzhHLFhQRTU0FoO35MkF7xC8speXOysoCOP68g34wzyFCA3vxS0SimLYm7+/8YqEg7owHwE7NrXTwIs6EExv8V25hQ/sp1jwQHY38UgPJMpdrg1GhN15qr37Y8NThFw9CEg7hzkszP2N2U3cDiyl/sR1EiKYMk4RHzDTwCnzxZpAmp90wXxbaG7AHLa6gPLEnq8TpV4GWOvPf1qVEj5rIPF8KUMm1/kMnMfOkuVniFxpos9zb+js74pXwnYMOG18R4YHhStm/wQHvIUiAqW3L1QxUbXkqKW5Q2mpGqMIdww8Appa1TFS7ngI3diWSInQpStgYHMNtXXpbvkoh6FRyeLRIb9hNtoU1AMMW38xSfeRpcldJKnPtMu+WZkqtG3Iopbn/tXQZFURf4uYz3dxTSzKb51MNnZY1VXParajuxuUJtFUzZDn3Gvo8U/kkNCU0o9CN84EqnhpgC1hE/SMzYZJ5ptIE2alK9I1yvYBL6rIgrf2gKzRIbtxQEk8zVSzYiOdOxAFyxngd7lt9kHKbtaEhkVjDjnarIBNn1uaqXHRBxhkB3lHNF+1HVNQNRYdn/aT94WWuVmpXo3nsHgPzj+mwnK8jD5aPQEfpuHu03aoqRzPpbzgfi1ndjwVIaDUrHpZ7j9r4ckljQqbsr9gXNjN7xl7JR1YxkWHD4BIUfxC69704UiD/okGSYyJlLdji41zFLk0l0zdi3O+/QVq3z7Fa5v/WfL8gHYG0JadqWeZTfl8AQ8J4dH3Er6x1MxL/iNb88Thjdhaw32 eXrbK4pU nNklC3LnXNHb5aYLfv5b0nDdU58C6aX06MNsvEOrrKQFJ19BGuikO6mav+CZdsoMMfHY1ZX4m0LZdDUxHnKx1TCSxpivI/TLxB60vKuoHhY87QbZhIVCR1XVN9/1nNb8cDOX9fshXbPdnlYR86TBYajqFrRIQIGe5iwULRi0sVU1sgIRRgJHSxsYl2ridbw8GjwL8AG0lqTUhtfk1WSrPqfML7F5pJrcJ8VYWxtshV3OYGtUx4GcL0dZXaLe9dXJZ7I/ln51VK+GNtTG0XkNN/7qKBaW8nidVwE9m+lSlCoUn8L1ZGkBydOJfXFfNVtR9vFOvOEwKk8gA9jVBqlJvof1gBftkFpHxV8Gbh1vNJfbueEw2Nwxz0uxb/dkaOxxyJ947N9QkienqsLUdnKgvo8X70g9RS+lfllyWVRScWq4ZJ9UyiCllruk2UQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri 12-04-24 17:24:38, Peng Zhang wrote: > From: ZhangPeng > > Since commit f1a7941243c1 ("mm: convert mm's rss stats into > percpu_counter"), the rss_stats have converted into percpu_counter, > which convert the error margin from (nr_threads * 64) to approximately > (nr_cpus ^ 2). However, the new percpu allocation in mm_init() causes a > performance regression on fork/exec/shell. Even after commit > 14ef95be6f55 ("kernel/fork: group allocation/free of per-cpu counters > for mm struct"), the performance of fork/exec/shell is still poor > compared to previous kernel versions. > > To mitigate performance regression, we use lazy_percpu_counter[1] to > delay the allocation of percpu memory for rss_stats. After lmbench test, > we will get 3% ~ 6% performance improvement for lmbench > fork_proc/exec_proc/shell_proc after conversion. > > The test results are as follows: > > base base+revert base+lazy_percpu_counter > > fork_proc 427.4ms 394.1ms (7.8%) 413.9ms (3.2%) > exec_proc 2205.1ms 2042.2ms (7.4%) 2072.0ms (6.0%) > shell_proc 3180.9ms 2963.7ms (6.8%) 3010.7ms (5.4%) > > This solution has not been fully evaluated and tested. The main idea of > this RFC patch series is to get the community's opinion on this approach. Thanks! I like the idea and in fact I wanted to do something similar (just never got to it). Thread [2] has couple of good observations regarding this problem. Couple of thoughts regarding your approach: 1) I think switching to pcpu counter when update rate exceeds 256 updates/s is not a great fit for RSS because the updates are going to be frequent in some cases but usually they will all happen from one thread. So I think it would make more sense to move the decision of switching to pcpu mode from the counter itself into the callers and just switch on clone() when the second thread gets created. 2) I thought that for RSS lazy percpu counters, we could directly use struct percpu_counter and just make it that if 'counters' is NULL, the counter is in atomic mode (count is used as atomic_long_t), if counters != NULL, we are in pcpu mode. 3) In [2] Mateusz had a good observation that the old RSS counters actually used atomic operations only in rare cases so even lazy pcpu counters are going to have worse performance for singlethreaded processes than the old code. We could *almost* get away with non-atomic updates to counter->count if it was not for occasional RSS updates from unrelated tasks. So it might be worth it to further optimize the counters as: struct rss_counter_single { void *state; /* To detect switching to pcpu mode */ atomic_long_t counter_atomic; /* Used for foreign updates */ long counter; /* Used by local updates */ } struct rss_counter { union { struct rss_counter_single single; /* struct percpu_counter needs to be modified to have * 'counters' first to avoid issues for different * architectures or with CONFIG_HOTPLUG_CPU enabled */ struct percpu_counter pcpu; } } But I'm not sure this complexity is worth it so I'd do it as a separate patch with separate benchmarking if at all. Honza [2] https://lore.kernel.org/all/ZOPSEJTzrow8YFix@snowbird/ > > [1] https://lore.kernel.org/linux-iommu/20230501165450.15352-8-surenb@google.com/ > > Kent Overstreet (1): > Lazy percpu counters > > ZhangPeng (2): > lazy_percpu_counter: include struct percpu_counter in struct > lazy_percpu_counter > mm: convert mm's rss stats into lazy_percpu_counter > > include/linux/lazy-percpu-counter.h | 88 +++++++++++++++++++ > include/linux/mm.h | 8 +- > include/linux/mm_types.h | 4 +- > include/trace/events/kmem.h | 4 +- > kernel/fork.c | 12 +-- > lib/Makefile | 2 +- > lib/lazy-percpu-counter.c | 131 ++++++++++++++++++++++++++++ > 7 files changed, 232 insertions(+), 17 deletions(-) > create mode 100644 include/linux/lazy-percpu-counter.h > create mode 100644 lib/lazy-percpu-counter.c > > -- > 2.25.1 > -- Jan Kara SUSE Labs, CR