From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B896EE49B0 for ; Wed, 23 Aug 2023 15:47:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 441C1280080; Wed, 23 Aug 2023 11:47:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3F0EA280056; Wed, 23 Aug 2023 11:47:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 291EA280080; Wed, 23 Aug 2023 11:47:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 15946280056 for ; Wed, 23 Aug 2023 11:47:35 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id CACB640199 for ; Wed, 23 Aug 2023 15:47:34 +0000 (UTC) X-FDA: 81155799228.16.143C22E Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf09.hostedemail.com (Postfix) with ESMTP id 91030140037 for ; Wed, 23 Aug 2023 15:47:32 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="m/Rutyao"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="DR/w1bq7"; spf=pass (imf09.hostedemail.com: domain of jack@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=jack@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692805653; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yon7qxBDLFnBXgKAsSaVBsa3rXFAPSE2gKqBEhqBxa4=; b=BDWvaIVVk1f+mcDH2Ee56b/GpEWwbrcztEDS+ScyqUhIReeeAtY15yoGDyu3lIiKBQyLvw 9oIPXFndlSlI0A0ujYiZthi1OpUASBFpRbrLW3MSItRNhe3PDqJKHNaxIFh8GmhuV7uq2s LwlsRCrZDks3KgCuN8wA+bj/LeHX7JA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692805653; a=rsa-sha256; cv=none; b=p0KzdUwiN3EG9C239YyaVdNvq1zxmjVu/QTyC29Rf980IFdgzFGODOqAOafmmQiqTKcCob /FI2BhwbVqGHcw2bh9jJjiRR6PaaWECiS7k6XMPHczQ4uXaujvwgKKQFDEOKI3InFSzEBL yy0/BLOGmijo/NxoHMd4W2wMvryrNCE= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="m/Rutyao"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="DR/w1bq7"; spf=pass (imf09.hostedemail.com: domain of jack@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=jack@suse.cz; dmarc=none Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id B00D820906; Wed, 23 Aug 2023 15:47:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1692805648; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=yon7qxBDLFnBXgKAsSaVBsa3rXFAPSE2gKqBEhqBxa4=; b=m/Rutyao5Bm1E61PJHAl1JaGFGKc5PfNRiwF2ObK9oI8kxB2PstBiucoDnaA23aJqmOfw/ Qlr+gSl+tUYjyWbW0EE9aiSj/yFflFMmKpVJq5kRDqE3qQ6hBo/3rzOHs+h1b/MS8LC7Od zX7R6KDcxbjrLmkn4Har/sq59BVkiTI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1692805648; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=yon7qxBDLFnBXgKAsSaVBsa3rXFAPSE2gKqBEhqBxa4=; b=DR/w1bq7MLKt8StQ59fHmQnP4WQWAKFxxDSWsq6pifLVtCBX1fMbWKOOZdFxJkIoLSqoi/ uTJo46zCoExxM2CA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 9FCDA13458; Wed, 23 Aug 2023 15:47:28 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id o4/3JhAq5mRaMwAAMHmgww (envelope-from ); Wed, 23 Aug 2023 15:47:28 +0000 Received: by quack3.suse.cz (Postfix, from userid 1000) id 397F2A0774; Wed, 23 Aug 2023 17:47:28 +0200 (CEST) Date: Wed, 23 Aug 2023 17:47:28 +0200 From: Jan Kara To: Mateusz Guzik Cc: Jan Kara , Dennis Zhou , linux-kernel@vger.kernel.org, tj@kernel.org, cl@linux.com, akpm@linux-foundation.org, shakeelb@google.com, linux-mm@kvack.org Subject: Re: [PATCH 0/2] execve scalability issues, part 1 Message-ID: <20230823154728.rpkw6fpwvwqbnnh3@quack3> References: <20230821202829.2163744-1-mjguzik@gmail.com> <20230821213951.bx3yyqh7omdvpyae@f> <20230822095154.7cr5ofogw552z3jk@quack3> <20230823094915.ggv3spzevgyoov6i@quack3> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 91030140037 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: onimcsopyhnojomnoz35yegbh9fu7i9g X-HE-Tag: 1692805652-481704 X-HE-Meta: U2FsdGVkX18EePnW4CxRH3JglvKwfBa/aK8DRhH1qbqC+wJgNUVU0M+RzrfDBTWx8OPGXM0x4ko2N358xsxe7rRZxEq9SpqvSR208FFY4Lg455XdK13WY+bXgTTtTZNRUtkyxIrHvKoe38tggpRmljJWDAHtGY/CCRMrKHemyBl/ZZIOro3aEnwFAGq0vQOg2IVm+cnMLdSazXjV7ctcUKhbymcOXLg4OACxScXrwJgwuLTOJQ8fRUi3pQaAq1o5IysxbNLtiEAz5JOef9BPKke5WoFUVUdq5LpQCtOE/TF7D8Hxwg1bUbQxKebXfljILZ0io2wyokJp7Oo6Y6dH3NjEzDil8MrcgJZ1ZucF6rpWZAs4w18bHI3xwHhX/52+58mojzHuOIbVgKy1dx+EgwqahLcVhhQHngfeJhcljJ7fgj4ehvt3uxFGNZZru2IyEzsnbUAYl5BC2BmB+XYhGTBXINg5eOgxBOx2gnCRbMo8cvXc6fnHEndNUz0RmKj8uAwqKUNb3LgB5aFdgFkaNtevZuXGPvALr+UPC9fsVogekTc0kjv3JuzwQCQ9F4z3S0FKO5XgqLyudLL9xAXELhaaZq3wjLMAUQfH90cS5HL0trh7tKIl8rUIgl71AlKgirtc56AZbC/kGkHT1x0BMXa9/rQiHrMo2flPW2/GF0AWy0VRfcxOYKloH9Ht15c7YuJrcXKZel3E22sef3saa0ihelyYGpJW/aExtGZn3ZTOLYPgbjzU+38GeR4JKWexbA1e8Si6LlVRE8FuAk2joH8axUDuhjlHrGbor2UNBFJBsZTfgpOw5fcRdLxqGK/0DkXec5dzV/jDG7haCsMGEVUr7n0Xn4ACx5sk0KNk45JMND0bb2tHGownurWj0+0nUz4bIggBaP1CnBzaOHnkYeuVEF+vGcVfE+SJfpROWSourEz5ftKjWwN3zP0rk1woKyFBPex99QFpTyYf9GD qF1y+Y01 F9RE4aclr06TbE4nQCWF56LOtrXJZbHwEM8MEZaL6++WsBAGPpNbCo/1jkdWT89tRqnof0icgK02VAOzEmW72bZBt7iSsKxYjr5GWXp0QD3r8WOfOWyk/866ft1quZ+V++y2WNdycXCG6bO2vSk6oFlUpEBhYlvuRW6T0334TazDkcF1Zsy4n5dQPAAszOcXsBxxvEfLhMn7taQzONbzwU6RUwOfDyTF0jbub1/MNHfP6rXGEtS4ieujeyq3ReRF8+/zUcLyW5i0RppR5nuoJBAi5yegFHYxe+bJmhXMyOyuwtTlAVhtum6SxPk+/SHgDCEwJYZwsrnc+cpM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed 23-08-23 14:13:20, Mateusz Guzik wrote: > On 8/23/23, Jan Kara wrote: > > On Tue 22-08-23 16:24:56, Mateusz Guzik wrote: > >> On 8/22/23, Jan Kara wrote: > >> Then for single-threaded case an area is allocated for NR_MM_COUNTERS > >> countes * 2 -- first set updated without any synchro by current > >> thread. Second set only to be modified by others and protected with > >> mm->arg_lock. The lock protects remote access to the union to begin > >> with. > > > > arg_lock seems a bit like a hack. How is it related to rss_stat? The scheme > > with two counters is clever but I'm not 100% convinced the complexity is > > really worth it. I'm not sure the overhead of always using an atomic > > counter would really be measurable as atomic counter ops in local CPU cache > > tend to be cheap. Did you try to measure the difference? > > > > arg_lock is not as is, it would have to be renamed to something more generic. Ah, OK. > Atomics on x86-64 are very expensive to this very day. Here is a > sample measurement of 2 atomics showing up done by someone else: > https://lore.kernel.org/oe-lkp/202308141149.d38fdf91-oliver.sang@intel.com/T/#u > > tl;dr it is *really* bad. I didn't express myself well. Sure atomics are expensive compared to plain arithmetic operations. But I wanted to say - we had atomics for RSS counters before commit f1a7941243 ("mm: convert mm's rss stats into percpu_counter") and people seemed happy with it until there were many CPUs contending on the updates. So maybe RSS counters aren't used heavily enough for the difference to practically matter? Probably operation like faulting in (or unmapping) tmpfs file has the highest chance of showing the cost of rss accounting compared to the cost of the remainder of the operation... > > If the second counter proves to be worth it, we could make just that one > > atomic to avoid the need for abusing some spinlock. > > The spinlock would be there to synchronize against the transition to > per-cpu -- any trickery is avoided and we trivially know for a fact > the remote party either sees the per-cpu state if transitioned, or > local if not. Then one easily knows no updates have been lost and the > buf for 2 sets of counters can be safely freed. Yeah, the spinlock makes the transition simpler, I agree. > While writing down the idea previously I did not realize the per-cpu > counter ops disable interrupts around the op. That's already very slow > and the trip should be comparable to paying for an atomic (as in the > patch which introduced percpu counters here slowed things down for > single-threaded processes). > > With your proposal the atomic would be there, but interrupt trip could > be avoided. This would roughly maintain the current cost of doing the > op (as in it would not get /worse/). My patch would make it lower. > > All that said, I'm going to refrain from writing a patch for the time > being. If powers to be decide on your approach, I'm not going to argue > -- I don't think either is a clear winner over the other. I guess we'll need to code it and compare the results :) Honza -- Jan Kara SUSE Labs, CR