From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41707C433EF for ; Tue, 28 Jun 2022 02:57:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 92E318E0002; Mon, 27 Jun 2022 22:57:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8DE908E0001; Mon, 27 Jun 2022 22:57:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7CDE88E0002; Mon, 27 Jun 2022 22:57:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 707D18E0001 for ; Mon, 27 Jun 2022 22:57:04 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 4106B20D98 for ; Tue, 28 Jun 2022 02:57:04 +0000 (UTC) X-FDA: 79626132768.21.C74E761 Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) by imf08.hostedemail.com (Postfix) with ESMTP id 3C58E160032 for ; Tue, 28 Jun 2022 02:57:01 +0000 (UTC) Received: by mail-qk1-f176.google.com with SMTP id b133so8731596qkc.6 for ; Mon, 27 Jun 2022 19:57:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:date:mime-version:user-agent:subject:content-language:to :cc:references:from:in-reply-to:content-transfer-encoding; bh=vSypklzz7sQ15cqD5TowlyIuG5uwkM3pNtpcIbosIO0=; b=By9ShoqlGRjvTLMhyFzje4xVZ5MGOzM82m2xhZ8ZXrXBdkYMrkGN/CdphYf983uv31 76xfUCgrkEt5AqybjOvfTgj4Y9CDTu6bDFQaJCwJTYL4jz2IvbG+4D0XhPoe+qJ4nx60 1as3xnBspvRlNE3fLIB+qh+doWOvHTG0/MiqkWpCKLJUIcIn9KYgz/rcnqpmhQXdy6fP tsqS8mRYEfqTKa8dscwHP9uXO+oKLfGsnGiWmnVfZl4NMd/ynvbg25F8mXWCBXCGOuwh /Vb+Moonsjj9s/qxO+8iYL5HMX8O3kzaZxvQygKVR/2iWxbjrZ0DmHGanYga40tWZGn7 O51g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=vSypklzz7sQ15cqD5TowlyIuG5uwkM3pNtpcIbosIO0=; b=isWJgj2AF4QWnvUX226aP8982G7Ip23sxhTpAmKdYg+oFx3SpnA65S4ZuPOPLUapyT RWodhSpVqenZALew6V4aidSkdmMv+qa462xIkvC6ba+5+CFUIWWmYgisYOTZwGph+9zd EOl9f5+fiRTFu/wD6avWh/5p/IgvZlATHaFUe0bwUg4d5rC8WO7Jehcdm5gH01hl/zka Pujfd3RBi3gWs1Lqn5R40XztlU3N53+IzaxGICal8eKhP84yzSmdyg+m8TlAO8uk0vrv iA7HvA9HgGNWjowuqXRhsESofpzCP11vxkLrO0eXetNc9QSw+dcIGsMAuW/l2wvBKTro MTNQ== X-Gm-Message-State: AJIora+YSy6fw01hMpmiyUV+i0Vn7geC7XBneMUp2uyzZMGlKW+nkfq7 iY2RqlklEFbtrH/V7V/4/359GEML7x5Zp+A1Qw== X-Google-Smtp-Source: AGRyM1ubUVsowPjUbNvilznEB1L5LyvNYUjvStiOmcudnPw8WNl3zSlw8UIISp0utPRmWyPQzyps1g== X-Received: by 2002:a05:620a:2892:b0:6ae:bdfa:1d0e with SMTP id j18-20020a05620a289200b006aebdfa1d0emr10246660qkp.235.1656385020464; Mon, 27 Jun 2022 19:57:00 -0700 (PDT) Received: from [192.168.1.210] (c-73-219-103-14.hsd1.vt.comcast.net. [73.219.103.14]) by smtp.gmail.com with ESMTPSA id bi32-20020a05620a31a000b006af3f3b385csm2148527qkb.98.2022.06.27.19.56.58 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 27 Jun 2022 19:56:59 -0700 (PDT) Message-ID: Date: Mon, 27 Jun 2022 22:56:58 -0400 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.10.0 Subject: Re: [RFC[ Alloc in vsprintf Content-Language: en-US To: David Laight , 'Linus Torvalds' , Joe Perches Cc: Andrew Morton , Petr Mladek , Steven Rostedt , Sergey Senozhatsky , Rasmus Villemoes , Matthew Wilcox , Miguel Ojeda , Andy Shevchenko , LKML , linux-mm References: <20220620004233.3805-1-kent.overstreet@gmail.com> <0a5901f8460f452a89c9b0cda32fb833@AcuMS.aculab.com> <20220620150514.3tjy5dv7pv5frcwd@moria.home.lan> <53d77ae6101a0f24cfb694174d4c7699424c57e8.camel@perches.com> <20220621005752.ohiq5besmy3r5rjo@moria.home.lan> <355e912490dbaef8fe4e12df0201c3f5b439565d.camel@perches.com> From: Kent Overstreet In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=By9Shoql; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf08.hostedemail.com: domain of kent.overstreet@gmail.com designates 209.85.222.176 as permitted sender) smtp.mailfrom=kent.overstreet@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656385021; a=rsa-sha256; cv=none; b=wtWjVf0VhOiLfO4Ihmhem/Y38CtwfRvEnKGieJ5wloWa/QaMhOIq0Coe1J8FVJS7cvrhFL Vur6SVgsUfQOHW3dEocjMvnYmbTugNZlwTJbNhtg89VS5sPXlCjZlqaX8iJ1Yyrc6cjC7c K5ULIbaJiCfBEjIzILAw7XdsT7AEcdc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656385021; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vSypklzz7sQ15cqD5TowlyIuG5uwkM3pNtpcIbosIO0=; b=U+ea16MlG7ZTX5R9ARpMlAQ5RF/ngL9cwBCVnRdU6F82LdXGdKi1Wtdzhs7pk60tmaU7NY Nk/ZM1xWtYEL108Ux119BKC2S+aUnMlyN42Z85xWOhemgxvvUkareNO0l0wKwTjhxN/vS6 EPmMyLQICeBdvc8NC8W7T15kmIE+Nrw= X-Stat-Signature: dx1o9ad83ez5uobtyk9ir55i39ibdpyx X-Rspamd-Server: rspam08 X-Rspam-User: X-Rspamd-Queue-Id: 3C58E160032 Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=By9Shoql; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf08.hostedemail.com: domain of kent.overstreet@gmail.com designates 209.85.222.176 as permitted sender) smtp.mailfrom=kent.overstreet@gmail.com X-HE-Tag: 1656385021-917695 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 6/27/22 04:25, David Laight wrote: > From: Linus Torvalds >> Sent: 26 June 2022 21:19 > .. >> That does require teaching the sprint_symbol() functions that they >> need to take a "length of buffer" and return how much they used, but >> that would seem to be a sensible thing anyway, and what the code >> should always have done? > > It needs to return the 'length it would have used'. > While occasionally useful I'm pretty sure this is actually > a side effect of the was that libc snprintf() was originally > implemented (sprintf() had an on-stack FILE). > > In any case it might be simplest to pass all these functions > the write pointer and buffer limit and have them return the > new write pointer. > It is likely to generate much better code that passing > a structure by reference. I've said it before, and now I'm going to say it again more forcefully: This obsession with perfect machine code in every situation, regardless of whether it's shown up in benchmarks or profiles, regardless of what it does to the readability of the C code that we humans work with, has to stop. Has. To. Stop. Full stop. We have to be thinking about the people who come after us and have to read and maintain this stuff. Linux should still be in use 50, 100 years from now, and if it's not it's because we _fucked up_, and in software the way you fuck up is by writing code no one can understand - by writing code that people become afraid to touch without breaking it. This happens, routinely, and it's _painful_ when it does. A big learning experience for me when I was a much younger engineer, freshly starting at Google, was working next to a bunch of guys who were all chasing and fixing bugs in ext4 - and they weren't exactly enjoying it. bcache uncovered one or two of them too, and I got to debug that and then had to argue that it wasn't a bug in bcache (we were calling bio->bi_endio in process context, which uncovered a locking bug in ext4). The codebase had become a mess that they were too scared to refactor, in part because there were too many options that were impossible to test - my big lesson from that is that the code you're scared to refactor, that's the code that needs it the most. And I could name some very senior kernel people who write code that's too brittle in the name of chasing performance - in fact I will name one, because I know he won't take it personally: the writeback throttling code that Jens wrote was buggy for _ages_ and at least one of my users was regularly tripping over it and I couldn't make out what the hell that code was trying to do, and not for lack of trying. Other code nightmares: - The old O_DIRECT code, which was a huge performance sink but no one could touch it without breaking something (I spent months on a beautiful refactoring that cut it in half by LOC and improved performance drastically, but I couldn't get it to completely pass xfstests. That sucked). - The old generic_file_buffered_read(), which was a 250 line monster filled with gotos - all in the name of performance, mind you - that people barely touched, and when people absolutely had to they'd do so in the most minimal way possible that ended up just adding to the mess (e.g. the IOCB_NOWAIT changes) - up until I finally cut it apart, then curiously right after that a ton more patches landed. It's almost like cleaning stuff up and prioritizing readability makes it easier for people to work on. - merge_bvec_fn was also quite the tale - also done in the name of performance, noticing a theme here? I like to write fast code too. Of course I do, I'm a kernel engineer, I wouldn't be a real one if I didn't. But that means writing code that is _optimizable_, which means writing code that's easy to go back and modify and change when profiling discovers something. Which means keeping things as simple as is reasonably possible, and prioritizing good data types and abstractions and structure. When I'm first writing code and thinking about performance, here's what I think about: - algorithmic complexity - good data structures (vectors instead of lists, where it matters - it often doesn't) - memory layout: keep pointer indirection at an absolute minimum memory layout matters - locking And honestly, not much else. Because on modern machines, with the type of code we feed our CPUs running in the kernel, memory layout and locking are what matter and not much else. Not shaving every cycle. I already demonstrated this with actual numbers in the printbuf discussion, to Rasmus - yes, the compiler constantly reloading is a shame and it shows up in the text size, and perhaps we'll want to revisit the -fno-strict-aliasing thing someday (I'm fully in agreement with Linus on why he hates strict aliasing, it was something the compiler people sprung on everyone else without discussion or a clear escape hatch or _tooling to deal with existing codebases_ but the tooling has improved since thing, it might not be complete insanity anymore) ...but if you look at the actual microbenchmarks I showed Rasmus? It turns out to not affect performance pretty much at all, it's in the noise. Waiting on loads from memory is what matters to us, and not much else.