From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2E12C43334 for ; Mon, 20 Jun 2022 00:42:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F25246B0072; Sun, 19 Jun 2022 20:42:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ED4556B0073; Sun, 19 Jun 2022 20:42:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D9BFE6B0074; Sun, 19 Jun 2022 20:42:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id CBDC06B0072 for ; Sun, 19 Jun 2022 20:42:37 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 9525160120 for ; Mon, 20 Jun 2022 00:42:37 +0000 (UTC) X-FDA: 79596763554.22.41E4FD4 Received: from mail-qv1-f42.google.com (mail-qv1-f42.google.com [209.85.219.42]) by imf13.hostedemail.com (Postfix) with ESMTP id D9417200A6 for ; Mon, 20 Jun 2022 00:42:36 +0000 (UTC) Received: by mail-qv1-f42.google.com with SMTP id 88so10491572qva.9 for ; Sun, 19 Jun 2022 17:42:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=u9dQ6/5I6Ax505mazhuw1Ffh+ePc0tjQgjDBMimjh8o=; b=CXxqJhfz4MfvCvDwPElTtqA/7CfxEoUx/4j41luaiyXTTW+Myif81Lvncz79doq7RV 4wLfLJVJX4CbV97/vJP+bBE+yfEAKR9tAZxRaiLHJ4aM5+helq7OvYwN+lrqbvRBxhsp JZOoJgDPZZPqNkis4sOqZLpSgiWwoXz7hyHmkRmU6U9oFT1mKp3wd9Rz0QQLybc9uVYQ hAlcW+CbaADICAKtljdE298GAi166keI1fb8+crSHbkwzP9UMaGpN6EWeQlXsvuHaVUs jf2pdZ3vBUYaqoImJiMtRzEXa4q7wwf60wTfBBoV8hKdcQjlveYmHswzby1zdsrDOfkw 2Tzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=u9dQ6/5I6Ax505mazhuw1Ffh+ePc0tjQgjDBMimjh8o=; b=Pa3UjehtNaakzliKcZoRc+BWAlwmNgL/556yHSlFNviB1dYvg9q09C/5bSyciaWwPw gukjnaH19CEJ+W9y+8Dj2EnPl99V6dbrnSjG03hOZ1/g7hri2alg536Jk6Sldp6Gshn+ g91LtyPmFYPQyqsC/ye+6muKuEK7Gp+y1Ylhgu8OvcShlNLoCXwNjB3MsV5I7oX6phvN 6mKodbc3Fd9RDTM6NWjJILqt5yc+nyZQcuCls5SJI7o9nxv08IJiNl+aTLECiacsemzH IijO3ua3oPZ6vSom3YCIJlT7LrN2D7mhje6QVmpG5Sr9Y1r4022mc2Omo2xtE5VhMGmh SBVQ== X-Gm-Message-State: AJIora/AUIDkYuG63UK0634+8kiatzmrsajtLXswtW6+8Tj8ubWci7jd uJQ5YjBr0YsQwl1ZteiK2Q== X-Google-Smtp-Source: AGRyM1vIWFMh9tYneqSwqWtx8vMubJTJJqtNl6QLXelNliQ8VAGPbTAojpbaay/Jn9iWxGxbukwyGg== X-Received: by 2002:a05:622a:1649:b0:306:6a7b:2c22 with SMTP id y9-20020a05622a164900b003066a7b2c22mr17970894qtj.66.1655685755990; Sun, 19 Jun 2022 17:42:35 -0700 (PDT) Received: from localhost (c-73-219-103-14.hsd1.vt.comcast.net. [73.219.103.14]) by smtp.gmail.com with ESMTPSA id h4-20020a05620a400400b006a6d4261f40sm12185580qko.111.2022.06.19.17.42.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 19 Jun 2022 17:42:35 -0700 (PDT) From: Kent Overstreet To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, pmladek@suse.com Cc: Kent Overstreet , rostedt@goodmis.org, enozhatsky@chromium.org, linux@rasmusvillemoes.dk, willy@infradead.org Subject: [PATCH v4 00/34] Printbufs - new data structure for building strings Date: Sun, 19 Jun 2022 20:41:59 -0400 Message-Id: <20220620004233.3805-1-kent.overstreet@gmail.com> X-Mailer: git-send-email 2.36.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655685756; a=rsa-sha256; cv=none; b=bZ4K2pvh3ElccbIUERQLCSNPyLGz5CjjPLb8mXf///0lTxoi+Kvkr2s4xZz7dWolRWRdBH 4em1bgJMD7+93U+bhLp6pakd+xKYIQvKadrQxAR6ThoTT7TrAUAvK7VQ5/QhrvF/+Qxh+m W5C7hnmgVayAZz5Dz/vJlrGaWK+2CYw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655685756; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=u9dQ6/5I6Ax505mazhuw1Ffh+ePc0tjQgjDBMimjh8o=; b=sqSiqqi/UkHqxaNpIbVaagAkS9sWEMs4r0XvvoErWzzFJaMc0nLZJ9gNi6DBS0mxnghJMW C4ZxU9la8EBnhSDAx73ShqOfoTfUzoT5yVwfzltjkXTr0+aXgB43Bm4SG7qP7LGNaSXKCe 5xxhve0Qn3ne/4QEKnLmf75JCqxHyQ8= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=CXxqJhfz; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf13.hostedemail.com: domain of kent.overstreet@gmail.com designates 209.85.219.42 as permitted sender) smtp.mailfrom=kent.overstreet@gmail.com Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=CXxqJhfz; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf13.hostedemail.com: domain of kent.overstreet@gmail.com designates 209.85.219.42 as permitted sender) smtp.mailfrom=kent.overstreet@gmail.com X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: D9417200A6 X-Stat-Signature: tp34uuq1mmdop7a6d5sk1gcqz1ofhz6u X-HE-Tag: 1655685756-856507 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Previous discussions: https://lore.kernel.org/all/20220419203202.2670193-1-kent.overstreet@gmail.com/ https://lore.kernel.org/all/20220519172421.162394-1-kent.overstreet@gmail.com/ https://lore.kernel.org/all/20220604193042.1674951-1-kent.overstreet@gmail.com/ Git repo: https://evilpiepirate.org/git/bcachefs.git/log/?h=printbuf_v4 Changes since v3: Bugfixes and performance improvements, the latest iteration of this patch series has been baking in the bcachefs tree and that shook out some bugs. Rasmus pointed out that -fno-strict-aliasing is going to cause gcc to generate nasty code, and indeed it unfortunately does but according to worst case scenario microbenchmarks it's not a problem for actual performance. Using memcpy() and memset() in the printbuf helpers _was_ a problem for performance, so that's been fixed. ----------- Core idea: Wouldn't it be nice if we had a common data structure and calling convention for outputting strings? The core concept this patch series is aimed at cleaning up and standardizing is that of a "pretty-printer", which is now a function like prt_foo() or foo_to_text(): void foo_to_text(struct printbuf *out, struct foo) What this patch series does or enables: - It becomes quite a bit easier to write composable pretty printers! This is huge. - A ton of code that works in terms of raw char * pointers and lengths (snprintf style, and many weird variations) gets cleaned up, with error prone raw pointers arithmetic replaced by proper helpers - A ton of code that emits either directly via printk() or to other places (sysfs, debugfs) can now output to printbufs, and becomes more reusable and composable - Countesy of Matthew Wilcox, the new and very cool %pf() format string, which allows passing a pretty printer function and its arguments to sprintf() and family. This means we can now call type specific pretty-printers without adding them to lib/vsprintf.c and writing a bunch of crazy parsing-and-dispatch code. For example, printk("%pd", dentry); becomes printk("%pf(%p)", prt_dentry, dentry); My OOM debugging & reporting patch series that builds off of this uses this to solve a very real problem that Michal Hocko brought up at LSF - with this we write shrinkers_to_text(), slab_to_text() which can _also_ now be used for reporting in debugfs (which Roman has been working on), as well as in the show_mem() report - the "%pf()" syntax lets us print the output of those functions without allocating (and having to preallocate) a separate buffer. - Some new formatting helpers: Nicely aligned text is much easier to read, and something that we want a _lot, but outputting nicely aligned text with printf() is a pain in the ass. Printbufs add tabstops, which can be used for right or left justification - simple, easy. prt_tab() emits spaces up to the next tabstop, prt_tab_rjust() advances to the next tabstop right justifying text since the previous tabstop. Printbufs also add an indent level, obeyed by prt_newline() which can be very useful for multi line output. In the future, \n and \t in format strings may learn to obey these as well. - Optional heap allocation - no need to statically allocate buffers on the stack and guess at the output size. - Lots of consolidating and refactoring This series replaces seq_buf, which does basically what an earlier version of printbufs did. A good chunk of lib/string_helpers.c, as well as lib/hexdump.c are converted (and simplified!). Pretty printers in lib/vsprintf.c previously outputted to buffers on the stack and then copied _that_ to the actual output buffer, that's all gone (replaced by proper helpers for outputting chars and strings), and they also used printf_spec for argument passing in ad-hoc ways. This patch series does a lot towards converting them to more standard pretty printers that can be called via %pf() instead of having to live in lib/vsprintf.c. Still to do: format string decoding for argument passing is a mess that's scattered all over the place. In the course of working on this patch series, I've spotted a _lot_ more consolidation and refactoring that needs to be done - we've got a ton of API fragmentation leading to lots of code duplication. But I'm already really excited about what this patch series enables. Cheers! Kent Overstreet (34): lib/printbuf: New data structure for printing strings lib/string_helpers: Convert string_escape_mem() to printbuf vsprintf: Convert to printbuf lib/hexdump: Convert to printbuf vsprintf: %pf(%p) lib/string_helpers: string_get_size() now returns characters wrote lib/printbuf: Heap allocation lib/printbuf: Tabstops, indenting lib/printbuf: Unit specifiers lib/pretty-printers: prt_string_option(), prt_bitflags() vsprintf: Improve number() vsprintf: prt_u64_minwidth(), prt_u64() test_printf: Drop requirement that sprintf not write past nul vsprintf: Start consolidating printf_spec handling vsprintf: Refactor resource_string() vsprintf: Refactor fourcc_string() vsprintf: Refactor ip_addr_string() vsprintf: Refactor mac_address_string() vsprintf: time_and_date() no longer takes printf_spec vsprintf: flags_string() no longer takes printf_spec vsprintf: Refactor device_node_string, fwnode_string vsprintf: Refactor hex_string, bitmap_string_list, bitmap_string Input/joystick/analog: Convert from seq_buf -> printbuf mm/memcontrol.c: Convert to printbuf clk: tegra: bpmp: Convert to printbuf tools/testing/nvdimm: Convert to printbuf powerpc: Convert to printbuf x86/resctrl: Convert to printbuf PCI/P2PDMA: Convert to printbuf tracing: trace_events_synth: Convert to printbuf d_path: prt_path() ACPI/APEI: Add missing include tracing: Convert to printbuf Delete seq_buf Documentation/core-api/printk-formats.rst | 22 + arch/powerpc/kernel/process.c | 16 +- arch/powerpc/kernel/security.c | 75 +- arch/powerpc/platforms/pseries/papr_scm.c | 34 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 16 +- drivers/acpi/apei/erst-dbg.c | 1 + drivers/clk/tegra/clk-bpmp.c | 21 +- drivers/input/joystick/analog.c | 23 +- drivers/pci/p2pdma.c | 21 +- fs/d_path.c | 35 + include/linux/dcache.h | 1 + include/linux/kernel.h | 12 + include/linux/pretty-printers.h | 10 + include/linux/printbuf.h | 253 +++ include/linux/seq_buf.h | 162 -- include/linux/string.h | 5 + include/linux/string_helpers.h | 8 +- include/linux/trace_events.h | 2 +- include/linux/trace_seq.h | 17 +- kernel/trace/trace.c | 45 +- kernel/trace/trace_dynevent.c | 34 +- kernel/trace/trace_events_filter.c | 2 +- kernel/trace/trace_events_synth.c | 32 +- kernel/trace/trace_functions_graph.c | 6 +- kernel/trace/trace_kprobe.c | 2 +- kernel/trace/trace_seq.c | 111 +- lib/Makefile | 4 +- lib/hexdump.c | 246 +-- lib/pretty-printers.c | 60 + lib/printbuf.c | 253 +++ lib/seq_buf.c | 397 ----- lib/string_helpers.c | 224 +-- lib/test_hexdump.c | 30 +- lib/test_printf.c | 33 +- lib/vsprintf.c | 1723 ++++++++++----------- mm/memcontrol.c | 68 +- tools/testing/nvdimm/test/ndtest.c | 22 +- 37 files changed, 2050 insertions(+), 1976 deletions(-) create mode 100644 include/linux/pretty-printers.h create mode 100644 include/linux/printbuf.h delete mode 100644 include/linux/seq_buf.h create mode 100644 lib/pretty-printers.c create mode 100644 lib/printbuf.c delete mode 100644 lib/seq_buf.c -- 2.36.1