From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F3D8D206BA for ; Wed, 16 Oct 2024 21:01:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C9D46B007B; Wed, 16 Oct 2024 17:01:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 97A1F6B0082; Wed, 16 Oct 2024 17:01:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 841D76B0083; Wed, 16 Oct 2024 17:01:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 63CC36B007B for ; Wed, 16 Oct 2024 17:01:17 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 6D7AD140BF3 for ; Wed, 16 Oct 2024 21:01:06 +0000 (UTC) X-FDA: 82680685500.23.19D3C6C Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf01.hostedemail.com (Postfix) with ESMTP id B88E64001D for ; Wed, 16 Oct 2024 21:01:07 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of "SRS0=6iTh=RM=goodmis.org=rostedt@kernel.org" designates 147.75.193.91 as permitted sender) smtp.mailfrom="SRS0=6iTh=RM=goodmis.org=rostedt@kernel.org"; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729112401; a=rsa-sha256; cv=none; b=cNd5KzjZ6p6Ot5eVc1ftwEy68i1G+0UTj4fUyeixtxk2+7LLiDMC2+8Au/8nCkBM1+dNoi CPGietLkGt7J45V8rZK6tz6ZF1F7B7atwI0fpuGdHC0GgQfttoFM0IxJiDs208tdSGPxq1 kNaocOFiaRnJfhdgV84ukJZM7Bc30UQ= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of "SRS0=6iTh=RM=goodmis.org=rostedt@kernel.org" designates 147.75.193.91 as permitted sender) smtp.mailfrom="SRS0=6iTh=RM=goodmis.org=rostedt@kernel.org"; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729112401; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Mf3R/3S0MCI5tF4BhQ2U7wtgZrq3J6u/Z05NpnyO0ck=; b=5tIp8NFiUk4hHOjH9FwrrOQFY7YUEdXfFpqbuZoENOYsqKpuBC9Wj6sWWakO6wpszIe6C4 UKVcBUcpYj25g484mdtO7y5Ei+C/ZOI4vg9+8RpVarx1+9R1S0Kr5AWnV1kOtuifKHYPMO hYlXhwpVucUuP37SYcUGhDHkPPB1zFU= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 7A965A427EA; Wed, 16 Oct 2024 21:01:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 368BFC4CEC5; Wed, 16 Oct 2024 21:01:07 +0000 (UTC) Date: Wed, 16 Oct 2024 17:01:28 -0400 From: Steven Rostedt To: Mike Rapoport Cc: Andrew Morton , Luis Chamberlain , Andreas Larsson , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Brian Cain , Catalin Marinas , Christoph Hellwig , Christophe Leroy , Dave Hansen , Dinh Nguyen , Geert Uytterhoeven , Guo Ren , Helge Deller , Huacai Chen , Ingo Molnar , Johannes Berg , John Paul Adrian Glaubitz , Kent Overstreet , "Liam R. Howlett" , Mark Rutland , Masami Hiramatsu , Matt Turner , Max Filippov , Michael Ellerman , Michal Simek , Oleg Nesterov , Palmer Dabbelt , Peter Zijlstra , Richard Weinberger , Russell King , Song Liu , Stafford Horne , Suren Baghdasaryan , Thomas Bogendoerfer , Thomas Gleixner , Uladzislau Rezki , Vineet Gupta , Will Deacon , bpf@vger.kernel.org, linux-alpha@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-kernel@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-openrisc@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-sh@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-trace-kernel@vger.kernel.org, linux-um@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, sparclinux@vger.kernel.org, x86@kernel.org Subject: Re: [PATCH v6 6/8] x86/module: prepare module loading for ROX allocations of text Message-ID: <20241016170128.7afeb8b0@gandalf.local.home> In-Reply-To: <20241016122424.1655560-7-rppt@kernel.org> References: <20241016122424.1655560-1-rppt@kernel.org> <20241016122424.1655560-7-rppt@kernel.org> X-Mailer: Claws Mail 3.20.0git84 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Stat-Signature: 8bhx88n334uejqzt1k1f7q98f8fhndqo X-Rspamd-Queue-Id: B88E64001D X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1729112467-83716 X-HE-Meta: U2FsdGVkX1+Ku80rfjkbhLJ+EN8bv1pN85KIvvqTdtC4AzW5/8Qw9IgccRC0hedxH2p+NOMRsYag2klbLwi6GKHpdXATyaSoYWHUfQldNE+sQfA+0DGHqIntSYxrvgrVcfs7ICiEG0kEUCk9SMzOKdsKmwlKSaHpzm2h0MxFzan8bGRmjpr5A4x7PUQp+8MP+qqSJSAR7zhd5HbKcJHBkI2yABkSyRZMU/YXZM4snJ5ZxiD9fs05QEJP4i5k4tD+vOqeQQ0hhZn+7kZQNwSVdbhWerJgwjv4jAfnGm+WfSNFG8FOGNslWzk9EnOsOU8aEr6NNWfVMN31TKyOSgxnHklc40r95LUIa5UyteF4Uyg3PTsJFvZv4aR1UC9rJ8kLjjF9oA3bDMmI4IiJGdfXItH6LfpdY1PRS8IFxWTpqDrWaE1e7AA8/cctpdtWOtHoiyb5YL3i4KPodaejg4KZPqMXJpO6dgs360xg4Wua42za5NkKxolLsAxeTmOYu5aaRlHZsc550NtdkwBO58blF25fs2WElhQw8MIJnjWK5AA8TNkmNBCiN7+MzR/pm3TiXTphR+fKKb+TWW4MSbhqAB0RtCYmV0OllUyQPqaEBSaVW9WN+uN3cxlLhyRdWdCgAJMt+7AcXgbJ/y8q98vPB0guR8Ixrleht10GqL97nloU+AQGA6neb3IUpFZNFOTPH4mZjzJ/VWOwx81kG8j1+W4EiyZlSJuOoQgm/NRiqAlrVrvmliChAbzvHAfumENv5F37nBGxnixdsY0H5OgB4DyXjP2mFyI0ClIyaoPcdMH6k4nBdEakEE7NnU997yuQGtZ8tBb1v2Sjsktzmy+yAZqi/+OHc5RxVD11R4PI6sXC4vE4RI1N1tx4aigqP2YVPd/WkuWhbDPv4J/cYuQLu5TfQ406ulzJU/CT4Mi5xhPcDPTrwDv5l5KZamSAZD71kn2CfmyZhXID1ZoEe/P SCk2YsP6 Yhif6hb8+/wgAn8myMd4gHprqteRKcSb87J14vsDttSkXdFs/elf2eX11CIJdu9oiGqlmJt/u6Zf5ngdGfoCiwVALi9eka3gc4MiK5Qj+oxzOGYgMv70u5I/QRl7a8JWyLdOM4/1BQkOWf2mKtnyqvkXpcCzlXbo14TgbdrvkW5WD4gB6qfCEWEL2hLIRVN4V2xf0ikRhA6iFq7gfTqgR/IYPE/9124Q/eQDfTONqr2JsPxACNvBW/BTTGJSvjWI1hYOVO+UfUID9f3ruzdk3CmCmNg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, 16 Oct 2024 15:24:22 +0300 Mike Rapoport wrote: > diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c > index 8da0e66ca22d..b498897b213c 100644 > --- a/arch/x86/kernel/ftrace.c > +++ b/arch/x86/kernel/ftrace.c > @@ -118,10 +118,13 @@ ftrace_modify_code_direct(unsigned long ip, const char *old_code, > return ret; > > /* replace the text with the new text */ > - if (ftrace_poke_late) > + if (ftrace_poke_late) { > text_poke_queue((void *)ip, new_code, MCOUNT_INSN_SIZE, NULL); > - else > - text_poke_early((void *)ip, new_code, MCOUNT_INSN_SIZE); > + } else { > + mutex_lock(&text_mutex); > + text_poke((void *)ip, new_code, MCOUNT_INSN_SIZE); > + mutex_unlock(&text_mutex); > + } > return 0; > } So this slows down the boot by over 30ms. That may not sound like much, but we care very much about boot times. This code is serialized with boot and runs whenever ftrace is configured in the kernel. The way I measured this, was that I added: diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c index 4dd0ad6c94d6..b72bb9943140 100644 --- a/arch/x86/kernel/ftrace.c +++ b/arch/x86/kernel/ftrace.c @@ -104,6 +104,8 @@ static int ftrace_verify_code(unsigned long ip, const char *old_code) return 0; } +u64 sdr_total; + /* * Marked __ref because it calls text_poke_early() which is .init.text. That is * ok because that call will happen early, during boot, when .init sections are @@ -114,6 +116,8 @@ ftrace_modify_code_direct(unsigned long ip, const char *old_code, const char *new_code) { int ret = ftrace_verify_code(ip, old_code); + u64 start, stop; + if (ret) return ret; @@ -121,9 +125,12 @@ ftrace_modify_code_direct(unsigned long ip, const char *old_code, if (ftrace_poke_late) { text_poke_queue((void *)ip, new_code, MCOUNT_INSN_SIZE, NULL); } else { + start = trace_clock_local(); mutex_lock(&text_mutex); text_poke((void *)ip, new_code, MCOUNT_INSN_SIZE); mutex_unlock(&text_mutex); + stop = trace_clock_local(); + sdr_total += stop - start; } return 0; } diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index c01375adc471..93284557144d 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -10738,6 +10738,11 @@ __init static int tracer_alloc_buffers(void) register_snapshot_cmd(); + { + extern u64 sdr_total; + printk("SDR TOTAL: %lld\n", sdr_total); + } + test_can_verify(); return 0; And did the same before this patch. I ran it three times and have the following numbers (all in nanoseconds): before: 11356637 11863526 11507750 after: 43978750 41293162 42741067 Before this patch, the total updates took 11ms. After the patch it takes around 42ms. This is because we are patching 59 thousand sites with this. # dmesg |grep ftrace [ 1.620569] ftrace: allocating 59475 entries in 233 pages [ 1.667178] ftrace: allocated 233 pages with 5 groups If this is only needed for module load, can we at least still use the text_poke_early() at boot up? if (ftrace_poke_late) { text_poke_queue((void *)ip, new_code, MCOUNT_INSN_SIZE, NULL); } else if (system_state == SYSTEM_BOOTING) { text_poke_early((void *)ip, new_code, MCOUNT_INSN_SIZE); } else { mutex_lock(&text_mutex); text_poke((void *)ip, new_code, MCOUNT_INSN_SIZE); mutex_unlock(&text_mutex); } ? The above if statement looks to slow things down just slightly, but only by 2ms, which is more reasonable. -- Steve