From: Rong Xu <xur@google.com>
To: Alice Ryhl <aliceryhl@google.com>,
Andrew Morton <akpm@linux-foundation.org>,
Arnd Bergmann <arnd@arndb.de>, Bill Wendling <morbo@google.com>,
Borislav Petkov <bp@alien8.de>, Breno Leitao <leitao@debian.org>,
Brian Gerst <brgerst@gmail.com>,
Dave Hansen <dave.hansen@linux.intel.com>,
David Li <davidxl@google.com>, Han Shen <shenhan@google.com>,
Heiko Carstens <hca@linux.ibm.com>,
"H. Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>,
Jann Horn <jannh@google.com>, Jonathan Corbet <corbet@lwn.net>,
Josh Poimboeuf <jpoimboe@kernel.org>,
Juergen Gross <jgross@suse.com>,
Justin Stitt <justinstitt@google.com>,
Kees Cook <kees@kernel.org>,
Masahiro Yamada <masahiroy@kernel.org>,
"Mike Rapoport (IBM)" <rppt@kernel.org>,
Nathan Chancellor <nathan@kernel.org>,
Nick Desaulniers <ndesaulniers@google.com>,
Nicolas Schier <nicolas@fjasle.eu>,
"Paul E. McKenney" <paulmck@kernel.org>,
Peter Zijlstra <peterz@infradead.org>, Rong Xu <xur@google.com>,
Sami Tolvanen <samitolvanen@google.com>,
Thomas Gleixner <tglx@linutronix.de>,
Wei Yang <richard.weiyang@gmail.com>,
workflows@vger.kernel.org,
Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>,
Maksim Panchenko <max4bolt@gmail.com>,
Yonghong Song <yonghong.song@linux.dev>,
Yabin Cui <yabinc@google.com>,
Krzysztof Pszeniczny <kpszeniczny@google.com>,
Sriraman Tallam <tmsriram@google.com>,
Stephane Eranian <eranian@google.com>
Cc: x86@kernel.org, linux-arch@vger.kernel.org,
linux-doc@vger.kernel.org, linux-kbuild@vger.kernel.org,
linux-kernel@vger.kernel.org, llvm@lists.linux.dev
Subject: [PATCH v5 6/7] AutoFDO: Enable machine function split optimization for AutoFDO
Date: Wed, 23 Oct 2024 15:44:05 -0700 [thread overview]
Message-ID: <20241023224409.201771-7-xur@google.com> (raw)
In-Reply-To: <20241023224409.201771-1-xur@google.com>
Enable the machine function split optimization for AutoFDO in Clang.
Machine function split (MFS) is a pass in the Clang compiler that
splits a function into hot and cold parts. The linker groups all
cold blocks across functions together. This decreases hot code
fragmentation and improves iCache and iTLB utilization.
MFS requires a profile so this is enabled only for the AutoFDO builds.
Co-developed-by: Han Shen <shenhan@google.com>
Signed-off-by: Han Shen <shenhan@google.com>
Signed-off-by: Rong Xu <xur@google.com>
Suggested-by: Sriraman Tallam <tmsriram@google.com>
Suggested-by: Krzysztof Pszeniczny <kpszeniczny@google.com>
Tested-by: Yonghong Song <yonghong.song@linux.dev>
---
include/asm-generic/vmlinux.lds.h | 7 ++++++-
scripts/Makefile.autofdo | 2 ++
2 files changed, 8 insertions(+), 1 deletion(-)
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index bd64fdedabd2..8a0bb3946cf0 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -556,6 +556,11 @@ defined(CONFIG_AUTOFDO_CLANG)
__cpuidle_text_end = .; \
__noinstr_text_end = .;
+#define TEXT_SPLIT \
+ __split_text_start = .; \
+ *(.text.split .text.split.[0-9a-zA-Z_]*) \
+ __split_text_end = .;
+
#define TEXT_UNLIKELY \
__unlikely_text_start = .; \
*(.text.unlikely .text.unlikely.*) \
@@ -582,6 +587,7 @@ defined(CONFIG_AUTOFDO_CLANG)
ALIGN_FUNCTION(); \
*(.text.asan.* .text.tsan.*) \
*(.text.unknown .text.unknown.*) \
+ TEXT_SPLIT \
TEXT_UNLIKELY \
. = ALIGN(PAGE_SIZE); \
TEXT_HOT \
@@ -589,7 +595,6 @@ defined(CONFIG_AUTOFDO_CLANG)
NOINSTR_TEXT \
*(.ref.text)
-
/* sched.text is aling to function alignment to secure we have same
* address even at second ld pass when generating System.map */
#define SCHED_TEXT \
diff --git a/scripts/Makefile.autofdo b/scripts/Makefile.autofdo
index 6155d6fc4ca7..1caf2457e585 100644
--- a/scripts/Makefile.autofdo
+++ b/scripts/Makefile.autofdo
@@ -10,6 +10,7 @@ endif
ifdef CLANG_AUTOFDO_PROFILE
CFLAGS_AUTOFDO_CLANG += -fprofile-sample-use=$(CLANG_AUTOFDO_PROFILE) -ffunction-sections
+ CFLAGS_AUTOFDO_CLANG += -fsplit-machine-functions
endif
ifdef CONFIG_LTO_CLANG_THIN
@@ -17,6 +18,7 @@ ifdef CONFIG_LTO_CLANG_THIN
KBUILD_LDFLAGS += --lto-sample-profile=$(CLANG_AUTOFDO_PROFILE)
endif
KBUILD_LDFLAGS += --mllvm=-enable-fs-discriminator=true --mllvm=-improved-fs-discriminator=true -plugin-opt=thinlto
+ KBUILD_LDFLAGS += -plugin-opt=-split-machine-functions
endif
export CFLAGS_AUTOFDO_CLANG
--
2.47.0.105.g07ac214952-goog
next prev parent reply other threads:[~2024-10-23 22:44 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-10-23 22:43 [PATCH v5 0/7] Add AutoFDO and Propeller support for Clang build Rong Xu
2024-10-23 22:44 ` [PATCH v5 1/7] Add AutoFDO " Rong Xu
2024-11-01 18:01 ` Masahiro Yamada
2024-11-01 20:08 ` Rong Xu
2024-10-23 22:44 ` [PATCH v5 2/7] objtool: Fix unreachable instruction warnings for weak functions Rong Xu
2024-10-23 22:44 ` [PATCH v5 3/7] Change the symbols order when --ffunction-sections is enabled Rong Xu
2024-10-23 22:44 ` [PATCH v5 4/7] Add markers for text_unlikely and text_hot sections Rong Xu
2024-10-23 22:44 ` [PATCH v5 5/7] AutoFDO: Enable -ffunction-sections for the AutoFDO build Rong Xu
2024-10-23 22:44 ` Rong Xu [this message]
2024-10-23 22:44 ` [PATCH v5 7/7] Add Propeller configuration for kernel build Rong Xu
2024-10-25 23:03 ` [PATCH v5 0/7] Add AutoFDO and Propeller support for Clang build Yabin Cui
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20241023224409.201771-7-xur@google.com \
--to=xur@google.com \
--cc=akpm@linux-foundation.org \
--cc=aliceryhl@google.com \
--cc=arnd@arndb.de \
--cc=bp@alien8.de \
--cc=brgerst@gmail.com \
--cc=corbet@lwn.net \
--cc=dave.hansen@linux.intel.com \
--cc=davidxl@google.com \
--cc=eranian@google.com \
--cc=hca@linux.ibm.com \
--cc=hpa@zytor.com \
--cc=jannh@google.com \
--cc=jgross@suse.com \
--cc=jpoimboe@kernel.org \
--cc=justinstitt@google.com \
--cc=kees@kernel.org \
--cc=kpszeniczny@google.com \
--cc=leitao@debian.org \
--cc=linux-arch@vger.kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kbuild@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=llvm@lists.linux.dev \
--cc=masahiroy@kernel.org \
--cc=max4bolt@gmail.com \
--cc=miguel.ojeda.sandonis@gmail.com \
--cc=mingo@redhat.com \
--cc=morbo@google.com \
--cc=nathan@kernel.org \
--cc=ndesaulniers@google.com \
--cc=nicolas@fjasle.eu \
--cc=paulmck@kernel.org \
--cc=peterz@infradead.org \
--cc=richard.weiyang@gmail.com \
--cc=rppt@kernel.org \
--cc=samitolvanen@google.com \
--cc=shenhan@google.com \
--cc=tglx@linutronix.de \
--cc=tmsriram@google.com \
--cc=workflows@vger.kernel.org \
--cc=x86@kernel.org \
--cc=yabinc@google.com \
--cc=yonghong.song@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox