From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qv1-f42.google.com (mail-qv1-f42.google.com [209.85.219.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 49F3618308A for ; Mon, 4 Nov 2024 04:50:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.42 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730695824; cv=none; b=X6Fpw6aj8XEtaqa7H2GHRdaPgdTj7Ccf05R7WwX5Suy6FGGmlbKAb9iJfSFgg1JjQYOvANG2ZOIhwn3ezkXIKZORGfIWsHO3ifP6jh8f/GPXf3YP0faNlAdhCq2gDyKqfrhzWagGLnYhuL1N23WRyH5XlJ3mLWtxYUclbu5RWnA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730695824; c=relaxed/simple; bh=yHLwBvRcDmvZBD3Q2o8biW6+ZyYq54VMM35NculsqtI=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type; b=jsSz1pZ6Om4DEMRoq+cx1zNlz3XClme4Um9u0DTqmXnrr8nKzcpzoXyHjq/QaFoq2meIkOlo/MFWdXG9oVggecZdAMGZe7U+B91Lbb1CkPwIlpsGKKsvlLCHTeR9lo0+ve/DxUCOldKECS0/3uQDyqQxIi36xE0W6BYznF5QHbY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=SWgK+I07; arc=none smtp.client-ip=209.85.219.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="SWgK+I07" Received: by mail-qv1-f42.google.com with SMTP id 6a1803df08f44-6cbe68f787dso25970926d6.2 for ; Sun, 03 Nov 2024 20:50:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1730695821; x=1731300621; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=xelsbw4CYPm0pAy0Wg832CMC7u06eSjN58hX3diMwfk=; b=SWgK+I07yOYt0VMvJrp8V1tWatT7l0GZGHvUcbfFY9XOrEXrq29WOiiFoOERhxojIC vks7mAumVQkjv0etl6NtH9QgA3nHzr3EcWjPDBIzQts8L2PeXmtqUdCN0s2t1Ao4RDpr cTtQjhO33lFRuSpQcehH06OqZ8rzqDJHurPKE28JyMM807KVP4C60ZgAfSAjN+zx0VC0 sfK6tA/wQr506Nu9oGGZK9gk0lTe98EswrunX7o1rZp6chwbJooQsZwd3QqsFziW0TwG KrnCLqTRNDFoi8EAkPCuWP2Jf386Du4pS/VT/6qrsT6jY+mpHrWRUY7IinHF1RsKZEXs ftVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730695821; x=1731300621; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xelsbw4CYPm0pAy0Wg832CMC7u06eSjN58hX3diMwfk=; b=Zydy10GN7JO0u0X2vOrm+aUrMtJ8TYEOE6z+uu24jNHUMtfmjn+Dq+wESOEC628Bqa ddi0g39KJ2ZTK1UDD8OC6LVjzctNja7KArwLuZu7dGgE/OSgE+Aaf53D1c53Bbl6WzpF 5EwNs6168rRkv31Co23sDmWnZ5GsUPBVHKvB9IJ8skkWLiw0F6Jh0aFTuL2sEx8XaYQm bt1I+XMBmuHNL+2wOKUrHvSty/YwA8ERjt/g53GcSBUkbVvCxq4Nm5SL+3sxN4H3vZ1m gUzHkKDsz3V9o2avMrnTJbdEd5y/fDQ1rpDIIq9/4r1ir82wo4I7EzE9K9Xp2TR7Kups DujA== X-Forwarded-Encrypted: i=1; AJvYcCUarA8tq4OLZWqUpWwo19FszZOPmEgR63ELuS95g+VhTV3p+RyqLtMBX71WfRFjumuy8N6WIo2mczI=@vger.kernel.org X-Gm-Message-State: AOJu0YzK3BpcG6J+GhlwFmw5QuHyxVEGVjuchdZN/qJ8xbGF6Cwv/k6G Oyw8a0xZDr7HhMQq+wnL0uLtGyyVe73sayvZ7bpF1iCElo6SmcCBe2wD6BTyw3k6X/NMzl12HKn yDeGkCfZcqt88YVyL5UnaOgUfctwqY7nxhdOa X-Google-Smtp-Source: AGHT+IF2tDFS5vlJFQGEA8PSXd+4TNc7lauPgQRDf2fcZBAWgQ+PxfptvywKq5K3S+F5pPhyHSNjQzY8mD3d0dhW7vA= X-Received: by 2002:a05:6214:5508:b0:6ce:2519:5dd7 with SMTP id 6a1803df08f44-6d351af4ccdmr195294736d6.35.1730695820784; Sun, 03 Nov 2024 20:50:20 -0800 (PST) Precedence: bulk X-Mailing-List: workflows@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: <20241102175115.1769468-1-xur@google.com> <20241102175115.1769468-2-xur@google.com> <09349180-027a-4b29-a40c-9dc3425e592c@cachyos.org> <3183ab86-8f1f-4624-9175-31e77d773699@cachyos.org> In-Reply-To: <3183ab86-8f1f-4624-9175-31e77d773699@cachyos.org> From: Han Shen Date: Sun, 3 Nov 2024 20:50:07 -0800 Message-ID: Subject: Re: [PATCH v7 1/7] Add AutoFDO support for Clang build To: Peter Jung Cc: Rong Xu , Alice Ryhl , Andrew Morton , Arnd Bergmann , Bill Wendling , Borislav Petkov , Breno Leitao , Brian Gerst , Dave Hansen , David Li , Heiko Carstens , "H. Peter Anvin" , Ingo Molnar , Jann Horn , Jonathan Corbet , Josh Poimboeuf , Juergen Gross , Justin Stitt , Kees Cook , Masahiro Yamada , "Mike Rapoport (IBM)" , Nathan Chancellor , Nick Desaulniers , Nicolas Schier , "Paul E. McKenney" , Peter Zijlstra , Sami Tolvanen , Thomas Gleixner , Wei Yang , workflows@vger.kernel.org, Miguel Ojeda , Maksim Panchenko , "David S. Miller" , Andreas Larsson , Yonghong Song , Yabin Cui , Krzysztof Pszeniczny , Sriraman Tallam , Stephane Eranian , x86@kernel.org, linux-arch@vger.kernel.org, sparclinux@vger.kernel.org, linux-doc@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Peter, thanks for reporting the issue. I am trying to reproduce it in the up-to-date archlinux environment. Below is what I have: 0. pacman -Syu 1. cloned archlinux build files from https://aur.archlinux.org/linux-mainline.git the newest mainline version is 6.12rc5-1. 2. changed the PKGBUILD file to include the patches series 3. changed the "config" to turn on clang autofdo 4. collected afdo profiles 5. MAKEFLAGS=3D"-j48 V=3D1 LLVM=3D1 CLANG_AUTOFDO_PROFILE=3D$(pwd)/perf.a= fdo" \ makepkg -s --skipinteg --skippgp 6. install and reboot The above steps succeeded. You mentioned the error happens at "module_install", can you instruct me how to execute the "module_install" step? Thanks, Han On Sat, Nov 2, 2024 at 12:53=E2=80=AFPM Peter Jung wr= ote: > > > > On 02.11.24 20:46, Peter Jung wrote: > > > > > > On 02.11.24 18:51, Rong Xu wrote: > >> Add the build support for using Clang's AutoFDO. Building the kernel > >> with AutoFDO does not reduce the optimization level from the > >> compiler. AutoFDO uses hardware sampling to gather information about > >> the frequency of execution of different code paths within a binary. > >> This information is then used to guide the compiler's optimization > >> decisions, resulting in a more efficient binary. Experiments > >> showed that the kernel can improve up to 10% in latency. > >> > >> The support requires a Clang compiler after LLVM 17. This submission > >> is limited to x86 platforms that support PMU features like LBR on > >> Intel machines and AMD Zen3 BRS. Support for SPE on ARM 1, > >> and BRBE on ARM 1 is part of planned future work. > >> > >> Here is an example workflow for AutoFDO kernel: > >> > >> 1) Build the kernel on the host machine with LLVM enabled, for example= , > >> $ make menuconfig LLVM=3D1 > >> Turn on AutoFDO build config: > >> CONFIG_AUTOFDO_CLANG=3Dy > >> With a configuration that has LLVM enabled, use the following > >> command: > >> scripts/config -e AUTOFDO_CLANG > >> After getting the config, build with > >> $ make LLVM=3D1 > >> > >> 2) Install the kernel on the test machine. > >> > >> 3) Run the load tests. The '-c' option in perf specifies the sample > >> event period. We suggest using a suitable prime number, > >> like 500009, for this purpose. > >> For Intel platforms: > >> $ perf record -e BR_INST_RETIRED.NEAR_TAKEN:k -a -N -b -c > >> \ > >> -o -- > >> For AMD platforms: > >> The supported system are: Zen3 with BRS, or Zen4 with amd_lbr_v= 2 > >> For Zen3: > >> $ cat proc/cpuinfo | grep " brs" > >> For Zen4: > >> $ cat proc/cpuinfo | grep amd_lbr_v2 > >> $ perf record --pfm-events RETIRED_TAKEN_BRANCH_INSTRUCTIONS:k > >> -a \ > >> -N -b -c -o -- > >> > >> 4) (Optional) Download the raw perf file to the host machine. > >> > >> 5) To generate an AutoFDO profile, two offline tools are available: > >> create_llvm_prof and llvm_profgen. The create_llvm_prof tool is pa= rt > >> of the AutoFDO project and can be found on GitHub > >> (https://github.com/google/autofdo), version v0.30.1 or later. The > >> llvm_profgen tool is included in the LLVM compiler itself. It's > >> important to note that the version of llvm_profgen doesn't need to > >> match the version of Clang. It needs to be the LLVM 19 release or > >> later, or from the LLVM trunk. > >> $ llvm-profgen --kernel --binary=3D -- > >> perfdata=3D \ > >> -o > >> or > >> $ create_llvm_prof --binary=3D --profile=3D= \ > >> --format=3Dextbinary --out=3D > >> > >> Note that multiple AutoFDO profile files can be merged into one vi= a: > >> $ llvm-profdata merge -o ... > >> > >> > >> 6) Rebuild the kernel using the AutoFDO profile file with the same con= fig > >> as step 1, (Note CONFIG_AUTOFDO_CLANG needs to be enabled): > >> $ make LLVM=3D1 CLANG_AUTOFDO_PROFILE=3D > >> > >> Co-developed-by: Han Shen > >> Signed-off-by: Han Shen > >> Signed-off-by: Rong Xu > >> Suggested-by: Sriraman Tallam > >> Suggested-by: Krzysztof Pszeniczny > >> Suggested-by: Nick Desaulniers > >> Suggested-by: Stephane Eranian > >> Tested-by: Yonghong Song > >> Tested-by: Yabin Cui > >> Tested-by: Nathan Chancellor > >> Reviewed-by: Kees Cook > > > > Tested-by: Peter Jung > > > > The compilations and testing with the "make pacman-pkg" function from > the kernel worked fine. > > One problem I do face: > When I apply a AutoFDO profile together with the PKGBUILD [1] from > archlinux im running into issues at "module_install" at the packaging. > > See following log: > ``` > make[2]: *** [scripts/Makefile.modinst:125: > /tmp/makepkg/linux-cachyos-rc-autofdo/pkg/linux-cachyos-rc-autofdo/usr/li= b/modules/6.12.0-rc5-5-cachyos-rc-autofdo/kernel/arch/x86/kvm/kvm.ko] > Error 1 > make[2]: *** Deleting file > '/tmp/makepkg/linux-cachyos-rc-autofdo/pkg/linux-cachyos-rc-autofdo/usr/l= ib/modules/6.12.0-rc5-5-cachyos-rc-autofdo/kernel/arch/x86/kvm/kvm.ko' > INSTALL > /tmp/makepkg/linux-cachyos-rc-autofdo/pkg/linux-cachyos-rc-autofdo/usr/li= b/modules/6.12.0-rc5-5-cachyos-rc-autofdo/kernel/crypto/cryptd.ko > make[2]: *** Waiting for unfinished jobs.... > ``` > > > This can be fixed with removed "INSTALL_MOD_STRIP=3D1" to the passed > parameters of module_install. > > This explicitly only happens, if a profile is passed - otherwise the > packaging works without problems. > > Regards, > > Peter Jung >