From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6EBF8C54E76 for ; Tue, 17 Jan 2023 18:15:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E75ED6B0072; Tue, 17 Jan 2023 13:15:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E25B46B0073; Tue, 17 Jan 2023 13:15:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CEDA36B0074; Tue, 17 Jan 2023 13:15:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id BBF566B0072 for ; Tue, 17 Jan 2023 13:15:28 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 807CA1202C0 for ; Tue, 17 Jan 2023 18:15:28 +0000 (UTC) X-FDA: 80365093536.09.DC27E6D Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) by imf08.hostedemail.com (Postfix) with ESMTP id E1417160017 for ; Tue, 17 Jan 2023 18:15:23 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=FpCwn17I; spf=none (imf08.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1673979326; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kjzPtu3zrCJ7LGN8rTmaJPE6Mz4bv33wdR1lSV8x46Y=; b=6BtZzjsfVKzSPfAhdiWE4QE3TFdLcsQb3d8pGXjmxbenk40nVpvNDZGnXqXOTIU+MHGsgh J+pgWqglOsWbnyl2HaaQe2rqgPwcoShsIRF1pmhtuB5pysCA8i4PiskJZ0Q6wiMrsQpVTQ +p+8/nGZXvLcxe+0p6FNOGBHZbCliOk= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=FpCwn17I; spf=none (imf08.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1673979326; a=rsa-sha256; cv=none; b=1JxM+oAYV1NHhjP1UYPbLngHLGTbxABhcEE9Rv/1KWLhy3FIQ7VlmYZPrU8Av83QneBtom Q9EdxAlVDtUGSssHHxDkX8EzsPVUn6IykxlcmV7OrR0YRW10ynCm1eoM7FS0szMaEgqh2L yFerPhtj7I3xkoYPXV6kHcAfoFLX3cI= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=kjzPtu3zrCJ7LGN8rTmaJPE6Mz4bv33wdR1lSV8x46Y=; b=FpCwn17IJKT76tn1oBj68F9qvT OjxGi/x1in4z37b9B+lVy0CNficyOfMmlE9g1rYdRRG1lTlmSEguXX5yxat4nL9STqETp6AJ6F69/ a/6HCtP2NNfgcV86cPQ4fijJtySW/24ehKr9nhK4DHtOFKw4+utyw5DWfkUvhNUUVENjufX119qHC JsHbdataE92XKLCPH2/FlxWYMURhOVS8ZzUb98kN+5H+P1z2JVNl5YTUwWKaZ02HJsZFkq9XvQMG0 kMDkLFN3j1excScPTSTM603ZRkLG3wZ7n/XRPRhWAiRkfsm0L1AMi3lrAHYzqnfV0DEmv6YEXwWdE FaB4n93w==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1pHqU0-0060fj-1T; Tue, 17 Jan 2023 18:14:53 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id AEFD93002BF; Tue, 17 Jan 2023 19:14:59 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 1000) id 9A12D202203E9; Tue, 17 Jan 2023 19:14:59 +0100 (CET) Date: Tue, 17 Jan 2023 19:14:59 +0100 From: Peter Zijlstra To: Linus Torvalds Cc: "Kirill A. Shutemov" , Dave Hansen , Andy Lutomirski , x86@kernel.org, Kostya Serebryany , Andrey Ryabinin , Andrey Konovalov , Alexander Potapenko , Taras Madan , Dmitry Vyukov , "H . J . Lu" , Andi Kleen , Rick Edgecombe , Bharata B Rao , Jacob Pan , Ashok Raj , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sami Tolvanen , ndesaulniers@google.com, joao@overdrivepizza.com Subject: Re: [PATCHv14 08/17] x86/mm: Reduce untagged_addr() overhead until the first LAM user Message-ID: References: <20230111123736.20025-1-kirill.shutemov@linux.intel.com> <20230111123736.20025-9-kirill.shutemov@linux.intel.com> <20230117135703.voaumisreld7crfb@box> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: E1417160017 X-Stat-Signature: yfk8cpy3zsnmk7xr6zeqo9hi4jgbgh33 X-HE-Tag: 1673979323-247720 X-HE-Meta: U2FsdGVkX18/7+Fx0WfmSuKOr4KCJKLV/NUz5LauRAcncUazQ29k/FT2RfB7ojQIJcigmNC/TTnNzD2/SXtMVktlgjtpKf3EtQMg8Uoye96rHlmStBfyIRV+0zE+yQ6IB4a3Y+PLqHrAzkE0Pzm3JXyZVI1/DjMOWAJzP5jfmdX9C7aN9KvZnFcdWf4Ysje5xadekoZ4ICfV5PBhUUu7B/GYpa8sn72WRlTO4NwC1ZVk4fmSMWZmuLGHULi808khR/GrX/84kiz6oNd/M8ringX661bHM6dsKl29CotKP/8Ys09TY/WFoATjzD8yvYAw6sevWNrFNEe8enJI6u4vzb/QdcM6mgsHf0TmnG+pq6m2URcA5iYHdJKwPNCAxLFewYwi3/Tfn2k6pS7TGb4npOzT/ak5y/ByXtshlaIBVOzXILEfHWCDRBf/km2g6VKNvj2Zs6uFMh8vpvSX5XBVXuGuGoaOCblMMVmeDei8HTcx52UZqk8CfLkYmKYmeb2eq06Mh3xJfQYuUNj9FRVrQb08bVBWfyfbhyDU8ZvmunwYoB9c2NF+gL0FIHRPYGau6Bz17SUUSZ4VyIe/WQwefymoqLfgcxHiYBLX1RRPV4ebui1/Bbxxwy9OzqebvZLbrHUPuI99jtz/p0sk39G4+JILQH0KfBg3VpKe4rYvzQWry6VtTsb81zlO8yj34BcPr02/uIuxQfxSgde7PASvimuZ3t1HBiZ5e8Mdmt9qXHCaalE03riDOiy7u4g76UuvOZYrtv/Tz7Ab7Rty01ULJm+VnlyP+LCcjqtWumQzH2YDPn5pAk9QiUWvJt9KEDwHDBU7zdYHORDr8pTUkxbVWBAl4Agetxt2iGWYq9TdIsDP5TCHqaACoMaRK7ESiJnDe7eMw/I9gwfgUqbiaDPJ6JKT/Ha8mgwvFWz0hzjV9iMd5l/9yTsIpMe21DXMLhh7w6/VGSbtHR6a21iLl4Y 1hWjaweK D2PAx X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jan 17, 2023 at 09:18:01AM -0800, Linus Torvalds wrote: > The reason clang seems to generate saner code is that clang seems to > largely ignore the whole "__builtin_expect()", at least not to the > point where it tries to make the unlikely case be out-of-line. So in this case there is only a 'likely' hint, we're explicitly trying to keep the thing in-line so we can jump over it. It is GCC that generated an implicit else (and marked it 'unlikely' -- which we didn't ask for), but worse, it failed to spot the else case is in fact shared with the normal case and it could've simply lifted that mov instruction. That is, instead of this: 0003 23b3: eb 76 jmp 242b 0005 23b5: 65 48 8b 0d 00 00 00 00 mov %gs:0x0(%rip),%rcx # 23bd 23b9: R_X86_64_PC32 tlbstate_untag_mask-0x4 000d 23bd: 48 89 f8 mov %rdi,%rax 0010 23c0: 48 c1 f8 3f sar $0x3f,%rax 0014 23c4: 48 09 c8 or %rcx,%rax 0017 23c7: 48 21 f8 and %rdi,%rax 001a 23ca: 48 b9 00 f0 ff ff ff 7f 00 00 movabs $0x7ffffffff000,%rcx 007b 242b: 48 89 f8 mov %rdi,%rax 007e 242e: eb 9a jmp 23ca It could've just done: 0003 48 89 f8 mov %rdi,%rax 0006 eb 76 jmp +18 0008 65 48 8b 0d 00 00 00 00 mov %gs:0x0(%rip),%rcx # 23bd 23b9: R_X86_64_PC32 tlbstate_untag_mask-0x4 0010 48 c1 f8 3f sar $0x3f,%rax 0014 48 09 c8 or %rcx,%rax 0017 48 21 f8 and %rdi,%rax 001a 48 b9 00 f0 ff ff ff 7f 00 00 movabs $0x7ffffffff000,%rcx and everything would've been good. In all the cases I've seen it do this, it was the same, it has this silly move out of line that's also part of the regular branch. That is, I like __builtin_expect() to be a strong hint. If I don't want things out of line, I shouldn't have put unlikely on it. What I don't like is that implicit else branches get the opposite strong hint. What I like even less is that it found it needed that else branch at all.