From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4944EB64D9 for ; Sun, 2 Jul 2023 21:11:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1E46F8E0084; Sun, 2 Jul 2023 17:11:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 194EA8E007C; Sun, 2 Jul 2023 17:11:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 085748E0084; Sun, 2 Jul 2023 17:11:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id ECD898E007C for ; Sun, 2 Jul 2023 17:11:22 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id B30B64020B for ; Sun, 2 Jul 2023 21:11:22 +0000 (UTC) X-FDA: 80967917604.08.145078B Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf12.hostedemail.com (Postfix) with ESMTP id E941840002 for ; Sun, 2 Jul 2023 21:11:20 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Lno3SkkQ; dmarc=none; spf=pass (imf12.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688332281; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RPrtYzed+Cadf70euVKoaSCFPnB2qjS8eTmUyZ83reY=; b=tUIFLkNjD9OxyVCdCNYymeguAKOwYJzv89rNyiwqsKDWIr7SncMcS1d4KyAUlI6MPYKklJ dDa/SULXieG0VRmaqCDTwjsXitR2yNBpHSRcmP1rjtMfgn9VoH10Lpo5+eIzy279QfTKgA oWqi+8Ikcdr7kuD/2yN8DpnJSgc1lKE= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Lno3SkkQ; dmarc=none; spf=pass (imf12.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688332281; a=rsa-sha256; cv=none; b=fX5HjO51o6Luqb7kOnKN0y61V0wInMk0lESwOCnTULB8qX3SbFLpspIn1m3a5CwNh6OWgM mbGA4a2qQJJ4vEYvaZfrj/gQjJMgEKK37PveV+avG+8PH1TSpdTlp2LQwoplHhytbfXqSe 7lvVUcFmqieSyZhXztaKQolSxzOocDA= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D153760C89; Sun, 2 Jul 2023 21:11:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BF697C433C7; Sun, 2 Jul 2023 21:11:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1688332279; bh=ozTYQx1eRJJJ+4EwI6iIprPJB77zziVTip4YVLzz3zM=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=Lno3SkkQL23XKhwFBx0qg26MXW3Io+5ES3F72CA/o+w0dAL3oBnu9VafN2JU7p4aH uUz+PZJRiP3DjqUClSiE3czwRX0fgA3S++quTOwSa8gpAKyUItDTvw1xqI1aHKGlTv H0j8q4p1heIRTU/5gWBWQ4UUlz3/MXLUZwuTM8j8= Date: Sun, 2 Jul 2023 14:11:17 -0700 From: Andrew Morton To: "Zhu, Lipeng" Cc: viro@zeniv.linux.org.uk, brauner@kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, pan.deng@intel.com, yu.ma@intel.com, tianyou.li@intel.com, tim.c.chen@linux.intel.com Subject: Re: [PATCH] fs/address_space: add alignment padding for i_map and i_mmap_rwsem to mitigate a false sharing. Message-Id: <20230702141117.d9827596dea4ca9d6c5d1fd3@linux-foundation.org> In-Reply-To: <20230628105624.150352-1-lipeng.zhu@intel.com> References: <20230628105624.150352-1-lipeng.zhu@intel.com> X-Mailer: Sylpheed 3.8.0beta1 (GTK+ 2.24.33; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: E941840002 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 7m5iabubyy1frk3bm1eu4ue3aggwnhwm X-HE-Tag: 1688332280-750544 X-HE-Meta: U2FsdGVkX18b9YUAQfTaDUlIoS/Y3qGbskYmXbQwoV4zx3GD0HrKQQsBkzM/RGrG1ZT6DQPSYu46Lb1VCMakkqDvSKfhju5WW8g3YqbseUADKAFplOoSr472exf4hfldByixnD0LffPZAMlQBAXAwSFkVriO+zbNogyY3+iV1zjN7lFt7aYtKOGlkuJLg5JcLd3x6BlYOD2FFA3jZAHVrksoLqXV2yOLsUtW4huGROqT0syOipdGS2RlsLMmfA6uzmnxfc07aRy+wG3QaHO/WV/R/UDgA4IPFmdLDOK9gFbAeqijrz53ZnLR+/VHQELunpzaNwInoGPVQYWDxIDCXT/wNJyptFKRpubqOsK3ZYvgqXD31V/sAzODLR/xk0Gz7y5Bk5FCGOoo/mb6NvR7IoAlpp1/UGZFM7KLFFmsbGAnTI71P7JSUhG7KwnJAXo1bBpOOqp8u6ZVhTVynZwwdj81W3CpM+DD9PpaO4MhBlr2lnIJzwYEzMDbnrB53H6eDPoQPsVMdCrJEWiSAkpZyob2MK3lLVSvw4AThCiIGSnzl/+vtKbliBd9Ey0VkJ4Qgxulf+SHIsKWfJcAVi+xuWVIgmvk5cTxrl8TgEQS/M5IgmOOtMFieGjnCNik2o4WhAL6zYUxay5JwR4jLz/H9H9Qxk7VWJ7o8PtMUFf3wDgmPtI7SFP1sgEjzTW/VaelXa98AzBkKaz3HXg6Xmx7k2f2uAdSDGyWr1L3vg6QdBx1xHmMQ29nPyV6RsLafXNiM83DeTx8sJcwyZdhrv1fFih8S9TfWGRGakUVGpu6GEZq98tuJ7O+S8e/rYPh3VlqYKtUWOlzhieQvBC+1kodWsoTohVOcn4MU11HTaMut7WUizNBUB83u0O12+fJUn0yzxLMpuZQiSr/yoqMJ0bdNjty6az6zLJ2W5LOVtEd7ww23jNzT5rjNYja+w4OguoWOYw/Cnx+93vI8Bd1pMh +ko8UrhK 2tXfG6FERo99mwyOXBJtiDhtimFgDlqLDhufwdJ5umsBrB8F+EnzlURPaRdwdPuMQH2SxOO1T79JSHAHrs5UjxVPu6V9kdaFtCLhQPbThhycPqnUZZdQIQE2K7EL7lZXG6JKO4F5APaDBlj3xeWzX//LMjVdIpxT0We9GGnXKykuu5AktvM3wVdel1EeZio29rBFu6s9FJTed5JCOF0DRDQQOLBML2WLkEPq3AYV+DGY5Dje7ZjPUd4O/tdp0DdljjlLu5rvQGOBCHaxo38DFFSlVPUtzyV+8XyJoO5xN5xsOIlu74FbevLLvQWVBypzzVT2ZJtO9d+LkSgWZjpNvaSrVaO+XDpWIEIDsvAZjQmLb+uqHJvQfzBot7JTJS/cS7R8AwpYq7+8/q8JexzL1wGpm5A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, 28 Jun 2023 18:56:25 +0800 "Zhu, Lipeng" wrote: > When running UnixBench/Shell Scripts, we observed high false sharing > for accessing i_mmap against i_mmap_rwsem. > > UnixBench/Shell Scripts are typical load/execute command test scenarios, > the i_mmap will be accessed frequently to insert/remove vma_interval_tree. > Meanwhile, the i_mmap_rwsem is frequently loaded. Unfortunately, they are > in the same cacheline. That sounds odd. One would expect these two fields to be used in close conjunction, so any sharing might even be beneficial. Can you identify in more detail what's actually going on in there? > The patch places the i_mmap and i_mmap_rwsem in separate cache lines to avoid > this false sharing problem. > > With this patch, on Intel Sapphire Rapids 2 sockets 112c/224t platform, based > on kernel v6.4-rc4, the 224 parallel score is improved ~2.5% for > UnixBench/Shell Scripts case. And perf c2c tool shows the false sharing is > resolved as expected, the symbol vma_interval_tree_remove disappeared in > cache line 0 after this change. There can be many address_spaces in memory, so a size increase is a concern. Is there anything we can do to minimize the cost of this?