From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C9C0EB64D9 for ; Sat, 17 Jun 2023 16:08:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E4C8A8E0001; Sat, 17 Jun 2023 12:08:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DD6706B0078; Sat, 17 Jun 2023 12:08:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C76668E0001; Sat, 17 Jun 2023 12:08:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B37106B0075 for ; Sat, 17 Jun 2023 12:08:11 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 924D6C01F5 for ; Sat, 17 Jun 2023 16:08:11 +0000 (UTC) X-FDA: 80912721582.02.2670452 Received: from smtpout.efficios.com (smtpout.efficios.com [167.114.26.122]) by imf28.hostedemail.com (Postfix) with ESMTP id 58551C002C for ; Sat, 17 Jun 2023 16:08:09 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=efficios.com header.s=smtpout1 header.b=mFQYEPED; dmarc=pass (policy=none) header.from=efficios.com; spf=pass (imf28.hostedemail.com: domain of mathieu.desnoyers@efficios.com designates 167.114.26.122 as permitted sender) smtp.mailfrom=mathieu.desnoyers@efficios.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687018089; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6OFGDK8VRGUQmUC9I9KXk9vi98Gs9/mKoPXyBlsIn1E=; b=keCxtObaWm5kAjnphsSnjo8UYSXdyIIR7ieW4aEVbVPgACztM6enPoMFMl7F96mGsMUZPS CP3v0px86NbYlM+8H7oZ9S0PE622xvQnHg8oruWl1gkpEVa0EMRd6stHLQgrCwYIgcYOuu MkMSEbZ3cMxn5HBCBuFSlKWjNOo5Ll0= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=efficios.com header.s=smtpout1 header.b=mFQYEPED; dmarc=pass (policy=none) header.from=efficios.com; spf=pass (imf28.hostedemail.com: domain of mathieu.desnoyers@efficios.com designates 167.114.26.122 as permitted sender) smtp.mailfrom=mathieu.desnoyers@efficios.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687018089; a=rsa-sha256; cv=none; b=YrnecRUsBnmMunk4VrouTcNveX7qHSa6Slg1tfkIvpLNmpVZtq3CXBZQJgvSsRXd//d+c8 st5Rjw04lUpAnRDjJKpVMpULaVi9q/VtjypP2DQ3HODg+FB75lPRLcrD8tXCAT9WkNVDYy KPZkbJh5ziFIM4hYt9SjxnvAtbJIJQQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=efficios.com; s=smtpout1; t=1687018088; bh=hgJzCXMmkqHB9zu6/hj+qfZRgZydP6kjIOD4e+oiFOA=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=mFQYEPED4gaJvcYd/O4g7kh85pD3HfCmBmGvIdkS/CeO1+GqM6lShnYTTr3gpOD2+ +iKvK1voevlX17Etylc0ncbsfG76IkRi6lPWDut+biCo0o6va+ldebHsmIFyR0xzty TO3f2nkKYAPn6oWlXhHYHC8Ht1IvR5SVTchaWB3V5GAHKaWalLKNsGygXSJQLt7hZf DaEMIT6rD4t+zxqIlThQLOyMXAdimtHB6OlVSTW5ezrhv8uSjbw7vcy1cJWeH1ef9q Gepq+uI6pknU4ySsG6OkjP6aKYS/SCcTBCF03JCrLxITrO0xwqw4+24U6fsSlP706T jAluni3D9LDyA== Received: from [192.168.18.28] (unknown [107.159.220.152]) by smtpout.efficios.com (Postfix) with ESMTPSA id 4Qk1CX03Ksz18k0; Sat, 17 Jun 2023 12:08:07 -0400 (EDT) Message-ID: <6bbbb1f9-fc2b-c95d-5ef1-c178cfaae1ba@efficios.com> Date: Sat, 17 Jun 2023 12:08:26 -0400 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: [PATCH] mm: Move mm_count into its own cache line Content-Language: en-US To: John Hubbard , Andrew Morton Cc: Peter Zijlstra , linux-kernel@vger.kernel.org, kernel test robot , Aaron Lu , Olivier Dion , michael.christie@oracle.com, Feng Tang , Jason Gunthorpe , Peter Xu , linux-mm@kvack.org References: <20230515143536.114960-1-mathieu.desnoyers@efficios.com> <20230616131639.992998157fe696eb0e0589aa@linux-foundation.org> <1634ca8f-2b22-712e-15f9-9980ba8a4e64@nvidia.com> From: Mathieu Desnoyers In-Reply-To: <1634ca8f-2b22-712e-15f9-9980ba8a4e64@nvidia.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 58551C002C X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: jxzp3wuyephjnyrtp9dgon4rqqsjb3jq X-HE-Tag: 1687018089-932568 X-HE-Meta: U2FsdGVkX18wFbRTJ+uP8zvb5XSSQ2fenM4Qm+CsrzXm6uyoIvdlfH/8y3TH1AEyPcFMGt4siRVgxLw97QE0HhAvDD3yy33eWndBnVl1DqelqkyYtNUEK5rtmtq11yFITcc+0fIHpunDK6lvq2UhlqEPqxXMDC7Z4d1rtgtLLrrXNRSubxdHp5XYepQjZqUObpRVbxSmZmtDW5Yo7FIDb1QsAq8KptNXTQlJSymH6goIVU36o2Hr+4ZWFHX8GlCfEy7kmPou43/D3EYVHxYhLhI2E1o++zgWreD8E4FCnqyuOt1uw40Mn0KOWpPqMym95zXT4TYIy7SwVvlq8ZUg5G4V3QXzIxglOkVMowJub8UoEgXeyT1UEiD8kYKziiISSZCnCKoRpb3XD5yVqvgqeuUYpjt5irPt56KmIpIzBASdfA9PityrZXMydog/chVCSVknOkn/jj/RyyepjV+YmDVeXIPUXX65JRLfiNDQoUu0HuTTVOYXiJUxZSZtcN9MYClIOnhrR0fGmN9ndPt4EKJQl3d7dKEoZydkA5iZsV4cQgM2DsldoF3jut4YWosYvm5ROkpspRUe7MVKaDQ+xPb9glh86aPPRqR3qugFnA7OAUhNJ5ZU7mTpMbGk1u8WBFj/9KShBat7m1XkEuVr1DGZRTqL+Y7EySoFNT3isZljE1YWFgopFyZY7nvsjE2WAo98yNGGOdY46E5jI47l/k8JuERr9+GKJTqGDUCigTiZG6mz81bVTwMdDb8KJCYhhPu73Hj8VxBZ7FKVoMIsh0ng0ueZ2EX94G/ia9RUzK/X5spwV2ffxF1bWvYxWiFkF8Wrqg9//6IOYNJX9ojxX7jYDqdAXAsn8TEam5mXB0aGJrxXLUHNk4JjhqTBbG7uEMP7ID2r4rpCl2jhPzyoaoaw67Myj6wMxk2/5F/9Sze6k/ewH1Uh2kVUhLJpFjEx0t9rK3w/6Jq3sPwJbGn nGLIOuF1 SGaLTRZ2GI7/Uenur5cokHqqwCgTwV36eykqjOGXVED6UvJ9ytJAYTmYttsQm5W6hqPf1eZnM8mCcgvqquVpGaXMLgxHXP90YVpQTocEdkmRMHN5PwlNfL7EmDRzEWetAamOndoDG4ISRaJSimZpjkipBA8FsM71d1mQECptTL8xzzeluvM1eCqXjksrEnsbz3vIxHL3yJZsvc8zSzyCWCVFlJN9YNqXRCu+xkQ03mFtYvm0H3jcVI/9M/PdmUsuNcx3UBsVnQM5lQ4k8N2jaCB2O0oAoWyQQ8X0Nyijz/xrMt6RLIlyMqabNaRCaXdw0kkLs51vNJi0VuHQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 6/16/23 18:35, John Hubbard wrote: > On 6/16/23 13:38, Mathieu Desnoyers wrote: > ... >>>> This comment is rather odd for a few reasons: >>>> >>>> - It requires addition/removal of mm_struct fields to carefully >>>> consider >>>>    field alignment of _other_ fields, >>>> - It expresses the wish to keep an "optimal" alignment for a specific >>>>    kernel config. >>>> >>>> I suspect that the author of this comment may want to revisit this >>>> topic >>>> and perhaps introduce a split-struct approach for struct rw_semaphore, >>>> if the need is to place various fields of this structure in different >>>> cache lines. >>>> > > Agreed. The whole thing is far too fragile, but when reviewing this I > wasn't sure what else to suggest. Now looking at it again with your > alignment suggestion, there is an interesting conflicting set of > desires: > > a) Here: Feng Tang discovered that .count and .owner are best put in > separate cache lines for the contended case for mmap_lock, and > > b) rwsem.h, which specifies precisely the opposite for the uncontended > case: > >  * For an uncontended rwsem, count and owner are the only fields a task >  * needs to touch when acquiring the rwsem. So they are put next to each >  * other to increase the chance that they will share the same cacheline. > > I suspect that overall, it's "better" to align rw_semaphore's .count and > .owner field so that the lock is optimized for the contended case, > because it's reasonable to claim that the benefit of having those two > fields in the same cacheline for the uncontended case is far less than > the cost to the contended case, of keeping them close to each other. > > However, it's still not unlikely that someone will measure a performance > regression if such a change is made. > > Thoughts? > I suspect that the purpose of b) is only to maximize the functional density of the data cache: only using a single cache line for the rwsem uncontended case has the smallest footprint on the data cache. However, if we look at the rwsem in the context of its use within another data structure containing it, I think we can do better by analyzing the access pattern of _other_ fields of that structure. I have faced a similar situation within liburcu's wfcqueue's API [1], where it's better for head and tail to sit on different cache lines to eliminate false-sharing between enqueue and dequeue. I solved this by splitting the head and tail parameters in the API. So the user can decide to place them other on the same cache line, or on different cache lines, depending on the use-case. The user also has the freedom to place both head and tail on the same cache line as _other_ fields based on usage pattern. By providing enough flexibility to place the rwsem fields so that the count is on its own cache-line, and owner is on the same cache line as other fields touched when the rwsem is held, I suspect we can both improve functional density _and_ eliminate false-sharing in the contended case. Thanks, Mathieu [1] https://github.com/urcu/userspace-rcu/blob/master/include/urcu/wfcqueue.h#L279 > ... >>> If the plan is to put mm_count in "its own" cacheline then padding will >>> be needed? >> >> It's taken care of by the anonymous structure trick. Here is an quick >> example showing the difference between alignment attribute applied to >> an integer type vs to an anonymous structure: > > Thanks for explaining very clearly how that works, that's really > helpful! > thanks, -- Mathieu Desnoyers EfficiOS Inc. https://www.efficios.com