From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A19D8D149E7 for ; Fri, 25 Oct 2024 19:46:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1EFBB6B007B; Fri, 25 Oct 2024 15:46:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 19F996B0082; Fri, 25 Oct 2024 15:46:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 08DDF6B0083; Fri, 25 Oct 2024 15:46:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id E085B6B007B for ; Fri, 25 Oct 2024 15:46:18 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 932AB1C6C6C for ; Fri, 25 Oct 2024 19:45:55 +0000 (UTC) X-FDA: 82713155406.26.C6278F9 Received: from gentwo.org (gentwo.org [62.72.0.81]) by imf04.hostedemail.com (Postfix) with ESMTP id 321E240008 for ; Fri, 25 Oct 2024 19:45:51 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gentwo.org header.s=default header.b=W4Jes7yt; spf=pass (imf04.hostedemail.com: domain of cl@gentwo.org designates 62.72.0.81 as permitted sender) smtp.mailfrom=cl@gentwo.org; dmarc=pass (policy=reject) header.from=gentwo.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729885498; a=rsa-sha256; cv=none; b=yRMqThhRNA/pS/u2cCGsi8yqmQCo//jluiPO2TM5WimgraOdBtZm5SW4S/gfT2Wtp8BWE4 RBdJYzKGtUMMKS8zyjmmPXyKzrJqeMnd2JYR/jxzvbaYAJtY+ZlKU3ZpbOKFrQCh3Fib/x kItzmY0vonPrkfYda1ICx/ZMmcCIVOA= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=gentwo.org header.s=default header.b=W4Jes7yt; spf=pass (imf04.hostedemail.com: domain of cl@gentwo.org designates 62.72.0.81 as permitted sender) smtp.mailfrom=cl@gentwo.org; dmarc=pass (policy=reject) header.from=gentwo.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729885498; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=x+Qq9Vx8YNRoZQjfWSXB0H8Q/jg1IHxnyqzO1wQC2po=; b=fPgfyW7Tqhi0tBc4HUF+2VM94DyAeEckEwO45kc/3tD6YA9aE9tVM9KS8mD5jZVWO69Uoo TlMHVCeXWWWKIG2KZBoRPXnk6mmpvi404kw9EOO2HmpO6djKDFemV4TDicrLQtRGMD6Qzs VyChCd2ebdMlJ9rkSa039tHJm/8wE9w= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gentwo.org; s=default; t=1729884988; bh=x+Qq9Vx8YNRoZQjfWSXB0H8Q/jg1IHxnyqzO1wQC2po=; h=Date:From:To:cc:Subject:In-Reply-To:References:From; b=W4Jes7ytlXV9rXvyZdJydttKL5xA9UZzAfrBUrbnFvkgcQ3do4auyKL2SppmTQzLk 4U/Oz2+zr97U7hbhOq9AXt0r5KitVEBSJKrRuCbP3LwYV2+6E5ueQhRzxscmpJZu2z zYDML7Iz7A1F9rWYQSf1xA3C8W5PPDdp7InrU9Kw= Received: by gentwo.org (Postfix, from userid 1003) id A35C6409CC; Fri, 25 Oct 2024 12:36:28 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by gentwo.org (Postfix) with ESMTP id A1F8640681; Fri, 25 Oct 2024 12:36:28 -0700 (PDT) Date: Fri, 25 Oct 2024 12:36:28 -0700 (PDT) From: "Christoph Lameter (Ampere)" To: Peter Zijlstra cc: tglx@linutronix.de, axboe@kernel.dk, linux-kernel@vger.kernel.org, mingo@redhat.com, dvhart@infradead.org, dave@stgolabs.net, andrealmeid@igalia.com, Andrew Morton , urezki@gmail.com, hch@infradead.org, lstoakes@gmail.com, Arnd Bergmann , linux-api@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, malteskarupke@web.de Subject: Re: [PATCH v1 11/14] futex: Implement FUTEX2_NUMA In-Reply-To: <20241025085815.GG14555@noisy.programming.kicks-ass.net> Message-ID: <887eadb6-6142-3edf-0a25-d33b2219b90d@gentwo.org> References: <20230721102237.268073801@infradead.org> <20230721105744.434742902@infradead.org> <9dc04e4c-2adc-5084-4ea1-b200d82be29f@linux.com> <20241025085815.GG14555@noisy.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Stat-Signature: ewhp33oyuiqje59djbean3x918xorx8o X-Rspamd-Queue-Id: 321E240008 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1729885551-821037 X-HE-Meta: U2FsdGVkX1+GBosrTD2aSRRQyKZhXdHDsbv4Rzku9O+t6Yohvy8oVhifIIfzR58p/rHvhDjqOk7FxslT9cMbhZcxkhF9/TrwCCmptevz15BDSJ4jZKtbLj79WarSDA9d+mvjtnra+o9J3nS/9wQCCvT6OqGLUes2XDjPW0oqO3dy5Gj/qwGDADChfEUGPqUluM7O/pWBpFB2iFGWSK8uuTAK0o/ANOCMPctSIEwS2Rr3C5EtNb+Urol/8hnhjdaRbOSK1EVFUmYstmWb/keuBAy6sGIMm4PER0LOG+UPkL3/wVit+yOJ5SpiFXU0tvBhSQt+UIUM5yinVc+wKd86IExIXiadUr+bBTCT1EbQ7/8jTE/mMHPduGgaRQePqwn53nmIAZBHaptHT4+DrG5V3xGNq/j5GxHK/L9gPVI9SdURPt81UaJFcNth1mBoz7/pa060vjQ7S+K581VsMVEJWn9KypvHiPIFmB0Hn4bAc4jVreURP5owH0BYZYNdtu7jrh9LYsbff3h0bNvvO1VR0NvxnkA86ep5dXr6wv42k5/Ogwyq1I2f7XXpdzH+k0Q83LZAXUD2KOOuN9v+o/GPG/sXtU3tIb+Js3yayzfia1QNtOUiaQWC60JHjixXF5To2Woe828C3lXZ3yG5sBcfVf0COpFE4jEwMcAq7goL+XJnkHHCtTcfc+wFixmYceRNBsR6TOGN+XF9zqpWuGGL78eZijfkWXqKCrgty5F0UWFCYtCtCopYqGGGOWIfUOUcvRvlp/bI5ZO7SAf9DjOFcbIuUEvtFZntZzhc8yyJZjJn5t0O3lJycSzaBZ6MnBHHVEc/0TlHIAfUDzO7Sg9IJXVTDvtOxrAF86Pu9+aIkrC6SUqDBXvjeELL6onZxHhkskSk/coNondFZeUpiBV0YlP+XCNyR68Ldi6N87NcT8gpc7LMXm6qC08hrHE1kOzGJTD3BlydAEUrNGJaPbw 3kApQ2ZV 7Z+9i3YKIF+PtWZRrtg/3h0KqhjMyXHAbKZIiuFUrZJKGWo4nWgm4wIbCOhn8nyMeSyyQkkXQSGlOZ8s4B+rbNXrg+pJxScj9FUuOZYKwrTxXSw+hSoYqPbLKXxQJ45aeRHheBHUqZHq+XoR/nZIn2OwWKXm+50IpTe3NbPiIcEvsLTX6Q1hGXS3LzJc62zdEkk1lcNDEjNfg222a/wNqDgdeEOzKbArh4JIVJkOAstJCbTjlnD55hecG3A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Sorry saw this after the other email. On Fri, 25 Oct 2024, Peter Zijlstra wrote: > > Could we follow NUMA policies like with other metadata allocations during > > systen call processing? > > I had a quick look at this, and since the mempolicy stuff is per vma, > and we don't have the vma, this is going to be terribly expensive -- > mmap_lock and all that. There is a memory policy for the task as a whole that is used for slab allocations and allocations that are not vma bound in current->mempolicy. Use that. > Using memory policies is probably okay -- but still risky, since you get > the extra failure case where if you change the mempolicy between WAIT > and WAKE things will not match and sadness happens, but that *SHOULD* > hopefully not happen a lot. Mempolicies are typically fairly static. Right. > > That way the placement of the futex can be controlled by the tasks memory > > policy. We could skip the FUTEX2_NUMA option. > > That doesn't work. If we don't have storage for the node across > WAIT/WAKE, then the node must be deterministic per futex_hash(). > Otherwise wake has no chance of finding the entry. You can get a node number following the current task mempolicy by calling mempolicy_slab_node() and keep using that node for the future. It is also possible to check if the policy is interleave and then follow the distributed hash scheme. > The current scheme where we determine node based on hash bits is fully > deterministic and WAIT/WAKE will agree on which node-hash to use. The > interleave is no worse than the global hash today -- OTOH it also isn't > better. This is unexpected strange behavior for those familiar with NUMA. We have tools to set memory policies for tasks and those policies should be used throughout.