From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4453CDB465 for ; Mon, 16 Oct 2023 12:40:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 258FB8D006E; Mon, 16 Oct 2023 08:40:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1DF9D8D0001; Mon, 16 Oct 2023 08:40:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 082268D006E; Mon, 16 Oct 2023 08:40:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id E863B8D0001 for ; Mon, 16 Oct 2023 08:40:14 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 9297340918 for ; Mon, 16 Oct 2023 12:40:14 +0000 (UTC) X-FDA: 81351282348.12.8A6CFB7 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf14.hostedemail.com (Postfix) with ESMTP id 6E396100023 for ; Mon, 16 Oct 2023 12:40:12 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of alexandru.elisei@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=alexandru.elisei@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697460013; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7A0wbU2zva4yUVI2ojIAha5iaTFNMMOR5zkRFB42juA=; b=cnZvoVINnA/fRBXkS6/SKZ/oHVXmJc+CNvZgLpdopAXBegewnxFc9l0m6Sn5kNzSvrvz/p mzTxwjW8uufaJhO5+1duTwl8YU6iRmY+YjrCg1wVf1nN3vf0zYpleJ6b/xHdZcDF3mnnNe sc2j1udFWJOI8e90EIuOcXG24qepGIE= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of alexandru.elisei@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=alexandru.elisei@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697460013; a=rsa-sha256; cv=none; b=IPklsFB67VH9QfmLwMQB+WEUE7kh0XJvRTjcs6WLvFzkTjX9G5o6qTK/thpn8fv8G4kWDP xm+uJvncLuk31G07pkyZi6nYkuhyqfKPHvVYjDkUVNgI5jfqxfD7ZAiK1yl8fZ/8K+zTUu /ZF+jJcyI6yznZL6mDhVpqm6esMkB7M= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DD7541FB; Mon, 16 Oct 2023 05:40:51 -0700 (PDT) Received: from monolith (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E092D3F5A1; Mon, 16 Oct 2023 05:40:05 -0700 (PDT) Date: Mon, 16 Oct 2023 13:40:39 +0100 From: Alexandru Elisei To: Hyesoo Yu Cc: catalin.marinas@arm.com, will@kernel.org, oliver.upton@linux.dev, maz@kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, arnd@arndb.de, akpm@linux-foundation.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, mhiramat@kernel.org, rppt@kernel.org, hughd@google.com, pcc@google.com, steven.price@arm.com, anshuman.khandual@arm.com, vincenzo.frascino@arm.com, david@redhat.com, eugenis@google.com, kcc@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org Subject: Re: [PATCH RFC 04/37] mm: Add MIGRATE_METADATA allocation policy Message-ID: References: <20230823131350.114942-1-alexandru.elisei@arm.com> <20230823131350.114942-5-alexandru.elisei@arm.com> <20231012012824.GA2426387@tiffany> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20231012012824.GA2426387@tiffany> X-Rspamd-Queue-Id: 6E396100023 X-Rspam-User: X-Stat-Signature: xnz4546qxor3sh3ok4y38j9jzs33uw8x X-Rspamd-Server: rspam01 X-HE-Tag: 1697460012-438560 X-HE-Meta: U2FsdGVkX1/GADYC6R4mxGMdetV1sNPkoHsX2FiQnpag77W1MNXDYkhwzIdXfpzmPDlhN6gOtdgYwl3dA3b+P5gTahXwY9C/exccu/CmnFMX4cPuvv2KwpPBhx7nNoY8y137eSgfDOE6OfkSDhTAHocUYdD9zscMZtso0YTky15aCAq6StS6Cl8wMGfIqqoT1C9tHP2i9GvWfdC8q1OU1zCQy3ZilG6Q2ldBYcDJiVKnol1F8ZGkzXd1g1iFqnh2UHB+rGRD+WeNjwB8c/NJMBu2rvXNyXn7HoHOV+IKHI/aJSKXW9AL2vF0kgNmozoiDp6TW0EFVvdhtEdrzm+ByuDkmFHNqREtiD2LqjH9AQClgycV+3VzP7mrSqZj86Ye8dVcqG2BxHT41iPI2omRVSdppHEPze1+Apd1E4aGwRVb/Cfhv8uJE0cR1SqEmYuYsY2IVHh2IQofXjrWZ/ZIWi+t7fFbeRsCs66Gk+WJtqg5enLVAsf3lzviyAKla1EN0FGAei8Jiv2GOQna1a5ov4AFDbGnYK4qi+kwDqlbxY0XPHHcUxsam71p34k9vpjZEmCiAagiT/xk848VuyXMFdfym/gI1aYiowPThgF8ULAEf0u3Hs6vixdWJ8mXSEHGjAV203BJ69IsJbwDoa9PLJcf0m6WcH3zQ83vrERHDHUZtJmGpvVW1OB6s/oCp2xdd4cNqSp9DFZ0xQaS9O0LuqvrEguh21nYlr6Xa6pepmwS3BeZkJm7Uh9bsuTAHuelh8QwC/17pQ1J8pgNfyieTh0eKa6juwLsBURSRLDn1mqqaOA2Z0ThQBpX8FY488sB/7uL36Dwe1QgXC+kAnt/gBV25+/3PWBQO+aEZvkAZVaM6EPP6PhelX96DSFpXNzeH9zvKTCXxOTwtw0C/U9zDmYQmGorT8tin/G8ToMirYQt1FxTuVGY1x3yyQbfzM1h+226lIPAmAZxgnwImE0 LaSRmTIr YxdFQo/bb6vOhDZ2wX1yb+WZ12NahuhxvgGSE1SEoyJODTzUY7Ne+4J6qs0Oe60gVtcWhN4QbnjqanBaTe48qfRwNDo6ogtvph6HAe4FgNGw106LVUxpph03adn0AaS89fNkMn6A0Zes9Q34stVs5xHFpkdfci3sspDvyoLatNyZBaLhKV5qix1bEHN84hum/YYc0X5EP/5vny8DxVIIHUhsdMw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hello, On Thu, Oct 12, 2023 at 10:28:24AM +0900, Hyesoo Yu wrote: > On Wed, Aug 23, 2023 at 02:13:17PM +0100, Alexandru Elisei wrote: > > Some architectures implement hardware memory coloring to catch incorrect > > usage of memory allocation. One such architecture is arm64, which calls its > > hardware implementation Memory Tagging Extension. > > > > So far, the memory which stores the metadata has been configured by > > firmware and hidden from Linux. For arm64, it is impossible to to have the > > entire system RAM allocated with metadata because executable memory cannot > > be tagged. Furthermore, in practice, only a chunk of all the memory that > > can have tags is actually used as tagged. which leaves a portion of > > metadata memory unused. As such, it would be beneficial to use this memory, > > which so far has been unaccessible to Linux, to service allocation > > requests. To prepare for exposing this metadata memory a new migratetype is > > being added to the page allocator, called MIGRATE_METADATA. > > > > One important aspect is that for arm64 the memory that stores metadata > > cannot have metadata associated with it, it can only be used to store > > metadata for other pages. This means that the page allocator will *not* > > allocate from this migratetype if at least one of the following is true: > > > > - The allocation also needs metadata to be allocated. > > - The allocation isn't movable. A metadata page storing data must be > > able to be migrated at any given time so it can be repurposed to store > > metadata. > > > > Both cases are specific to arm64's implementation of memory metadata. > > > > For now, metadata storage pages management is disabled, and it will be > > enabled once the architecture-specific handling is added. > > > > Signed-off-by: Alexandru Elisei > > --- > > [..] > > @@ -2144,6 +2156,15 @@ __rmqueue(struct zone *zone, unsigned int order, int migratetype, > > if (alloc_flags & ALLOC_CMA) > > page = __rmqueue_cma_fallback(zone, order); > > > > + /* > > + * Allocate data pages from MIGRATE_METADATA only if the regular > > + * allocation path fails to increase the chance that the > > + * metadata page is available when the associated data page > > + * needs it. > > + */ > > + if (!page && (alloc_flags & ALLOC_FROM_METADATA)) > > + page = __rmqueue_metadata_fallback(zone, order); > > + > > Hi! > > I guess it would cause non-movable page starving issue as CMA. I don't understand what you mean by "non-movable page starving issue as CMA". Would you care to elaborate? > The metadata pages cannot be used for non-movable allocations. > Metadata pages are utilized poorly, non-movable allocations may end up > getting starved if all regular movable pages are allocated and the only > pages left are metadata. If the system has a lot of CMA pages, then > this problem would become more bad. I think it would be better to make > use of it in places where performance is not critical, including some > GFP_METADATA ? GFP_METADATA pages must be used only for movable allocations. The kernel must be able to migrate GFP_METADATA pages (if they have been allocated) when they are reserved to serve as tag storage for a newly allocated tagged page. If you are referring to the fact that GFP_METADATA pages are allocated only when there are no more free pages in the zone, then yes, I can understand that that might be an issue. However, it's worth keeping in mind that if a GFP_METADATA page is in use when it needs to be repurposed to serve as tag storage, its contents must be migrated first, and this is obviously slow. To put it another way, the more eager the page allocator is to allocate from GFP_METADATA, the slower it will be to allocate tagged pages because reserving the corresponding tag storage will be slow due to migration. Before making a decision, I think it would be very helpful to run performance tests with different allocation policies for GFP_METADATA. But I would say that it's a bit premature for that, and I think it would be best to wait until the series stabilizes. And thank you for the feedback! Alex > > Thanks, > Hyesoo Yu.