From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC782C6FD18 for ; Wed, 19 Apr 2023 02:08:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4C7168E0002; Tue, 18 Apr 2023 22:08:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 477B28E0001; Tue, 18 Apr 2023 22:08:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 33EFE8E0002; Tue, 18 Apr 2023 22:08:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 2119D8E0001 for ; Tue, 18 Apr 2023 22:08:19 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id E338D1C66AD for ; Wed, 19 Apr 2023 02:08:18 +0000 (UTC) X-FDA: 80696505876.18.65C62E6 Received: from mail-qt1-f180.google.com (mail-qt1-f180.google.com [209.85.160.180]) by imf23.hostedemail.com (Postfix) with ESMTP id DD52914000A for ; Wed, 19 Apr 2023 02:08:16 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=F+uP6RWj; spf=pass (imf23.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.180 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681870097; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2i54J70P7BHws90mr0mZ0tSyG0lrFeeDQmHC/zLnk6E=; b=AMgUDEN27Q2K+woIoMZ5U+cJtMxyQ6AIlvrMWymX1xOz+cOaggBXXZzs7OXjxU6ElkHETh n/4vfz0XL0L5J5X7f2ORqq7NeDTrUY+HmYqGyjzi4buUmMePLOVFBVbrijo9FrC6kag1Np B4MppIecWZUhhoHmv2LD4UBwMWynXnc= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=F+uP6RWj; spf=pass (imf23.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.180 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681870097; a=rsa-sha256; cv=none; b=PS8YAGkM/wfRIUR4UBw6pLRajdKVBfp/+6n8PimZJEeD/nMQmzMIWJBJzZ4/MnOr76IsxQ aQquT7LTPn8Q8uVSXOn22guEwDrNvYLCvNeQuewKp8Kds64OwFcemi8P6w5wyDln728u4h cJr/mz0e5hDpvouJ54K+mqQwy1/zmnY= Received: by mail-qt1-f180.google.com with SMTP id ei23so3144017qtb.2 for ; Tue, 18 Apr 2023 19:08:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20221208.gappssmtp.com; s=20221208; t=1681870096; x=1684462096; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=2i54J70P7BHws90mr0mZ0tSyG0lrFeeDQmHC/zLnk6E=; b=F+uP6RWjfrt7M5J+rVPxz+azqdWmSkNnUt8Rc9nzB0D8OBAvY2zC5YzLZHbP6CtrY0 RI9UnC3Lsq9zr+GA+CXa5zK7tOxIGBhdMz8fCEj3WKIUL03JByPY1frjAXAaFa6sxi+q id4d3K1qRZHwO0m+JIjLX3DD4OcA+c0gp85eXj7oZeOEt2oeWNUQALi5aubsk1PTA0A4 ywgK1FqREJm6t49JtmuZj1id4MS8kGBV7sdu62dPyWyWfVi0Q/M485OauQB2R4VVORzZ iAVBS/z37wTW5nXHsWqvtlX+NTDGe9LQED2QiIRCPqvA6WOfW4bLej0F1I2QtRaIFUOp KisQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681870096; x=1684462096; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=2i54J70P7BHws90mr0mZ0tSyG0lrFeeDQmHC/zLnk6E=; b=MAA36QqfAbnAxs7oea4Vu64ONF5ufxjzYn0qbRqIh5lhDb5DqQ3/4Wkugfe20HqtUp 5v+JmN5IMET/EWFVufP52+EKnkWonSj3YXSddt7Ccsfc6gf/cSx+wMOQTD4SjNMKzgKY 0KJCUjZUGCXKEQogMWTfCVYBxHytCyATolkgBqN/qsYxuGi2hpJ0d6yZCRpAvnMbLJUD cSOmf5/74s5caMrJonYDPO6voDETNPnHr/fyaHYy9PiQJd2ZDGo2kwYwyRfndzoUYyUm NP077L7x11MjE2Hek2P9K2O0iFobtsdEhFSer9xOKrbQEpiMCmbfQ45NITc6vAOQbsoH tthw== X-Gm-Message-State: AAQBX9cylX/96bNP9Mb44bvDgtjP1lzDnDZVk+vXFvqjPrJ6psMGfNDz HaeYQZqiI5nwz57xYlzuio63oEdaWyxZQHGQebA= X-Google-Smtp-Source: AKy350Zb5g9QoBeP8dMxH/J5F1Lj3IrlRKoEL6OoEmZjYedwJVSJCUuA3wb1faeRrAklioPZUgmo3w== X-Received: by 2002:a05:622a:1010:b0:3e6:55b2:35e with SMTP id d16-20020a05622a101000b003e655b2035emr4017850qte.26.1681870095830; Tue, 18 Apr 2023 19:08:15 -0700 (PDT) Received: from localhost ([2620:10d:c091:400::5:e646]) by smtp.gmail.com with ESMTPSA id v9-20020a05620a0a8900b0074abe1e1457sm4333376qkg.76.2023.04.18.19.08.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Apr 2023 19:08:15 -0700 (PDT) Date: Tue, 18 Apr 2023 22:08:14 -0400 From: Johannes Weiner To: "Kirill A. Shutemov" Cc: linux-mm@kvack.org, Kaiyang Zhao , Mel Gorman , Vlastimil Babka , David Rientjes , linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: Re: [RFC PATCH 00/26] mm: reliable huge page allocator Message-ID: <20230419020814.GA272256@cmpxchg.org> References: <20230418191313.268131-1-hannes@cmpxchg.org> <20230418235402.lq7mxrrre2kl6vsf@box.shutemov.name> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230418235402.lq7mxrrre2kl6vsf@box.shutemov.name> X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: DD52914000A X-Stat-Signature: 5qdjyyysznhm3dtmdkzo31wtp78g8dd7 X-HE-Tag: 1681870096-444931 X-HE-Meta: U2FsdGVkX1/jLMlTiau9lLQ578FMlFSYfBDshklnVXAdSU30DZeklay9X6fMz5vVMxUYi7P5i7MyasvTHhW186M1LSvRUM+7Cdl2l74fg1cQ2XAS/nWlwUHZok4ohnyErRKKyYIrbK6tuwB9kLGvi9QfNFiV8R6Uskm+nlwxbEZwVBhRkkuNRmzESqaMZXiIEX3IJLlRms+QQuVlqSF/mT+t5rord7HF6w2maGDuu3kQdxszSKXMVJXGn9ijQI8bI6xQ4nsCE08NYEaRAwcOi0bD4sli0SeZZR87W3BAYXHEMUPFY6nOt679N7IywRItJnCnvuyAIqJ2fM+wMxG2X3nr8K8xFTPiJhkEOM1p42K0GbWVgR+qtd0WdEk4yN0uRpmhY9UIuZc4YQLDxcXoW0qQGmLYCwVJKE21jDLchb/p79vJsZr/sUvWpMw6ry5SZm3PfvWgcazXhhBx5lijofjhDtL4BOpWN1D/wKOy6S8KjArygNnuWR5MM26gZo2mMqbJ/4Iq20tvomppmJ35yMOTyr4sD9EqoLFfG1wVmBCJTDeEm5nNiYWEUpMJNgahM1RTuWvcxl4rVulopyjy0L9iet6MMhKPRGhc/ei+Aauc4RFEMjVqpPOOLnCZLVgKrsoqiS9dAdij3ojBd9Ga20ZpRNSM6DlH+TfL+DWioT2XyorBci6iIoP4UxWXz5kDZY4026zGKo7ESXP3WyM9SuIg4IoVqw/qnyoaDgyJgJ81SgJZvrI7kLNmxYHpAbPLzjMijIFZFhtUc8Pg2uB9aX7tWFrnTtFaLOTw3EJrjQIN1BcqHbd5E4TqCz8phlRwKAjQhQ0FD5bVgYuIBipd510eYRMP+ngUyrX7vDPXBgNMdBqU3tFffPWTmQrvj9Txl+ZOiptqpuIU9IQY0OxBlA71nzRy4rrxggq0/wmMJoLpXdQ8jrgs7anevAO+fzttu3Z9u25NaxhbXs56HE4 OMd/5hgl b7Di1UML2sUcb/rxWYZi6vQMmCgATDlbnNH7dbDEKMMmfuKM5aAhFyJ1ah59no9GzGUzQUNqrwZid5nh2Zerl4hwJET9Dz4qmFxwTXzw6vWUPAWb5FRBrklBzVhPZaC596rbM779AfS12pst0nQ/U6WFaIaLfnGfjNUur/Kah8tOu6yP3UhKpIJNrUHdqJzoH+nyg+TgmQfpkjiBeLVUWSvtY4gWzcs9Je+1uGaWl6Eh+VyZHraxGkC7YPrwVlW2+Pv/gUq4ElTepSEU3QTwrq8+t6cBzPpLYdPdL1Wz5k09IAELNneaJUd3GA8H83La/O3ThZgG73fCCIjyICGhBoWb9Iy5K1fpsHdFSxhoWHoajICkl1UcoD0y3HviX8A3KK1rtYBj5DCMrdYdyntzY0ANVZgxnVmV0QH5MUcIprDJZdR+W5cOMn1DP10YmX4Phlyxn X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Kirill, thanks for taking a look so quickly. On Wed, Apr 19, 2023 at 02:54:02AM +0300, Kirill A. Shutemov wrote: > On Tue, Apr 18, 2023 at 03:12:47PM -0400, Johannes Weiner wrote: > > This series proposes to make THP allocations reliable by enforcing > > pageblock hygiene, and aligning the allocator, reclaim and compaction > > on the pageblock as the base unit for managing free memory. All orders > > up to and including the pageblock are made first-class requests that > > (outside of OOM situations) are expected to succeed without > > exceptional investment by the allocating thread. > > > > A neutral pageblock type is introduced, MIGRATE_FREE. The first > > allocation to be placed into such a block claims it exclusively for > > the allocation's migratetype. Fallbacks from a different type are no > > longer allowed, and the block is "kept open" for more allocations of > > the same type to ensure tight grouping. A pageblock becomes neutral > > again only once all its pages have been freed. > > Sounds like this will cause earlier OOM, no? > > I guess with 2M pageblock on 64G server it shouldn't matter much. But how > about smaller machines? Yes, it's a tradeoff. It's not really possible to reduce external fragmentation and increase contiguity, without also increasing the risk of internal fragmentation to some extent. The tradeoff is slighly less but overall faster memory. A 2M block size *seems* reasonable for most current setups. It's actually still somewhat on the lower side, if you consider that we had 4k blocks when memory was a few megabytes. (4k pages for 4M RAM is the same ratio as 2M pages for 2G RAM. My phone has 8G and my desktop 32G. 64G is unusually small for a datacenter server.) I wouldn't be opposed to sticking this behind a separate config option if there are setups that WOULD want to keep the current best-effort compaction without the block hygiene. But obviously, from a maintenance POV life would be much easier if we didn't have to. FWIF, I have been doing tests in an environment constrained to 2G and haven't had any issues with premature OOMs. But I'm happy to test other situations and workloads that might be of interest to people. > > Reclaim and compaction are changed from partial block reclaim to > > producing whole neutral page blocks. > > How does it affect allocation latencies? I see direct compact stall grew > substantially. Hm? Good question. There are 260 more compact stalls but also 1,734 more successful THP allocations. And 1,433 fewer allocation stalls. There seems to be much less direct work performed per successful allocation. But of course, that's not the whole story. Let me trace the actual latencies. Thanks for your thoughts! Johannes