From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0A51C54EE9 for ; Thu, 22 Sep 2022 23:14:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 09261940008; Thu, 22 Sep 2022 19:14:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 04167940007; Thu, 22 Sep 2022 19:14:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E7247940008; Thu, 22 Sep 2022 19:14:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id D8E6E940007 for ; Thu, 22 Sep 2022 19:14:31 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 924DA406DF for ; Thu, 22 Sep 2022 23:14:31 +0000 (UTC) X-FDA: 79941277542.27.BE9A7B1 Received: from mail-qt1-f170.google.com (mail-qt1-f170.google.com [209.85.160.170]) by imf08.hostedemail.com (Postfix) with ESMTP id 445D3160012 for ; Thu, 22 Sep 2022 23:14:31 +0000 (UTC) Received: by mail-qt1-f170.google.com with SMTP id ay9so7463671qtb.0 for ; Thu, 22 Sep 2022 16:14:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date; bh=mgFYnY1RdRGyspxYgyA20AVsPMGDCNSztuoVIa2jt1Y=; b=Q2he6+yxlkaDq4CUH97E1DzltZLwL3xOzRh/bSc6Ji13ZjmqEQaAcvHP/OvwuriJpc MdnFXLVdHWMiHMBbJbd1GcXCjx4mJ+EZ3fUYCifjZr53/irUafcEQQXeqxdGfC7D93py 8j3AFtKjVhjdtaKFrrD6RgFLs9mI/KNyodxlApcyNkze+9Ug6yHT8X4Iw7bXnXxpLfuM 9SDiyPRx2EL4c+w397FmAjWCcBIAwIteiZL/CiQCWhgbOh5dK9qYUzohtocos9q3i8ib HrZyiAASO9QKecD5xLxxNAQx+sbWEo5pZIRJZxhvd74IVb8XI2sEk7b4Fd/g8wBPeqUI xPnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date; bh=mgFYnY1RdRGyspxYgyA20AVsPMGDCNSztuoVIa2jt1Y=; b=RPaI4X27UsFX5BWI/uqt8Sbd1Zb/hjrkrvv5iYfHU+qo1S6IlyuZhBJTnnxqw3VB/V sTqHKcS5voKCxr+CrGZIy9RG39IW9kCNBJeac0RR37MSLWIu7D+4de5pVqVh9vFHOLGE +pC7RUcDGFvzH09ktfVhKdodvoZy4mQuD2tabx67JNDiLPOOT8r0RxxBUBu68KBPSRNP Vt2Ftq0zE4mJ6kmacyEd7WHTqHU5tNnUi+ANbDQ+y19W5rnngmhT/rG5dE29TWjvu5/l BkLPAGRv/0Lp7o6MJS/brDysPfizcqHtDPBmQ0L50npTry3HNdhPrBMYON+/AhfGst7a oDAw== X-Gm-Message-State: ACrzQf1QjMNmPb1fec2eJfUuAqJg16yfckAFiFpFfZSASvynZtvFVg8s Xzt3f4S1cwQp0eFGndl2Nds= X-Google-Smtp-Source: AMsMyM5Q8Xv8RIOWfBcgbIGfXDkaa9DDYJtjJIhk4bs3mCpL99XI74671Nl33Q61ZaAkXP2xIpjwHw== X-Received: by 2002:a05:622a:13cf:b0:35b:bb29:c1e6 with SMTP id p15-20020a05622a13cf00b0035bbb29c1e6mr5059822qtk.687.1663888470462; Thu, 22 Sep 2022 16:14:30 -0700 (PDT) Received: from [10.69.40.226] ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id d8-20020ac85348000000b003445b83de67sm4219209qto.3.2022.09.22.16.14.28 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 22 Sep 2022 16:14:29 -0700 (PDT) Message-ID: Date: Thu, 22 Sep 2022 16:14:27 -0700 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.1.2 Subject: Re: [PATCH 0/3] mm/hugetlb: hugepage migration enhancements Content-Language: en-US To: Mike Kravetz Cc: Andrew Morton , Muchun Song , Oscar Salvador , Michal Hocko , David Hildenbrand , Florian Fainelli , linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20220921223639.1152392-1-opendmb@gmail.com> From: Doug Berger In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663888471; a=rsa-sha256; cv=none; b=VINpMiJCQqK5XW97s1CWfxR6am54AKgMpkEk33UEZZ6lpB5y+KHmSBJWeJSXhnpq22Iwke vlQ3MYnQvV3Yc0YqpoesPW5VTB1wCm9ADzUWF88wWyFQir9SM9y7XDSR+q9ovqTE9qNP2w gE33y8BUgENapnydV4oqHp4af7urUlo= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Q2he6+yx; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf08.hostedemail.com: domain of opendmb@gmail.com designates 209.85.160.170 as permitted sender) smtp.mailfrom=opendmb@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663888471; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mgFYnY1RdRGyspxYgyA20AVsPMGDCNSztuoVIa2jt1Y=; b=7Jk1fMwcIpxPF0T/BRpbvJ4784cndgjTHZBB4BrDpcsz2/Kwx/b1+W0+eQyAJDjZAz8lK1 /f/NJkkGYK42vXmKSQ13ZamKech4qpJ7pxjRyyIFt9yfxlx9MLgFcGwDucBRVYfqdZHwNM 0mctKaZ9LX/m3utnpl/3NANDn6cLFA8= X-Rspamd-Server: rspam04 X-Rspam-User: X-Rspamd-Queue-Id: 445D3160012 Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Q2he6+yx; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf08.hostedemail.com: domain of opendmb@gmail.com designates 209.85.160.170 as permitted sender) smtp.mailfrom=opendmb@gmail.com X-Stat-Signature: 1ttpbnjy6t6oq87sut7awecuoj97y6b9 X-HE-Tag: 1663888471-163879 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 9/22/2022 1:25 PM, Mike Kravetz wrote: > On 09/21/22 15:36, Doug Berger wrote: >> This patch set was included as patches [04/21-06/21] in my >> previous patch set to introduce Designated Movable Blocks [1]. >> They were included there for context, but they have value on >> their own and are being resubmitted here for consideration on >> their own merits. >> >> The alloc_contig_range() function attempts to isolate the free >> pages within the range and migrate the data from non-free pages >> within the range to allow those pages to become isolated free >> pages. If all of the pages in the range are on isolated free >> page lists they can be allocated to the caller. >> >> When free hugepages are encountered in the range an attempt is >> made to allocate a new compound page to become a replacement >> hugepage and to dissolve the free hugepage so that its pages >> within isolated pageblocks can be added to the isolated free >> page lists. Hugepages that are not free and encountered within >> the range must be migrated out of the range and dissolved to >> allow the underlying pages to be added to the isolated free >> page lists. >> >> Moving the data from hugepages within the range and freeing the >> hugepages is not sufficient since the allocation of migration >> target hugepages is allowed to come from free hugepages that may >> contain isolated pageblocks and freed hugepages will not be on >> isolated free page lists so the alloc_contig_range() will fail. > > Thanks for looking into this! I am adding Oscar, Michal and David on > Cc: as they have looked at similar issues in the past. Thanks for helping to get the right eyeballs looking at this. > > Before jumping into the details of your changes, I just want to make one > observation. When migrating (or dissolving) hugetlb pages that are in a > range operated on by alloc_contig_range(), we should not change the number > of hugetlb pages available as noted here. This includes the number of > total huge pages and the number of huge pages on the node. Therefore, > we should allocate another huge page from buddy either as the migration > target or to take the place of the dissolved page. > > For free huge pages, we do this via alloc_and_dissolve_huge_page. IIUC, > there are no issues with this code path? Yes, I did not observe any issue with the the free hugepage handling except that your recent improvements with freezing allocated pages (thanks for those) will make merging patch 1 more difficult ;). > > As noted above, for pages to be migrated we first try to use an existing > free huge page as the target. Quite some time ago, Michal added code to > allocate a new page from buddy as the target if no free huge pages were > available. This change also included a special flag to dissolve the > source huge page when it is freed. It seems like this is the exact > behavior we want here? I wonder if it might be easier just to use this > existing code? Yes, the temporary page flag can be made to work here and I did experiment with it, but it is dependent on the current design decisions. I decided to move to the approach suggested here because I believe it could conceivably scale if support for migrating gigantic pages is desired in the future. The circular dependency on alloc_contig_range will likely keep that from happening, but that was my thinking. Thanks for taking the time, -Doug