From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4BD2C433E0 for ; Tue, 12 Jan 2021 20:12:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 512D22311B for ; Tue, 12 Jan 2021 20:12:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 512D22311B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 996246B00C3; Tue, 12 Jan 2021 15:12:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 947016B00C6; Tue, 12 Jan 2021 15:12:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 80F9A6B00C7; Tue, 12 Jan 2021 15:12:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0001.hostedemail.com [216.40.44.1]) by kanga.kvack.org (Postfix) with ESMTP id 688506B00C3 for ; Tue, 12 Jan 2021 15:12:22 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 22F07180AD815 for ; Tue, 12 Jan 2021 20:12:22 +0000 (UTC) X-FDA: 77698220124.11.sun59_120019427518 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin11.hostedemail.com (Postfix) with ESMTP id D2A35180F8B87 for ; Tue, 12 Jan 2021 20:12:21 +0000 (UTC) X-HE-Tag: sun59_120019427518 X-Filterd-Recvd-Size: 7174 Received: from aserp2120.oracle.com (aserp2120.oracle.com [141.146.126.78]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Tue, 12 Jan 2021 20:12:21 +0000 (UTC) Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10CK9oEL032853; Tue, 12 Jan 2021 20:12:15 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc : references : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=corp-2020-01-29; bh=laFX/HAfU+qzq2ofvncNg04yx1ExXeNRXIbgc+vDoeQ=; b=ErpmWMRe1ncknNKD84nF8AtT1rjqLV6bvWfOH+jjM28E/p8kk2S1ep6Z1jQvt3Olsb4k YJjLIiUDEdVGjFxXKM9J1HmDyL8BZEHveKDyI9mYyfhXDsHaHjsMv1pfvLI0/ycOkhFE F74o9uzorMgIZS6n9BJ4Qbaq6bk64kMgwABQeKW59XdrxuKJ/6yMyJABe+DExuBpPhrc ZRFWttMnpk2OjyH4UwmdhkBFdr2qjUgAdJeDt2Pyu7WAuHixvHd7IObFvCoq/xw4oqEc fVOg7czlh+FXv2l0tghyUWcQRCDre80ZKpU92ETQC0hwUZQsS1iLeVfJQbt6eZwBkgfj Cw== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by aserp2120.oracle.com with ESMTP id 360kcyr9p3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Jan 2021 20:12:14 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10CKA4F8102785; Tue, 12 Jan 2021 20:12:14 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserp3030.oracle.com with ESMTP id 360keybfcy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Jan 2021 20:12:14 +0000 Received: from abhmp0004.oracle.com (abhmp0004.oracle.com [141.146.116.10]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 10CKCA6P026278; Tue, 12 Jan 2021 20:12:11 GMT Received: from [192.168.2.112] (/50.38.35.18) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 12 Jan 2021 12:12:10 -0800 Subject: Re: [PATCH v3 1/6] mm: migrate: do not migrate HugeTLB page whose refcount is one To: Michal Hocko , David Hildenbrand Cc: Muchun Song , akpm@linux-foundation.org, n-horiguchi@ah.jp.nec.com, ak@linux.intel.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yang Shi References: <20210110124017.86750-1-songmuchun@bytedance.com> <20210110124017.86750-2-songmuchun@bytedance.com> <1b39d654-0b8c-de3a-55d1-6ab8c2b2e0ba@redhat.com> <20210112112709.GO22493@dhcp22.suse.cz> <20210112121643.GP22493@dhcp22.suse.cz> <20210112142337.GR22493@dhcp22.suse.cz> <52ec4899-80df-4cbe-41f1-e0a29e838afa@redhat.com> <20210112145325.GS22493@dhcp22.suse.cz> From: Mike Kravetz Message-ID: <70b5cf44-9f8c-2149-0920-823045be3fe1@oracle.com> Date: Tue, 12 Jan 2021 12:12:09 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.1.1 MIME-Version: 1.0 In-Reply-To: <20210112145325.GS22493@dhcp22.suse.cz> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9862 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 malwarescore=0 suspectscore=0 adultscore=0 spamscore=0 mlxlogscore=999 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101120119 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9862 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 phishscore=0 impostorscore=0 bulkscore=0 adultscore=0 suspectscore=0 malwarescore=0 lowpriorityscore=0 clxscore=1011 mlxlogscore=999 mlxscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101120119 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 1/12/21 6:53 AM, Michal Hocko wrote: > On Tue 12-01-21 15:41:02, David Hildenbrand wrote: >> On 12.01.21 15:23, Michal Hocko wrote: >>> On Tue 12-01-21 13:16:45, Michal Hocko wrote: >>> [...] >>>> Well, currently pool pages are not migrateable but you are right that >>>> this is likely something that we will need to look into in the future >>>> and this optimization would stand in the way. >>> >>> After some more thinking I believe I was wrong in my last statement. >>> This optimization shouldn't have any effect on pages on the pool as >>> those stay at reference count 0 and they cannot be isolated either >>> (clear_page_huge_active before it is enqueued). >>> >>> That being said, the migration code would still have to learn about >>> about this pages but that is out of scope of this discussion. >>> >>> Sorry about the confusion from my side. >>> >> >> At this point I am fairly confused what's working at what's not :D > > heh, tell me something about that. Hugetlb is a maze full of land mines. > >> I think this will require more thought, on how to teach >> alloc_contig_range() (and eventually in some cases offline_pages()?) to >> do the right thing. > > Well, offlining sort of works because it retries both migrates and > dissolves. It can fail with the later due to reservations but that can > be expected. We can still try harder to rellocate/rebalance per numa > pools to keep the reservation but I strongly suspect nobody has noticed > this to be a problem so there we are. Due to my time zone, I get to read all the previous comments before commenting myself. :) To be clear, this patch is handling a very specific case where a hugetlb page was isolated for migration and after being isolated the last reference to the page was dropped. Normally, dropping the last reference would send the page to free_huge_page processing. free_huge_page processing would either dissolve the page to the buddy allocator or more likely place the page on the free list of the pool. However, since isolation also holds a reference to the page, processing is continued down the migration path. Today there is no code to migrate free huge pages in the pool. Only active in use pages are migrated. Without this patch, 'free pages' in the very specific state above would be migrated. But to be clear, that was never the intention of any hugetlb migration code. In that respect, I believe this patch helps the current code work as designed. David brings up the valid point that alloc_contig_range needs to deal with free hugetlb pool pages. However, that is code which needs to be written as it does not exist today. I suspect nobody thought about free hugetlb pages when alloc_contig_range was written. This patch should in no way hinder development of that code. Free huge pages have a ref count of 0, and this code is checking for ref count of 1. That is a long way of saying that I still agree with this patch. The only modification/suggestion would be enhancing the commit message as suggested by Michal. -- Mike Kravetz