From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A0F7EE14D0 for ; Wed, 6 Sep 2023 21:27:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4F7E98E0010; Wed, 6 Sep 2023 17:27:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4A7C38D0005; Wed, 6 Sep 2023 17:27:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 36FB58E0010; Wed, 6 Sep 2023 17:27:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 23E178D0005 for ; Wed, 6 Sep 2023 17:27:35 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id E8A4E1CAC20 for ; Wed, 6 Sep 2023 21:27:34 +0000 (UTC) X-FDA: 81207459228.16.12182E4 Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) by imf08.hostedemail.com (Postfix) with ESMTP id 2326A160006 for ; Wed, 6 Sep 2023 21:27:31 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=i5TgRyBE; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf08.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.128.44 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694035653; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HxYYC1e0cfCBncvIB+q/JicAk64sR5rIAn7aK80eSuk=; b=3amcMku9EQp/ZZ9+H7ITsaKID122mBLPmNnmIz4x2RceTJuSCQ+L72usSbv+acaUQhfUCl 8A0bmk8sbx2oUUHZ/6qc7p1CocvtgqcFzqgtcyqP1X4yZb7thZRxY5GaLnQTYhSXkSorou iSNeBrddGsIvNmoEYVS1tNWDnz1/f/o= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=i5TgRyBE; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf08.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.128.44 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694035653; a=rsa-sha256; cv=none; b=Lr86nVlwfLetiYRxSGrISG8no1d2OWRBqFL8L+hCuY2tf3Lqpu16xp//83g25IfMIIskBO 1QxDQgngquS0Th06+LvjFMtUlcLKUUuNAoOyP6JCQq5AVBwLJINJj55oh3AElLAKjrc+Y+ gDUK1XrDygvJpx67LOlnHe/0De3/TWU= Received: by mail-wm1-f44.google.com with SMTP id 5b1f17b1804b1-401b3ea0656so3148115e9.0 for ; Wed, 06 Sep 2023 14:27:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694035650; x=1694640450; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=HxYYC1e0cfCBncvIB+q/JicAk64sR5rIAn7aK80eSuk=; b=i5TgRyBEFXTzJzhLHxUN25fC81zswGYlkpla+aAi43AkWKSbXG3qA4gMAx3q5G7nmE 9PkbjfleGeX/guF5AbWDYmE+kmxyWqiyqYqPuo1mcPqe/3dgd8TuzjfK6MXfBJRZSKrP F/T0HBVcWjDvRB3pXcSPg16+Gek1zit6v7NRVubEd5EJys8hetkoixuG7l2t+zlfTNSI BBPsMOXhy2rzBsZknmPmdhrJcLPf/wtZmuiknORbusLMsbZ1m143Ftm0mirDRFbJO136 s050pox4AHEByGm3gfS5sKILktpkc8U1cXOqyCmwHYOLxkv9XZxSDmauq88M/t/BIjjC IdjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1694035650; x=1694640450; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=HxYYC1e0cfCBncvIB+q/JicAk64sR5rIAn7aK80eSuk=; b=iplv9Xe1wxMqQhzB/0nsXHXHgWikGz+pZ2GKTt3cyNDF3rsEgZKKcjfnZBtz1iomC2 D5mD9iL27L1acrw2aoaf32reEF4ZHS4U+24FOGK2UPAdDnLlwU67mqQbmACsFIxrB+17 Hnnvr8B8gzUG+wht+jWLf062c25beGRamVDaBmNkiNaBtxsufdMotECAnsgwx1Z4/gu7 xsABnDWZFasUmcvZQrkjjDlJ48plISk3/3RSwyBsRQYp81nO6M4DmOj9n9finV1dPvCu pXwOPgH0uzLtW+KYUgGDL0L8pxmHnhMqEYyZ0WpCD2uHN6YBogPHzM+7eOEL9sNnhKWa uzow== X-Gm-Message-State: AOJu0YzcqPjG00D8oUFderfbqPw/s0d0/lX9VSrRUzaaIaARR/6Py6Dm G/5C8hvJroyERx154yPE2bV49A== X-Google-Smtp-Source: AGHT+IGweuxCL65GjEbvRg7LXcDrDXb/vGQIiScSnLJkVi//NfzygtqJpeTWrOEJlEkDppJMQS6/Eg== X-Received: by 2002:a05:600c:2308:b0:401:d803:6243 with SMTP id 8-20020a05600c230800b00401d8036243mr3284465wmo.32.1694035650376; Wed, 06 Sep 2023 14:27:30 -0700 (PDT) Received: from ?IPV6:2a02:6b6a:b5c7:0:6274:c7ab:26e2:b113? ([2a02:6b6a:b5c7:0:6274:c7ab:26e2:b113]) by smtp.gmail.com with ESMTPSA id k20-20020a7bc414000000b003fd2e898aa3sm995774wmi.0.2023.09.06.14.27.29 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 06 Sep 2023 14:27:29 -0700 (PDT) Message-ID: <57c8dd7f-d1a0-37c4-1d3b-d6374e92ffa1@bytedance.com> Date: Wed, 6 Sep 2023 22:27:28 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: [External] Re: [v4 4/4] mm: hugetlb: Skip initialization of gigantic tail struct pages if freed by HVO Content-Language: en-US To: Mike Kravetz Cc: linux-mm@kvack.org, muchun.song@linux.dev, rppt@kernel.org, linux-kernel@vger.kernel.org, songmuchun@bytedance.com, fam.zheng@bytedance.com, liangma@liangbit.com, punit.agrawal@bytedance.com References: <20230906112605.2286994-1-usama.arif@bytedance.com> <20230906112605.2286994-5-usama.arif@bytedance.com> <20230906181016.GA3612@monkey> From: Usama Arif In-Reply-To: <20230906181016.GA3612@monkey> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 2326A160006 X-Stat-Signature: 63s7zxh1oijzg87qt9chhkazetc6uwqf X-HE-Tag: 1694035651-105782 X-HE-Meta: U2FsdGVkX1/yh9yMW4nZuqyON6YyNRghwmvoxGdRiL5/BD1gFRSUgFJRdIvOQIcuQtIkA98rx2ISQhFX0G3RD3b/olgIpg4c7FCW3eYviOVCSV5TawROYxZCkU00+IouJ0A4vH/VRhB0pnOUNVGT6W7YQAlKFwXJXTpiZsTdkFZ6wnzDNUqiJTOUjji2Vhkso7hEuw4bpYQCu98SlB2H4RYukKOaDF6Mp5PptfBpmBOpYwCuvyOBH4N5gR4jaOCYbjrVzcWKH7saZtU6SvqOPZfLKQVo+QYbBJ5/XnyRDbn5H9HvafmRNjhKCoFOQ3/pWDhLNsF/Rouw5RIl7xN9MHDyWex7neFCc3fQAPCiEqcHicIEoBjySrv27frCYazIKkcuM6N1awJ7tOLtrSAOmrQJE2/3sLRipbHr8ppbmMbTNxwq5dYa67WaXGWxD6OFsXy7O8WkKYtGgZHxCVGzqaWjYBuSa0u6eLPsBZmV3vTvbtZAYeg+mpTvDAoE9ipLoA/5Gl/fCSmEAGkI6j2KkoUUOV1ieNchwktFhI/Z5tRR2KRJIttUyehqeen5CtNxs5v9QdFQr4TV5gWgRn+ZWxpbpdiECFq6FHLD/Qjnk5Da8C82n+/S9vAIiPErfClfCHrPrxMXBzBXcbqdskRRO7NGht4SrKIV7aKoTn+c+xxEuIgtdgNWrx/dH+F9zmt/sKGlJSfQDlYh/MuzkFFA5h3UL1y35TnRcQSF/bZ/kogHsl8iHBVl0IR8oY93O2skwKiPdjPx/EmJWqkPolK01QyBNc6LNJTRxz1BKwfHtGo2re13Yrd0j1lN8uLld5LAkTjsggA+e3yuCT/3vLfNURzyrXrJIw8SIJiH5lalSnvOpIOMRW5k7l6f/bChqUfpnPxwZQsWU15ocCin+5arnMwDzGpxGD+wz3g/zi/lNTNdDYi7VTXxqdTOgI/4WBTMNR7DyxJye7Dr7a4NOmY fF6vtWGl xuUQdcygD2g8Y6spkaFgOG/DWSV/reZCN524E1tRUCR7o/JefP24kb9xCJa3fFg20deWXnBMaFkTq9+95Gn9RJWt1MRU3Or4kKHDawnCvlN6+sTkbWi0pT+/shBhikDwDkYZgnuLbYYjPhOGEIAQeif2YpaE3gIFklbjBnho1W5qwOTPADn7JXO4UQ0kWA7Kf32HmvdyAdXHg6dlfBFqTj1iuO/1+UJaHkp0PrJ1QBukEfkEHPLeuFX1sEpCZUt6beEEhs0xzHhUybTwOxdzJs1Vt73MewRNh9iagMHp9vFFklaQekXR7Tr7sm3hvE+yrzuK3ZPlJHJo38sF6C28XIKQKccKv0Tk6V+5rwCB/33/T0SXEdG+cyGsn3Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000011, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 06/09/2023 19:10, Mike Kravetz wrote: > On 09/06/23 12:26, Usama Arif wrote: >> The new boot flow when it comes to initialization of gigantic pages >> is as follows: >> - At boot time, for a gigantic page during __alloc_bootmem_hugepage, >> the region after the first struct page is marked as noinit. >> - This results in only the first struct page to be >> initialized in reserve_bootmem_region. As the tail struct pages are >> not initialized at this point, there can be a significant saving >> in boot time if HVO succeeds later on. >> - Later on in the boot, the head page is prepped and the first >> HUGETLB_VMEMMAP_RESERVE_SIZE / sizeof(struct page) - 1 tail struct pages >> are initialized. >> - HVO is attempted. If it is not successful, then the rest of the >> tail struct pages are initialized. If it is successful, no more >> tail struct pages need to be initialized saving significant boot time. > > Code looks reasonable. Quick question. > > On systems where HVO is disabled, we will still go through this new boot > flow and init hugetlb tail pages later in boot (gather_bootmem_prealloc). > Correct? > If yes, will there be a noticeable change in performance from the current > flow with HVO disabled? My concern would be allocating a large number of > gigantic pages at boot (TB or more). > Thanks for the review. The patch moves the initialization of struct pages backing hugepage from reserve_bootmem_region to a bit later on in the boot to gather_bootmem_prealloc. When HVO is disabled, there will be no difference in time taken to boot with or without this patch series, as 262144 struct pages per gigantic page (for x86) are still going to be initialized, just in a different place.