From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD112EB8FA5 for ; Wed, 6 Sep 2023 11:26:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 22596280005; Wed, 6 Sep 2023 07:26:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1D5738E0014; Wed, 6 Sep 2023 07:26:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 09E22280005; Wed, 6 Sep 2023 07:26:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id ECB858E0014 for ; Wed, 6 Sep 2023 07:26:17 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id BD368140D10 for ; Wed, 6 Sep 2023 11:26:17 +0000 (UTC) X-FDA: 81205943994.20.2FB5DD8 Received: from mail-wm1-f51.google.com (mail-wm1-f51.google.com [209.85.128.51]) by imf28.hostedemail.com (Postfix) with ESMTP id 7061CC0029 for ; Wed, 6 Sep 2023 11:26:15 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Fp3oTigS; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf28.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.128.51 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693999576; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=GSQpDONv/OZHUkjn9Ge1fW9m2KPxZyl+hgiPVQzyIw4=; b=feydzmcQCkOwVU1L7Kw47XTCsGLsHdsPpKGBDBDdnlhPJZeI2E9QCgGVXLmxr/o0qGh+fr 9JzF2jb59aGIChYk3CQ2+gKyfkeHiTGWSKYPtIG1ecXy5etYuy1wb36ZA8rqBGp6/2Ci1H +520RNfgUEdFYWK6LLVDsee0wAkEuJQ= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Fp3oTigS; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf28.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.128.51 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1693999576; a=rsa-sha256; cv=none; b=CUQ31iGQsOcvXREJ1HXERYkwpC6s+N4mnK3Y1KY8dMOkll2OxXRJuABX5Bxcdg+9n2XnkS jYxb/nDd7/Gh3M+G7no9nRxOs9t2eoHTSjdcKr2CU0wf5GX+gBzeTWcXafVhlWfBkdjmb5 npIJwY0u8pU5OE0M4l0vgvowxWGS83I= Received: by mail-wm1-f51.google.com with SMTP id 5b1f17b1804b1-401da71b7faso35225985e9.2 for ; Wed, 06 Sep 2023 04:26:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1693999573; x=1694604373; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=GSQpDONv/OZHUkjn9Ge1fW9m2KPxZyl+hgiPVQzyIw4=; b=Fp3oTigSVvS82+uzS0AuN1TRVQo+xzmtNEHUszof03vhgn3zA0iXvCc4fUF+QuhVlA Ye6fSH0UKIai0qzJpXyhALT/RZhBy2zGfL5LLLBBL0hYA+XxX36dY2u6cIQRFNfRJLNJ HiiOWBwBQcjuqvmP10cIs1vTGokMYXbhTFEex5d4DHU+BDBItq1G/QxHh2SruL4TEq+l KfFuwWStifYMDMgM8jAhA4FN3AAtlOsEzgWlVQYOqEecEOyfrYUffdMjg3JYzqCmUzGc CQh+1F2poG1Y6F0zGN3iNVNkfxxHX5HTwtZWyZoSmrqRszeFS8uTGWP4o8k2DWqcFs1V gFgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693999573; x=1694604373; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=GSQpDONv/OZHUkjn9Ge1fW9m2KPxZyl+hgiPVQzyIw4=; b=MnCiUmaT2jUKTCPGR+pcSMi0Nknb1Og/h6wPFEx1ClBYqUTiOCVDpMErzXzITFBXXT hn5UNqn8RLWXEX1gG/RIcaBx5Qu58LaFpktu6pb05ke7yWezDG+aJn0g/Otlk4x0Ch9O muJlV1RfSdsk9NKXxVhA0b85RF0eOyZEdRJA53G9eieY+9+Yw7Zvid36LzafZZvJu6Mn m7ioRIyLcIU7YW7/StFgAlYTKH1MziBH6zUX3E8HmRPC6ZLS3izk6bPcdkH0C0uVvLDf NX5SkBESaQcj85/gs2wIWEx+AaCpVgL8cnbEjV7s34OjtNIEIFp3eBQMs6vBwA2e6qhe J8VA== X-Gm-Message-State: AOJu0Yx3BPX5muF6Xs8b7hs/VLSuWvyZmk8Kwq5VT8Osk2MfyQzDnv/6 WNr6UWRi7ziim4TedKuFwv7o6w99DSzgHbeZb2A= X-Google-Smtp-Source: AGHT+IHmK/c6SqzH+hydtTNXTTg8Wlugag5Vv0VewURFx+im28phCmq40JCuwfBwAphdZILfVzaHIg== X-Received: by 2002:a1c:6a16:0:b0:401:c8b9:4b8a with SMTP id f22-20020a1c6a16000000b00401c8b94b8amr2092803wmc.29.1693999573276; Wed, 06 Sep 2023 04:26:13 -0700 (PDT) Received: from localhost.localdomain ([2a02:6b6a:b5c7:0:6dce:31d8:efb1:5d81]) by smtp.gmail.com with ESMTPSA id n9-20020a05600c294900b003feff926fc5sm19517038wmd.17.2023.09.06.04.26.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Sep 2023 04:26:12 -0700 (PDT) From: Usama Arif To: linux-mm@kvack.org, muchun.song@linux.dev, mike.kravetz@oracle.com, rppt@kernel.org Cc: linux-kernel@vger.kernel.org, songmuchun@bytedance.com, fam.zheng@bytedance.com, liangma@liangbit.com, punit.agrawal@bytedance.com, Usama Arif Subject: [v4 0/4] mm: hugetlb: Skip initialization of gigantic tail struct pages if freed by HVO Date: Wed, 6 Sep 2023 12:26:01 +0100 Message-Id: <20230906112605.2286994-1-usama.arif@bytedance.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 7061CC0029 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 8cb8mqp7f8cpwn64wpsdqmrj3m6m9oer X-HE-Tag: 1693999575-623400 X-HE-Meta: U2FsdGVkX18NMasJTJ0HPyxe3xD7gYtIYJK7RQEU5VskwUuMZIZ1cYEkRrW6ihHdVMtOlVWKohpx3DglqgRI/uoflGpL/xKSzsub+bpyjNylWlZisZX8Ms3eD95VypNN4UeERXV1/ubweCV6c7eRStJvOPAWBH008n2Y1/hj4buSkEeTxmN4mFRRl4uNaNMeSHS4dQD395MAoqWmZnNLBZ+02jzrkAJsk641bxEO/zwKKuIDdERe5ndCNZG4WHiUYwA3uiDkkNwOXzqTq+GKS2u3n+49ORT9ip+hs6mau8P9bz0uRBgRDMrQufP9oU4sgalTfxtEnb8yrgAjjfGPEMQtIWzppuADlXaYapRK1uFuxXeFknp9GL2mPMhjguVvSDFHvE72Q9ziMj48V6AZe1y8uNWzJj1umNsZaeKl+Fank34sZwQH4mgYthOIYHh5TIANGVUuABmce4yqHorW93GnqD4qHQCdEhzk3Mg8B0oyYWcm4GKdSlbVrmZf94mxqaPyyg1arduI+j0CE2gHFFg1054CRPa3U5HdKZapkCKzb3WTK+UVTkR2f+W71vXK2Peg6B8zXyRsWcpOX8HE08CDYcEvUwZAh7UggxCyVo0hz8Zf2QfpoevjySDPZaK23FA0dd6RFYFNpmcCIyDB/lczNjZYUqmR8lFf8JU1JjVKIKT7ngv3AOe/tJ0LH/2/C17E31x/VroZ/wSUH7hGHOO8V/Lxh7FrN/O0e5xHlO+EGktg1Y1B9lJGX6sS21nt15nSj+vmD+qBVs9h1iOnnYxY/hJ8yVfCnf8b5CdXGD5PvX7qbCqRgt7L45n4socbBnr8nTSMMxvf1vcS+QCBwds7AxI4WzU4VrskvT53vjX/7+VDe1cNlV/+iQhsw+UiQQ7EIRcghVUGHjLP5PcNtcei9kHFkEKDS3sePN6tFQuyH/zt6tOSZfdrBw+qkKDRQk3lj3rH4RALtvRJNYT 2fyDQ923 CBq2laXziwOAyLQQInKsqEK7nsu/UjKZ/lUzD7sSWhcQn93rAarxkXvPl2s3XHqB6SrfKRUWO++upIVcj3Xucb9ERYV65u6eAvAcATe6sVA7pj+MZ87SxhhOOeU3PIqGgSMCbu+CJf8pnwiyjz9hXVCSCEePR5F+malLyd9/9+IC7SAx2PTQgZv941FLY4l+r61lyPc8OexqrAzteJvpzqbdgZpu+iqGpb6Jp+6a17wxcDRvYQfugarKpgdythPTJUihBctv7oJiqKE4mXy8D3L8LVyH+IdCe7XzUluhd5hiQSk5SS8BIkMpv7DoyJHibZOW6IImNCcVghcAqFs58uHMkYQGQU9Rj3wBwDCkXm4C8oXY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This series moves the boot time initialization of tail struct pages of a gigantic page to later on in the boot. Only the HUGETLB_VMEMMAP_RESERVE_SIZE / sizeof(struct page) - 1 tail struct pages are initialized at the start. If HVO is successful, then no more tail struct pages need to be initialized. For a 1G hugepage, this series avoid initialization of 262144 - 63 = 262081 struct pages per hugepage. When tested on a 512G system (which can allocate max 500 1G hugepages), the kexec-boot time with HVO and DEFERRED_STRUCT_PAGE_INIT enabled without this patchseries to running init is 3.9 seconds. With this patch it is 1.2 seconds. This represents an approximately 70% reduction in boot time and will significantly reduce server downtime when using a large number of gigantic pages. Thanks, Usama [v3->v4]: - rebase ontop of patch "hugetlb: set hugetlb page flag before optimizing vmemmap". - freeze head struct page ref count. - Change order of operations to initialize head struct page -> initialize the necessary tail struct pages -> attempt HVO -> initialize the rest of the tail struct pages if HVO fails. - (Mike Rapoport and Muchun Song) remove "_vmemmap" suffix from memblock reserve noinit flags anf functions. [v2->v3]: - (Muchun Song) skip prep of struct pages backing gigantic hugepages at boot time only. - (Muchun Song) move initialization of tail struct pages to after HVO is attempted. [v1->v2]: - (Mike Rapoport) Code quality improvements (function names, arguments, comments). [RFC->v1]: - (Mike Rapoport) Change from passing hugepage_size in memblock_alloc_try_nid_raw for skipping struct page initialization to using MEMBLOCK_RSRV_NOINIT flag Usama Arif (4): mm: hugetlb_vmemmap: Use nid of the head page to reallocate it memblock: pass memblock_type to memblock_setclr_flag memblock: introduce MEMBLOCK_RSRV_NOINIT flag mm: hugetlb: Skip initialization of gigantic tail struct pages if freed by HVO include/linux/memblock.h | 9 ++++++ mm/hugetlb.c | 61 ++++++++++++++++++++++++++++++++++------ mm/hugetlb_vmemmap.c | 4 +-- mm/hugetlb_vmemmap.h | 9 +++--- mm/internal.h | 3 ++ mm/memblock.c | 48 ++++++++++++++++++++++--------- mm/mm_init.c | 2 +- 7 files changed, 107 insertions(+), 29 deletions(-) -- 2.25.1