From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDFB2C7EE2C for ; Fri, 25 Aug 2023 11:18:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3FF6B28008A; Fri, 25 Aug 2023 07:18:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3AC7D8E0011; Fri, 25 Aug 2023 07:18:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2741628005F; Fri, 25 Aug 2023 07:18:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 152028E0011 for ; Fri, 25 Aug 2023 07:18:53 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id D70D28062E for ; Fri, 25 Aug 2023 11:18:52 +0000 (UTC) X-FDA: 81162379704.13.8DED678 Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) by imf12.hostedemail.com (Postfix) with ESMTP id 5434C40008 for ; Fri, 25 Aug 2023 11:18:50 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Xjcfvt+p; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf12.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.221.42 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692962331; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=udAIvwx6neT1efGb/oM3DTSLzJIa/JJt0wDiTHZ/T+g=; b=wEqpygDRwe/YqGuIZ8o7/1lyT3onwdEWLaP7E7kTtAOxDOHwSYMjPEZj+tPyW6qisORuGG cEfw0Z7nc8wmu+doyGseTI9kwi6HXhjHsRhtyVhl5UgqiRkRtvXAunpuTntL9TpyEcbf7f 8MTir5yRKdZ00ze7Bcdam35sXecYIKs= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Xjcfvt+p; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf12.hostedemail.com: domain of usama.arif@bytedance.com designates 209.85.221.42 as permitted sender) smtp.mailfrom=usama.arif@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692962331; a=rsa-sha256; cv=none; b=Q8rdZ4zzQtVbQyOd1MKFqPtUtzsIKOn9/Jk6Ne0g+jnuS8etzAmMCcwVhvc8DXqZ7qibgY v/jENkIJX8Ip5rkiZOzMGCxNrio2vuPpH4lBKqbU9MZgXbabrJTKujpRKktTQgaILGhqq/ vdBae3LlR5JSbsJ3131ybDXNKXYdBsI= Received: by mail-wr1-f42.google.com with SMTP id ffacd0b85a97d-31c5cac3ae2so617999f8f.3 for ; Fri, 25 Aug 2023 04:18:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1692962328; x=1693567128; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=udAIvwx6neT1efGb/oM3DTSLzJIa/JJt0wDiTHZ/T+g=; b=Xjcfvt+pNpSD5aqiZlmN3nzUFcFvOxgryKI4rPyfk4iPzOPmROvK2JzTCa+4qq9thA iEqhHND7JXyd/QPB7n6Bf321bxRlgapCiJ/s/A68K6t6Ig+G5BYOgBTPEk/Hh783eUrF fHJgcdstjelwiZ55EM22t2pK7QiX9OsI01X6lyHHpgj3mN1qGBnZ84UsPAMeL7xSOod1 B6nI39otAs49zFtnRPBKTUtLArezZD2Q4YYiMAzcgkAWO6rnEi40gLXN/P00lpts6XhA D2QmjzN3pniBCtOt23pHbzxhsWOz5eesOKx/nMzgI9u0wYuss8iZxpOBkasS0p3mMKmP qfug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692962328; x=1693567128; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=udAIvwx6neT1efGb/oM3DTSLzJIa/JJt0wDiTHZ/T+g=; b=jsnxlAIrOB+R/xNe8j9eGKv0Bbd9MmhaEdFhO3878FN5ZFHnNan1lVanUySWDMCd+V Te75/pYUEF3sJtexfu4co8wejyp4zddolRVj7bcEyxrMaicUxDwedmvTE64uIdk7jX2w OvSkgFJ9FTiRjosPCZ/5Onl/hVKWf776g2qtz9uiZggNTh9vJqBADhva9FJgO/d63tW/ OpuYpMQsqa54EJdk6dLYUcFH9Yb4rwKk1wkrD6MZOfm5wWYd/tJlJx4RMSbKaeBFYMWe OwQb/Sp8n8a1jyzrkSisa42xdnYE1VgNvjnTXegIUV0BTmvViNpCaoSuzN/0qKTJI3nC DoMw== X-Gm-Message-State: AOJu0YzafNxVK1hwEbIz+6wfATc/Sg3hlrv352hFOsslsEZFqdR/MCx/ bgh2ep1Dexr8vgqz1ZtJI/eps+jW/JZvkJ//oKs= X-Google-Smtp-Source: AGHT+IFbzBQK2y0ZFfjnyaMV0QpwjJwtZGusGMD0WFsVP9UJ1VpMAySZOL7UT6fA9D9lUh9r1W6yTA== X-Received: by 2002:adf:de03:0:b0:313:f1c8:a968 with SMTP id b3-20020adfde03000000b00313f1c8a968mr15529115wrm.2.1692962328191; Fri, 25 Aug 2023 04:18:48 -0700 (PDT) Received: from localhost.localdomain ([2a02:6b6a:b5c7:0:9000:8d13:d0bb:ba7d]) by smtp.gmail.com with ESMTPSA id g9-20020a056000118900b0031ad5fb5a0fsm1939502wrx.58.2023.08.25.04.18.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Aug 2023 04:18:47 -0700 (PDT) From: Usama Arif To: linux-mm@kvack.org, muchun.song@linux.dev, mike.kravetz@oracle.com, rppt@kernel.org Cc: linux-kernel@vger.kernel.org, songmuchun@bytedance.com, fam.zheng@bytedance.com, liangma@liangbit.com, punit.agrawal@bytedance.com, Usama Arif Subject: [v3 0/4] mm: hugetlb: Skip initialization of gigantic tail struct pages if freed by HVO Date: Fri, 25 Aug 2023 12:18:32 +0100 Message-Id: <20230825111836.1715308-1-usama.arif@bytedance.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 5434C40008 X-Stat-Signature: mowtfie3fuj9gmmsxhg16omd8s4xtg6n X-HE-Tag: 1692962330-899710 X-HE-Meta: U2FsdGVkX190jrKzR3q+oon7QTGvBRGgCiVa6os8FiKFbVjHV5aXHd6/QCmsCuOW6GuEPRGtpw+C5LkEMHQ2RBtWvMxxtmvRsWsnCaX2oJWosyb+7ym7jxeF01Meq0KHMIm7Vo9US4aAHBrcXR+t9JfCR1x1zRb5WQO1V7p4P6kMmIjlCH0QEeLNtCDq05ruuJyJt5Jch3p3M6i5ldTdO0L/YXGuVBDKhdn0VoIJUmo7noVj3NFNIzpDmh6lMdW1D4jMF4iqFoAGkuEbLM/OuQxdoG7phnCR03d9ezjNf8v17c3/qCBHRzwbZF6NlMfAJSWhmUkik8UHZy/pONjuV2DkNY2og0fGH8nWrodkBD/13YmNi3ZNwQTMRoWfurzu1PJgbxVlprRhPyt/b9w5kVpWGolDo4D5W/IlECDmAiNKxGZmN3H2FqhFtcW3D21v1CLcpqgSbhpeFgBSzM9HGbDbWnb7ZIPMVFlmrhVC/0MswsWP2CyN42/6XBrHnmjuoIriN5PtVhDDmgQE8MCwwo7qYAK5QscPMXBVZs3QaBzyiqx1d4BzmH7vZGtkjnN2FXs7zCTnQuPLNNNaKaJndPIzO+MPJOEwmEeeRCp7eIsyavlJZ3RZ0oUReLGYr3BABjK/zwlASTzB/MbLtInSfnXr2eoqGg3u+NV6M8J1tYB2jzWbw+bpC2yynsAR4mfTZpvD932KzunCe5lCkE0SGtwGoKkv6WAzRexyam6gLz66b8LZl1zHA1goXT09FkRrWMEZUILJtv7pgO7aCeVWrKPLR3xw6YfQ/SdFek0eVUXq0KacxR2xwiOzwZ84NOiOCL2MLyWmc/+WippOzTNr1I75f/BL0xQcisguEPm0of8IRc6PtfgX4bRXIPdHYHVR4E282QiItJPG3QdWgccSM7xSBCbNJQd+fbSKlKGucVeRtPbpCQFKC/hbdc79c65CutH+jwqQAA3tENBySja IuyiWFpR 2orjHjNCqNprxWdY3y6kgmzrKRPp8YPcF3hmH5i04v/ED4CYwqJ8gSYDUFOn5jPiwoYcOnGqAKhYPILQn12IwER8mLgLYlhylbns949HB+MQbCIw12YRAuKRvWMa7URmdvPoS64xb7nkrvivZGdGfQc4EDMVopXviPyfQc2/y2ZfeB23m3HSnY6CFbgAgI7uiMQKLcuG/f8uBnyIltdACTtIYu6NawQNRxYzW7Q9FF1egwUzcNAEBbqzqttZetVDkUXOhgKtV3u3nlvw+TCtIkDvys3dOPrKtAFGnQGx+CgnxL1XbltnFKNm4Hi50fiuTijevkfRZA+OlJgGhD9ZE01E6jQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This series moves the boot time initialization of tail struct pages of a gigantic page to after HVO is attempted. If HVO is successful, only HUGETLB_VMEMMAP_RESERVE_SIZE / sizeof(struct page) struct pages need to be initialized. For a 1G hugepage, this series avoid initialization of 262144 - 63 = 262081 struct pages per hugepage. When tested on a 512G system (which can allocate max 500 1G hugepages), the kexec-boot time with HVO and DEFERRED_STRUCT_PAGE_INIT enabled without this patchseries to running init is 3.9 seconds. With this patch it is 1.2 seconds. This represents an approximately 70% reduction in boot time and will significantly reduce server downtime when using a large number of gigantic pages. Thanks, Usama [v2->v3]: - (Muchun Song) skip prep of struct pages backing gigantic hugepages at boot time only. - (Muchun Song) move initialization of tail struct pages to after HVO is attempted. This also means that the hugetlb command line parsing does not need to be changed. [v1->v2]: - (Mike Rapoport) Code quality improvements (function names, arguments, comments). [RFC->v1]: - (Mike Rapoport) Change from passing hugepage_size in memblock_alloc_try_nid_raw for skipping struct page initialization to using MEMBLOCK_RSRV_NOINIT flag Usama Arif (4): mm: hugetlb_vmemmap: Use nid of the head page to reallocate it memblock: pass memblock_type to memblock_setclr_flag memblock: introduce MEMBLOCK_RSRV_NOINIT_VMEMMAP flag mm: hugetlb: Skip initialization of gigantic tail struct pages if freed by HVO include/linux/memblock.h | 10 ++++++++ mm/hugetlb.c | 52 ++++++++++++++++++++++++++++++++-------- mm/hugetlb_vmemmap.c | 2 +- mm/hugetlb_vmemmap.h | 8 +++---- mm/internal.h | 3 +++ mm/memblock.c | 47 ++++++++++++++++++++++++++---------- mm/mm_init.c | 2 +- 7 files changed, 95 insertions(+), 29 deletions(-) -- 2.25.1