From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F1FFC433F5 for ; Thu, 14 Apr 2022 04:04:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 612E96B0071; Thu, 14 Apr 2022 00:04:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 59B976B0073; Thu, 14 Apr 2022 00:04:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 43C286B0074; Thu, 14 Apr 2022 00:04:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0247.hostedemail.com [216.40.44.247]) by kanga.kvack.org (Postfix) with ESMTP id 318B36B0071 for ; Thu, 14 Apr 2022 00:04:39 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id D328218356242 for ; Thu, 14 Apr 2022 04:04:38 +0000 (UTC) X-FDA: 79354143036.26.6A12F96 Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) by imf27.hostedemail.com (Postfix) with ESMTP id 9AEDF40007 for ; Thu, 14 Apr 2022 04:04:37 +0000 (UTC) Received: by mail-pj1-f45.google.com with SMTP id mm4-20020a17090b358400b001cb93d8b137so8180436pjb.2 for ; Wed, 13 Apr 2022 21:04:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=1Yvw2OdMv8Wer1zau2Qqgf9K5ZU91nVYs0auzDDWTAE=; b=bS3J8H3+QDcELENjNFvFu9mMbmF7yl/dm/cHNSZD77rUAfzCe9dg8fvSJjVfy269p/ F4pXukydfJ+IYEmIARGce2SXHwi/Xobw+QcE4aHJrIhLdJY8E8Rztp1YdOdMz+J04beH kPjNyaNw6Rh10lB//HPNtNk6g2BOFgaBO1xUXewq9A3m45cttZbx3vvDqw7U40OvnyOS Ox+suMvZmkJo/95JiXjRp3k1+Sf9J6cKoqVuWoiWR0t9fJdzjz845b531wtTg54Qb7vP dnDh81pBKAw5qMMCcXvvX4KLfYAcZuTRisyE1z7mYpz30pc//2Npj+3wenshfAsYtbsq sLmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=1Yvw2OdMv8Wer1zau2Qqgf9K5ZU91nVYs0auzDDWTAE=; b=6LYsPf7ErE9QKoVtfC7+oLDZraBoRXKVKb740zaqMdp83gXhBO4Or3CpY1leIVO/ji PYGfpK/wTpq8MNJfPPicUTO8MeK4GPNxBXqqs5IW/osqbp0iwujHdd7yO65vegBajwFs 2uFfj340yvy/qNf9xJjucHzhYYcfXYvm7xZl24VuzYla15EFV5qH5yp+3Zqt13yhhsPp d56tydT9QsBJm5ct+IxZPpxG/41LYp16FDOAnvWxx7V0e39QfJgS31OP8nCDFHWFSX3I 9I6i/fzanZIT+lTLEmuMNeoh6cRNnNcENVF9cQKSASFV7Yl+hsLvIU8qEYL1ebUeQFHV emhA== X-Gm-Message-State: AOAM532ieSaxVqZB0JkWBNazyW6DkMg3XHFfDLztqF3InArMnw9n5Q4s tmYfMoXY77OygSJBiYKYbG+QIA== X-Google-Smtp-Source: ABdhPJxGWI1CZqiLhhdt7xudKn9fMZwBuVLqm+dNGrQaijILM1Hww8DNvntXLZstY4WN8dDcRWI99w== X-Received: by 2002:a17:903:284:b0:158:8e21:2108 with SMTP id j4-20020a170903028400b001588e212108mr9878098plr.37.1649909076271; Wed, 13 Apr 2022 21:04:36 -0700 (PDT) Received: from localhost ([139.177.225.245]) by smtp.gmail.com with ESMTPSA id w14-20020a63474e000000b0039cce486b9bsm565631pgk.13.2022.04.13.21.04.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 21:04:35 -0700 (PDT) Date: Thu, 14 Apr 2022 12:04:32 +0800 From: Muchun Song To: Andrew Morton Cc: corbet@lwn.net, mike.kravetz@oracle.com, mcgrof@kernel.org, keescook@chromium.org, yzaikin@google.com, osalvador@suse.de, david@redhat.com, masahiroy@kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, smuchun@gmail.com Subject: Re: [PATCH v8 4/4] mm: hugetlb_vmemmap: add hugetlb_optimize_vmemmap sysctl Message-ID: References: <20220413144748.84106-1-songmuchun@bytedance.com> <20220413144748.84106-5-songmuchun@bytedance.com> <20220413121051.a363193c726451115c634a69@linux-foundation.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20220413121051.a363193c726451115c634a69@linux-foundation.org> Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=bS3J8H3+; spf=pass (imf27.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.216.45 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-Stat-Signature: sspwr88mseiqt3ua99ih6teh11dmqn6f X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 9AEDF40007 X-HE-Tag: 1649909077-378919 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Apr 13, 2022 at 12:10:51PM -0700, Andrew Morton wrote: > On Wed, 13 Apr 2022 22:47:48 +0800 Muchun Song wrote: > > > We must add hugetlb_free_vmemmap=on (or "off") to the boot cmdline and > > reboot the server to enable or disable the feature of optimizing vmemmap > > pages associated with HugeTLB pages. However, rebooting usually takes a > > long time. So add a sysctl to enable or disable the feature at runtime > > without rebooting. > > Do we really need this feature? Really? What's the use case and what > is the end-user value? > We know that this feature is disabled by default without passing "hugetlb_free_vmemmap=on" to the boot cmdline. When we (ByteDance) deliver the servers to the users who want to enable this feature, they have to configure the grub (change boot cmdline) and reboot the servers, whereas rebooting usually takes a long time (we have thousands of servers). It's a very bad experience for the users. So we need a approach to enable this feature after rebooting. This is a use case in our practical environment. > Presumably CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP worsens things for some > setups/workloads? Please tell us much more about that. What is the > magnitude of the deoptimization? > Most workloads are allocated HugeTLB from the pool not the buddy allocator, which are the users of this feature. However if the use case is that HugeTLB pages are allocated 'on the fly' instead of being pulled from the HugeTLB pool, those workloads would be affected with this feature enabled. Those workloads could be identified by the characteristics of they never explicitly allocating huge pages with 'nr_hugepages' but only set 'nr_overcommit_hugepages' and then let the pages be allocated from the buddy allocator at fault time. I don't know what the specific workload is. But I can confirm it is a real use case from the commit 099730d67417. For those workloads, the page fault time could be ~2x slower than before. I suspect those users want to disable this feature if the system has enabled this before if they don't think the memory savings benefit is enough to make up for the performance drop. Do you think it's ok if I put those information in the commit log or vm.rst? Thanks.