From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BCB8C6FD1C for ; Thu, 9 Mar 2023 22:38:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 106BE280002; Thu, 9 Mar 2023 17:38:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0B7BC6B007B; Thu, 9 Mar 2023 17:38:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EE917280002; Thu, 9 Mar 2023 17:38:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E093A6B0078 for ; Thu, 9 Mar 2023 17:38:42 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id BB76CAAC5A for ; Thu, 9 Mar 2023 22:38:42 +0000 (UTC) X-FDA: 80550825684.20.AD83160 Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) by imf06.hostedemail.com (Postfix) with ESMTP id EA2CE180011 for ; Thu, 9 Mar 2023 22:38:40 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ifpidnJU; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf06.hostedemail.com: domain of zokeefe@google.com designates 209.85.208.49 as permitted sender) smtp.mailfrom=zokeefe@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678401521; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=iIVP+U9IXmg9igNMbR9KE+dlRtqJIh5HRgs4QRiTQQA=; b=IZ2W2eJ1jstXSW7b6N3HBGZKRU4Mxf2L5WrSWEznuqfmeLRbxYJX3hFHnHaxY9GC1qRH08 FQPGXF5gg/GTW1qgsFVE5vimdHDdL59qDZxhlA7FmstqhCZZHP3s9iij+U/wmBQGJoKoNi zmXYpJFBy4Fec88WPZhvZigjT+bTgw8= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ifpidnJU; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf06.hostedemail.com: domain of zokeefe@google.com designates 209.85.208.49 as permitted sender) smtp.mailfrom=zokeefe@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678401521; a=rsa-sha256; cv=none; b=Lc/EMNkHP7SCWVG/8lVgGOvl9ufKlFEjfAEGu2tfoCSWLs/Xfx2UNfHEuMBpDZkZjiPOJG 7883jEbjFUq/laaPGUHbI/EMPi8l6wXOUuKbUIgw8ynM1mLUqb4Gn1tNuT29BKm6WrUjl2 zN0Ufib/Z/uDzg6ysaFx2q/74FOtEWE= Received: by mail-ed1-f49.google.com with SMTP id j11so13258623edq.4 for ; Thu, 09 Mar 2023 14:38:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1678401519; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=iIVP+U9IXmg9igNMbR9KE+dlRtqJIh5HRgs4QRiTQQA=; b=ifpidnJUCwCO7tXzTto2idUUHuYawl7SNcbA1YLMw7oJweI8S0ykSH+vudNquLAV2f r0T4JfyTZY54X4tZHvAWag8GZTNQkcHDzfijfkkoRGVPOlZb5iC6PKVK7AkyR911Uyse 3zXzBs+xhdCa2r7E/zroPX110xJxmWp00O5P2o/mxZ6WQ5DMNIwr7URcmr7KG/no6Qu2 pkPcM9F05p0G5shNzx+TH8Oxeu0sosJCuqhHwoj+u7zhMhUC8C1OhsHg1pSgkbpHvBD/ GTe6P8k5PYEq6uFqEK6rBR9/DXkGvR86aoS3XTH4et7vWLcE/DJszY7ARrtX1hDmAVxS 4iEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678401519; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iIVP+U9IXmg9igNMbR9KE+dlRtqJIh5HRgs4QRiTQQA=; b=Q5anN8eha3kbshTBLU1IDHdpxJUnyO9lnDHzCQvNMpkC+wLdWgu+kSNE/Vo4MnSVie JtjLur57FOUYs13Txd7dKJV80z6Ajk+4g2isO9FKRB42H/SzPMgCGA+E+dYB8j50QlAz pDnQeSiNWsBDd3YgJ8fPTz9qmKaPtM1tp6sqcWF7zl2OJq4z9Do2fdlYL9U2Vvc48aQS IJdVlPNf9JTn06lqiLNrMeNKIfs8QqIHsfrxbZ906RwksHFjgCj4MFiHSlU9D6LjQWJ9 EDMtvxuZQqP3ciZ8hEsk+i72TcZp5aH9V6sVL7D+EPLGUlgbD0df1Wy6nYikJUVWJ84v 9UQw== X-Gm-Message-State: AO0yUKXHXMrzwuVeIeRV9HCyFN+T19dbpAWFO32+7jzCrsgBX8uOOhCs VNU41SF+yNL6ixriCQqcRIxiOQfg9pzhMR3lVx90sA== X-Google-Smtp-Source: AK7set9ZCtHMECp6dFXoSY1Mc/M0yJkIFeY9zLiltN6dJ2EEYBuUWVxA6LVpRPp/SJpk68hCxQcWkM9w/3Uen4BjbbY= X-Received: by 2002:a50:d581:0:b0:4c2:ed2:1198 with SMTP id v1-20020a50d581000000b004c20ed21198mr12985289edi.4.1678401518988; Thu, 09 Mar 2023 14:38:38 -0800 (PST) MIME-Version: 1.0 References: <20230306235730.GA31451@monkey> <20230307004049.GC4956@monkey> <20230308190206.GA4005@monkey> In-Reply-To: <20230308190206.GA4005@monkey> From: "Zach O'Keefe" Date: Thu, 9 Mar 2023 14:38:01 -0800 Message-ID: Subject: Re: THP backed thread stacks To: Mike Kravetz Cc: Peter Xu , David Hildenbrand , Rik van Riel , Mike Rapoport , linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: EA2CE180011 X-Stat-Signature: 86iat7k5hmbo3cj8hh9by5boc4wca191 X-HE-Tag: 1678401520-986091 X-HE-Meta: U2FsdGVkX19OKUo7xTln1m5OoiNXCnjhFKi5HiyU+mXagNxpKg7mcBpNooC/3bSSzCRuTOLe6SywJRqeHGOEtLBzdSMU/xwAr7pZV+30lAmZ3jHtQwQjEmX+8+dnWmqMVWdnd7X5Ul6GoLgsvGJqec7NZdM1QcMSAuluIRKp3XNp8asbLBJXsL1hV8AoSJ9pRUqZ67tOo2MnyaEhfLnEOLyfadslL9qY8z2wCOGdHoo9AHGEgG9LmIRDYUxXo5ap/ouEvlG327QK8rNJj3/N6wnESkmJJM50zfntaHlEkMb2tLQ2gmFny1/5kjkfclmRQyHhJe23Ib677ZWOi7Q2MaBCrfIN8y//p/im8+AIDQiZ3hs38/KtFWOP5yRfmFmxPWuPglffQWu5YaBzuEk6ZAtZ6OzluwbExwUQEhVbmLGKqoCMiOrlcglvFcWpIcTCL/ODDScHPzaIcdJTaZZeDI0AoHSYV+0a7HVXFn2dbzInzX2szVFuNz6KDENmBpuDyxRxBM9NfKzjSwU3NMoVWFofHSOhNAaMQzJWwwRPF1nXEvEYXKJ8wHFUYEb4+f2nogGXPCLDixZ5z61u6hwmrOP8FPNBTx0yrUHEjHx91fjAt7TWKpuOtH1rRsd/ni9ZaGUV24cK0kLyMjx7Xa8X6tfQtKjTZIT2IY3C5/3TRSc8ak8Wr9ZM+a/U4Ll/hZP/DeKrhxxNcxL2pKLvIC8X+9ke6h6SmCxRWiyMQRUtHHcd9P4Uko3yRPt2IGtDXrghuKHY7IwHxf8Ptr+JCXltluRBEdGoylav+07u4fuNKjT4ABTJiinBD39PdADZKu87kt1EUWNM8DI4PABKR+U2+DEYn4wyICqlvf2tcgY75EXNUgLOQ6+Kzl6bpZuGYXhZJ3BtmJzdAVkG9HdxMvb4/hZJxi8/KPSllQxTMgSnYy1EgI72A6pnNxtxIDOaq0iMis111gf7/Tmd4OtCmd7 OobJ1EgX wG/JcGKyGXPWfyLM2F/4YCpdo0yiI1pOJQDK2gL8rSriWHF0u/SJMMr0QXHm/FyfHUnJYIyWOf3BzQfbBasx5t0O3YHF46jfyT4Y/pnW11C6sZ//RelVIlXdN97sMnxfiapwzl6sc5SE2hQ5Ad/eajOtZ8NRbR8TTkmWXeM9AcgiTgbJICiewikbxOwL6wfEW7+7NtSJw0j/GGCsu4djdAn331onI/DbBVs3o8gmLCHCrS68TTU+OLnu5g3oSQLD/D9MFDTXknRABJoQeOACSeihQc4cwPy9zVIu8LOOWl6ITwFk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Mar 8, 2023 at 11:02=E2=80=AFAM Mike Kravetz wrote: > > On 03/06/23 16:40, Mike Kravetz wrote: > > On 03/06/23 19:15, Peter Xu wrote: > > > On Mon, Mar 06, 2023 at 03:57:30PM -0800, Mike Kravetz wrote: > > > > > > > > Just wondering if there is anything better or more selective that c= an be > > > > done? Does it make sense to have THP backed stacks by default? If= not, > > > > who would be best at disabling? A couple thoughts: > > > > - The kernel could disable huge pages on stacks. libpthread/glibc = pass > > > > the unused flag MAP_STACK. We could key off this and disable hug= e pages. > > > > However, I'm sure there is somebody somewhere today that is getti= ng better > > > > performance because they have huge pages backing their stacks. > > > > - We could push this to glibc/libpthreads and have them use > > > > MADV_NOHUGEPAGE on thread stacks. However, this also has the pot= ential > > > > of regressing performance if somebody somewhere is getting better > > > > performance due to huge pages. > > > > > > Yes it seems it's always not safe to change a default behavior to me. > > > > > > For stack I really can't tell why it must be different here. I assum= e the > > > problem is the wasted space and it exaggerates easily with N-threads.= But > > > IIUC it'll be the same as thp to normal memories iiuc, e.g., there ca= n be a > > > per-thread mmap() of 2MB even if only 4K is used each, then if such m= map() > > > is populated by THP for each thread there'll also be a huge waste. > > I may be alone in my thinking here, but it seems that stacks by their nat= ure > are not generally good candidates for huge pages. I am just thinking abo= ut > the 'normal' use case where stacks contain local function data and argume= nts. > Am I missing something, or are huge pages really a benefit here? > > Of course, I can imagine some thread with a large amount of frequently > accessed data allocated on it's stack which could benefit from huge > pages. But, this seems to be an exception rather than the rule. > > I understand the argument that THP always means always and everywhere. > It just seems that thread stacks may be 'special enough' to consider > disabling by default Just my drive-by 2c, but would agree with you here (at least wrt hugepages not being good candidates, in general). A user mmap()'ing memory has a lot more (direct) control over how they fault / utilize the memory: you know when you're running out of space and can map more space as needed. For these stacks, you're setting the stack size to 2MB just as a precaution so you can avoid overflow -- AFAIU there is no intention of using the whole mapping (and looking at some data, it's very likely you won't come close). That said, why bother setting stack attribute to 2MiB in size if there isn't some intention of possibly being THP-backed? Moreover, how did it happen that the mappings were always hugepage-aligned here? > -- > Mike Kravetz >