From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7883CC54E5D for ; Thu, 14 Mar 2024 19:34:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 04242800DD; Thu, 14 Mar 2024 15:34:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F3346800B4; Thu, 14 Mar 2024 15:34:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD596800DD; Thu, 14 Mar 2024 15:34:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id CEDF9800B4 for ; Thu, 14 Mar 2024 15:34:43 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 63EA880A58 for ; Thu, 14 Mar 2024 19:34:43 +0000 (UTC) X-FDA: 81896646846.20.8AB8DFE Received: from mail-qt1-f175.google.com (mail-qt1-f175.google.com [209.85.160.175]) by imf29.hostedemail.com (Postfix) with ESMTP id BCCB112000F for ; Thu, 14 Mar 2024 19:34:41 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=soleen-com.20230601.gappssmtp.com header.s=20230601 header.b=dK30jRVV; spf=pass (imf29.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.160.175 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=pass (policy=none) header.from=soleen.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710444881; a=rsa-sha256; cv=none; b=kybFxfvWODk37Xr+ByzrVof8Ok+KBWQgDzcM5GHv15E0ojyaj/+Xv1h23O172NQ4Vo0DfE WvuarxEAjdMgWZ93Cb0B1spsme1H5A7I8/4cne7ytuUInyvDc1z0r0ft1aIjipNJ+gsLUT iKKpDz7Xjms8KsAli9UGdzNplKTf3Sw= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=soleen-com.20230601.gappssmtp.com header.s=20230601 header.b=dK30jRVV; spf=pass (imf29.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.160.175 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=pass (policy=none) header.from=soleen.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710444881; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Gu8ndvCosLfR13D/hsYz/cv9ogsH/tqd6hyUv+jcRaM=; b=ke7EVRD+JP7PU2WcR54YxT0e9X90bIVuXTiG/Z+XDVFe2EJQz8lg3O7ItC48ali7I9yTT9 pRQvxhDCwFuI22Kp2lQFZMTUxZGOuVhvXdXgER2Vbj3Itf/4x9iXf/ZBA3Yi1txQsrqnP4 yYEgysaqgw1z/WL65I6uBVYks1yP3vI= Received: by mail-qt1-f175.google.com with SMTP id d75a77b69052e-42a029c8e62so8959031cf.1 for ; Thu, 14 Mar 2024 12:34:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen-com.20230601.gappssmtp.com; s=20230601; t=1710444881; x=1711049681; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Gu8ndvCosLfR13D/hsYz/cv9ogsH/tqd6hyUv+jcRaM=; b=dK30jRVVAtb0wqlm3kTF3QFQcNtT6sUv4qaHF0XEywhyRKVBeRgCiwiJfLUPTeoaOw UOiw/ge7Kn9fjFiiTVnOZtJgKwTxTHU7Cy7pGLSi9eXkG8GvR5ZWA55/9uJciXGfSt6z j5YR9ouo0j0uW1i2L87qbEXxC9CfAtQhZFh3U14T6gWIKYpZq2itt7OgWw7kcAXQqukr zCANZ58kF75nR+4bcqhYYjoDwKNc8H+v6VWYrE9ZkvEgu+LMvsLPV4ZaPTwZJqKjUgxN +pW+dhAf1DVZORl3rvmLf6G/O1yEq19wHvzWcL0V7sZpqqDMnAvgWUYcd+3z5Gr9rFCx Vd+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710444881; x=1711049681; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Gu8ndvCosLfR13D/hsYz/cv9ogsH/tqd6hyUv+jcRaM=; b=jTz20uwzL7cfhmXWf2blbpf8I0EpkrDX7Z+etI54yTIdy/i3A1r5+t8lJllxFvtgyW rbwhdSypOsPLtecykjDZTrwKUVBfG4RHp1Hnznp/zhLsLbzbCWqWLdmlqoPH24Avhm8K 6nx6C10XKPNliEp9DgXMiLnQfTEvPKPRqC6tPZOJCHU6NapflYE/Vj575p7UseZsn2NI veq0emlyxXlWCUqpYFW9EKlevl6aHzfKyvSiJPrHx4K82Bf/d3xmk5FinySmJ3VIEQZL i9TcjyteCQo97gqpILY4A/HkC2iLSno+q7A3wxSS2xf1OcC9lMdvqXXAMYhxj7ySodE8 gtYw== X-Forwarded-Encrypted: i=1; AJvYcCU7ziHcXS91eUFuc+b2Zq8JEr6MGW+2YONjMtx/70Tag5oBv6xyTFJmTbF0O4BOzvF80DEORHgSIoXm9Bz4E+wnPcU= X-Gm-Message-State: AOJu0Yy72Kv+jP1m03JrvcirVC1TlZRNN/sBoUYes0kYQl+XSUnV+eZt gQzMfQukIyQNyAABHpX08GI6S2S46lHHQtieMVdpnlXmIh58QIdBvXkQEEq5x8FGY+S8Xn5uLDQ bdhQJMX0x3GdCgq30lwt0ZyKZu4ORN+Vg5d4HhA== X-Google-Smtp-Source: AGHT+IFYUmp8vuQtAaVQJsJwK0pf9SP/QzZlup80TTakr/1z6jjrHWw1ZTmVhdIG/sfZHB8mMBDgjcN1TuW0c5bY+6I= X-Received: by 2002:ac8:5f8e:0:b0:42f:30bd:fc8b with SMTP id j14-20020ac85f8e000000b0042f30bdfc8bmr1956069qta.63.1710444880851; Thu, 14 Mar 2024 12:34:40 -0700 (PDT) MIME-Version: 1.0 References: <20240311164638.2015063-1-pasha.tatashin@soleen.com> <2cb8f02d-f21e-45d2-afe2-d1c6225240f3@zytor.com> In-Reply-To: From: Pasha Tatashin Date: Thu, 14 Mar 2024 15:34:03 -0400 Message-ID: Subject: Re: [RFC 00/14] Dynamic Kernel Stacks To: Kent Overstreet Cc: "H. Peter Anvin" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, x86@kernel.org, bp@alien8.de, brauner@kernel.org, bristot@redhat.com, bsegall@google.com, dave.hansen@linux.intel.com, dianders@chromium.org, dietmar.eggemann@arm.com, eric.devolder@oracle.com, hca@linux.ibm.com, hch@infradead.org, jacob.jun.pan@linux.intel.com, jgg@ziepe.ca, jpoimboe@kernel.org, jroedel@suse.de, juri.lelli@redhat.com, kinseyho@google.com, kirill.shutemov@linux.intel.com, lstoakes@gmail.com, luto@kernel.org, mgorman@suse.de, mic@digikod.net, michael.christie@oracle.com, mingo@redhat.com, mjguzik@gmail.com, mst@redhat.com, npiggin@gmail.com, peterz@infradead.org, pmladek@suse.com, rick.p.edgecombe@intel.com, rostedt@goodmis.org, surenb@google.com, tglx@linutronix.de, urezki@gmail.com, vincent.guittot@linaro.org, vschneid@redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: BCCB112000F X-Stat-Signature: hyy5gu9ti8oj7t1chzibdzr51aeqtgc9 X-Rspam-User: X-HE-Tag: 1710444881-821557 X-HE-Meta: U2FsdGVkX18tBwOGbhuvx25NsVNQZOyQxHNnb9m4rnH5XnKaviAELCSAiTFDHYP009AQcCgsOTnaxPx9oNAOxIMPeCnKsPvlL39Bx0M880JC1NXqS2XQ3lGN+Kjhre3V45RAUyTkSseeFOy7Mv4M1sZZbSduQVB/xpgoI2XcVWrSJ5oalSVi7ADXLEA67pTxxiX0/ngYpDc3Da2UermVuHBjgj+6T9d/zPKzqw8Kw+36iSN+48rMALfmdevFsDwCTtQPUCXr3Viaa4Q2TUFLWAUUKH4uZ0Yzaa16KaEfFKbSWcENCLctvNYTXozCMPjGBLW/NI7tAuBknJeGhhgaBgcUTrT+6rGHqxkJTF9nwwkA2HD0NQxmlz1X3pfw3ksHb1bsHHaLnrBHi5yZDjJ/QQ1NJXmw06GQ3lkzH4WRY1IwNVF/bvWvWhH8LOU1a1sLROEZOdqRNqGValvea8JmWf78pi/EvtCDhTxuKiT/ZPHfXix647JfAMTgDQJ8MKVAHFMmMd9tBU24CSJeQC1Cm1hUL1BYMVAI3KDppNNW+swV2xLZSZUu2kIuUll9lN7XSnICGjb1lVeY7nSupFSiYW7pfGnY8B/kPhh+uBVkaktp0PuwBMo4BD+iR99gXVjTNMZRShFEzyaLofGBERN/MiiVxxbUpYUTs3uy7b1l0zyLTNYyQiCm4yOHzjZuQaooLpKAZxDP0OcDlrYpXKVh/tlwuji/NobIvvLHn7MzyAOos4bLs433D/XNev61uzacb5072IGH1rP9dngga37ImSPLH02J//I4/YOTpvKCtSvNIvKJNX6GoVcvvqqmhNQdb1sC28ZDy29JtP2LXgTJokBshoAGIe2FWORmwmIGMHR/uVPyvkclH05qadcCtaa6ifLyXt2/N6fG8uskLzS6CIO3dzCb4YS6KGQ1wHJa8Mxp4wPU2ygGPhA6DvhjZb/y4nhMHUi4L0XcVHUkRK+ vEwZJGLp S+jV5g7FCk1783wMQ5HEtFni2Hdp+DiFY2aWvr5SWEwNloMPM1qWxupT1ww== X-Bogosity: Ham, tests=bogofilter, spamicity=0.051170, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Mar 14, 2024 at 3:29=E2=80=AFPM Kent Overstreet wrote: > > On Thu, Mar 14, 2024 at 03:23:08PM -0400, Pasha Tatashin wrote: > > > > > > > > My point is that what matters is total memory use, not just memory = used in > > > > the kernel. Amdahl's law. > > > > > > If userspace is running a few processes with many threads and the > > > userspace stacks are small, kernel stacks could end up dominating. > > > > > > I'd like to see some numbers though. > > > > The unused kernel stack pages occupy petabytes of memory across the fle= et [1]. > > Raw number doesn't mean much here (I know how many machines Google has, > of course it's going to be petabytes ;), percentage of system memory > would be better. > > What I'd _really_ like to see is raw output from memory allocation > profiling, so we can see how much memory is going to kernel stacks vs. > other kernel allocations. I've heard there is memory profiling working that can help with that... While I do not have the data you are asking for, the other kernel allocations might be useful, but this particular project is targeted to help with reducing overhead where the memory is not used, or used in very extreme rare cases. > Number of kernel threads vs. number of user threads would also be good > to know - I've been seeing ps output lately where we've got a lot more > workqueue workers than we should, perhaps that's something that could be > addressed. Yes, doing other optimizations make sense, reducing the total number kernel threads if possible might help as well. I will look into this as well to see how many user threads vs kernel threads we have.