From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99880C54E60 for ; Thu, 14 Mar 2024 19:05:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E0B57800DA; Thu, 14 Mar 2024 15:05:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DBD8D800B4; Thu, 14 Mar 2024 15:05:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C83AA800DA; Thu, 14 Mar 2024 15:05:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B93BE800B4 for ; Thu, 14 Mar 2024 15:05:57 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id EB9E9A0326 for ; Thu, 14 Mar 2024 19:05:56 +0000 (UTC) X-FDA: 81896574312.19.67F23AB Received: from out-173.mta0.migadu.com (out-173.mta0.migadu.com [91.218.175.173]) by imf08.hostedemail.com (Postfix) with ESMTP id DDB4716001D for ; Thu, 14 Mar 2024 19:05:54 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=lHtoAmqU; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf08.hostedemail.com: domain of kent.overstreet@linux.dev designates 91.218.175.173 as permitted sender) smtp.mailfrom=kent.overstreet@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710443155; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=troX3X8HuvTilW6NgLCfSK9nrQd3wvTH0v9DPQHyR8M=; b=KRCX4uR7WPYfoEsrCOXLc+/XcW4IWz3S50wcnyGQRUp3wnzjG8ZQifJ2qCpVgmvLBUH9T2 OM4TIPt01a+mvEWtOjXPn1aVR2MtkanMhyDSegHn16xTv2DuRSwdP+4Vfa71fmjvB370Zd VMp1APJFAwKBh98t19ihoAuypNS/nCs= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=lHtoAmqU; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf08.hostedemail.com: domain of kent.overstreet@linux.dev designates 91.218.175.173 as permitted sender) smtp.mailfrom=kent.overstreet@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710443155; a=rsa-sha256; cv=none; b=Ts+lZRBrHDdtuXKWsDarJAT2I4V6f2M0q1teUJkHNmploZd3WTXBrA2cUFa9pJLLQHAXN1 3dLzappKnP32Q56n+QIu0R2kb4MyB6v52avfH+rV0qjnxzJX9z6L8F1aSf+he0mof2NUiE nq2aiN3I+ZUgsVsbT9M5DFGf47RjxDA= Date: Thu, 14 Mar 2024 15:05:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1710443152; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=troX3X8HuvTilW6NgLCfSK9nrQd3wvTH0v9DPQHyR8M=; b=lHtoAmqUIojCK2jPtn9uqGtrJW+6IvhqHqUKqyQDf7NHJmuqJjy/W+x2a7AAXH+7hkUNzH Br5u6MBOqEW3aFoO2R+YLGVNy6A+mUgZ+78EHK5t21DNKyfXdAXuYn02RoQqPJi6KcAeeJ jn15DnwPDOe9CHsRu4tkcVsZdEBBQdg= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Kent Overstreet To: "H. Peter Anvin" Cc: Pasha Tatashin , linux-kernel@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, x86@kernel.org, bp@alien8.de, brauner@kernel.org, bristot@redhat.com, bsegall@google.com, dave.hansen@linux.intel.com, dianders@chromium.org, dietmar.eggemann@arm.com, eric.devolder@oracle.com, hca@linux.ibm.com, hch@infradead.org, jacob.jun.pan@linux.intel.com, jgg@ziepe.ca, jpoimboe@kernel.org, jroedel@suse.de, juri.lelli@redhat.com, kinseyho@google.com, kirill.shutemov@linux.intel.com, lstoakes@gmail.com, luto@kernel.org, mgorman@suse.de, mic@digikod.net, michael.christie@oracle.com, mingo@redhat.com, mjguzik@gmail.com, mst@redhat.com, npiggin@gmail.com, peterz@infradead.org, pmladek@suse.com, rick.p.edgecombe@intel.com, rostedt@goodmis.org, surenb@google.com, tglx@linutronix.de, urezki@gmail.com, vincent.guittot@linaro.org, vschneid@redhat.com Subject: Re: [RFC 00/14] Dynamic Kernel Stacks Message-ID: References: <20240311164638.2015063-1-pasha.tatashin@soleen.com> <2cb8f02d-f21e-45d2-afe2-d1c6225240f3@zytor.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: DDB4716001D X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 6owseembtixx36cxuswn7rufei73zdcf X-HE-Tag: 1710443154-248932 X-HE-Meta: U2FsdGVkX181hSxD9Hm43xOVXedqwzcfwBjtBxZkeFNWuyAvV/f2WKRdqblbg0m9u1AkoZ8HPZl5/16DTB/arJWZfIuJPZ/uGIT4e8/x6hdH5XQJcBNoCnHM4eSiU7YkwqP/HUdYlFERaZBVaqkSo7CFBKuF7PFZ0iHzKPTJqwqLYafJAFXUY5bD3OdT/HFAA2OQ4w7CW+ej/47JLAjGlss5cpI5TjIzG83pWlqSa7g6OR14QuOPwcMGSImD8lHcaxfIzBlhl+pGVTevddYkqgLjBdLOGED+SATSvh7MmZY9NcOQ+O7fxTHF/P6Ki8UhPiiBshkH1DUiIf6XmCIPfgbbKYgJkC6h8dgksiiKfI9Eaji5egbRko+MBbdm/jdBWhjfdQiIvQMofm77S0ETuq2MgvrAi4dQiU9hbB/pDZaWmfQTN0yLhvyReeiP9L/H5MyAiIUm/FmrUcux6FjHTnxC1//OvLzGxqxF2hHuwERzW3p2/vMemrgEfgGvrOGa2qhitPdUlt6Apik2Fp0h2BxnWLj3kxjfCdHrJJ/jrs/6RupKAqcLKoeN+7Vc3hX9BhXOisGJye6EKSE7NEZPEBWAwl7wcStLzciaX9A38C0QNdj05c4MQo6aIW27qxs+yuyz1Bm/VSN+kEyKgqhu3iBD/kLXnZQwFkG/MecKhhy5Sl1uHXchmBatMSny0QoWN0CvJCpIszhIlmCOOtRM1m9FE2bvhZBXMEeEi+GdDN2d0409u6BkpswvxJHPDAiFc61ioIBEP5f+9bHf7HXE68qMncnwm8ynGCMM6SjCai24M7psszncuSMj/5xIihvsBw7oEX9RfU2DvdmUviirOG8bKmM7hjz5w9lJKxa4zPIYiSIAUpm4ZlUzSnlUqh0MHakWDSug7wbHTECx6JYZ6yYQGWZunAsdcid7iv9ZiKI3G9+MDNmELzgqXU3aMLiytBrMnjEfejP/58YRTso AkGjIUbZ sj9krtmmdt5yTAXLmw0H6nEYopdUC2mNDxWvOvd4IsyhuOwwNS7A/lkwF58SqoHjLxQLcPux2sc/r1Ps= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Mar 12, 2024 at 02:36:27PM -0700, H. Peter Anvin wrote: > On 3/12/24 12:45, Pasha Tatashin wrote: > > > > > > Ok, first of all, talking about "kernel memory" here is misleading. > > > > Hi Peter, > > > > I re-read my cover letter, and I do not see where "kernel memory" is > > mentioned. We are talking about kernel stacks overhead that is > > proportional to the user workload, as every active thread has an > > associated kernel stack. The idea is to save memory by not > > pre-allocating all pages of kernel-stacks, but instead use it as a > > safeguard when a stack actually becomes deep. Come-up with a solution > > that can handle rare deeper stacks only when needed. This could be > > done through faulting on the supported hardware (as proposed in this > > series), or via pre-map on every schedule event, and checking the > > access when thread goes off cpu (as proposed by Andy Lutomirski to > > avoid double faults on x86) . > > > > In other words, this feature is only about one very specific type of > > kernel memory that is not even directly mapped (the feature required > > vmapped stacks). > > > > > Unless your threads are spending nearly all their time sleeping, the > > > threads will occupy stack and TLS memory in user space as well. > > > > Can you please elaborate, what data is contained in the kernel stack > > when thread is in user space? My series requires thread_info not to be > > in the stack by depending on THREAD_INFO_IN_TASK. > > > > My point is that what matters is total memory use, not just memory used in > the kernel. Amdahl's law. If userspace is running a few processes with many threads and the userspace stacks are small, kernel stacks could end up dominating. I'd like to see some numbers though.