From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01137C4332F for ; Mon, 7 Nov 2022 17:39:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 78E6E6B0073; Mon, 7 Nov 2022 12:39:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 73DF58E0001; Mon, 7 Nov 2022 12:39:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 62CD76B0078; Mon, 7 Nov 2022 12:39:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 550366B0073 for ; Mon, 7 Nov 2022 12:39:14 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 07CF3120531 for ; Mon, 7 Nov 2022 17:39:14 +0000 (UTC) X-FDA: 80107357428.13.5E64DEF Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf12.hostedemail.com (Postfix) with ESMTP id 9381E40002 for ; Mon, 7 Nov 2022 17:39:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=aHuggXqSZgXb21UbEzsKOKhxQygaz7VUuywqzT42yGc=; b=OCuu9bB9n/ZKIVsn5QDuWO5roj lmo3bv9sY7nJeKPbM2xiQFrS7NMiwFZKAlPuyaviXp8XLFkgbhDuiRPozaihJ512T3mqIz8MW2zDK 3MHOH12/pGhO2pmPyeZPkiieGdnxhkpfBDPpQCR2xwJ3HnIx4syndgtNtyuYXK2Q9dYY0neKQ6IDN tWZloHy/7Ty9bWyN4h8mL0dIGl1S3y/D8dbfhqIRiup7fOiiz10khDxhB1LDdZQghuFBHmSR2DPdR Sr4ocKOUSoZAbiTv/7Jqc0ZURHqYUcx6s+vYg7QLzKq/jcs3NNC1DTeglO3mNh3GZuu5puOz58K/S B8UK4pJQ==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1os65P-00GkOZ-A8; Mon, 07 Nov 2022 17:39:03 +0000 Date: Mon, 7 Nov 2022 09:39:03 -0800 From: Luis Chamberlain To: Aaron Lu Cc: Song Liu , bpf@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, x86@kernel.org, peterz@infradead.org, hch@lst.de, rick.p.edgecombe@intel.com, dave.hansen@intel.com, rppt@kernel.org, zhengjun.xing@linux.intel.com, kbusch@kernel.org, p.raghav@samsung.com, dave@stgolabs.net, vbabka@suse.cz, mgorman@suse.de, willy@infradead.org, torvalds@linux-foundation.org, Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: Re: [PATCH bpf-next v1 RESEND 1/5] vmalloc: introduce vmalloc_exec, vfree_exec, and vcopy_exec Message-ID: References: <20221031222541.1773452-1-song@kernel.org> <20221031222541.1773452-2-song@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667842751; a=rsa-sha256; cv=none; b=XFDTOWt2QHeLSUlBeZDzuW+O+d/yHvNmAzcfMnZiQZg41OkAORzyJJIS32fPUP/eB0slbk 7G6Ph004uWr9XNU70T/vyAv37ujfMPl6+gqwO6jla2I7mznqzazQOHAcOHIqMyvnTKJJj0 60C8m6JoKvbRSB773y4iJ/lauPIXSEs= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=OCuu9bB9; spf=none (imf12.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667842751; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aHuggXqSZgXb21UbEzsKOKhxQygaz7VUuywqzT42yGc=; b=H+K+cT62FdcGnTjkDTjJVy7JOax0EYsE4Yq5d4KGRTUjUM+QKLt3CV11puBRzRCg5o/w3g x8+yAAATjbD5jJ5xIE6ga++qX0ooYfW5McaJXH8cXBP5J9ILvNRerxIpH/gVEV/ogR9Fg/ 5Wr3zuxMO+EUn0L7NcXGLsu2Dhbgy7o= Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=OCuu9bB9; spf=none (imf12.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) X-Rspamd-Server: rspam12 X-Rspam-User: X-Stat-Signature: stb4ce399cdbqt9cjdhmwkrizhjw6z5n X-Rspamd-Queue-Id: 9381E40002 X-HE-Tag: 1667842751-886639 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Nov 07, 2022 at 02:40:14PM +0800, Aaron Lu wrote: > Hello, > > On Wed, Nov 02, 2022 at 04:41:59PM -0700, Luis Chamberlain wrote: > > ... ... > > > I'm under the impression that the real missed, undocumented, major value-add > > here is that the old "BPF prog pack" strategy helps to reduce the direct map > > fragmentation caused by heavy use of the eBPF JIT programs and this in > > turn helps your overall random system performance (regardless of what > > it is you do). As I see it then the eBPF prog pack is just one strategy to > > try to mitigate memory fragmentation on the direct map caused by the the eBPF > > JIT programs, so the "slow down" your team has obvserved should be due to the > > eventual fragmentation caused on the direct map *while* eBPF programs > > get heavily used. > > > > Mike Rapoport had presented about the Direct map fragmentation problem > > at Plumbers 2021 [0], and clearly mentioned modules / BPF / ftrace / > > kprobes as possible sources for this. Then Xing Zhengjun's 2021 performance > > evaluation on whether using 2M/1G pages aggressively for the kernel direct map > > help performance [1] ends up generally recommending huge pages. The work by Xing > > though was about using huge pages *alone*, not using a strategy such as in the > > "bpf prog pack" to share one 2 MiB huge page for *all* small eBPF programs, > > and that I think is the real golden nugget here. > > I'm interested in how this patchset (further) improves direct map > fragmentation so would like to evaluate it to see if my previous work to > merge small mappings back in architecture layer[1] is still necessary. You gotta apply it to 6.0.5 which had a large change go in for eBPF which was not present on 6.0. > Conclusion: I think bpf_prog_pack is very good at reducing direct map > fragmentation and this patchset can further improve this situation on > large machines(with huge amount of memory) or with more large bpf progs > loaded etc. Fantastic. Thanks for the analysis, so yet another set of metrics which I'd hope can be applied to this patch set as this effort is generalized. Now imagine the effort in cosnideration also of modules / ftrace / kprobes. > Some imperfect things I can think of are(not related to this patchset): > 1 Once a split happened, it remains happened. This may not be a big deal > now with bpf_prog_pack and this patchset because the need to allocate a > new order-9 page and thus cause a potential split should happen much much > less; Not sure I follow, are you suggesting a order-9 (512 bytes) allocation would trigger a split of the reserved say 2 MiB huge page? > 2 When a new order-9 page has to be allocated, there is no way to tell > the allocator to allocate this order-9 page from an already splitted PUD > range to avoid another PUD mapping split; > 3 As Mike and others have mentioned, there are other users that can also > cause direct map split. Hence the effort to generalize. Luis