From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 967DDC43334 for ; Fri, 8 Jul 2022 13:41:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DFD506B0072; Fri, 8 Jul 2022 09:41:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DAAD36B0073; Fri, 8 Jul 2022 09:41:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C9BB56B0074; Fri, 8 Jul 2022 09:41:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id B98746B0072 for ; Fri, 8 Jul 2022 09:41:55 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 94881324EE for ; Fri, 8 Jul 2022 13:41:55 +0000 (UTC) X-FDA: 79664045790.10.20EB1E0 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf01.hostedemail.com (Postfix) with ESMTP id 1EC914001C for ; Fri, 8 Jul 2022 13:41:53 +0000 (UTC) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id AA42221F9E; Fri, 8 Jul 2022 13:41:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1657287712; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=yrGC8i4SCMF9Esn1IzCdcU+AzOqu29pzLCK+893dvLQ=; b=PN0Xw+gEITOJWp4eEE1LIkOtrr1oAaxu+ZJi+JGJWMGkKEzb6McuEau9VButlqBitQ8F6N Zg558SQTkljutj1834QDNsv6aQXCXBZ+Kf83HfUtbfzRTBNjXg3P7jG6EurccNLZhehLBr kTggSvmglRVAp3BqvmXJb7lTtXjH4DE= Received: from suse.cz (unknown [10.100.201.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 67BE72C141; Fri, 8 Jul 2022 13:41:51 +0000 (UTC) Date: Fri, 8 Jul 2022 15:41:47 +0200 From: Michal Hocko To: Alexei Starovoitov Cc: Matthew Wilcox , Christoph Hellwig , davem@davemloft.net, daniel@iogearbox.net, andrii@kernel.org, tj@kernel.org, kafai@fb.com, bpf@vger.kernel.org, kernel-team@fb.com, linux-mm@kvack.org, Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Vlastimil Babka Subject: Re: [PATCH bpf-next 0/5] bpf: BPF specific memory allocator. Message-ID: References: <20220623003230.37497-1-alexei.starovoitov@gmail.com> <20220706175034.y4hw5gfbswxya36z@MacBook-Pro-3.local> <20220706180525.ozkxnbifgd4vzxym@MacBook-Pro-3.local.dhcp.thefacebook.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20220706180525.ozkxnbifgd4vzxym@MacBook-Pro-3.local.dhcp.thefacebook.com> ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1657287714; a=rsa-sha256; cv=none; b=Ndw31Gbb2doPitLOg0DNlvt9c0WQJZCkVRjuFPx5pYt9MoKjBUL65oQk6F8yqVaP7zKabr MVK+SNMpv3khDYdmdXAssi1/+88SZOmB6m2MRrmiCmL+eI1fXBgvi6rmOnZr3TvjSIqcoB QIFsqhVbdbdAgGnFc5jXvrkk5TVs3ks= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=PN0Xw+gE; spf=pass (imf01.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657287714; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yrGC8i4SCMF9Esn1IzCdcU+AzOqu29pzLCK+893dvLQ=; b=V/V0ubKd+aHO1gdmo95jibL4rhlSbraSkMgwcJLoKlh1NnHU9SU/vo4CuraE9n5S1asXWq im977lYcZy5LneGV0nROQFfQpkLBzxEctpT5otMQ/UgOhHFH+DSNgvo8KSSa2dhJ0ktNVO fF4Nki8xfcKSw0OPtTijvq0SL2cwpxQ= X-Stat-Signature: zricx5mssm9g3d67djzn51j5fftepkoq X-Rspamd-Queue-Id: 1EC914001C Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=PN0Xw+gE; spf=pass (imf01.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1657287713-971242 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed 06-07-22 11:05:25, Alexei Starovoitov wrote: > On Wed, Jul 06, 2022 at 06:55:36PM +0100, Matthew Wilcox wrote: [...] > > For example, I assume that a BPF program > > has a fairly tight limit on how much memory it can cause to be allocated. > > Right? > > No. It's constrained by memcg limits only. It can allocate gigabytes. I have very briefly had a look at the core allocator parts (please note that my understanding of BPF is really close to zero so I might be missing a lot of implicit stuff). So by constrained by memcg you mean __GFP_ACCOUNT done from the allocation context (irq_work). The complete gfp mask is GFP_ATOMIC | __GFP_NOMEMALLOC | __GFP_NOWARN | __GFP_ACCOUNT which means this allocation is not allowed to sleep and GFP_ATOMIC implies __GFP_HIGH to say that access to memory reserves is allowed. Memcg charging code interprets this that the hard limit can be breached under assumption that these are rare and will be compensated in some way. The bulk allocator implemented here, however, doesn't reflect that and continues allocating as it sees a success so the breach of the limit is only bound by the number of objects to be allocated. If those can be really large then this is a clear problem and __GFP_HIGH usage is not really appropriate. Also, I do not see any tracking of the overall memory sitting in these pools and I think this would be really appropriate. As there doesn't seem to be any reclaim mechanism implemented this can hide quite some unreachable memory. Finally it is not really clear to what kind of entity is the life time of these caches bound to. Let's say the system goes OOM, is any process responsible for it and a clean up would be done if it gets killed? Thanks! -- Michal Hocko SUSE Labs