From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39661C54EBC for ; Tue, 10 Jan 2023 19:08:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AE1688E0003; Tue, 10 Jan 2023 14:08:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A91908E0002; Tue, 10 Jan 2023 14:08:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9596C8E0003; Tue, 10 Jan 2023 14:08:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 816248E0002 for ; Tue, 10 Jan 2023 14:08:19 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 455E7161EFB for ; Tue, 10 Jan 2023 19:08:19 +0000 (UTC) X-FDA: 80339825118.17.F9DFB21 Received: from mail-yw1-f172.google.com (mail-yw1-f172.google.com [209.85.128.172]) by imf12.hostedemail.com (Postfix) with ESMTP id 9CC834003F for ; Tue, 10 Jan 2023 19:08:16 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=kNneCXnU; spf=pass (imf12.hostedemail.com: domain of tjmercier@google.com designates 209.85.128.172 as permitted sender) smtp.mailfrom=tjmercier@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1673377696; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eB6zDLr3jptFznjCY+cJ6LV+lGxSv6mlgWXUb4GRVLE=; b=FociQPV2fPiTYzkvWP6OkBJIRByl7HZCvwuBeCRFCIZZcwwC/twNnfoYxq4WS3m/7GU6UW 8jnme/7oByVYs04gIV4Q/5cPKzgLlhlnUblhcE+9olys9BHX7PzzceJq62UiivGttGcwsQ l2gWIrVB5Ekf75iJgY0UlTG//U6XaBA= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=kNneCXnU; spf=pass (imf12.hostedemail.com: domain of tjmercier@google.com designates 209.85.128.172 as permitted sender) smtp.mailfrom=tjmercier@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1673377696; a=rsa-sha256; cv=none; b=7YtnSpu42HbrsJu4ofsGGh1amYZjdp5skuvsttrcOFHWqlHI6m858+aJ2Q/XIl/mOWI/W3 XyhNNMjrj0GpHSCv7qIeOcmki2zv0pymxoGrsbStZJ1zZUTB6GIXqyBxzSNXplEY0/9FFJ D7UXM/7qirwYhRjaPd7ZC9bR5OsivzY= Received: by mail-yw1-f172.google.com with SMTP id 00721157ae682-4d0f843c417so52773327b3.7 for ; Tue, 10 Jan 2023 11:08:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=eB6zDLr3jptFznjCY+cJ6LV+lGxSv6mlgWXUb4GRVLE=; b=kNneCXnUbGtaezFZly8EELiFvrMI8uCxra4hUNp5F8h55KzRcraX6HTcL/eoX2slxU 5eIut2BW8V2KxE4ECFGrtM+afRxG/KTeduZVHs5S8WSaTnrTAKUpQ9LGjuGWxJrrW+Rb HL3W/mk5/ZWXlu153IFY6bWd5ku/c5DQ1P/WEPBZYa8NWqLe/FC6eZ+nVcj+NfNmapAJ hhVfZ0STKoz+rrAswTAQZmVDSFu+39MDK4KoXl6hbj290qVSa10/hwUiXT3+ysS6+4M+ Fn8gtLHQRlRNe7nlS5w5XbzHPW/6rq2M3KUaxPGuT+9WamrHs6La624Sedcyvl8v1GSa pxXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=eB6zDLr3jptFznjCY+cJ6LV+lGxSv6mlgWXUb4GRVLE=; b=beJX2z+jthU7KP+Y8X74g3c07DhMwYmFi7DcFvJJDaVIK4MtdQWxPEu9uEYAMmAIeS 0cHgLZYGlJGqbuDuntdpOgQ8JIyGJX9qyht8jSc0TbZYiluIPR90FVHLQlnv080BzTCn VA6EASKgvLWcEm1uvZlih5FSCN/bWpnYBKl4g4j/eoZIZFt2DtQ9subDLwcKL34D8+ZL Y7nOVCfCZNTak64GmiqV+Hsj3mWKkf6QpplP3f41NAzBXlfEaA7L6Y865cz63P9pRj18 Bch4Zw3bsuQakgdEJM40YbneIL5e2S1fcLoEeQARm8lz85KKja1OYxfEJElH6zRasm2c rw0g== X-Gm-Message-State: AFqh2kruCPKQzB3PhZgroSlZFNA8kxva3g+jbn6fdzpKAmOmXvyQqWqy 4QHGp6kpAUDAkb5R4wtQptp9uubd3Rr8osR8tk50Gw== X-Google-Smtp-Source: AMrXdXuWt8GL2cvwM/11siCYrRBKZLbeyNUoNJAwEvUjJQicN0t0jFrxPGHJ56q+FKvR1iOeW+XS0IBv7xTbyCMXxzs= X-Received: by 2002:a05:690c:b88:b0:3e3:866c:a51b with SMTP id ck8-20020a05690c0b8800b003e3866ca51bmr218703ywb.439.1673377695560; Tue, 10 Jan 2023 11:08:15 -0800 (PST) MIME-Version: 1.0 References: <20230109213809.418135-1-tjmercier@google.com> <20230109213809.418135-2-tjmercier@google.com> In-Reply-To: From: "T.J. Mercier" Date: Tue, 10 Jan 2023 11:08:04 -0800 Message-ID: Subject: Re: [PATCH 1/4] memcg: Track exported dma-buffers To: Michal Hocko Cc: Tejun Heo , Zefan Li , Johannes Weiner , Jonathan Corbet , Sumit Semwal , =?UTF-8?Q?Christian_K=C3=B6nig?= , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , daniel.vetter@ffwll.ch, android-mm@google.com, jstultz@google.com, cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: 3k59jps9ipzsn7x74gnkui7tzxmzokmk X-Rspam-User: X-Rspamd-Queue-Id: 9CC834003F X-Rspamd-Server: rspam06 X-HE-Tag: 1673377696-63926 X-HE-Meta: U2FsdGVkX1/1k2TmKzTIgzrqT9jucFq8xnAeLielPnUjEVe7oOykupp12OdUIzlte4/U8UrEA6csPzuw5hcKZmWLLseAx7Eq31kt/yQJ6fliYi6k04LyKya2TS4w79VVEfAbm6lZzB17iUA4TA+DrCUbiZ8S1H/eL361WpjyEwMyNeYq9DbJQIsZmf3lODdGmE7sIox2r6R0Y+Nb5aqNSE3kJefDlPfYqzMHxMeqdZL59TQTfs4KP90RdKUVS+bDJVQyifA/ZTivH0fua1TeG7JHtYnru2dBZCzf1M8oPRabipGpOtwd0wAq4K4w2LRGcc4rSEqQ6icQA4PLJq92Q5Mqmojtju6L0QAmLt6vv/F7BOHDldr88+PCwFKDZ9pfBcDEU8yInfCaKgkm1nr5QuWR1bjUXOGAsHYQB6LF/61fx7NhkTc5xxKqxTIARXPHcADCOeaNK7gx02l9OuwvXJ1bNuNL6RwDpkPfRu1S1e5Hp7fu+LnXdzr51nqGjMjlISnnX5hyzvpnaMnMYA122O/EnYEflRmlipToassAujNelarX/qqXsGITQ+OVKq+fWmHf5DwBs6eiYadrbn++D7kLSTcYVTyXS2RP9SVLywffNKPlbPPq30HK9N+F8VvowgfWkILED5RLMz2/l2bMeN8JXVL/jDJVcDCFfXTBy/xdNYaQV0XPUh9tClXBQKaECYqmmAeBUzrQyJBmdIA6GuPI1AdfbKuuoeQ7fARZUGt25u/DJsUOX78kQcfsu4ybXMG8XLLgoQvs1bMogh+CVwzaAXuSC/fHdpDkoIdBqNOK1R49OAXN/7vkjJxwoqa+/ZSB77V4n35zCtNX6XgS/xWuVxmQKyzWmxz1eU0ElVFJDl76sLQewO4TL3rtk/QTeVnyfCZh7cGCTc4RGc+lSmdWNKYyUuglKY/C2Vfd99MZjWeThMTF5mU8bmFq14zgnk612xfYJuD0jX60oQG GYZ/fGk0 nzSivY4Ogk/L06r0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jan 10, 2023 at 12:58 AM Michal Hocko wrote: > > On Mon 09-01-23 21:38:04, T.J. Mercier wrote: > > When a buffer is exported to userspace, use memcg to attribute the > > buffer to the allocating cgroup until all buffer references are > > released. > > > > Unlike the dmabuf sysfs stats implementation, this memcg accounting > > avoids contention over the kernfs_rwsem incurred when creating or > > removing nodes. > > I am not familiar with dmabuf infrastructure so please bear with me. > AFAIU this patch adds a dmabuf specific counter to find out the amount > of dmabuf memory used. But I do not see any actual charging implemented > for that memory. > > I have looked at two random users of dma_buf_export cma_heap_allocate > and it allocates pages to back the dmabuf (AFAIU) by cma_alloc > which doesn't account to memcg, system_heap_allocate uses > alloc_largest_available which relies on order_flags which doesn't seem > to ever use __GFP_ACCOUNT. > > This would mean that the counter doesn't represent any actual memory > reflected in the overall memory consumption of a memcg. I believe this > is rather unexpected and confusing behavior. While some counters > overlap and their sum would exceed the charged memory we do not have any > that doesn't correspond to any memory (at least not for non-root memcgs). > > -- > Michal Hocko > SUSE Labs Thank you, that behavior is not intentional. I'm not looking at the overall memcg charge yet otherwise I would have noticed this. I think I understand what's needed for the charging part, but Shakeel mentioned some additional work for "reclaim, OOM and charge context and failure cases" on the cover letter which I need to look into.