From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27C71ECAAD3 for ; Fri, 9 Sep 2022 11:33:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BA54A8D0006; Fri, 9 Sep 2022 07:32:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B54E88D0002; Fri, 9 Sep 2022 07:32:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A1CBC8D0006; Fri, 9 Sep 2022 07:32:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9053F8D0002 for ; Fri, 9 Sep 2022 07:32:59 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 664021613E8 for ; Fri, 9 Sep 2022 11:32:59 +0000 (UTC) X-FDA: 79892335278.07.8639DCA Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) by imf18.hostedemail.com (Postfix) with ESMTP id 034F71C0078 for ; Fri, 9 Sep 2022 11:32:58 +0000 (UTC) Received: by mail-ej1-f54.google.com with SMTP id go34so3311037ejc.2 for ; Fri, 09 Sep 2022 04:32:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date; bh=SZHsUiERmS1OiRclpw++WSMWCMLSWrTWxLeIYpyoO6Q=; b=AtOK6o3UiZ8x4mhjnDJ/pBpLJ1neGtLe4RE39J5d0HHYeXSt/LOPJzFHYk2zvuknsZ T3f/PJxyPN1HEmdG7CdvH/8VI4PwA8Uja31v9vOlzqb2Kf++b0LaC5tpi7MM/P3XyXxE NUa6u09UvQf7gvibLmyaa6BoIYwApwyDrvudeJfV7bWJ6948xUawLw7ZUp2Yxh56vVZb ogbwxtRwFTZy+f/G4glFWWrs/vEHHbbiA3qy/QbN7PCeAn5R96Sr82z02bZAkLAAjbov v/KJz6yLif0oDWy4Ghk97sGQraIKKC8GaF8Qxx+BskzQYvNjR4h4LbXWYZOLAvJ9ab8K 9jMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date; bh=SZHsUiERmS1OiRclpw++WSMWCMLSWrTWxLeIYpyoO6Q=; b=mD/qxT+7lZ78RFdV1Ekh9wV+rYdmpGg9E5ZU0l13cQRUsBR2O9H9kjcqRnjjzxEX/B lO7o8U1Z+nei5oA/89u/LIXotZT7k6+ymMAxkkjaf76G9J7ejq/YUQosN/rR/YNuYN8w rHNZ+KVMAmBwc9n6aq0K0d5JE40Yl1R+txcBI6JHApV38Apo6bRTNljyRiS7qapsdM/x xu/gB+JLBdQXB6FwJfS/hQBkkcwC8jzb0R33UWe1r3E9D0xoNKWzsN5syqU/SVPv1Kzi ebpsIQXkMXl94KM7qQd2cPvX7V6ZRst95qwjoahdoBsHcS51apHZj3va2H9T70ykGRTE 4Ltw== X-Gm-Message-State: ACgBeo2GfbKX3JpXLLYwOJD+CC97GeKueru8YOcVrhIusOFFdSBruKm2 vIRPzWlsfXSjzDxTn/24Z7w= X-Google-Smtp-Source: AA6agR4cLAreyhuyrrh8jrieDYpfhc8cSDUsUhnRs4sOP40THG/uswRlmdpSVWucNK4IrbedMFbCNA== X-Received: by 2002:a17:906:7954:b0:742:7a6:b179 with SMTP id l20-20020a170906795400b0074207a6b179mr9416830ejo.679.1662723177505; Fri, 09 Sep 2022 04:32:57 -0700 (PDT) Received: from ?IPV6:2a02:908:1256:79a0:2ad1:9592:69ea:f12b? ([2a02:908:1256:79a0:2ad1:9592:69ea:f12b]) by smtp.gmail.com with ESMTPSA id 17-20020a170906201100b0073dc5bb7c32sm163295ejo.64.2022.09.09.04.32.56 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 09 Sep 2022 04:32:56 -0700 (PDT) Message-ID: <068ab330-dbf7-8aa8-8bed-156ca01be48b@gmail.com> Date: Fri, 9 Sep 2022 13:32:57 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.11.0 Subject: Re: [RFC PATCH 0/5] GEM buffer memory tracking Content-Language: en-US To: Lucas Stach , linux-mm@kvack.org, dri-devel@lists.freedesktop.org Cc: Daniel Vetter , David Airlie , Andrew Morton , Michal Hocko , linux-fsdevel@vger.kernel.org, kernel@pengutronix.de References: <20220909111640.3789791-1-l.stach@pengutronix.de> From: =?UTF-8?Q?Christian_K=c3=b6nig?= In-Reply-To: <20220909111640.3789791-1-l.stach@pengutronix.de> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1662723179; a=rsa-sha256; cv=none; b=nv1sfM39XZrDSnSe/2gCiVTY/SB4uv3HZrVurIhNMRqPDTRpg+ZochtsnbxCSJyXiAYrqR UpkCwR00BLLKJxqq7b5eRzz2y32/MSw5pIeE15q8vCuiX2fLNfNp1N8pks3e/GoJ39FZMj FsVYtb2W4dXGwmAjsq7pkhlV3A9/pr0= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=AtOK6o3U; spf=pass (imf18.hostedemail.com: domain of ckoenig.leichtzumerken@gmail.com designates 209.85.218.54 as permitted sender) smtp.mailfrom=ckoenig.leichtzumerken@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1662723179; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SZHsUiERmS1OiRclpw++WSMWCMLSWrTWxLeIYpyoO6Q=; b=pQL7oBvECh56VwPwfwUBbfkhyp0c47/x+PQJIT9EDVVkY4NQS91RUxtY/XuzBFrTUbHzZN bxvHi6iQZfqtjrqLHTB3EMjsHvrG6qvH2ebXiMQyAn5Beegwl7NXA1ywu+5YMBsO7Eri9K tKi3eMpBXqdkhDTOg2mj1OxYw0+cHK4= X-Stat-Signature: bq6o3o79q4exdezrq9xpqee6kk6tpy7c X-Rspamd-Queue-Id: 034F71C0078 X-Rspamd-Server: rspam11 X-Rspam-User: Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=AtOK6o3U; spf=pass (imf18.hostedemail.com: domain of ckoenig.leichtzumerken@gmail.com designates 209.85.218.54 as permitted sender) smtp.mailfrom=ckoenig.leichtzumerken@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1662723178-18967 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Am 09.09.22 um 13:16 schrieb Lucas Stach: > Hi MM and DRM people, > > during the discussions about per-file OOM badness [1] it repeatedly came up > that it should be possible to simply track the DRM GEM memory usage by some > new MM counters. > > The basic problem statement is as follows: in the DRM subsystem drivers can > allocate buffer aka. GEM objects on behalf of a userspace process. In many > cases those buffers behave just like anonymous memory, but they may be used > only by the devices driven by the DRM drivers. As the buffers can be quite > large (multi-MB is the norm, rather than the exception) userspace will not > map/fault them into the process address space when it doesn't need access to > the content of the buffers. Thus the memory used by those buffers is not > accounted to any process and evades visibility by the usual userspace tools > and the OOM handling. > > This series tries to remedy this situation by making such memory visible > by accounting it exclusively to the process that created the GEM object. > For now it only hooks up the tracking to the CMA helpers and the etnaviv > drivers, which was enough for me to prove the concept and see it actually > working, other drivers could follow if the proposal sounds sane. > > Known shortcomings of this very simplistic implementation: > > 1. GEM objects can be shared between processes by exporting/importing them > as dma-bufs. When they are shared between multiple processes, killing the > process that got the memory accounted will not actually free the memory, as > the object is kept alive by the sharing process. > > 2. It currently only accounts the full size of them GEM object, more advanced > devices/drivers may only sparsely populate the backing storage of the object > as needed. This could be solved by having more granular accounting. > > I would like to invite everyone to poke holes into this proposal to see if > this might get us on the right trajectory to finally track GEM memory usage > or if it (again) falls short and doesn't satisfy the requirements we have > for graphics memory tracking. Good to see other looking into this problem as well since I didn't had time for it recently. I've tried this approach as well, but was quickly shot down by the forking behavior of the core kernel. The problem is that the MM counters get copied over to child processes and because of that become imbalanced when this child process now terminates. What you could do is to change the forking behavior for MM_DRIVERPAGES so that it always stays with the process which has initially allocated the memory and never leaks to children. Apart from that I suggest to rename it since the shmemfd and a few other implementations have pretty much the same problem. Regards, Christian. > > Regards, > Lucas > > [1] https://lore.kernel.org/linux-mm/20220531100007.174649-1-christian.koenig@amd.com/ > > Lucas Stach (5): > mm: add MM_DRIVERPAGES > drm/gem: track mm struct of allocating process in gem object > drm/gem: add functions to account GEM object memory usage > drm/cma-helper: account memory used by CMA GEM objects > drm/etnaviv: account memory used by GEM buffers > > drivers/gpu/drm/drm_gem.c | 42 +++++++++++++++++++++++++++ > drivers/gpu/drm/drm_gem_cma_helper.c | 4 +++ > drivers/gpu/drm/etnaviv/etnaviv_gem.c | 3 ++ > fs/proc/task_mmu.c | 6 ++-- > include/drm/drm_gem.h | 15 ++++++++++ > include/linux/mm.h | 3 +- > include/linux/mm_types_task.h | 1 + > kernel/fork.c | 1 + > 8 files changed, 72 insertions(+), 3 deletions(-) >