From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6DA8CCA47B for ; Fri, 10 Jun 2022 10:58:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1AC226B00B8; Fri, 10 Jun 2022 06:58:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 15C2B6B00BA; Fri, 10 Jun 2022 06:58:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0248F8D0087; Fri, 10 Jun 2022 06:58:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E6A066B00B8 for ; Fri, 10 Jun 2022 06:58:58 -0400 (EDT) Received: from smtpin31.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id C354A1201CE for ; Fri, 10 Jun 2022 10:58:58 +0000 (UTC) X-FDA: 79562028756.31.D1CBC84 Received: from mail-ed1-f50.google.com (mail-ed1-f50.google.com [209.85.208.50]) by imf08.hostedemail.com (Postfix) with ESMTP id 41C3A160074 for ; Fri, 10 Jun 2022 10:58:58 +0000 (UTC) Received: by mail-ed1-f50.google.com with SMTP id g7so470346eda.3 for ; Fri, 10 Jun 2022 03:58:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:date:mime-version:user-agent:subject:content-language:to :cc:references:from:in-reply-to:content-transfer-encoding; bh=w6F5upqNbqcjOOBJM+5BezuuZ8DymaMUCrQuFA3GOnA=; b=YZZv8NAd4bFAUuTNFuwIlDcmKTp86LFMvcog24DagojMbNLE5Se1DmFMVmJ87uRvVn DHg3DsyZl5P8aCUmB2QRs6OBqSILElVCVRD7bc68R64iLg9kYb342KPpCwWXWO184NBc HQcHyAdUhV8Q0WhynDc+yQ4/sZ7PQcZevq6VBL5SzqV6KYGeklmIRe38n9bvYY5Q1+JK WhZSTnmlqR3Jo6PN7avr1sSEwFMtRIB63k6sjGq6yDCPtUFoSFNusdQKAtNJh/AlfSLS +4BnHWFH5oLHuBp1MpFR5IFuUXStqITHPSeYQd5mB00paW4tZfj70WmiPHpMoU2dylt+ VQsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=w6F5upqNbqcjOOBJM+5BezuuZ8DymaMUCrQuFA3GOnA=; b=BBn7HrnQChe+bNSSjzrXilU2g/X+L45RgNvxSM4FBp86eTV64OdWuCBE4j2GR0EFG0 SlJkmSiaPzPz+a2akFF+NZDstuhxoVCXhWBJStYOkPSOuEFTa2wnwoK0Vv63/OD9J8IZ eImkR7nKseogDa6OSdh7EO7ZkfqjZfcnYesAxNW1sEqsO5FuqbgUGJVzqr4iNYGDlkUw qrWy/J0zR9pcx74JxpYWPOs3LQdHNjRLHocj75XZb7i9uq9F75nNYNMht9KUNKyW+HEE W+L134AuEhtCojiW2Arj0G76FBH72pP6VeapEy51+6lO6g8MnT69L+c0p/CIeUjJCO9J eAUA== X-Gm-Message-State: AOAM5323UmQwNBn/9VMe4UzAHbfQaF1bb2Vu1e9E87IWmlnC5YI2takM IHT/a8ANspyIzi+sIaTPKQ0= X-Google-Smtp-Source: ABdhPJwRYA/7eDJK0xNPvbtH+OzO5lQ4tB0JbSJQSmwOmyloSbsX0o7RtK6TGGmCWnVQA/XG1FkVRA== X-Received: by 2002:a05:6402:368a:b0:42d:ef42:f727 with SMTP id ej10-20020a056402368a00b0042def42f727mr50676107edb.204.1654858736837; Fri, 10 Jun 2022 03:58:56 -0700 (PDT) Received: from [192.168.178.21] (p5b0ea02f.dip0.t-ipconnect.de. [91.14.160.47]) by smtp.gmail.com with ESMTPSA id fi20-20020a1709073ad400b006fec8e8eff6sm12055062ejc.176.2022.06.10.03.58.54 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 10 Jun 2022 03:58:55 -0700 (PDT) Message-ID: <3f7d3d96-0858-fb6d-07a3-4c18964f888e@gmail.com> Date: Fri, 10 Jun 2022 12:58:53 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.9.1 Subject: Re: [PATCH 03/13] mm: shmem: provide oom badness for shmem files Content-Language: en-US To: Michal Hocko , =?UTF-8?Q?Christian_K=c3=b6nig?= Cc: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, intel-gfx@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-tegra@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, alexander.deucher@amd.com, daniel@ffwll.ch, viro@zeniv.linux.org.uk, akpm@linux-foundation.org, hughd@google.com, andrey.grodzovsky@amd.com References: <20220531100007.174649-1-christian.koenig@amd.com> <20220531100007.174649-4-christian.koenig@amd.com> <77b99722-fc13-e5c5-c9be-7d4f3830859c@amd.com> <26d3e1c7-d73c-cc95-54ef-58b2c9055f0c@gmail.com> From: =?UTF-8?Q?Christian_K=c3=b6nig?= In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1654858738; a=rsa-sha256; cv=none; b=skncYaUrA3jLTZNVXhibxbZx97DRp1e5NDNr6oT0xerw7NQ0BnSpDSuDOnamsPUSGabIgQ WfkHXHVqpF+l4Vsnam/077LGMk01wtlAFfSIktLmQzP2rsxPECtDdRL/kKwYUn6epRvaCB X2NZRV0zM82ZhF9lTm3KBZ2mdv5JW10= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=YZZv8NAd; spf=pass (imf08.hostedemail.com: domain of ckoenig.leichtzumerken@gmail.com designates 209.85.208.50 as permitted sender) smtp.mailfrom=ckoenig.leichtzumerken@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1654858738; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=w6F5upqNbqcjOOBJM+5BezuuZ8DymaMUCrQuFA3GOnA=; b=OEvq7ez7vXHrebdubrpugnWvX06BDVr03JSitF2PqObaEuHk5QYsKKAdk5+H6PUdsR9Zyj UFNpYP/K8yPODR87+9Q7oHb0TA8sdRgvz9iI+Hvldszfi7RvFzXoWdjjMrUOTi/IC2tkLZ xScgntKoyaOUXb5cD5u9ou1PzlMLaeo= X-Rspamd-Queue-Id: 41C3A160074 Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=YZZv8NAd; spf=pass (imf08.hostedemail.com: domain of ckoenig.leichtzumerken@gmail.com designates 209.85.208.50 as permitted sender) smtp.mailfrom=ckoenig.leichtzumerken@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Rspamd-Server: rspam06 X-Stat-Signature: gj1r8n6ypaqr5ugc1nr49465s681iwcz X-HE-Tag: 1654858738-178936 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Am 09.06.22 um 17:07 schrieb Michal Hocko: > On Thu 09-06-22 16:29:46, Christian König wrote: > [...] >> Is that a show stopper? How should we address this? > This is a hard problem to deal with and I am not sure this simple > solution is really a good fit. Not only because of the memcg side of > things. I have my doubts that sparse files handling is ok as well. Well I didn't claimed that this would be easy, we juts need to start somewhere. Regarding the sparse file handling, how about using file->f_mapping->nrpages as badness for shmem files? That should give us the real number of pages allocated through this shmem file and gracefully handles sparse files. > I do realize this is a long term problem and there is a demand for some > solution at least. I am not sure how to deal with shared resources > myself. The best approximation I can come up with is to limit the scope > of the damage into a memcg context. One idea I was playing with (but > never convinced myself it is really a worth) is to allow a new mode of > the oom victim selection for the global oom event. It would be an opt in > and the victim would be selected from the biggest leaf memcg (or kill > the whole memcg if it has group_oom configured. > > That would address at least some of the accounting issue because charges > are better tracked than per process memory consumption. It is a crude > and ugly hack and it doesn't solve the underlying problem as shared > resources are not guaranteed to be freed when processes die but maybe it > would be just slightly better than the existing scheme which is clearly > lacking behind existing userspace. Well, what is so bad at the approach of giving each process holding a reference to some shared memory it's equal amount of badness even when the processes belong to different memory control groups? If you really think that this would be a hard problem for upstreaming we could as well keep the behavior for memcg as it is for now. We would just need to adjust the paramters to oom_badness() a bit. Regards, Christian.