From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B976EC05027 for ; Wed, 8 Feb 2023 22:00:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4EBB06B0071; Wed, 8 Feb 2023 17:00:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 49B996B0072; Wed, 8 Feb 2023 17:00:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 363926B0074; Wed, 8 Feb 2023 17:00:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 2965F6B0071 for ; Wed, 8 Feb 2023 17:00:57 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id D87E9A0EF8 for ; Wed, 8 Feb 2023 22:00:56 +0000 (UTC) X-FDA: 80445495312.10.845FE8F Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf05.hostedemail.com (Postfix) with ESMTP id CC993100032 for ; Wed, 8 Feb 2023 22:00:54 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=XFi5Jdbh; spf=pass (imf05.hostedemail.com: domain of minchan.kim@gmail.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=minchan.kim@gmail.com; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675893654; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Fx0i8cxtjtRwzsSINFE91pP/WWK899fcc6eF3oZCP2A=; b=5mEFzFDNoicx42W37L7RyL9CxTrkVi0LZ3QERC8cxrjebrB6+XtreDJOKRG2OFsD6k4hCN GiVmBbKe/x8Rnb44Liwd5FSQvz+woUO17MK/oBbq6HepmcAucUdLv6fDcyf8mzuQVm/ccQ mhu230b/aWjIlYk1rq89LfFctTXi7ik= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=XFi5Jdbh; spf=pass (imf05.hostedemail.com: domain of minchan.kim@gmail.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=minchan.kim@gmail.com; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675893654; a=rsa-sha256; cv=none; b=ljZOkNlnMoNHZRE3y/J/z2QHT6PSmlDrluR5WUdO7LvSTyQEKTbNnnz8nVypBtfy8mEyxH BOjB/aRuRHTolWOp1OxYNF3n2Nbk/fGB3ZbmBPbXRmoRQr5dZ9QAxScGmIJ0laEArW7Ybl GB+oeHJYdxm5qL/9RfxoWmgBR+TaTQY= Received: by mail-pl1-f177.google.com with SMTP id g17so563731ply.11 for ; Wed, 08 Feb 2023 14:00:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:from:to:cc:subject:date:message-id :reply-to; bh=Fx0i8cxtjtRwzsSINFE91pP/WWK899fcc6eF3oZCP2A=; b=XFi5JdbhXqFD0z6v3p9TBIezSPPMnBzq+IpKkQ1PCztw001AQ6EMTF/zmdKiD1Oktn b/v6Vf1lclAMPE5dh55At030Ic9tso3/Y4e8VI2jbSvx6uV1rj2pz3OELwEe9R1dUD8q HqWJgNKVQxmDtzbTUq8VRHJmr3fZRz2QJBSti0Xt6BmBoy9SvbtQjoT30UakEP1gHH8L eMcAIkhdQx4KXZusdt/FQ8birkKqru4MvUZAb/7raiJcAxWIRjp+OuBVw/QNz2lfa93F JtAttgNGO8WjVKYuSSGB6qfqFNbZvcfqnV3yh4vsUoRi9919TOhPeLlmq0Re9f93M8h3 6lig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Fx0i8cxtjtRwzsSINFE91pP/WWK899fcc6eF3oZCP2A=; b=PRU2DdnvPy5jamkjtd16oP/PXOInlkfGRqCErcoRb4rVhK9WQIpkUo0seSoXIa6Dzg RS/N+MM3MTHG5ceZpJ74NNmoEuTRnWN0Qn1Isz/7PzKg6Dzuu8woCDRh4FAsps4JivGm vHVxuPwl6RPgH5GVXa1S12Hlxzxb1dnODp6/2+pJZq0PubApTkJ/CcqCdNFtWrt9bLjs dx61HkAtLBUxDS0FfD//eGIjTqNrWW5pJzMaoTtVE/f/DOTwn0YBEWwXEMUbgZJ8fkhv UsgcNqt/OJgvEAMHe/oZ76nc2YMpQ4uHDdQlcFzrO4Vi7RiHGCsoPo3qREroH1u/4wah UsBQ== X-Gm-Message-State: AO0yUKXC/x4hqp/A+y7Yfn2RF6i5B3qCwswn7KKyQacUZ+EZmZtU0TxH kFrlqHYM0YrQh+1AiMQGg5w= X-Google-Smtp-Source: AK7set86TYpVcITgJXJm4s/OX9vpmb5LfCz010PfmJd7Uh/wCgxDc/XDm5BFHxmnF0Wn8wX5b+e00A== X-Received: by 2002:a17:902:e30a:b0:198:d8a4:1ece with SMTP id q10-20020a170902e30a00b00198d8a41ecemr6346283plc.8.1675893653517; Wed, 08 Feb 2023 14:00:53 -0800 (PST) Received: from google.com ([2620:15c:211:201:d4e2:8ad2:b455:fc1b]) by smtp.gmail.com with ESMTPSA id io23-20020a17090312d700b00198a845fbbasm1754051plb.84.2023.02.08.14.00.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Feb 2023 14:00:52 -0800 (PST) Date: Wed, 8 Feb 2023 14:00:50 -0800 From: Minchan Kim To: Chris Goldsworthy Cc: Roman Gushchin , Sukadev Bhattiprolu , Andrew Morton , Rik van Riel , Roman Gushchin , Vlastimil Babka , Joonsoo Kim , Georgi Djakov , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH] mm,page_alloc,cma: configurable CMA utilization Message-ID: References: <20230131071052.GB19285@hu-sbhattip-lv.qualcomm.com> <20230131201001.GA8585@hu-sbhattip-lv.qualcomm.com> <20230201040628.GA3767@hu-cgoldswo-sd.qualcomm.com> <20230206052139.GA21897@hu-cgoldswo-sd.qualcomm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230206052139.GA21897@hu-cgoldswo-sd.qualcomm.com> X-Rspamd-Queue-Id: CC993100032 X-Stat-Signature: r6n7eouwrr7z39tgattssw1gb487ygpz X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1675893654-446747 X-HE-Meta: U2FsdGVkX19wJ6bEwC3/+Jr1vM6ghfiUTCEJKAyd/fFM0N9lnPg0RljwYy++OfUwfzMj517JKaxT/88FHyUFF8PIuO7PHT55nqd3HBoIwadJrz2HeG7wF6c+khFS0o1IO+AJZTpg4P6gKF4YboGLRyzgCPdHnjWqJtT5MidfWRjydPy/WJUfmAp4dSsTGRJ8CVBWim0Tp5+rDJ6e4cLb7ufFtUczUUwNWAqjCAEo75OjIRZ418x4hmm0es2tG3tjnI1MjAubX3eZzJcFjtTTIMY/fxZzzR8WomVpw3najPAUVOxnZlfvtWUZMrx16GPltFY5Mbt66EpflhJ6weS0nbSk/r+2V8f4/PGImbytIK6ZXGU1TD7uuV3sCUunTWBKezKIbkWGJhQ9C0Lt5Dm+lGb8WLbhAhxisyFpdyXoB4gipq8IWmLpLhIg9QXZDY8I+K8A1UNHzDApuJt4vuF2Pf0+xXve2Q8e+knrn1zEiH3Q2I4E4OdyOdUafBCwi8bZ8uRUxW3dLaFgdc23d1weZIbg337zTsi9YDNLEE8rzqCfHQIKDmB/VcDFghSw2gMFhPwgf+VxEWiR4klYPQY1KSbPCXVgODFdAgA+dWofNyJSAf4f044ARRAamNAqvbgYh2EbAgRhwstsb5mJWLyXJ2EvKJrLrD0z7pqlObRBRS94R2lPvHcgF+S/6LSh2kvSOGSJChAeh9XRLvYvVZxXKpUFhzegU4/9F4JgqDOcsId+1jvIExJxFDny4FbmdoTtPDvL6MAbFalvOP8JFx0EJ1Vt/EwifEHl7nkGM1DbTIuiRaDKw7Ndp2sLi8GCF+pW5i1ECZjTsHgiKmxv7ugQfEBGt51OsDM5agGF0ihXaKdz+PoAq7IadXX8TlOgpT3Z8dPpsavItIdR7kFJdLxdVLFNm1PkwXt4oWLciaiAsaKd6lHuWNf41JhiwEIEe6viVFmxDtjLiHUdOFFXemJ GUllKiQ5 wyTmPf3H1KcxK0jaeluR+7lAG/5lJT8awtVNt2e8Jx3j/jaIW23VwcjpnGvbufrjds9/Z5vA+slfaI08YJj7OZuaOYwwkwUYxgIvSDq4Q4VuueerpQr8Oy+FQbGCyXiWkaCHn1TvigGxyBaZrAXJEufEjnh+PfSMsjDhCAQUePgESVBndhOhY7fCswAarenqMdfIGwQKEP0hhHRR/hDAn+WKcdw6nHNM0tAqpZp5KUVEIbw3yLp5DzWOkDOjdenRatl26aTCIw73YWZieU1DtysAq+3bib/uzGmjWQNSCFdsabyY5ElnfVw+aReiGOQY+8zT86ty2cbHltN86XUp58PQSCz+yhp/RWlAS+ET2ZciskofDRCaGhW+VnzbeA2liUEH6gbhMO5NkA2hdqVQhS7VCjOC6le0ov+QG6jhD55pp1WY9z7Od8MVKcZFfIA+LUIIpbho3hIpZ6DYGbLa4+ONtH4FRCgwpcwhdE2CfXXJSjvpdDWW78thJdc7tF8sMeWeyDGfElGYrmX9S1Ktv0ZNNVeBeu9TtO9xjEiPpvPDerrYr9MxyT8anYse7S4xbqBgXaCIc+/87n/3bXExTzYfpSrkd3acu67S+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.061435, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sun, Feb 05, 2023 at 09:22:28PM -0800, Chris Goldsworthy wrote: > On Wed, Feb 01, 2023 at 03:47:58PM -0800, Minchan Kim wrote: > > Hi Chris, > > > > On Tue, Jan 31, 2023 at 08:06:28PM -0800, Chris Goldsworthy wrote: > > > We're operating in a resource constrained environment, and we want to maximize > > > the amount of memory free / headroom for GFP_KERNEL allocations on our SoCs, > > > which are especially important for DMA allocations that use an IOMMU. We need a > > > large amount of CMA on our SoCs for various reasons (e.g. for devices not > > > upstream of an IOMMU), but whilst that CMA memory is not in use, we want to > > > route all GFP_MOVABLE allocations to the CMA regions, which will free up memory > > > for GFP_KERNEL allocations. > > > > I like this patch for different reason but for the specific problem you > > mentioned, How about making reclaimer/compaction aware of the problem: > > > > IOW, when the GFP_KERNEL/DMA allocation happens but not enough memory > > in the zones, let's migrates movable pages in those zones into CMA > > area/movable zone if they are plenty of free memory. > > > > I guess you considered but did you observe some problems? > > Hi Minchan, > > This is not an approach we've considered. If you have a high-level idea of the > key parts of vmscan.c you'd need to touch to implement this, could you point me > to them? I think the problem is not specific with CMA but also movable zone. If movable pages are charged into non-movable zones, the problem wil happen. So what I suggested was if reclaimers(e.g., background/direct reclaimers) found the request was GFP_KERNEL but there are not enough free pages in the zone and lower zones but has movable pages in there, migrate them into the CMA area and/or movable zones to make room for the GFP_KERNEL allocation before the final failure. It needs touch wakeup_kswapd/kcompactd to trigger the migration and reclaim/compaction needs to deal with the commmand. I couldn't say where are good places to change until I look at further details but I thought it's more general solution. > > I guess one drawback with this approach is that as soon as kswapd starts, > psi_memstall_enter() is called, which can eventually lead to LMKD running in > user space, which we want to minimize. One aim of what we're doing this is to > delay the calling of psi_memstall_enter(). LMKD running would be not a problem, I think but you are worry about LMKD decide killing apps due to wrong signal? I think it's orthgonal issue. Actually, it's long time problem for userspace memory manager since they don't know where the memory pressure comes from with what constrains. This is the GFP_KERNEL constraint but LMKD can kill apps which consumes much memory for movable zones or CMA area so cannot help the memory pressure. Furthermore, LMKD has bunch of knobs to affect decision to kill apps. PSI is just event to wake up LMKD, not decision policy. > > It would be beneficial though on top of our change: if someone called > cma_alloc() and migrated out of the CMA regions, changing kswapd to behave like > this would move things back into the CMA regions after cma_release() is called > (instead of having to kill a user space process to have the CMA re-utilized upon > further user space actions). > > Thanks, > > Chris.