From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 308E9C05027 for ; Wed, 1 Feb 2023 19:00:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8BAC86B0072; Wed, 1 Feb 2023 14:00:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 86A286B0073; Wed, 1 Feb 2023 14:00:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 759726B0074; Wed, 1 Feb 2023 14:00:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 66EA16B0072 for ; Wed, 1 Feb 2023 14:00:41 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 0FDA0120721 for ; Wed, 1 Feb 2023 19:00:41 +0000 (UTC) X-FDA: 80419639482.19.E0E4636 Received: from out-85.mta0.migadu.com (out-85.mta0.migadu.com [91.218.175.85]) by imf17.hostedemail.com (Postfix) with ESMTP id D379A4002B for ; Wed, 1 Feb 2023 19:00:38 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=dgUfIgYx; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf17.hostedemail.com: domain of roman.gushchin@linux.dev designates 91.218.175.85 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675278039; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nYaJBwWv04VyrNu+X4EgPv0Xb15I2c3WvPYD3xQxlig=; b=1SEjs5cuPkNVqXJjyEs94FgDa/bLqsFEZFydWqdRrECSrRYcAHxW+plT/b/9hiHA1ZkAnc Ml3kE62KeZq7WGqyDilDkDvspt+NEz02tqsL0A7s+oyN64n3jDpDdyhgX3vOji123wZkMq 8rZaY7cbcqxMPFjN2/KlTt67uYcZERA= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=dgUfIgYx; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf17.hostedemail.com: domain of roman.gushchin@linux.dev designates 91.218.175.85 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675278039; a=rsa-sha256; cv=none; b=cjGU7kWhzrVFlCC5hg/zRITCzQYGv731Rh9vuvrAG2eRhKM+c2AqF5jCI/oGvqFxq9k24e ZXuQ5P5/Z4XFTVOgpmFYYfWUyqPlnk9nN/CnRuIoso+mdwamSjNN01GKtc0wHBttVJdY/m mM7O9X6w027tLfRf/4BtvejvlwH5RJM= Date: Wed, 1 Feb 2023 11:00:25 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1675278035; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=nYaJBwWv04VyrNu+X4EgPv0Xb15I2c3WvPYD3xQxlig=; b=dgUfIgYxZt9wZbnGKSDK2uiNaZZ/5zUS5bhvJ5luv/cRV8MxqT28jSeyHUkeRbs/VPvSZT QvWHYE9Dj1WHn0PdlnWgWFBx1C7x/fVWfKqcuVcq5es654OVipyrBV1RVeVOn51zpxvWv4 o22/GhZRvsJkMFHf6ifH2R6v4EqvoJA= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Roman Gushchin To: Chris Goldsworthy , Rik van Riel Cc: Sukadev Bhattiprolu , Andrew Morton , Rik van Riel , Roman Gushchin , Vlastimil Babka , Joonsoo Kim , Minchan Kim , Georgi Djakov , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH] mm,page_alloc,cma: configurable CMA utilization Message-ID: References: <20230131071052.GB19285@hu-sbhattip-lv.qualcomm.com> <20230131201001.GA8585@hu-sbhattip-lv.qualcomm.com> <20230201040628.GA3767@hu-cgoldswo-sd.qualcomm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230201040628.GA3767@hu-cgoldswo-sd.qualcomm.com> X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: D379A4002B X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: 4rnhzkhso8ne8mpk6cqsxxy4pdzcqm8c X-HE-Tag: 1675278038-510168 X-HE-Meta: U2FsdGVkX18syIviBoSotkUK28dftpMR/CW9cPXO40GMKGdL5epYbULR/msF89Kk2h7D0pCMS+M31rUghOxwsqNuE5do8cAJbDr6QmriBeytkQx4d2RaJR0mWMsops1bEnTQrc1ctJZAAlo+F0Aaf1oOH1r76lwIeytN4BGWh6eQybdm2YQBSIgC4G6j57R8kAAayTmtsguSa29j49hTbIYM+JYoKmacfLWmnEGyHV8lCavoYQooczCgfLZmW1HXH/Mvn1Z2WmuxUSU3ayCb9nzh59TzpA+KyQURE+lO/R+1OPk5WutmvDHgJrE9Stzju6C0SUNSZDtX2mjaxGg+rjpqN+GMyWhVHnOQKYVgU/XCVE/JZzn4CtnM+d1kElnCgmHX2HYKymTEpzkl1VvI7/+ZyTxcQooS1ImvPbk7LpLq71hlCEG/xWYe3oNHcNYbpOQg/hRo2/1nc+dJzVEsikGo+Nld8gTJ4l/M+KzcDsI+CyNRQAVN6GI/YjUECKylePbb0bg3Y08tn85+M0/LT3rgUo5qbfabeUSxv9m6K7MgzRXXQOSwzYPQqEd/AyTtSYz45vXsomzxFsqqpcHnwqPRTwxQPPF9SEcMmS9o2AXhqhoRg6gdCujhzRr60bb9DdeJKXOK0PkIbxgP7YIF7LDOWWO4v8ecc2QYxTPdTJ3UoXHt9f2oKDzFrD6vuLuIrrDZ6u7TN07WjnE9L3bBScRWEzpPk/sZUuZhQ6mdp1hpMHwhp7QIemImIH2BEsycvizbu8af7WNwuM5fe3NY9y92WmnUWUVa5/La3FOl4zvIuQFzbHVWUudqvnGABtr2iUh2zyoXJ58Gh64LahcYayP3nmanVLfXp8Q9R6izqdJm5AnouNJ0/hB3oGkTaB107yyKc22qhpx6rZ0S6fr/FWkKd4fGL6AtD4+qB5pl/WRDeRJUJ7MIEn/8oLEk142hEv+WXonJS+9OV6/1d04 81BxGz4q YSwltsp9OPFnSt3LMga98vOKdI9HpOUKMSReIzW8kzu1GUZIGrgTUN16lKXtArvDxkVouPM7zUYIiE03yaUpKgw67mBzKY5ooxvR6pZZz3CO1WvH7beezaN3olY5aZZa/hG9lCvwuJpolPVrASU5+PhFhSnPY7XwMss6KFMBlSgsRxyxxu/xoolOZN1ZIfGduTKjR X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jan 31, 2023 at 08:06:28PM -0800, Chris Goldsworthy wrote: > On Tue, Jan 31, 2023 at 03:59:36PM -0800, Roman Gushchin wrote: > > On Tue, Jan 31, 2023 at 12:10:01PM -0800, Sukadev Bhattiprolu wrote: > > > On Tue, Jan 31, 2023 at 10:10:40AM -0800, Roman Gushchin wrote: > > > > Hi Sukadev! > > > > > > > > Can you, please, share a bit more details about your setup? E.g. what is > > > > the zone size, the cma area size and the value you want to set your sysctl to? > > > > > > Hi Roman, > > > > > > I currently have a device with 8GB Zone normal and 600MB of CMA. We have a > > > slightly different implementation and use up all the available CMA region. > > > i.e. going forward, we intend to set the ratio to 100 or even higher. > > > Hi Roman, > > > It means you want allocations be always served from a cma region first? > > Exactly. > > > What's the point of it? > > We're operating in a resource constrained environment, and we want to maximize > the amount of memory free / headroom for GFP_KERNEL allocations on our SoCs, > which are especially important for DMA allocations that use an IOMMU. We need a > large amount of CMA on our SoCs for various reasons (e.g. for devices not > upstream of an IOMMU), but whilst that CMA memory is not in use, we want to > route all GFP_MOVABLE allocations to the CMA regions, which will free up memory > for GFP_KERNEL allocations. > > > The idea behind the current formula is to keep cma regions free if there is > > a plenty of other free memory, otherwise treat it on par with other memory. > > With the current approach, if we have a large amount of movable memory allocated > that has not gone into the CMA regions yet, and a DMA use case starts that > causes the above condition to be met, we would head towards OOM conditions when > we otherwise could have delayed this with this change. > Note that since we're > working on Android, there is a daemon built on top of PSI called LMKD that will > start killing things under memory pressure (before an OOM is actually reached) > in order to free up memory. This patch should then reduce kills accordingly for > a better user experience by keeping a larger set of background apps alive. When > a CMA allocation does occur and pages get migrated out, there is a similar > reduction in headroom (you probably already know this and know of the FB > equivalent made by Johannes Weiner). I see... Thank you for the explanation! So the problem is that movable allocations are spread between cma and non-cma evenly, so that non-movable allocations might fail. And the idea is to use the cma area more actively for movable allocations to keep a headroom for non-movable allocations. Is it correct? Then _maybe_ a new knob is justified, at least I don't have better ideas. Rik, do you have any input here? Let's then define it in a more generic way and _maybe_ move to the cma sysfs/debugfs (not 100% sure about this part, but probably worth exploring). Thanks!