From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73F09C3DA7F for ; Wed, 31 Jul 2024 16:17:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EEF476B0085; Wed, 31 Jul 2024 12:17:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E9F8D6B0088; Wed, 31 Jul 2024 12:17:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D67876B0089; Wed, 31 Jul 2024 12:17:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id BAE1B6B0085 for ; Wed, 31 Jul 2024 12:17:25 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 3BDA11C18FC for ; Wed, 31 Jul 2024 16:17:25 +0000 (UTC) X-FDA: 82400552850.27.4CD3B8D Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf05.hostedemail.com (Postfix) with ESMTP id 8F77410001D for ; Wed, 31 Jul 2024 16:17:21 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Z1zHSZM4; spf=pass (imf05.hostedemail.com: domain of akpm@linux-foundation.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722442587; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tQbKPKRxOVIsa2+VBj52HNpH+buHDv4JWSuGX1FhOjc=; b=8FrvmqB0SMR66X10vF2m5ntkl4fNIR0zw5lCAe/mTAuWCo3D17G+7Pgj9TyeYTsEDh/VTN MiAL1ojkCxrOnoy343Zr+uoSWdz2o03ChfrH9kic+yIAT+6O30FWLEhXqFNlUKSQjIpt/i WdpjAtfxB9EF3C7FgLAgnfpsnsx9V38= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722442587; a=rsa-sha256; cv=none; b=bOJF9+PjSkbA1wy3yApJ/lSjH6Q8rh2iUMGk4M1Dj5E5bCaYyRXJw/jJkw001X9fGWAF5M zfgWVJMqYKBLninN4+DrSOvNySRS+Cy8soacCkODTXI7wdcP1EkoKXM1XDPmTVt+/JvuHl gM8k27EONYOGLP7Ca8Gj/D7JdKmNWoA= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Z1zHSZM4; spf=pass (imf05.hostedemail.com: domain of akpm@linux-foundation.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 9C088CE1723; Wed, 31 Jul 2024 16:17:17 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 34F2FC116B1; Wed, 31 Jul 2024 16:17:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1722442636; bh=o4Mg7vGFFDdh5GzpJdq3GqHcTrDwf5S/DobsTje7KZ4=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=Z1zHSZM48wVFFlJqF1StVZOLw9IG/5ID/2e4qXWkhfUgK1S62W74TkucFn3eixaa6 BBsstP1IUCZyQMPE/DAvSqZJq5diHwxXSiPTHLoH7Xcm7M6qIbfz1XbnT4p+Bgi84d C5Up9oY/oVPtVn4kfqnCL1+L4+URhAF2nXLQFQBk= Date: Wed, 31 Jul 2024 09:17:15 -0700 From: Andrew Morton To: Zhiguo Jiang Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Will Deacon , "Aneesh Kumar K.V" , Nick Piggin , Peter Zijlstra , Arnd Bergmann , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , linux-arch@vger.kernel.org, cgroups@vger.kernel.org, Barry Song <21cnbao@gmail.com>, kernel test robot , opensource.kernel@vivo.com Subject: Re: [PATCH v2 0/3] mm: tlb swap entries batch async release Message-Id: <20240731091715.b78969467c002fa3a120e034@linux-foundation.org> In-Reply-To: <20240731133318.527-1-justinjiang@vivo.com> References: <20240731133318.527-1-justinjiang@vivo.com> X-Mailer: Sylpheed 3.8.0beta1 (GTK+ 2.24.33; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 8F77410001D X-Stat-Signature: jc83qizosg4kzorpju1duqxajd1icm45 X-HE-Tag: 1722442641-533675 X-HE-Meta: U2FsdGVkX1/ZVbkww727aAUUaSzBwTpAXdAteYP5wYn6Nm4Tv/a2Rt4IUHTNeLrLCVW9hzBDBv0bBCemaGLUuXFVBCTuoj39g3Zv0uFYYgK/HDbk4tVvsPjysyqMqjN97IfPTsTRUPDfsUIUNuWabM5qrlEEXuzmDe++1ZY9Bb8eS1hVdv6k/jaRBmjzgBp/RE/9cw32v0g0ZJGgTrH4yIIUFKMNIZmFfARyHIzJbGsVT9yQwdyeWlpc7bX45e3RmNpz23SN8a5X56VkStkv0+5WnaHgzLZp63EHOg8cDg/qfMJKgTU45Y91CZtGuLmP/mrVDADyH+8OIvZIuchwL1uWZ/QdJBDctGpmwR2H5EBQZmXQ9GSNnE82lqVwi93GXFgu99b1FaM7iBcpKdTUItKhHPvqkL9Y/X/42cNFwjRf46sJoQUdCuRR6eN03l2QX5+SDwIRx+4sksBQ8voNyx13LdXEqtBfnXvlOzxPUsKYcTA97QwDp0Fbs0DDYGGVc56koIQ4bLEJ+gK704CzU4tTRBb5crV11GfVtnTJGXJfVrPHV3Gzh/Kq6ZW2WTK2nY2C8iu8x/Ow/DA3o6Tb9boK81sHlc0XlfFFF0n6TBm21crCxdIH/NrCZlY34DJ+yReDdcb7rY7XB7LywBnBwSi9XMs0zaaCJX+BAz+QewI/YhQtoi+yV0vm/TeFQMXNCxBW8CDJqvdoM+K7YKKtaXRfvXWTAgEwKqDa/PX+wjg8YxtwUkDBCMXIaF96Fe3Ep6iVk1mNtx2PpWDE3+jr+/ZfnWfHPbXBPA84p2cBYrMwa5xN54UqS3Xc2Su2AdwQmLPEmpXHvikOYN7eaVtTL3+td3/MY26MQsxW56gA6Yj8LfOCGJ832Ebe6/WoWIstwKbiTXnGfwjK/Ix9kYf2Jst7ErgkoGLWe2c/qUmskK12fK9tASaCPHnRUtkti5WAV/KgG0PateF8gmwjPjk bxf8PQv4 hlhXLSkQZQZcOESb3oE3ATX9trDhB9+J8C5ANbL2m5TW2i+6WLuCALaJIFxn+rXaIpS8ahtIjXVuurpX9BQ9PrXxVawnD+3VgldCxxq3gSh/+QTvDgau3B7ZJzqx9qLv2L0f7CECBYj9Z1l8uW4xr9UA5ndyO1N8q+SI2AMp5hcA1vmKiFNC0pfPAc+bQ5/Dl70OT1EwmCvn0GZ8R6RmXzsnMSyGHXxb/cYvQ9eLhptHYfhI+GpQV94hDzSafBFAyjjzutI8RUxMXqXaV9WC3dRMTjKpcV1BghaJw X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, 31 Jul 2024 21:33:14 +0800 Zhiguo Jiang wrote: > The main reasons for the prolonged exit of a background process is the The kernel really doesn't have a concept of a "background process". It's a userspace concept - perhaps "the parent process isn't waiting on this process via wait()". I assume here you're referring to an Android userspace concept? I expect that when Android "backgrounds" a process, it does lots of things to that process. Perhaps scheduling priority, perhaps alteration of various MM tunables, etc. So rather than referring to "backgrounding" it would be better to identify what tuning alterations are made to such processes to bring about this behavior. > time-consuming release of its swap entries. The proportion of swap memory > occupied by the background process increases with its duration in the > background, and after a period of time, this value can reach 60% or more. Again, what is it about the tuning of such processes which causes this behavior? > Additionally, the relatively lengthy path for releasing swap entries > further contributes to the longer time required for the background process > to release its swap entries. > > In the multiple background applications scenario, when launching a large > memory application such as a camera, system may enter a low memory state, > which will triggers the killing of multiple background processes at the > same time. Due to multiple exiting processes occupying multiple CPUs for > concurrent execution, the current foreground application's CPU resources > are tight and may cause issues such as lagging. > > To solve this problem, we have introduced the multiple exiting process > asynchronous swap memory release mechanism, which isolates and caches > swap entries occupied by multiple exit processes, and hands them over > to an asynchronous kworker to complete the release. This allows the > exiting processes to complete quickly and release CPU resources. We have > validated this modification on the products and achieved the expected > benefits. Dumb question: why can't this be done in userspace? The exiting process does fork/exit and lets the child do all this asynchronous freeing? > It offers several benefits: > 1. Alleviate the high system cpu load caused by multiple exiting > processes running simultaneously. > 2. Reduce lock competition in swap entry free path by an asynchronous > kworker instead of multiple exiting processes parallel execution. Why is lock contention reduced? The same amount of work needs to be done. > 3. Release memory occupied by exiting processes more efficiently. Probably it's slightly less efficient. There are potential problems with this approach of passing work to a kernel thread: - The process will exit while its resources are still allocated. But its parent process assumes those resources are now all freed and the parent process then proceeds to allocate resources. This results in a time period where peak resource consumption is higher than it was before such a change. - If all CPUs are running in userspace with realtime policy (SCHED_FIFO, for example) then the kworker thread will not run, indefinitely. - Work which should have been accounted to the exiting process will instead go unaccounted. So please fully address all these potential issues.