From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31C79C4167B for ; Wed, 29 Nov 2023 03:14:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C7EA36B0394; Tue, 28 Nov 2023 22:14:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C2F596B0396; Tue, 28 Nov 2023 22:14:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF71B6B0397; Tue, 28 Nov 2023 22:14:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A085C6B0394 for ; Tue, 28 Nov 2023 22:14:18 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 508E51C0311 for ; Wed, 29 Nov 2023 03:14:18 +0000 (UTC) X-FDA: 81509523396.16.15A87B6 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.100]) by imf05.hostedemail.com (Postfix) with ESMTP id EC486100020 for ; Wed, 29 Nov 2023 03:14:15 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=McPzxdOo; spf=pass (imf05.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.100 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701227656; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kl/CXikQKFVbW61BR1C9N0Z7iWtCmYqW8FDjY1Ua0v0=; b=62SzBNdDuVrJmrL9yMUU99TRebdMXH+yo0+TZQaMeP2qKCHioHuDKebZtqcmVJdQf5BG9r HqMUA32MbYWuAEYMd7hdWNATgGCe+IZBHqYExx3FP5SwSGqNdSGvdVQwi9xxFXgxfGxsdR FazG2ecUz+W7WENv3rdzg2BzNizMncU= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=McPzxdOo; spf=pass (imf05.hostedemail.com: domain of ying.huang@intel.com designates 134.134.136.100 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701227656; a=rsa-sha256; cv=none; b=SFB6+PjpT8a9MzcMvo05WiBoAFwiXf1DA/bE9m0wTo92ci6Eb8Plfg+24uukJZQ0JeqxHN TfY3GCMesFU/ap9GfyFKffYBHLn7jybpSk/v9EI6V6cw1jgvZlxDQaFz9ob2s+4jgxd6g/ W0O08mTVPC6K/HWKYb4SFtc4NA+/KtM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1701227656; x=1732763656; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version:content-transfer-encoding; bh=f7evfONOTRdDT8DGaj9/5LbR0Nw4iuL7vChGwPlt1Ds=; b=McPzxdOotk5SYPKrX1DmaTTTo2UOZ0legK8/K4N0bpwQZpDXvCo4r4He BvJSgc+/2dgw3xUdT7DJet/8oSG/QrKzNQOxoSjoX07BY9vxz0iQEJXqv bJjG50evo51QEuHyOISbKGi+hyze/RPd2dqDYqnPIdxyhf0kUAJWpCABK VcApJjwdlgLDawai01m4l8twJgJSug/CY4nfBepjpZImdhWMU/aJAcA4Q Vl7hBA5epqHebpUjlBKSjHleMTFOyO+/ppYEmHv/O1W6JCubDZQS4X7Yi YcARyoyFVTp/iqZVWrj479a00v+KDQR/21juPEokrnyjE5u9AFQ5xF306 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10908"; a="459584460" X-IronPort-AV: E=Sophos;i="6.04,234,1695711600"; d="scan'208";a="459584460" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Nov 2023 19:14:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10908"; a="797771575" X-IronPort-AV: E=Sophos;i="6.04,234,1695711600"; d="scan'208";a="797771575" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Nov 2023 19:14:04 -0800 From: "Huang, Ying" To: Minchan Kim Cc: Yosry Ahmed , Chris Li , Michal Hocko , Liu Shixin , Yu Zhao , Andrew Morton , Sachin Sant , Johannes Weiner , Kefeng Wang , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v10] mm: vmscan: try to reclaim swapcache pages if no swap space In-Reply-To: (Minchan Kim's message of "Tue, 28 Nov 2023 14:37:28 -0800") References: <87msv58068.fsf@yhuang6-desk2.ccr.corp.intel.com> <87h6l77wl5.fsf@yhuang6-desk2.ccr.corp.intel.com> <87bkbf7gz6.fsf@yhuang6-desk2.ccr.corp.intel.com> <87msuy5zuv.fsf@yhuang6-desk2.ccr.corp.intel.com> Date: Wed, 29 Nov 2023 11:12:03 +0800 Message-ID: <87leah45j0.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: EC486100020 X-Rspam-User: X-Stat-Signature: zeqzpdysgsazmoo8ug6k8gtbd59sbaph X-Rspamd-Server: rspam01 X-HE-Tag: 1701227655-43467 X-HE-Meta: U2FsdGVkX1+tlQHmt8Gt+9YnNzESIUXETCF2MFO5+2WXcS4rIQjISB6StjsNbamUHpGs3IGw/NLQzhkA1jFCjdroHdU//Pwf9bXKWrJwTTkulZ6TuCTmVJZ2jc02op+vqLaiPHQODQiNgsbOaT6A7olWY1QrS/2A3/3XmFxCL1yicHtrdUehamHjU/kqSliVzi5zj9TTz52YgV4yNwJdH9tCVM2WOmaRqdL1yss2ES/OqHp2auewpJHoKxF+m+tbxJYzcsejcNqDrmsv2dC+n1CCNbYc1/uzdx4vzTD1y89h6oRvLAQrj/prOWqkTi/bAoPCAEGS+0s2NDMusy/vpF7WJ6UIsJZGaV+EfG3yPI3Ftpv563FfEEdys1MvCr9Yz8MeC9uOeOqFv7uo3799G5QNQ+IwDJhUblFPI/n6mq7N6jLsV4uTnDWCqwCs/QnecPhtQiYsQybR+Tt6TRWTsWeH2cGexRv33yFFrVaWxhrKAAFd0gv095j7cEjo9hSyht4vAY894U3/N29rDF8JNZWGDo6I8ZbsKG0y2st+6HhFsln92umB6fXQCEeaDUmx4luJW8LsJp67SjS0pjWOKGVisHrpCE0CtwehOfax52z6amx20MYcgRdHs/qmhK2GeCVC1/L5sN5zPlHKOcfHhlF9B2RKgge3H0aUw6FqkGFbJUGjsyW8ynkI4sWWrOAeC1iWpx/aPADfQAmKDFOalS7acpbsNgehhZWYr7of6FkAuevgKW8nH7V4WTjNoOH7Qe66j643ojgUJW4kouVkvUzQiIleHB9R3nn/FjEQIo4ce/65OYWTJlcGZJ75vW/x44E9WJxhUo3j+sz0im0OQYM/cHbeg5T18+qt3d0JeyEMjUbYFsQzbkCwgEmQwpLpM9WdDp4v2mKXvY4KOKcWuRGeG+ocJUJmlwzDd6fduWFPHkHlHNI9zVG/l44CuSYOlDzJujkhBKz+/6999SM CPKZDh8w OtA36UkL5HteD67kBSFgu6lDzPZzYVV1GQVrxT8W2lxm+iCETfnlW1x65313z+Qa7oda37zAJ06LCJfYvkgDjY8au5slnMHiHpix2Tl7DalivZuSqUQn9U5PxdRJvXSyg0BWfrMRZ8Cjwe2YIDfb5pI+UZWWyIePFxAWafQWmCW0TU1PbJc3qptSeB4R8oPOs7ShEVE3FqvWs4oo9lXK2SkCMZln7yS4UIH0gP8IMJU/Nsb7MdnHqVfUwSyRuimx7+/C8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Minchan Kim writes: > On Tue, Nov 28, 2023 at 11:19:20AM +0800, Huang, Ying wrote: >> Yosry Ahmed writes: >>=20 >> > On Mon, Nov 27, 2023 at 1:32=E2=80=AFPM Minchan Kim wrote: >> >> >> >> On Mon, Nov 27, 2023 at 12:22:59AM -0800, Chris Li wrote: >> >> > On Mon, Nov 27, 2023 at 12:14=E2=80=AFAM Huang, Ying wrote: >> >> > > > I agree with Ying that anonymous pages typically have differen= t page >> >> > > > access patterns than file pages, so we might want to treat them >> >> > > > differently to reclaim them effectively. >> >> > > > One random idea: >> >> > > > How about we put the anonymous page in a swap cache in a differ= ent LRU >> >> > > > than the rest of the anonymous pages. Then shrinking against th= ose >> >> > > > pages in the swap cache would be more effective.Instead of havi= ng >> >> > > > [anon, file] LRU, now we have [anon not in swap cache, anon in = swap >> >> > > > cache, file] LRU >> >> > > >> >> > > I don't think that it is necessary. The patch is only for a spec= ial use >> >> > > case. Where the swap device is used up while some pages are in s= wap >> >> > > cache. The patch will kill performance, but it is used to avoid = OOM >> >> > > only, not to improve performance. Per my understanding, we will = not use >> >> > > up swap device space in most cases. This may be true for ZRAM, b= ut will >> >> > > we keep pages in swap cache for long when we use ZRAM? >> >> > >> >> > I ask the question regarding how many pages can be freed by this pa= tch >> >> > in this email thread as well, but haven't got the answer from the >> >> > author yet. That is one important aspect to evaluate how valuable is >> >> > that patch. >> >> >> >> Exactly. Since swap cache has different life time with page cache, th= ey >> >> would be usually dropped when pages are unmapped(unless they are shar= ed >> >> with others but anon is usually exclusive private) so I wonder how mu= ch >> >> memory we can save. >> > >> > I think the point of this patch is not saving memory, but rather >> > avoiding an OOM condition that will happen if we have no swap space >> > left, but some pages left in the swap cache. Of course, the OOM >> > avoidance will come at the cost of extra work in reclaim to swap those >> > pages out. >> > >> > The only case where I think this might be harmful is if there's plenty >> > of pages to reclaim on the file LRU, and instead we opt to chase down >> > the few swap cache pages. So perhaps we can add a check to only set >> > sc->swapcache_only if the number of pages in the swap cache is more >> > than the number of pages on the file LRU or similar? Just make sure we >> > don't chase the swapcache pages down if there's plenty to scan on the >> > file LRU? >>=20 >> The swap cache pages can be divided to 3 groups. >>=20 >> - group 1: pages have been written out, at the tail of inactive LRU, but >> not reclaimed yet. >>=20 >> - group 2: pages have been written out, but were failed to be reclaimed >> (e.g., were accessed before reclaiming) >>=20 >> - group 3: pages have been swapped in, but were kept in swap cache. The >> pages may be in active LRU. >>=20 >> The main target of the original patch should be group 1. And the pages >> may be cheaper to reclaim than file pages. > > Yeah, that's common for asynchronous swap devices and that's popular. The= n, > How about freeing those memory as soon as the writeback is done instead of > keep adding more tricks to solve the issue? > > https://lkml.kernel.org/linux-mm/1368411048-3753-1-git-send-email-minchan= @kernel.org/ > > I remember it's under softIRQ context so there were some issues to change > locking rules for memcg and swap. And there was some concern to increase > softirq latency due to page freeing but both were not the main obstacle to > be fixed. Thanks for sharing. It's good to avoid to add the pages back to LRU, then isolate them from LRU. I have concerns that is it possible that too many pages are reclaimed? For example, to reclaim a small number of pages, too many pages were written to disk because the performance difference between CPU and storage. Originally, we still only reclaim requested number of pages although much more were written. But with the change, we may reclaim them all. >>=20 >> Group 2 are hard to be reclaimed if swap_count() isn't 0. > > "were accessed before reclaiming" would be rare. If page reclaiming algorithm works well enough, that should be true. >>=20 >> Group 3 should be reclaimed in theory, but the overhead may be high. >> And we may need to reclaim the swap entries instead of pages if the pages >> are hot. But we can start to reclaim the swap entries before the swap >> space is run out. > > I thought the swap-in path will reclaim the swap slots once it detects > swapspace wasn't enough(e.g., vm_swap_full or mem_cgroup_swap-full)? Yes. You are right. But before swap space wasn't enough, we may keep quite some pages in swap cache. But these pages may becomes hot later. Then we have no opportunity to reclaim these swap space. So, we may need to add some code to check this situation at appropriate places. For example, when we scan pages in active list, or activate pages in inactive list. >>=20 >> So, if we can count group 1, we may use that as indicator to scan anon >> pages. And we may add code to reclaim group 3 earlier. -- Best Regards, Huang, Ying