From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7273DC3271E for ; Mon, 8 Jul 2024 21:34:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CFD1B6B0098; Mon, 8 Jul 2024 17:34:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CD4216B0099; Mon, 8 Jul 2024 17:34:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC32B6B009A; Mon, 8 Jul 2024 17:34:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A072D6B0098 for ; Mon, 8 Jul 2024 17:34:58 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 5FA281614D0 for ; Mon, 8 Jul 2024 21:34:58 +0000 (UTC) X-FDA: 82317890676.25.63547E0 Received: from mail-ua1-f48.google.com (mail-ua1-f48.google.com [209.85.222.48]) by imf20.hostedemail.com (Postfix) with ESMTP id 75EA01C000F for ; Mon, 8 Jul 2024 21:34:55 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.48 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=kernel.org (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720474465; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GFhnV5KJxxo4N1WnwlDGnMUQsfNE/JI9ypR7AtiPbkU=; b=7gjT5qHr7ps0oO7a/8skisQ12g038IIwXEzcwqTNge8Ftx+VmE2eSpHj3wZ9vfNhZsXFNG +p9sL98vzRJRL3OfwEds27z5pNcnqxOwpHhDFxUXGai4soF8zOTeTvWzG1Lix5fjGBq9hH hGKgtREuAALMkqIApISMxmQjqtHuP6U= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720474465; a=rsa-sha256; cv=none; b=cGBKRgyFRC+HDbOIDurPqYXnlW/eWGEEYgMtUCCnnpO+coTp9huGEZodL155b87JlJw/kJ qx7MHfVJnJTmAWqV8JNvUPy4kpn9L9ntjUiSCcwzIHO54MV9wO6p14V3Kq5OzMaCw6janR JI9u2yCjgpFyWoVrXk7wPb9uxiZzTqI= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.48 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=kernel.org (policy=none) Received: by mail-ua1-f48.google.com with SMTP id a1e0cc1a2514c-80fa1509989so2393007241.0 for ; Mon, 08 Jul 2024 14:34:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720474494; x=1721079294; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GFhnV5KJxxo4N1WnwlDGnMUQsfNE/JI9ypR7AtiPbkU=; b=kqw6kNmi6bgltEhmEHtjgM0+Dr6Sm2fpv0yMbDDSPhgPhFr6qflYmSUa3fB0I+rB94 xC2fCVQbIIB8RAIvZnZWKEU+XCbdoWNFMl/Ssfb6C6WUz5aglDM7/0hPDUNhf8DoQCUT 5TGfE5zVSRJq3sZWTChy+P4R2I2JpAqriqKUEh4adVENfML3gLUXqXCg7pErnA+oH+sV g8QRa/fgoUq9KnPMS0YQolbR6aA6cNkw2jUJtxSAw+dUfInHLJltQNXH4Vlu1LWUtsvE cbUVy1Cr2xpzMN7B4Y9oJMWAdMuSgkQg7UB7foFqbYoxMLuCJByW1J9+nKL1Z0HOIrw7 WqNw== X-Forwarded-Encrypted: i=1; AJvYcCXI5d+Zs32YxIWbuZH5gDETLqmfnAPV0YdxlrY90tDIUg2cVmjvjHEijtw//8y4xAf0dvkGWhgcWC5zQ/nadNrknrQ= X-Gm-Message-State: AOJu0YyWFPI0cTzb8P7UZsZHu3FsB5tcEBFVXs9tQUGEfU+fv1uDh0X3 o57JWn+VC2imJ33xTfaNJXBqQ0bLg/xchf+oyqEnxfiH3lc4v0f1ymZEtj37ur2IPDWjMdvogtD kpaXmfpWnQ1CIX7ChDbDFsaCEM6c= X-Google-Smtp-Source: AGHT+IEpXynBb6aFguiwD0qTNstWNUzPyQU1RvKkv9S1j86ZSg61lCV0xMV3MiqmtkAzC6wdK+YdLTD3e2hWx1PrQkU= X-Received: by 2002:a05:6102:4b83:b0:48f:b5c1:7269 with SMTP id ada2fe7eead31-49032ff4e42mr358919137.0.1720474494331; Mon, 08 Jul 2024 14:34:54 -0700 (PDT) MIME-Version: 1.0 References: <20240708090413.888-1-justinjiang@vivo.com> <122bbe20-f2ab-4d44-97ac-76df8469d3fa@vivo.com> <4ca9836c-4f03-4792-9be8-f7db71a2b162@vivo.com> In-Reply-To: From: Barry Song Date: Tue, 9 Jul 2024 09:34:43 +1200 Message-ID: Subject: Re: [PATCH v5] mm: shrink skip folio mapped by an exiting process To: zhiguojiang Cc: Andrew Morton , linux-mm@kvack.org, LKML , opensource.kernel@vivo.com, David Hildenbrand , Matthew Wilcox Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 75EA01C000F X-Stat-Signature: mh8ywj4anhnafbcha98tcdwdumny3mjy X-HE-Tag: 1720474495-393081 X-HE-Meta: U2FsdGVkX1+BmZQFX4tObLgpwwlESqKH2C4FZ1nT7Tm/ClQznYxu3iiQc5QOjQz9hQcpxwvPI+4nFzvYMsROxlrJ4Z/sMhFS3zrhMgI+HHUqZpYQL23sPr5Q2hJ6joU/YPUPEMMb7QSFKT4OMX+aNTZJ5TwIcXOvpwAv+tRINQIygj0g7oaPuGnVPDMzQkodagK82CWTlzHACcb4mX+r0eBwvNEmZNg94J/QqJ6/WyOw5MLKY07xvNeFRQo1MFJq57cQxT4LD1TcLQz75knOA1lRn7v7CDJt/qBKfRPEYxc+XREHjjw3aMKuRKaKamqsgFp4ZYru2BW1xNXpMh3zYBCgOTNqqS9mMAjRimNFVYhuHrS2R4P3eLsCdnCxI9wslOO5EaY5M+ucJM5pN6pHz7z023kCYd9PllOderspS19rIPKYAkkLTagoYShg+042Dj/CGv1j938+uLoJIEpuyRuRXb2YrdF5BSpJfw2JPVKVjQ71/ce+NlHJShB9VAwtDZ0uWzHI67+BJWad1yonPC/fJQLTDYHS59G0Mb4bc6U7f0HdefS97iOkoSZGQBSmdkjYhuXsY8nZwlBccO7Lud4qhxwpJrT062CnKfC31it3Bh9w80Ssudx5OtOi/6141lL/XmbYj+YcMITEUBkob/OK+ivHfn377qOmySX7s9XuJmvUfZvdKY5dfQWrndTPYUocv1noYqiLJOPqnncqyxxony2zLyAQaoumV2Pb54bTdyrNyVG3T901cTuLWrWEYmeQdJFKBxOSOjv89hOOzx5FdlPgEzseBXXrrxjEro1BxrSzW11h8k+EDFb0sOD0yPauVeWqBT7AJOvh7I7X2bLynWddHmsJFXDIk0pSZ279qZfMvrOHi952wodTrL3umD+HqD0xAFKvz4EPnezBqwgp5UeseBNKhaisLD9rsSu7J58WzI6WTygBd+X837FS2Y0avUj5M7C60HbY9nt bBiqZ0E/ yoi1Czx2eRSadGc0U1AUUXmjNGtro0qh+OwB45eorsV64Q9EmPMG3ACpauZ/RAOot80tGsio6ybFyBLPOb4KNr0vTm6SOlBw5GFBOcUerTVTmAH0ijVkTMuDGRCmkfabJM6ZPQ9gu0qgsMxrxr0Z/we9lHUU6cxrpEOscFjQKGTTjbKdSZL8UjOJ6lE1TiT2hqLImbH5C3+icMR8a121g+VWPV1f80889DmwWlFqPzYI7qV4sZEvh15fseZ2Hx8RYH+CTfDJYfV1Zv4QSX554GmiiwywEVyCHO4387ff81wYPgwc+6zuAvHkMlFPNiv0oVKj3/y3OzabPqgHdCee8XQpC4Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Jul 9, 2024 at 1:11=E2=80=AFAM zhiguojiang w= rote: > > > > =E5=9C=A8 2024/7/8 20:41, Barry Song =E5=86=99=E9=81=93: > > > > > > zhiguojiang =E4=BA=8E 2024=E5=B9=B47=E6=9C=889= =E6=97=A5=E5=91=A8=E4=BA=8C 00:25=E5=86=99=E9=81=93=EF=BC=9A > > > > > > > > =E5=9C=A8 2024/7/8 20:17, zhiguojiang =E5=86=99=E9=81=93: > > > > > > > > > =E5=9C=A8 2024/7/8 19:02, Barry Song =E5=86=99=E9=81=93: > > >> On Mon, Jul 8, 2024 at 9:04=E2=80=AFPM Zhiguo Jiang > > >> wrote: > > >>> The releasing process of the non-shared anonymous folio mapped > > >>> solely by > > >>> an exiting process may go through two flows: 1) the anonymous > > folio is > > >>> firstly is swaped-out into swapspace and transformed into a > > swp_entry > > >>> in shrink_folio_list; 2) then the swp_entry is released in the > > process > > >>> exiting flow. This will increase the cpu load of releasing a > > non-shared > > >>> anonymous folio mapped solely by an exiting process, because > > the folio > > >>> go through swap-out and the releasing the swapspace and swp_ent= ry. > > >>> > > >>> When system is low memory, it is more likely to occur, because > > more > > >>> backend applidatuions will be killed. > > >>> > > >>> The modification is that shrink skips the non-shared anonymous > > folio > > >>> solely mapped by an exting process and the folio is only releas= ed > > >>> directly in the process exiting flow, which will save swap-out > > time > > >>> and alleviate the load of the process exiting. > > >>> > > >>> Signed-off-by: Zhiguo Jiang > > >>> --- > > >>> > > >>> Change log: > > >>> v4->v5: > > >>> 1.Modify to skip non-shared anonymous folio only. > > >>> 2.Update comments for pra->referenced =3D -1. > > >>> v3->v4: > > >>> 1.Modify that the unshared folios mapped only in exiting task > > are skip. > > >>> v2->v3: > > >>> Nothing. > > >>> v1->v2: > > >>> 1.The VM_EXITING added in v1 patch is removed, because it will > > fail > > >>> to compile in 32-bit system. > > >>> > > >>> mm/rmap.c | 13 +++++++++++++ > > >>> mm/vmscan.c | 7 ++++++- > > >>> 2 files changed, 19 insertions(+), 1 deletion(-) > > >>> > > >>> diff --git a/mm/rmap.c b/mm/rmap.c > > >>> index 26806b49a86f..5b5281d71dbb > > >>> --- a/mm/rmap.c > > >>> +++ b/mm/rmap.c > > >>> @@ -843,6 +843,19 @@ static bool folio_referenced_one(struct > > folio > > >>> *folio, > > >>> int referenced =3D 0; > > >>> unsigned long start =3D address, ptes =3D 0; > > >>> > > >>> + /* > > >>> + * Skip the non-shared anonymous folio mapped solely by > > >>> + * the single exiting process, and release it directly > > >>> + * in the process exiting. > > >>> + */ > > >>> + if ((!atomic_read(&vma->vm_mm->mm_users) || > > >>> + test_bit(MMF_OOM_SKIP, &vma->vm_mm->flags)) && > > >>> + folio_test_anon(folio) && > > >>> folio_test_swapbacked(folio) && > > >>> + !folio_likely_mapped_shared(folio)) { > > >>> + pra->referenced =3D -1; > > >>> + return false; > > >>> + } > > >>> + > > >>> while (page_vma_mapped_walk(&pvmw)) { > > >>> address =3D pvmw.address; > > > Sure, I agree with your modification suggestions. This way, > > using PTL > > > indeed sure > > > that the folio is mapped by this process. > > > Thanks > > >> As David suggested, what about the below? > > >> > > >> @@ -883,6 +870,21 @@ static bool folio_referenced_one(struct fol= io > > >> *folio, > > >> continue; > > >> } > > >> > > >> + /* > > >> + * Skip the non-shared anonymous folio mapped > > solely by > > >> + * the single exiting process, and release it > > directly > > >> + * in the process exiting. > > >> + */ > > >> + if ((!atomic_read(&vma->vm_mm->mm_users) || > > >> + test_bit(MMF_OOM_SKIP, > > >> &vma->vm_mm->flags)) && > > >> + folio_test_anon(folio) && > > >> folio_test_swapbacked(folio) && > > >> + !folio_likely_mapped_shared(folio)) { > > >> + pra->referenced =3D -1; > > >> + page_vma_mapped_walk_done(&pvmw); > > >> + return false; > > >> + } > > >> + > > >> if (pvmw.pte) { > > >> if (lru_gen_enabled() && > > >> pte_young(ptep_get(pvmw.pte))) { > > >> > > >> > > >> By the way, I am not convinced that using test_bit(MMF_OOM_SKIP, > > >> &vma->vm_mm->flags) is > > >> correct (I think it is wrong). For example, global_init can > > >> directly have it: > > >> if (is_global_init(p)) { > > >> can_oom_reap =3D false; > > >> set_bit(MMF_OOM_SKIP, &mm->flags); > > >> pr_info("oom killer %d (%s) has mm > > pinned by > > >> %d (%s)\n", > > >> task_pid_nr(victim), > > >> victim->comm, > > >> task_pid_nr(p), p->comm); > > >> continue; > > >> } > > >> > > >> And exit_mmap() automatically has MMF_OOM_SKIP. > > >> > > >> What is the purpose of this check? Is there a better way to > > determine > > >> if a process is an > > >> OOM target? What about check_stable_address_space() ? > > > 1.Sorry, I overlook the situation with if (is_global_init(p)), > > > MMF_OOM_SKIP is indeed not suitable. > > > > > > 2.check_stable_address_space() can indicate oom_reaper, but it > > seems > > > unable to identify the situation where the process exits normally= . > > > What about task_is_dying()? static inline bool > > task_is_dying(void) { > > > return tsk_is_oom_victim(current) || > > fatal_signal_pending(current) || > > > (current->flags & PF_EXITING); } Thanks > > We can migrate task_is_dying() from mm/memcontrol.c to > > include/linux/oom.h > > > static inline bool task_is_dying(void) > > > { > > > return tsk_is_oom_victim(current) || > > fatal_signal_pending(current) || > > > (current->flags & PF_EXITING); > > > } > > > > > > no. current is kswapd. > Hi Barry, > > It seems feasible for check_stable_address_space() replacing MMF_OOM_SKIP= . > check_stable_address_space() can indicate oom kill, and > !atomic_read(&vma->vm_mm->mm_users) > can indicate the normal process exiting. > > /* > * Skip the non-shared anonymous folio mapped solely by > * the single exiting process, and release it directly > * in the process exiting. > */ > if ((!atomic_read(&vma->vm_mm->mm_users) || > check_stable_address_space(vma->vm_mm)) && > folio_test_anon(folio) && folio_test_swapbacked(folio) && > !folio_likely_mapped_shared(folio)) { > pra->referenced =3D -1; > page_vma_mapped_walk_done(&pvmw); > return false; > } > Yes, + David, Willy (when you send a new version, please CC people who have participated and describe how you have addressed comments from those people.) I also think we actually can remove "folio_test_anon(folio)". So It could be, @@ -883,6 +871,21 @@ static bool folio_referenced_one(struct folio *folio, continue; } + /* + * Skip the non-shared swapbacked folio mapped solely by + * the exiting or OOM-reaped process. This avoids redundant + * swap-out followed by an immediate unmap. + */ + if ((!atomic_read(&vma->vm_mm->mm_users) || + check_stable_address_space(vma->vm_mm)) && + folio_test_swapbacked(folio) && + !folio_likely_mapped_shared(folio)) { + pra->referenced =3D -1; + page_vma_mapped_walk_done(&pvmw); + return false; + } + if (pvmw.pte) { if (lru_gen_enabled() && pte_young(ptep_get(pvmw.pte))) { > Thanks > Zhiguo > > > > > > >> > > >> > > >>> diff --git a/mm/vmscan.c b/mm/vmscan.c > > >>> index 0761f91b407f..bae7a8bf6b3d > > >>> --- a/mm/vmscan.c > > >>> +++ b/mm/vmscan.c > > >>> @@ -863,7 +863,12 @@ static enum folio_references > > >>> folio_check_references(struct folio *folio, > > >>> if (vm_flags & VM_LOCKED) > > >>> return FOLIOREF_ACTIVATE; > > >>> > > >>> - /* rmap lock contention: rotate */ > > >>> + /* > > >>> + * There are two cases to consider. > > >>> + * 1) Rmap lock contention: rotate. > > >>> + * 2) Skip the non-shared anonymous folio mapped solely= by > > >>> + * the single exiting process. > > >>> + */ > > >>> if (referenced_ptes =3D=3D -1) > > >>> return FOLIOREF_KEEP; > > >>> > > >>> -- > > >>> 2.39.0 > > >>> > > >> Thanks > > >> Barry > > > > > > Thanks Barry