From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3EBACC47422 for ; Fri, 26 Jan 2024 10:57:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B7C846B0081; Fri, 26 Jan 2024 05:57:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B2C646B0089; Fri, 26 Jan 2024 05:57:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9CD3A6B008A; Fri, 26 Jan 2024 05:57:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 85A6D6B0081 for ; Fri, 26 Jan 2024 05:57:19 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 51F7E1C19A5 for ; Fri, 26 Jan 2024 10:57:19 +0000 (UTC) X-FDA: 81721160598.05.C25FF4D Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf03.hostedemail.com (Postfix) with ESMTP id 2EA6820008 for ; Fri, 26 Jan 2024 10:57:16 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=G8tAny3F; dkim=pass header.d=suse.com header.s=susede1 header.b=G8tAny3F; spf=pass (imf03.hostedemail.com: domain of mhocko@suse.com designates 195.135.223.131 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706266637; a=rsa-sha256; cv=none; b=NP3fxrIMoKdX58iNyU9vN0wmmx8g/G9bEId1yB1qQAUgsB2UYD9gPlsZJU7AUSDTLGhlch uxXyoIaulY+ZtAGdRh22YeoxvhfZ9q+kwUMtaSvK5h573KV7QAthBmXxwvJ3pa7n9VlJhn 6HST5z/52/wyEcYBaXXkQpR1kqA0w6s= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=G8tAny3F; dkim=pass header.d=suse.com header.s=susede1 header.b=G8tAny3F; spf=pass (imf03.hostedemail.com: domain of mhocko@suse.com designates 195.135.223.131 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706266637; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QJkySAAGmHdAQNOm7UkH9kJ80ptWT6pGsZVtoPtZ6kw=; b=72EckBOJtRuwiRh5UAZGxPBxkdt3/lTTVK5WKwwIumW2f86Ab2oKARkHdzdzmNoJuBgaDc 5IDyVu6TfvSqRilBP/MjYiTM91+TaraSmAKozAUPQ/fY9KCF38VpJfNi9KDS+qTEYKHrmq L8nxMiP2u3rwRB1/bgxw17zTR8+uBLU= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 7AF0C1FD9E; Fri, 26 Jan 2024 10:57:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1706266635; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QJkySAAGmHdAQNOm7UkH9kJ80ptWT6pGsZVtoPtZ6kw=; b=G8tAny3FgdJv1kzbj03ZHA1YNTXB9ES1Ve2yiSEpPnzpjtMMPvvxuB06VYIL4bwlcEyk3G 7InyOukpfF/LAHbjY5MPbAT09kbk+5YzofzgeJ8EGFWqZUb3E52N/U6G4Xx0zoCX0b7Z7d +8VpW19RrIewxJEZg2WaAyZqu64Qm0w= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1706266635; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QJkySAAGmHdAQNOm7UkH9kJ80ptWT6pGsZVtoPtZ6kw=; b=G8tAny3FgdJv1kzbj03ZHA1YNTXB9ES1Ve2yiSEpPnzpjtMMPvvxuB06VYIL4bwlcEyk3G 7InyOukpfF/LAHbjY5MPbAT09kbk+5YzofzgeJ8EGFWqZUb3E52N/U6G4Xx0zoCX0b7Z7d +8VpW19RrIewxJEZg2WaAyZqu64Qm0w= Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 72C9813A22; Fri, 26 Jan 2024 10:57:15 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id GpwEHAuQs2XVKgAAD6G6ig (envelope-from ); Fri, 26 Jan 2024 10:57:15 +0000 Date: Fri, 26 Jan 2024 11:57:15 +0100 From: Michal Hocko To: Zach O'Keefe Cc: Charan Teja Kalla , akpm@linux-foundation.org, mgorman@techsingularity.net, david@redhat.com, vbabka@suse.cz, hannes@cmpxchg.org, quic_pkondeti@quicinc.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Axel Rasmussen , Yosry Ahmed , David Rientjes Subject: Re: [PATCH V3 3/3] mm: page_alloc: drain pcp lists before oom kill Message-ID: References: <5c7f25f9-f86b-8e15-8603-e212b9911cac@quicinc.com> <342a8854-eef5-f68a-15e5-275de70e3f01@quicinc.com> <5adb12eb-8403-5860-28eb-5f6ab12f3c04@quicinc.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5adb12eb-8403-5860-28eb-5f6ab12f3c04@quicinc.com> X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 2EA6820008 X-Stat-Signature: xmicthwcydgra9i7h8xjyjc7h4y4duoj X-Rspam-User: X-HE-Tag: 1706266636-272454 X-HE-Meta: U2FsdGVkX18SWtPBJUB4b6GrvNm4ykWAVeQ33+bikVh/Ct+QVsCsKE8IAHKNGhlxd0cVzcb134mdxffQJkF0TpNp6BegS4lgdkcyGDKdJgzPCimq4PP0eJkn5jFrXAJXVaYD341mSysm75cuLipSFPackat71j+Y64dplwroqoM9uaqsQoPKZ1Pg5tT3QX3lZEaZS0jUnXiHoZQt8X19Uen/KqMu3rgWv4lV4fW2m6x0w9cTP4urnDcnZBg+0hlxDuucuzFHB8U80EWGAavZTPnC1iybLdikBzC5XGfzP1eumw4H3f3Z9MNlGPl9lNSvoMRaUnL3A64kxDAP2U2XaerPz60BF5499wmiiaRe8ROOzBqKwD+FnVqQDC76k4ZwzU05Ocsl9gHSxqgir4IVurbbUjUSVgc4NW0fRTd14QsR1ZTbrlwZY6Nrm85cEmEvC0JO59BXPS5yjVZCKIdKKYxXic4v/4q5FNlk4vpDmbWSlzHTN6H/Gghvre7juT54vhAGVMKC/m4lQYirAMUanp0WJcyD3qfdvB6hbEChRCPNUkQWfsXTZUN9Nw2IMNyzQJZedQWOvB8bb1f11QezClX86BNXSzCHt+OtNoZX0MpCFrhXA4CQ651AH4s2Y9/N2MlwxYjDoszoxEWi+RfLlo2y4tkhXgTLAnDhov7c/81aQuOt3y4WQjIbuM6osOd3UPOUIUbz8j34AX1T5gMrxAk514iMnoHjrB60HAMLCCueGjJPsD2hmpd9bhyqXvF+pg9tXlPYMewmAlFuiLDZvm6WbGJ4Izku6FLY4B1oNw1uf3eqEXKxrkfSOINFan2/hxFN4k5Xph2MNVxU6JCz62wANw1r+nh9HRYKxtcq3U1T1clgoeP6jnxcKrk2ae6uQN366F+4zYiLCyrgNyD9Lkr5bvcM0HyFLkUoHLzBqMsSEIbXddg7qQ+keSrglJgDjtAI6cWQMoGvhaVCxlE KcRSGiZ6 rK02TrLGr89JM2JesmyEqZARt7cUyUHZFQmA3lQImaxe/6lX6qvXRRB3QHQ8uQqXK+KHvNOI+AGgo9zIOLL/F5CIs8kXHuvPLXs8PS+gGxE1jrgFMubyLMs5Eo2n4LqiGKGgmuhd6uAgX41ECbAV7vvL8ShHX2OveknXhdEuAD0HOjy0w5qQrrCmRTRK785zwPolRt9Kz48tQfSCsjIRKJlPz/U28AfFsha6zwI9/DB39hO/fIM7K0LoHKrjflWVPPx6j80Fr2+0t48Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri 26-01-24 16:17:04, Charan Teja Kalla wrote: > Hi Michal/Zach, > > On 1/25/2024 10:06 PM, Zach O'Keefe wrote: > > Thanks for the patch, Charan, and thanks to Yosry for pointing me towards it. > > > > I took a look at data from our fleet, and there are many cases on > > high-cpu-count machines where we find multi-GiB worth of data sitting > > on pcpu free lists at the time of system oom-kill, when free memory > > for the relevant zones are below min watermarks. I.e. clear cases > > where this patch could have prevented OOM. > > > > This kind of issue scales with the number of cpus, so presumably this > > patch will only become increasingly valuable to both datacenters and > > desktops alike going forward. Can we revamp it as a standalone patch? Do you have any example OOM reports? There were recent changes to scale the pcp pages and it would be good to know whether they work reasonably well even under memory pressure. I am not objecting to the patch discussed here but it would be really good to understand the underlying problem and the scale of it. Thanks! -- Michal Hocko SUSE Labs