From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4B7FC3DA5D for ; Mon, 22 Jul 2024 21:58:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2FFFC6B0089; Mon, 22 Jul 2024 17:58:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2AFF16B008A; Mon, 22 Jul 2024 17:58:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 19E046B008C; Mon, 22 Jul 2024 17:58:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id EDD956B0089 for ; Mon, 22 Jul 2024 17:58:02 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 6FA6C14179A for ; Mon, 22 Jul 2024 21:58:02 +0000 (UTC) X-FDA: 82368752004.18.24E0B50 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) by imf05.hostedemail.com (Postfix) with ESMTP id A460710000B for ; Mon, 22 Jul 2024 21:58:00 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=rxC3rDEO; spf=pass (imf05.hostedemail.com: domain of rientjes@google.com designates 209.85.214.181 as permitted sender) smtp.mailfrom=rientjes@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721685457; a=rsa-sha256; cv=none; b=wgcocHPYLhgPDddT4Vy4USHPc2FT2gGxEHkdnFTmFAivkjoYRW1pUWe8cSATJ6xOXGRYqv bXH9JRMBLSG2oj0mxdYIQQ0nwpain+Ay1PdBbHSRLMzsxZVv636aI2utXenpfhRf6C43D8 v5J0c0JY2NkAyE3moA9+CgyumjRJ7CY= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=rxC3rDEO; spf=pass (imf05.hostedemail.com: domain of rientjes@google.com designates 209.85.214.181 as permitted sender) smtp.mailfrom=rientjes@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721685457; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/ZEcCqiUZDjOjxnaamPFPIyO33gwlR7/wTRa/ysbchE=; b=IgU9qTxiskPCUCRcWTuKbei4kYoD2WP6xKcohCQRuMOFIEGOuVdmDw0AiUT1+PWCxGO1hd Juhx6bEoldb1vsYvuoxp5+6kTVUOlhNgCr8KS2M2cLqZzJXzD/FZmujr8IwO3GjeJG5Mwr ugxUSM8KKOEepEX3GKjsiXz7KLfSDNw= Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-1fd7509397bso66025ad.0 for ; Mon, 22 Jul 2024 14:58:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1721685479; x=1722290279; darn=kvack.org; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=/ZEcCqiUZDjOjxnaamPFPIyO33gwlR7/wTRa/ysbchE=; b=rxC3rDEO08LcLg0DjUzFvuMGZcv343/93m3crBRmnwST5M5KdhruUL1GN/dHk8b8xN Y//iW5FtmPy4nQZobgPQ5ZfbwYQuUf6anfZwv/Jr20HRADPZ9FpUpYine6Dv5e1Pt4Gj BB/7BI3ikfGXEv+DArJ8cl1C/roxVgUNpXcCtEOzffdmmYqJBfodWx5dSbIlEnRzyjn7 Dhv8FcPMX9YhFrZvIXvK9TiWD/o369ZtTLEbkGPz3DBZCb8+2+JBlopXxmNx3hVLQvhO f0pVoUDRvgvNkFTXY2h39+KQdXt8S+34mkUHEvMyP6LNdk05JpOeWy05XbYUnUOsbgUg 2dOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721685479; x=1722290279; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/ZEcCqiUZDjOjxnaamPFPIyO33gwlR7/wTRa/ysbchE=; b=HYVZtmaLcJUmkWbFzynNCNifh/urezNf6ZUXhCKo/DoHKt7s7zXY5VDoHeySHoOryj XcQoQYUOln6I4SGmELpn5bsz61EZzf6b3dHumhlCp4rgRng1KhdNZoSgBWm6r/YwjP1i sB1IFI3zocc1FLVftfmnAztgGP85UTwMWEaCNq/cCKE6XnpYDx82AUAErVe1RrsWgtMo ivRiCgLrNR+ZhlIJ0EX0O3mf/IHXnyAeNQyepj8ZMqa6Mkf+ZNeUqQ0LSJcUvpuHbHe7 DcaXLzFK/jtl9LmXO+bwWuO9tgLalrzo3URE7J3mtZyPjM5sf1uFJc8y8ahrE1UBRKJL FF2A== X-Forwarded-Encrypted: i=1; AJvYcCWVDz+IR1Wqydkto+mHuNuHoxBi75QxV4u3YcMX7VVk/UPd1+Qdb/suLxpC/qQDytefGBE0mZKc/IQ8gZDzihX2zco= X-Gm-Message-State: AOJu0YwJnDn//unfoTNrJM8yKNfN+zQIecpESOKiHgluyOn/sZvoEJEB 1jwNq9k3/AclRtmxCnEa8yvaNio19IxIt0uTXcn44YWybBFmvcINPiDmL7UQoD7b9xga3brPmf0 MCA== X-Google-Smtp-Source: AGHT+IGCJyRbJXAlSUQb22mQlMIVWIDqcYuFa3iXcn611BiRvkWvVs7yDTUGIn2bFa/c2kjDAk3ARA== X-Received: by 2002:a17:903:181:b0:1f7:1c96:d2e8 with SMTP id d9443c01a7336-1fd7961a1cemr5827585ad.10.1721685478704; Mon, 22 Jul 2024 14:57:58 -0700 (PDT) Received: from [2620:0:1008:15:19a4:e2d1:106d:852a] ([2620:0:1008:15:19a4:e2d1:106d:852a]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70d1502429dsm3721447b3a.111.2024.07.22.14.57.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Jul 2024 14:57:58 -0700 (PDT) Date: Mon, 22 Jul 2024 14:57:57 -0700 (PDT) From: David Rientjes To: "Vlastimil Babka (SUSE)" cc: Michal Hocko , Andrew Morton , Mel Gorman , Balbir Singh , Peter Zijlstra , linux-mm@kvack.org Subject: Re: Tools for explaining memory mappings/usage/pressure In-Reply-To: <549a3199-2eb5-41b1-a03b-9cc571e72e03@kernel.org> Message-ID: References: <29c27dab-a590-5df2-c840-279bf9dff090@google.com> <549a3199-2eb5-41b1-a03b-9cc571e72e03@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Stat-Signature: ejfrdinma46zhsrxs63e8mnb38s1p8h3 X-Rspamd-Queue-Id: A460710000B X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1721685480-661593 X-HE-Meta: U2FsdGVkX1/ufT/khZh3M+K+QYxxTqO1IoexUD4GAKg1lJYL9QDuEOcPyz2fRtWGk+evulUIRlOrXL+zILnyMkN/C1vhvVKoNiWCrq/cNpE6g+Z1NsWng3NnK7p8idH865JC+cLagHaNlb1I0GNnScDjq94b3r2YijXwYQtncag6RhxvOOH12M5enSlqd3Gpsvz8z30Zx5eO7rI8PILOeslKqoBFxcYBpwHDh63gxG5Sp6ebDHkGUQDXGIvJ3CINpO5wjKsu0eWIQ9le5UibsCnIs5GsUwqo6Gcjpkf0yHzqriuz4NNTXxuAhEF30TclU/TK8Jk2/+roK92QwKuc2bfqNeKYJ9IZeJg+dEVTYQwRzdKiMWuR35zXAEX40TIRdKD/dCl1e/mPQZMSSBSP3iT07zo7JlIdWforFxMH5F4HzHXBNp8MZoo4BLYVdK2152N+DJGT/8H9bF0Qq/mhWb0dL0M+Q9KdiV2cNuQqbIg6BGAHEoLgVBsN2dQMffWfPQoef385vo5tJrwQ8Iy9sFUBJP1acNjufpfqeItWeQJPNmjk4uo8BKWRHFTe8zhv3XuLRlqcS4I+baQ1lwgYKqSppQwiXeGKkEAgyMfHb0cUEEOo+nRa2ny6RyVPmMf/tjWwCiiAkX/woIuZd3JWORhQ3HT68XysbdS4YfOfzZJO9SGjbq9NGewoY0viNDmecAnRkWlpChBya+U1osiKbicheOVcbYNlfndyFqKIQITSsOyF43q75Rrt76IWMYBAL6RUU7LL7sLri7NwTaQiPvkZp0+m09y/OmduLkIzzDMCiYo1aMIlPn1PlHQkiPcj8oWL2B3yOQrpjyPDSo5CJWRdgptopjUSacDCyiweVJWiWA+0ZIMNVUay22YHdEr5gfSQQkPOcEpilX9rm/HeXcOiX7uqOnishW+ZYTSTuvKdw1LBbM+Jv5J2HsC/kG+ZwKl4tkHWQv6G4Vk+Yj+ 03EM8vFH X63guX57Q+FdHyFZcNOsBqqzPDkfpTVAuRduZjNomMcfZQtgHtD/NmWrDvuDraDC8nlbFfiVyQ+M9Q/onYIXAguCzhJ1jlwQO5NV0M6S2oLuKz5edXbAQ58G9e1fWZyHTlTkolOR2G8PCAP/blkuL+McczEYolnsCq/3SrApsNqQbdc2rFn9jTo9LUuRRowh8I95X6mAJwMetai+DSXFAT/lBX5tjDdXKsPcTUnI3a+6kcWPL+zo5YMAgOsiFMtlijVGu1waiEvNqKfqogMMXldQcuKCHhaKEdMMf/a2InRPYW2y0npNu9NWujsk/PKgw5yVKsA2owv/5pMofjW9omN+IqA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, 22 Jul 2024, Vlastimil Babka (SUSE) wrote: > On 7/6/24 10:55 PM, David Rientjes wrote: > > Hi all, > > > > I'm trying to crowdsource information on open source tools that can be > > used directly by customers to explain memory mappings, usage, pressure, > > etc. > > > > We encounter both internal and external users that are looking for this > > insight and it often requires significant engineering time to collect data > > to make any conclusions. > > > > A recent example is an external customer that recently upgraded their > > userspace and started to run into memcg constrained memory pressure that > > wasn't previously observed. After handing off a hacky script to run in > > the background, it was immediately obvious that the source of the direct > > reclaim was all of the MADV_FREE memory that was sitting around. > > Converting to MADV_DONTNEED solved their issue. > > BTW, was this reported/fixed upstream? Sounds like a bug to me that would > better be fixed than suggesting the MADV_DONTNEED workaround to everyone > from now on. > >From the kernel perspective, it was working as intended: nearly half of the customer's memory (32GB VM instance) was lazily freeable under memory pressure and they were not expecting any reclaim stalls for future page faults. A recent upgrade of their userspace had switched heap freeing in a library from MADV_DONTNEED -> MADV_FREE implicitly without the user's direct knowledge :/