From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3281DE77188 for ; Wed, 8 Jan 2025 22:41:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 966F76B0089; Wed, 8 Jan 2025 17:41:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 916FF6B008A; Wed, 8 Jan 2025 17:41:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7DE5A6B008C; Wed, 8 Jan 2025 17:41:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 19F456B0089 for ; Wed, 8 Jan 2025 17:41:51 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id B2450A146C for ; Wed, 8 Jan 2025 22:41:50 +0000 (UTC) X-FDA: 82985758380.23.FDC2219 Received: from mail-qt1-f170.google.com (mail-qt1-f170.google.com [209.85.160.170]) by imf25.hostedemail.com (Postfix) with ESMTP id CA48AA000B for ; Wed, 8 Jan 2025 22:41:48 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=AxFLX4lb; spf=pass (imf25.hostedemail.com: domain of yosryahmed@google.com designates 209.85.160.170 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736376108; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FcGGz/xGaeMwbtLCLDiFDj6fCD6ZdSSx4WX8VwJ9hXk=; b=LUZ/mjGrSQZMWxL/Ccq0Rudt2R7HJ4Be4lztMa+JFHvdLUOwI8aCRwQ719OaC3jCK52tcb azLQrL5wQRoVqLATpeieXxPf8FdTCgs/3kn3VzXH0KuyYOrpyaLEPMehS7+JSB4a2VbKG4 sChpEdWd8cD52HtQSEM2InB+/6wsga8= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=AxFLX4lb; spf=pass (imf25.hostedemail.com: domain of yosryahmed@google.com designates 209.85.160.170 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736376108; a=rsa-sha256; cv=none; b=Wv/9DXw2xXT308rmIkNXIFvI7sxAL1IjM/T/B5tOhAToH+j/AET9yrFJzhzSfGvXUWX4lD fcl2JtJ1aSyFxzYss5wVwcJekA7K0F9GjLX0mN/utVlRyUR6oHVMsXLPmtWMLzr1xzE3fA zJwDyhrEJ6MXHGf2bkXniB5+7L5hY9w= Received: by mail-qt1-f170.google.com with SMTP id d75a77b69052e-467838e75ffso2643851cf.3 for ; Wed, 08 Jan 2025 14:41:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736376108; x=1736980908; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=FcGGz/xGaeMwbtLCLDiFDj6fCD6ZdSSx4WX8VwJ9hXk=; b=AxFLX4lb7rKLWvZNJPJFOagWotYMNi6wvzUO/kzJcsLBqqy4MlEzNmO8xlhXGPJRRQ s9sNX4A0vbsB76wCa2xRZpfrIHImJyczkI9VFA+CEzTVdhit0Npq8L/3EL6imG7JkWjC cVMT/4jaQfgYqLzpRnxVor3mpTBwlIEAYDH/Ki70UWwv6cjh+HbwQBWQWnWv5dq+bEgM 9Eb8ZZ61Xv62SEPXCCU5DR0I667sSvpOyiboGseb45+98SPTsSRfkh81SDCw/KwWVc0J kVXWBSdYwfwwbtA+NMhIDshHcFzq0uZDjZLMGH+BDLdr6NPKgQ5YSqflTebZtkqFUJrS SQuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736376108; x=1736980908; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FcGGz/xGaeMwbtLCLDiFDj6fCD6ZdSSx4WX8VwJ9hXk=; b=tnGEG7yDZUaDebNJQNJmus59wtM/x4rj1jTq3N68aWGdsYtCKED3R5EQPZGDsQ5IH5 Ap+zmceIkYxAF9UCBna3xdN8bIqkDXhPQUQo4kK4L80nxd/RKJfSzRBBBeTxtu1XhWnN 28sVD/Il0xJO+gJkmra1TI9pTrhuJHgEakwX30+8bBJUbMv7Sta2WhgSijMi+5mEfkza 3ReY8Q6hs1ixyEUSIqXgY67erS4vSJ7wdH1f9GAW6XzP9QT9FPsH1otiso5m5TDc2GJG EaYI854wpBGBBof3QMHbOtIYTX2Py4qw2lGhzDnjbTfhxg6r9DeXNOsdowpqaSbXyXaw 0odA== X-Forwarded-Encrypted: i=1; AJvYcCWFgvQAjEE/joP3EiLkQi3wvH388RA6mSIuVqfjjbvSaXDdTjLv1ybQC+OP/UoovpeJvEWgwooN7g==@kvack.org X-Gm-Message-State: AOJu0YyrrrJP7+WOjW66qy0l/URVD4vyC4WD1fxPDvxV2VMJ5RZDII8E NB0X7z1if7msuyiPX7lgO9bl83I0mNI9HmxhYYkP86cEVMa8irMNyTHmctmDbFAF5cIe7h//RVz yd6uNbSA8PIL2A/SID3D3BgsIWCd+dpDulpFVkGyh8r9f35MV0w== X-Gm-Gg: ASbGncvhRPjzDLdO1uLK6nmGaBseBCEQMTE2d8G9FGj4rMFAkXCc/UKouYzjFvzxKBs JkUPRwYEP1K/9/YUN2aLREB4TmmIyKgkt/RA= X-Google-Smtp-Source: AGHT+IF0IzhOD4hrWUdv1lZLgki2J9NEZ55HY8If0i8CgRUfQanpUfuqdHXUY2wu8wWC+my8bWb8sk5T6WTRgVXd1Jc= X-Received: by 2002:a05:6214:f6c:b0:6d8:a61c:bb58 with SMTP id 6a1803df08f44-6df9b339470mr66509646d6.46.1736375617512; Wed, 08 Jan 2025 14:33:37 -0800 (PST) MIME-Version: 1.0 References: <83bebb7f-f157-4179-b7ec-b25b2ee4270d@lucifer.local> In-Reply-To: <83bebb7f-f157-4179-b7ec-b25b2ee4270d@lucifer.local> From: Yosry Ahmed Date: Wed, 8 Jan 2025 14:33:01 -0800 X-Gm-Features: AbW1kvZ4tBXJP-Tm8eH-ELdkpkUgzJq9zQUjmExrWEgJA3SZkLdXKfJIxgl7IxE Message-ID: Subject: Re: [LSF/MM/BPF TOPIC] Physical LRU scanning feasibility To: Lorenzo Stoakes Cc: lsf-pc@lists.linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Matthew Wilcox , Axel Rasmussen , Guru Anbalagane , Wei Xu , Yuanchu Xie Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: CA48AA000B X-Stat-Signature: nsoinoxhownnx7489fwmnri8i7yzdu3y X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1736376108-20439 X-HE-Meta: U2FsdGVkX1+BfKHIMV2WCKCXew0xADTf1cKF8VBlJZdcwRvZ80LV3FSZ3D6NCCT1rXGM6+/HWt1+vyH0T4kwOVexEJG9EDBXdB5g/Ae7nFLYmSCgPsupjMIQY7QGVGuuHKS8+LyPvYJMB7JsRfj9Jf5HeerOWNAuj9WCli5ejm5RJzLbXRX2SMaMmXLq1lWCz3fNQQfuLcCPveQh/2jyT3YdBXdJ4Z2qR8FEAi6eI/TW/6qpQblu9512ZhD3y+m8hJEWFdcoeMRmk+jnmHPGsc0FMpq6DEOm9oDu46eAVimIUDUcNJCQTHjnQd8ELCt7bT48V4YwnRVK9e+VIJGuRBwk00S8X6d8IkT8VnlwyPytPYmIyuMf4H59PHA/WLQun3bZkKGSVsefY5y1f6I239grNDo4WK++K/AicneZWMRddmL5lHUwHlJfdHm1fE2Zy/LBORD174ROcNrb8e3lfa7xeqmC0NP8BqFjrV8dbbCA8nlmU7yFijsl02ohWLfYc5Gk/LA4MTfSwaxlbp/YDZ4ninAy8MIGfJQsFen0efrr9j3IoxAbR9/TfMjBHhigPif3sDnsd3TxWAyTs+A4rrd2eYjl+JvPZyd8aOTcEz8AKcW0BJdU37FzzD8LPfJ1UeuOkye3tmhmIM3Lzn0Hj7C++p4sI6HF7eWyNW/TYy7a6frYs6qdFtPOBBDxAY9PfqNQOVJCwX3rSYlg2RwSauj8CQCo7kVuMGM4151hPJpmYUj1ih4JW7q/abhT4mGL0Xut5xZ1tDk1UoMeeslvxX1P+K7s1RjFmsT9DxfYzJtFz/xQo69PAg2kIfzCwSTxJOqgxCE1fAAkFDhHpxYWU0DF2fCP98nE1JhQd/cZ5mZ7wlAoJ3B37d0X7KMDBr/VBFgCaCfCZgfvre49kT8VfPOEI5l/EB2MmprKRDREld6hS/T7SfdF/EjYMHlLAFhSMmv429BVEyEup+EGYvi p2bpsLum PJGfhGR+TkI80uN95U6HIQaR9QoExUhOYChl91RFgkGuTZ1xy/pczN9StvDHRDBp0mtTdqBB0fROIjkmShZxrXla7cUP/SQ73t7XibQcXZ2q1XSNuWfI3geGOPTOznHImkhvd7djBA9a9cIegg1eTsY4t8usN0swbpo1T6gnWFbHV2N+XZKZG0jAik0FMWIq8XYzpvl3LlwK5dqSk0M4xiLWyQg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.070050, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jan 8, 2025 at 1:46=E2=80=AFPM Lorenzo Stoakes wrote: > > Hi all, > > Not too long ago I took some time to investigate the possibility of > scanning physical memory directly by traversing the memory map directly > rather than the LRU linked list. > > This was inspired by a post from Matthew [0] wherein he demonstrated just > how significant the difference is between traversing arrays of contiguous > data on a modern system vs. the almost worst-case scenario of traversing = a > linked-list. > > I tested how this might look by implementing code which simply traverses > and filters the memory map for LRU pages, simplifying as much as possible= . > > However no matter which machine (ranging from 16 GB - 192 GB) or whether > virtualised or real hardware, I found unfortunately disappointing results= - > the act of having to scan such a large range of memory resulted in > performance significantly less than a typical LRU scan at low memory > utilisation and performance at best matching LRU scanning at high memory > utilisation (simulating higher memory pressure). > > There are a number of factors at play here, and perhaps the shrinkage of > struct page (allowing for denser placement in cache lines), or an improve= d > algorithm might lead to more promising results. > > Having discussed this with Matthew, he suggested I put forward a proposal > to discuss this area in order that we can learn from this should it appea= r > this approach is unworkable or perhaps determine whether there might be > something to this that we might still salvage. > > I intend to do some more research and generate some more specific numbers > (feel free to give feedback here) before LSF so we can have something mor= e > specific to talk about. > > I always envisioned this approach being somehow integrated with MGLRU and= I > wonder if some hybrid means of integrating this approach with the MGLRU o= ne > might make sense, which could also be another area of discussion. When I read this proposal the first thing that came to mind was memcg reclaim. While it seems to me that it is already inefficient to scan all physical memory looking for a possible reclaim candidate, it seems even more inefficient to try to find a possible reclaim candidate within the needed memcg. We'll also probably do a lot of repeated scanning as we iterate memcgs. The per-memcg per-node LRUs save us from this. Do you have an idea about handling memcg reclaim efficiently?