From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84876C46467 for ; Thu, 12 Jan 2023 01:38:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D9AA08E0002; Wed, 11 Jan 2023 20:38:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D24258E0001; Wed, 11 Jan 2023 20:38:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC4C28E0002; Wed, 11 Jan 2023 20:38:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id A836C8E0001 for ; Wed, 11 Jan 2023 20:38:51 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 79CAB1207DE for ; Thu, 12 Jan 2023 01:38:51 +0000 (UTC) X-FDA: 80344438062.23.BE77302 Received: from mail-yb1-f173.google.com (mail-yb1-f173.google.com [209.85.219.173]) by imf04.hostedemail.com (Postfix) with ESMTP id DE16F40014 for ; Thu, 12 Jan 2023 01:38:49 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ew490uHS; spf=pass (imf04.hostedemail.com: domain of yuanchu@google.com designates 209.85.219.173 as permitted sender) smtp.mailfrom=yuanchu@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1673487529; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AvPtEwtZM9M8vhjcnDBhdGXaa7R7IpCHINyiVw91Tqw=; b=T1FXh/2Kqlqgq3z0gEhVLcRLnFh+mBn6I0BvHkTpf1/rqiG4GaIezt3wqeKjMts20yd1Ez NruN9dJSX/6smgZ1ksQjd5bafthKeU2C9U7CMSCiIQpAw0B+l2W3Yd/sKsJPs0e/tKIdIN 9rWhI1WdLNyDyzUVGLrrFMNshWSLZZE= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ew490uHS; spf=pass (imf04.hostedemail.com: domain of yuanchu@google.com designates 209.85.219.173 as permitted sender) smtp.mailfrom=yuanchu@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1673487529; a=rsa-sha256; cv=none; b=na47tIjUj1ihAwfoCM9VVBqhzNZeIrDBS9BEfHAbbq97CWk1+bOAwTx7viqyszgACx0eGE Tlah7AmYE+29/Zubgr3jcAGi1k6s8faH5vXOk8sMJ5IMv/oh4ghGKxnAmzfmcK7r6NHUFh FBq6Dj4l7ul206Q8gQ1ESUZg3b3Znu8= Received: by mail-yb1-f173.google.com with SMTP id p188so17077095yba.5 for ; Wed, 11 Jan 2023 17:38:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=AvPtEwtZM9M8vhjcnDBhdGXaa7R7IpCHINyiVw91Tqw=; b=ew490uHSuUDb6IZiNj7J2omgVWofPd8fWgOYael7TpZBTXDtsHSAd1MktEsOv9faYX dk/AonZsRunyMpaJLLzXaM8J0rTiwnXf8VG29KNWe62a/gal2kTOWFjr0sequymp1DCY deryNvmfVbw3MfE8u6IwwSbV2nln5PFQpFtFcpyOWSMd9CXS/AKtS1p1YOGlfMpw79jP l5NkVrrTWEoDSakAg1zFF/4YwIyh4Rbq1+qoZZdl/j2jVgBkjX4nVsmfV9VLo3uChPQg JBL++ZZpMvU46kwYRyxClLlCAHYkOfjPCY0EBI0/s73cr3oKqru9ihboYjhKi2PyyHda So2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AvPtEwtZM9M8vhjcnDBhdGXaa7R7IpCHINyiVw91Tqw=; b=wGu8CKUeLbyTtsqyBE4uoJjUnER6rZjLz3LlIXeavbd68YeVg0/R42/42w676AnxoQ IbcgB94TP9C5/4FwH5eoR+p/nFozFnOeSGZpn8evjbqOzIr4Ex61r7GIX4Xvej636JBj PwM8S0gh42tnHKEfsdjEKfs4Ro8U1jZnRE7vXVVdE5PAXDOQ7+TI/rIecPSTwI++1quj ZhBmX3iywKZlJF8DWBxoi9cXxgV3h3RoQOfiMXEwa/1nikSmh1XPb9ygxCue9iAkUU7C ocyAF0IwFERkOIt2y3UrmlMmXQ+UGrTDxiLNjimDBqnzfkDKD85S9fBK7m0SPhMokqtZ iRZg== X-Gm-Message-State: AFqh2kpO95clV7uNPVtQLP4JINtXTZXR3AFP5qwDL35Qr5CajsuHrMcL Ap+m3a4fE05PV/4kVCWKdEb8uJbK71rIqhPH1jB8v6VrWuoJ2Q== X-Google-Smtp-Source: AMrXdXvR3AqsbCqjGKXvVsjswPX5UqCPBEwDxqz1k68HMcWKzjDj13JnbxiAoe4wa4zNk7Y/0aKbvez2mpNXiWGJ/+s= X-Received: by 2002:a05:6902:3cb:b0:6f7:dc52:d2cc with SMTP id g11-20020a05690203cb00b006f7dc52d2ccmr8940765ybs.292.1673487528875; Wed, 11 Jan 2023 17:38:48 -0800 (PST) MIME-Version: 1.0 References: <20221214225123.2770216-1-yuanchu@google.com> <20230111141716.GA14685@blackbody.suse.cz> In-Reply-To: <20230111141716.GA14685@blackbody.suse.cz> From: Yuanchu Xie Date: Wed, 11 Jan 2023 17:38:37 -0800 Message-ID: Subject: Re: [RFC PATCH 0/2] mm: multi-gen LRU: working set extensions To: =?UTF-8?Q?Michal_Koutn=C3=BD?= Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Yu Zhao , Andrew Morton , Shakeel Butt , Muchun Song , linux-kernel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: DE16F40014 X-Stat-Signature: dg4h8zi4abpmnc4obyt9xcdgaatirar4 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1673487529-275413 X-HE-Meta: U2FsdGVkX1+2LSETnsA/tEOVVmt1L3+UlO73lFgCdcZvjAid3QcRG3d9X8vBAENE4JnnYhSP3E7e1mJocjKH2+o4xDJrVeQMDprZj0xMHD6CxKO/YRyg+ilKJcTzCr1zifJw+Z6FdxdiJFJFh5V19fpds6XeyiHLP71h0iU3PS5a8XZqZbS/PcRFIPUxiCrPTbUPzPwIENJ7CF9uqcj1CNk5f6WnCi0nHOyT7OwI+MK0Slmc9AR1MCDpmxfL5oSNjw+o99Tyuv0zF0o4pOabIdz476JznIDO624CeaY1fDdWuexsLaXFKybLVQVXeGnFmfRxqVFBB+EoODqk70YmTOiozIhf7jnSLzmnAqfeiN47wHAavU377VSUCd8FqD12y4k4UvTdzBrN7kW9+fU+xOGSTKu/q7Oy4zhPubhuiWZnSzpzQL2SOqtLydXw73inHE8J8jMJpBpZldgr2I21J4buCpaX0mpAau7zOS7gnhVnsmVxjq8UtSS2nTZOAjJbjm5b5roeRBI8pE/T+0nMCafUEFJjjbcRaMEev1eGu+VUTNiEsQ97LI5HPPu+kb/p2O3T3WJHjosOq5IfemOt1d9nqRgWT4ESY+Kv/HpszovLtnuJZBgRzpXquWDloZbnxMCglxPEkTP7AEbm8WhYJbGm7CBGFmSAqVWa3y98ndtoPkMIopm5DneEb9dAr8QyQel6OyLs3DTwV2AYp7/wAFYpiL71oy0eqSP6g5UAZORWsxyEudshG2HFJNtjQGaW17KUQ++XmauUdubMeGUEKz3u3i5KF+Nm3HHRYZASFBWk4fBSdvucbPyyd6KYszwGhLU6gFCYtx/1NiYwHbbcXatX6L6pU8VjFElLNX9VcPUzwfsc/iiZrMYY5dhgGucK1LArrLDVxL1kc3rZOInk/6MMcLqEkjiCWPTFVIYj60kt5XWuzyqUbuxaJEz7t5c87sCvwBy4xCg5daG3YCK tjw1Y6Ij WD1ENT8RDtGMaERY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Jan 11, 2023 at 6:17 AM Michal Koutn=C3=BD wrote= : > > On Wed, Dec 14, 2022 at 02:51:21PM -0800, Yuanchu Xie wrote: > > that's frequently used. The only missing pieces between MGLRU > > generations and working set estimation are a consistent aging cadence > > and an interface; we introduce the two additions. > > > > Periodic aging > > =3D=3D=3D=3D=3D=3D > > MGLRU Aging is currently driven by reclaim, so the amount of time > > between generations is non-deterministic. With memcgs being aged > > regularly, MGLRU generations become time-based working set information. > > Is this periodic aging specific to memcgs? IOW, periodic aging isn't > needed without memcgs (~with root only) > (Perhaps similar question to Aneeh's.) Originally, I didn't see much value in periodic aging without memcgs, as the main goal was to provide working set information. Periodic aging might lead to MGLRU making better reclaim decisions, but I don't have any benchmarks to back it up right now. > > > Use case: proactive reclaimer > > =3D=3D=3D=3D=3D=3D > > The proactive reclaimer sets the aging interval, and periodically reads > > the page idle age stats, forming a working set estimation, which it the= n > > calculates an amount to write to memory.reclaim. > > > > With the page idle age stats, a proactive reclaimer could calculate a > > precise amount of memory to reclaim without continuously probing and > > inducing reclaim. > > Could the aging be also made per-memcg? (Similar to memory.reclaim, > possibly without the new kthread (if global reclaim's aging is enough).) It is possible. We can have hierarchical aging, invoked by writing to memory.aging with a time duration. For every child memcg, if its young generation is older than (current time - specified duration), do aging. However, now we need a userspace tool to drive the aging, invoking this interface every few seconds, since every memcg is aged at a different cadence. Having a kthread perform aging has the benefit of simplicity, gives a source of truth for the aging interval, and makes the feature more accessible. The application developers, if they want to take a look at the page idle age stats, could do so without needing additional ceremony. Thanks, Yuanchu