From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18981D64074 for ; Fri, 8 Nov 2024 18:01:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6F2656B00C2; Fri, 8 Nov 2024 13:01:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6A1E96B00C5; Fri, 8 Nov 2024 13:01:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 542306B00C6; Fri, 8 Nov 2024 13:01:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 33FAA6B00C2 for ; Fri, 8 Nov 2024 13:01:16 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id CFFEE1A023F for ; Fri, 8 Nov 2024 18:01:15 +0000 (UTC) X-FDA: 82763693376.26.05E3679 Received: from mail-qv1-f54.google.com (mail-qv1-f54.google.com [209.85.219.54]) by imf04.hostedemail.com (Postfix) with ESMTP id BC49540035 for ; Fri, 8 Nov 2024 18:00:26 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b="EKd0z/2G"; spf=pass (imf04.hostedemail.com: domain of gourry@gourry.net designates 209.85.219.54 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731088822; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Zs23C+5/amwqfDvV9LtWZn4mgzjfoDFSDR2Scoi1E+Y=; b=2H1S5Ew4R3C9IODNVDA2YfBe+1zvsSWZkRCtjb0lbYsQNtE/px9y6Lka1enOB7hlRTNpkH 5uyN4T5gMIfG5XXyjjkG3EabGamvsE2s7YpNEGgjyv7cO7RUcgHxquQ7p9xxR7ElRLZG8q Kwv0C5WojEXe1PepeYSUVG8DhdI+qaw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731088822; a=rsa-sha256; cv=none; b=3OQ3fesLCmZMh8VyTNWA2enpuERwhRDMYhW0ZHJPC+C+9N1u1IbugkWS/Hx8yvCbqWGWKF Re2RFUDrrvUmYVWjJX7rmAHbhLv/2bprPsb422lVuAEzFzx691tnRYqCFwtscEa0HjEnPM 4Hk4iTFUVY2a2ZG/vJzpRKg8CVkBoPE= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b="EKd0z/2G"; spf=pass (imf04.hostedemail.com: domain of gourry@gourry.net designates 209.85.219.54 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none Received: by mail-qv1-f54.google.com with SMTP id 6a1803df08f44-6cbd1ae26a6so15528166d6.1 for ; Fri, 08 Nov 2024 10:01:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1731088873; x=1731693673; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Zs23C+5/amwqfDvV9LtWZn4mgzjfoDFSDR2Scoi1E+Y=; b=EKd0z/2G5Us3oXxiYPfmfHM/dl8+dQ9Hq9ytJ/9z9Dv9Ydl+qnZ937vMvnqBiMwm/P mzah3hxf2LCx/P6GDjdlq+kwGyKTxs0qic3kxZpa36lxZgODqpmbHM3PTuZsF/Y/D5AD Y3h8rZ3lW7Po82wjY4gY3GAlyOIQMnT80gnO6thrxbVdJ2Sav1k9lE6OdR7s3HqEJOdG EZYrAWt/mHn7sgxn4eIfwBxCnSwtGKiKbgd9yp+Po5BOH7UqOTcg5NOBa15zRaXj5rTF eafiHJDIzJGte7JoFlVqXGvMTTNFnB6VjRQK0k4HWMis3hfG/Hk8Y/lwrSH5Nq26R0NV OGGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731088873; x=1731693673; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Zs23C+5/amwqfDvV9LtWZn4mgzjfoDFSDR2Scoi1E+Y=; b=eAElDuzn2P8e1wngXCvv8ottfadtzhYsZsceD2HjkFV2cP+FE2MBZ+SWncZW0/HEu5 ytIT3isUs/Ais3yrr/u0iOHaw7T5dtZ/tcMAsCvUeO88XRR/IPmxpRePay/Cctx+7jFM HO9YAdcG+miCUKdGTv9fz1eK0pVOMALw6VgSLg4IaN/CM9PEwWDzLSvQ17fCR5h4pOZu IK7lthvUvRRfguBLeWsOvJs2IsSUbiWNzOg0m0h1NWCGGd0E0qNd0NXf8l7qxnAs8Evy QkHSiEeHglut+TpUt1C48BQNvxspP5ATbLalUBDdlVcqiZRS/xosVtmUM9ZoyUnIrcjL /gfw== X-Gm-Message-State: AOJu0YzBM6nt3xMKQsuYpRiAS42s1oNW3Loq7q3lBVRTS4MH9S/cAZ5U l2xWOwBrUD7DEuKmy2Erf1hY08YfsjuIhD74AN2oCeXzHnAJHoWALGgx/ChoslA= X-Google-Smtp-Source: AGHT+IEPJrXsNFfGXnjpLXjxgjcfwfpF1PgAEkcM7oPTJS9Mf06xfBmic5tTTDSyk700PE5cHO04lg== X-Received: by 2002:a05:6214:46a1:b0:6cb:f79a:cb38 with SMTP id 6a1803df08f44-6d39e107cfcmr48997816d6.5.1731088872653; Fri, 08 Nov 2024 10:01:12 -0800 (PST) Received: from PC2K9PVX.TheFacebook.com (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6d3966317f3sm21688726d6.118.2024.11.08.10.01.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 10:01:11 -0800 (PST) Date: Fri, 8 Nov 2024 13:00:56 -0500 From: Gregory Price To: "Huang, Ying" Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, david@redhat.com, nphamcs@gmail.com, nehagholkar@meta.com, abhishekd@meta.com, Johannes Weiner , Feng Tang Subject: Re: [PATCH 0/3] mm,TPP: Enable promotion of unmapped pagecache Message-ID: References: <20240803094715.23900-1-gourry@gourry.net> <875xrxhs5j.fsf@yhuang6-desk2.ccr.corp.intel.com> <87ikvefswp.fsf@yhuang6-desk2.ccr.corp.intel.com> <87jzdi782s.fsf@yhuang6-desk2.ccr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87jzdi782s.fsf@yhuang6-desk2.ccr.corp.intel.com> X-Stat-Signature: 5omnh84wg1rzq881h31hyiqw5r1drmfs X-Rspam-User: X-Rspamd-Queue-Id: BC49540035 X-Rspamd-Server: rspam02 X-HE-Tag: 1731088826-285463 X-HE-Meta: U2FsdGVkX1/dvgtzK5OqkTZT7WHUhUrF3bw4Y87QTDGbwZEPtWlMiQxTqLOICiUlB4BDAp2/tEdrOtFRjiv7kCZ2dN7nQ0DAuUB5mD6r2+z683u3VENaIJl8l+x9HYDwRBLBD0Xk3zcZMrlN3eutKwb6sIOAjW13QIrV2zAkqq/lFBVLVTBDEpfcVaOowIb0vjGTFlMfcTobKyehIRQUmuObIyg11IRxzLc/ORYkVgTZYt/cGzdRYcyq9v2weA30lh51Xp+Wj4q0uzspodMZ/u9UaHVS1ogRJ50Cb6SVcnNI9x3EV57JfS3zlKc3gN6+VNaQww9M6CCHA2+ibv8xiv97coiIR60sal414JxU4Zuq0sEsxlO//tY8kE0T38S/CcYevbZJ6hHb64ZRzy9T9RqNeXev4yyl6F8d+bUughQY6Ok+rnBNGQIVEkdvjth9OnK7lcb6dHjenf/06jZGYbrZuyjqadn2kCYxSosUHDj3CuUCWwBYM6N7JrwocZi+2cVxuJ+oXd992uo2TlSJoovXbR8OisX9+hpbkOez9SI5QbEy1XXhSVQbnn4eWureGieDLXpuljrZiWrj/TtNBSk0eWfcT1/odALLVCVVXheRgQP/5szRN4qY3kLPmpwMrYGrJyYpnOVo6o5s6sLR+smHcaYMVgapBjMMyk0wTdlz7qtoD5KGzQEYQJukJBrNJN0vN0zqY3SKW4wupNjG2rl1Mh/cJ1BP9epwnGPSmOcypYLkM21HWWKu5tthlXvFFXJdAF7WGxofag6Xka0ix1ylT3p1ilpcjBBnsTDf+2SHd4I4QLfkUlDNpEcSzhDc2NOeC/1zX/AB1Yt2NDgEazUmvoM19TLk0NPQ8m3TOli1gPDT5QyGMu+bXTlCiZ8JgywTSsd1ScS6kSL4ywFKUwUF9007i2Ac366oBUI61ZwoLkXjKHpyAiRhWEG5iWZIV6xKUSUFDPu3x/Qf3rn t6GnkYpY GI/LWmnBsU9A85mZ9PD9apkdlLULZT4mIRrvqF8twgo2bXUa4hwLMpItexD2j5cUjMgxmmApBDzy223LSYi1pGuCjC1+J7TErNRGwmyZqlTA+bGhP2r79ud0oBLAlmxm2Mvo1V1JDM7ScKMBSN+bAErsn7SEuxH9wWeGD9nEmmGMIVrXvYfPDKWIVU5B6IARpn8dwc3/HrllH/ghFldLP/lS/sxaByhnxqZA5opxf9GXQKl+s3z30DPD3S0TMXxnSSi5GxA471K3mSeVDjVSebCljtutu6vaEWlmWbqCm6Z9YUZcskJkBJg4kT7qRtpdr21+7 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Nov 05, 2024 at 10:00:59AM +0800, Huang, Ying wrote: > Hi, Gregory, > >> > >> Several years ago, we have tried to use the access time tracking > >> mechanism of NUMA balancing to track the access time latency of unmapped > >> file cache folios. The original implementation is as follows, > >> > >> https://git.kernel.org/pub/scm/linux/kernel/git/vishal/tiering.git/commit/?h=tiering-0.8&id=5f2e64ce75c0322602c2ec8c70b64bb69b1f1329 > >> > >> What do you think about this? > >> > > > > Coming back around to explore this topic a bit more, dug into this old > > patch and the LRU patch by Keith - I'm struggling find a good option > > that doesn't over-complicate or propose something contentious. > > > > > > I did a browse through lore and did not see any discussion on this patch > > or on Keith's LRU patch, so i presume discussion on this happened largely > > off-list. So if you have any context as to why this wasn't RFC'd officially > > I would like more information. > > Thanks for doing this. There's no much discussion offline. We just > don't have enough time to work on the solution. > Exploring and testing this a little further, I brought this up to current folio work in 6.9 and found this solution to be unstable as-is. After some work to fix lock/reference issues, Johannes pointed out that __filemap_get_folio can be called from an atomic context - which means it may not be safe to do migrations in this context. We're back to looking at something like an LRU-esque system, but now we're thinking about isolating the folios in folio_mark_accessed into a task-local list, and then process the list on resume. Basically we're thinking 1) hook folio_mark_accessed and use PG_ACTIVE/PG_ACCESSED to determine whether the page is a promotion candidate. 2) if it is, isolate it from the LRU - which is safe because folio_mark_accessed already does this elsewhere, and place it onto current->promo_queue 3) set_notify_resume 4) add logic to resume_user_mode_work() to run through current->promo_queue and either promote the pages accordingly, or do folio_putback_lru on failure. Going to RFC this up ~Gregory