From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 572D5D13595 for ; Mon, 28 Oct 2024 07:33:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9A96F6B0085; Mon, 28 Oct 2024 03:33:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9327F6B0088; Mon, 28 Oct 2024 03:33:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7D2CF6B008C; Mon, 28 Oct 2024 03:33:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5B9696B0085 for ; Mon, 28 Oct 2024 03:33:00 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id CFEB041542 for ; Mon, 28 Oct 2024 07:32:47 +0000 (UTC) X-FDA: 82722194058.11.399C6AA Received: from mail-yw1-f176.google.com (mail-yw1-f176.google.com [209.85.128.176]) by imf19.hostedemail.com (Postfix) with ESMTP id 77BB81A0014 for ; Mon, 28 Oct 2024 07:32:28 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=g-cbnu-ac-kr.20230601.gappssmtp.com header.s=20230601 header.b=udVJITKH; spf=none (imf19.hostedemail.com: domain of chminoo@g.cbnu.ac.kr has no SPF policy when checking 209.85.128.176) smtp.mailfrom=chminoo@g.cbnu.ac.kr; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=cbnu.ac.kr (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1730100620; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UjBEEbazuN+oDMl64tlmxo8mAN8ol20TnWZ4NamQN5w=; b=HqihooVPzKsv866liQlxg/JmBCk3I4LemkWJXCIJAJSiOgHaa7rVGVRzxeVe14GU4g5a3+ t9ygfmG7Hzqf2ra0TjCVYciQCnDrSAXIzKOgGu2Uo7kHcOyn4E5cHm2yYFITQgMC425O/G zujzQ0VuvrE03e//j7urIaJKmfviiyI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1730100620; a=rsa-sha256; cv=none; b=A+iwZqq22tHnv58wuSY7RBbaZDuAR8VlpL9AcgwA087M39hxQeQLAX3Bc4Y9AKkJaZgocx wKeHP53SeRwhwRDUzMe1lRGc4UKLrEmSdYoalXrarsK6W8RgQ9+b9lZenIF9/3eRCWuxtr V0U7rfoPPc5iO4L7pMtoQQRT2bEjGeQ= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=g-cbnu-ac-kr.20230601.gappssmtp.com header.s=20230601 header.b=udVJITKH; spf=none (imf19.hostedemail.com: domain of chminoo@g.cbnu.ac.kr has no SPF policy when checking 209.85.128.176) smtp.mailfrom=chminoo@g.cbnu.ac.kr; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=cbnu.ac.kr (policy=none) Received: by mail-yw1-f176.google.com with SMTP id 00721157ae682-6e35fb3792eso37083697b3.3 for ; Mon, 28 Oct 2024 00:32:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=g-cbnu-ac-kr.20230601.gappssmtp.com; s=20230601; t=1730100776; x=1730705576; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=UjBEEbazuN+oDMl64tlmxo8mAN8ol20TnWZ4NamQN5w=; b=udVJITKHmGA0uL2qTZOrYdKCt1SKfsLp4dZ5FBIFIoIpKDPB/RprMMKaeEkwWzxsUs iYjX8gdt5g4P1yOrvHY70yOj82EZdFpy/RsSjHYpAQ4xYopFX2EgufhWRc8aMeDUbtA6 svVcglVgPkSzrBxZnbcmKaWazDk31deHVad+zvGDYopSo+FZkOgvBvg35sNPTzuK8CME 66dMruhY42EukMTAU2ZiXiL2H5XGaFId/P1bQOZBQ33wv4xz2Wt+4p5zCPnFd+K/Y9ox B35CVE2/hsi8hECzKHHd2X9PeQaQ1/lxA6EfDL+UxdkzBzTVIGfBWX3uNfhhYiaMsnVw FbMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730100776; x=1730705576; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UjBEEbazuN+oDMl64tlmxo8mAN8ol20TnWZ4NamQN5w=; b=JZbFG8njLqXUw41NGbVy/T6Gw7LCTPRm/YJG/Hvo8n+lPMTs3hiWuK98I0pL1LM1GP 9eJW8aq7MUnKz5gV0me2M3qFDG7jfn9J1L6oZqZfutLVhw2/ToCsuJEmyj8VwyzcdLWc +ieJuJUFhYGtD1+/mweh2NUJedYZRpAz8C6m1mhKZ1dVwudQGWOFpom96LyC+oeJMhx/ b62sdhX4/jl6zMkSlYePWzqixeisKjMTsYFCuoKsTbkRb+NXH6MAYftr2uclDFRPFQK4 NQ7HnflKSE6wAa/BX/BZKNJ44OCofhXhUf0vE0x2kyymjC+Ff+Cyl49VIcRY0Hwa/7Fj hgEg== X-Forwarded-Encrypted: i=1; AJvYcCURH2YWc+I1MdeJI6WKQUYAQIXT4R9hA+NE3f3ZDwXZw/90+R433hSBTIc87iqwM8OU4Lad2E0b2A==@kvack.org X-Gm-Message-State: AOJu0YwUy55+A1GPkbZdkm8rjCjZv5TVIet5iZAmbbt5S3X+TQxRMQEG IShXtL5dgLs2lrJPKmvsabGF2p1P5u+4bIkUqcAxGxZUVoTZtnLWQgFJKIO4ucVrNzUiLLYalAG rYICspQ2oynplk/2U5mXFzXCY95b7Zg+BwaRdnA== X-Google-Smtp-Source: AGHT+IHhY/KnuIcvsCBPuDFhzhTzUmM9Bzy0lOuEfeDsW3ETDxp6VdFnT0HuXluMl5sCoqQV623VJg/MFn+Wo5IEPes= X-Received: by 2002:a05:690c:fc5:b0:63b:f919:2e89 with SMTP id 00721157ae682-6e9d89947cemr62801977b3.2.1730100774261; Mon, 28 Oct 2024 00:32:54 -0700 (PDT) MIME-Version: 1.0 References: <20240802000546.322036-1-chminoo@g.cbnu.ac.kr> <20241008185013.64990-1-sj@kernel.org> In-Reply-To: <20241008185013.64990-1-sj@kernel.org> From: Minwoo Jo Date: Mon, 28 Oct 2024 16:32:40 +0900 Message-ID: Subject: Re: [PATCH] Hitshield : Something new eviction process for MGLRU To: SeongJae Park Cc: akpm@linux-foundation.org, rostedt@goodmis.org, mhiramat@kernel.org, willy@infradead.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 77BB81A0014 X-Stat-Signature: 5yp5zwnxzqmyscdpjzsa3iqdq6kbw4g1 X-HE-Tag: 1730100748-123119 X-HE-Meta: U2FsdGVkX18obSX0fHxh+C/DGRcAaNEAnFj6iPqelZBqEkYKx1WXv7xvGaGojPnjUXcdzkwSoPIWsM/ZiDhn6VufqXO40Di5NIjkElNy26AHuVA8QJm+RjgHbDpMNYEoDla/8yNWmCiAA9XQuipJNZXUkBTRQEtrUiqBaB6yl6XPLIQ8dijL9ypTFrMEcAtoFNMAbJsgjD95rRoZqut3/cMmMLBrX1/13oSoSohRnQAj6+giu3PE42NAcvtruPfJPictZMvk7E0373xrUMjle46mjAqDzTBg8uPZ3gwkF1wifiD6NqQwlt/qHRmcrxHJlIo2GfirYV1Ln0JsIbTUxplNQKnQ6YEaQzfPyFHvkfvhZ9EmdmZ/MmLAjkr6JYm8xGGFKOn4DbAS4tmAZobFrn9mvy8DZc7HD7L5IAx0m6OIjCgUUbyil8toYRcSF+RHgvisO36uA7w6PQoujSuTtaJ06tbQ4RnsuGimqUgMBzHbDDU7m9Dehm5xEPgnJjOYicsx5fcDnQjmoq90JOlTGtUyINjuux0K74TjO1q83iez26KRueA8AwBS98PhXFBEMW/hcx8+JHCzT38+mw/o/AqDHtwcEC9B4a+YDcn57CMIHjm9NLY8IFmU1gcVV3uPuOt8HmxOlqvFGxgspd14V0vK4hiNMdqZ9VO8+Kfq7UCFwF5w/+S0bH+vUoRCkpmu8voJnxOaqldTrIWf55wxvvKQ5icIsjFimhUO3B8xY8yKTHxI4hl3Q42tyGo/uHEDQGEG/AVFvI31X2ocVD0oivN1n//mvXVIM9yVnEOJdR0Q2yy09TkEvfD6ZFEMrT2D0dToVTdH0d5u0oL59rHH9GK4eZ2KTspPLjedRBjaWt88+5FAtge8O1hrfQpkbP26le8L7IgbK5jspWE23zTlU0TNgdxfOie5rLD/PsLA5CvauU5fw9tDZrQ+qPZQ5hBaYe8jeihlSbGAswpczmi v3LLNx8k BaNiBiYXNE/D2fKHt1LTXT/3OXVACs6YJ/pMzYyJRSwVjdnqBS1iAqhL4S9A7ZyF1YSJjPV+HaxCQ4MvkXixl09ToKi+mWROaDVbbmp5YJH091nW7ssd5nLaFt0Sen8Ygz0J0rj5EeUrrgfg9WnLZNeoGN9RGfimicnAsF4MS7LV5W5UJ80FGeo5KzFZ2APG2XbH408UD1IW26mL87zdUry4GaWI1Czem8HS2wYwH8F3Gsqne/c4Zc7AWnjMprmpDtARB8wZtpBA+Q5/6iYvcerLfG5yte7vOHTafAMNQr/VGCM1cJ5vhn/s60HuJ2zhprOi9ec9gBT7vZgTgAIm/bjpEGJwHdD12sjsF6b+aAzlINkFEEfDECc0N/ZqPHk+cw3L7rHHlwYBIgdGDjJIk4k0KOukm2tleN7orDcRsDjKaMSxjOAdWaQ3RSg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.007714, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: 2024=EB=85=84 10=EC=9B=94 9=EC=9D=BC (=EC=88=98) =EC=98=A4=EC=A0=84 3:50, S= eongJae Park =EB=8B=98=EC=9D=B4 =EC=9E=91=EC=84=B1: > > Hello Minwoo, > > On Fri, 2 Aug 2024 00:05:46 +0000 Minwoo Jo wrote: > > > Signed-off-by: Minwoo Jo > > > > This commit addresses the page space occupancy issue that arose > > in the previous commit with the HitShield technology. > > > > The HitShield technology is based on the observation that MGLRU > > does not take into account the state of the folio during eviction. > > > > I assumed that if a folio, which has been updated only 1 to 3 times, > > has increased in generation until the point of eviction, > > it is likely to be referenced less in the future. > > So, my humble understanding of the point is that, you want to improve the > kernel's reclamation logic by making it awares not only recency, but also > frequency? > > Given existence of several previous frequency based page placement algori= thm > optimization researches including LRFU, CAR and CART, I think the overall= idea > might make sense at least in some use cases. I'm not sure whether the > threshold value and the mechanism to measure the frequency make sense and= /or > can be applied to general cases, though. > > > > > Therefore, I implemented this technology to protect frequently updated = folios. > > > > I added the hitshield_0, hitshield_1, and hitshield flags to the page f= lag. > > > > Upon my review, I confirmed that the flags occupy positions 0 to 28. > > > > Thus, I believe that allocating 3 flags, or 3 bits, is sufficient. > > As others also mentioned, I think even three bits could be challinging to= add. > > In my humble opinion, frequency monitoring could be done using data struc= tures > and access check mechanisms other than folio/LRU and reclamation logic. = Then, > the monitoring mechanism could manipulate LRU lists based on the frequenc= y > data. Modifying reclamation code to refer to the frequency data could al= so be > another way. > > DAMON_LRU_SORT [1,2] could be an example of such approaches (driven by th= e > monitoring mechanism without reclamation code modification). Of course, > DAMON_LRU_SORT would have many limitations and rooms for improvements. I= 'm > curious if you also had a time to consider that kind of approaches? If y= ou did > so but found some critical problems and resulted in this patch, it would = be > great if you can further share the found problems. > > [1] https://lwn.net/Articles/905370/ > [2] https://www.kernel.org/doc/html/latest/admin-guide/mm/damon/lru_sort.= html > > > Thanks, > SJ > > [...] I apologize for the late response to your message. Based on my limited understanding, the LRU manipulation in DAMON appears to assist LRU by tracking data access to identify Hot Pages and Cold Pages, subsequently lowering the priority of Cold Pages. Additionally, this approach goes beyond simply measuring frequency, as it analyzes patterns, which I believe could lead to greater performance. By appropriately utilizing the lru_prio and lru_deprio mechanisms of DAMON'= s LRU_SORT, I think it could also be implemented in MGLRU. While HitShield differs from DAMON_LRU_SORT in that it checks counters during internal operations rather than monitoring the internals, I believe that by appropriately modifying this approach, we could implement the HitShield mechanism without altering the folio's internal structure or using flags. Thank you for your insights; I will definitely consider them. Thank you. Minwoo, Jo