From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50A3EC4708D for ; Fri, 6 Jan 2023 04:00:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C52418E0002; Thu, 5 Jan 2023 23:00:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C02358E0001; Thu, 5 Jan 2023 23:00:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AC9E78E0002; Thu, 5 Jan 2023 23:00:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 990698E0001 for ; Thu, 5 Jan 2023 23:00:29 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 6B2D11C5EF7 for ; Fri, 6 Jan 2023 04:00:29 +0000 (UTC) X-FDA: 80323022178.02.9CA228D Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf16.hostedemail.com (Postfix) with ESMTP id 9BA56180009 for ; Fri, 6 Jan 2023 04:00:27 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=RiI1TTlG; spf=pass (imf16.hostedemail.com: domain of akpm@linux-foundation.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1672977627; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GSYJH/6nlnr5IcnexjplCugPICd7oNbbGxMmz/gcY3A=; b=YOGYwD01U+zQHLuLaoeSnT1/NSzeK/dTU8QhEBn3SA94eoyg6uc3kKNyavsR9TZj3rjyfq 9QYPlJSbndpOWq3nWRS/Yp68xBEyaoFrY/SQOShDlGrXXR+++TiCaKfhla3ue7dM/SYh2e jOz0XJVUbaeBAF/HGYL/bHhvw/B1aO8= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=RiI1TTlG; spf=pass (imf16.hostedemail.com: domain of akpm@linux-foundation.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1672977627; a=rsa-sha256; cv=none; b=P/J2ds9w8vJFKPcO3/eyWcMqKNBuIB78ttDJXpr/7vRCEhBmJ3nbLFhOFU6Ukv6MOslNRW VteuoicEHlaF9/8kFChWUrMAWvdL6LhoNMBglJgEdM9nGMRS4d4o5oOJ9hz+zASELUqArU 1BfZsZvE5SreH7QW+j6sBGVWtzveKeg= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 0186AB81BA9; Fri, 6 Jan 2023 04:00:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 53430C433D2; Fri, 6 Jan 2023 04:00:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1672977624; bh=+UPdXkytzKMtZ3ZWHr4CnIskWe+T5fdDfosPD98BIPQ=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=RiI1TTlG8cChT7br22VuUsCCJFUjhv1+2U6wbep8GVwXIEmQZQqtkiVFnz49/AiI1 oMzPMIkX+w0SlJpfyZ+MrlzPWmOyc1FG7DOJDsPxKlFGiwlpUT9KGdYNXEbV8Dp3HV 5gXx+czAystFOUsVKop9Yjv8GXgV9V7DIFNk64yY= Date: Thu, 5 Jan 2023 20:00:23 -0800 From: Andrew Morton To: Yu Zhao Cc: Alexander Viro , Andrea Righi , Johannes Weiner , Michael Larabel , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@google.com Subject: Re: [PATCH mm-unstable v2 1/2] mm: add vma_has_recency() Message-Id: <20230105200023.ac9f34f5b7738eae4fd940d6@linux-foundation.org> In-Reply-To: <20221230215252.2628425-1-yuzhao@google.com> References: <20221230215252.2628425-1-yuzhao@google.com> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; x86_64-redhat-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 9BA56180009 X-Stat-Signature: m39i15ock4orx867xqtf1qu718dcru4k X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1672977627-951211 X-HE-Meta: U2FsdGVkX19QdFqq0KWirq8kZiaOLpULvH3z4pychzCGNySDeouDXzxN72xtSKorBTWbTe/390O/KoBMSqyg5whOXEIgzF52o05CWuFjuzTnhqbT66oz1T/TRpwXxktylfduupqa8AftnDw1lr8R2mjk54ZGSVnBwImJKO7a3xJJfT/dfvJYjV1e5DktfV0EhVgR5bokQ5BDTWNE8SKw/cfqnAPVsQZx75RgETSzJrWsCBxvRFYBdO6x3/n7VWdguW2OqvVjD1gtqwn5E25dmCKZEcz1oqeTr1JJpVmtplvL6HsQ2warI3tv/ydrxh8AMohrlarowwuQUVp4gSNVk9MZO9YUiQODjsktu/s/RJr9hBDkMfso0F9qJkBCjby3Kv46ZEs8O6mLGFEEYECBDEBtTqBgQ85bD3ctucvl70pjUINWfDBPZXJm0fXeChPjQB3rTUcDl4358xKZjsA/zCplTzB64FGTxecnYQb/pXAwo1fjBhhWsr+ePhdq3Xuz3Uesb3YEVeySWuu7W3M9F+MIdSbDOLwH+CE5HkMRau+tq4HKMkRvBN8Mc7EgN1+kmhoBqLRf9hbaNPbCt8JgQSDE9IkKwFaTVAFNPvyFyVNml9JK6Ie9MdPGfUCa9T5UEiyC6DCLc41xDdVMxETV5HoKGckQ871KpY70lLHDWd4XdPw8FkPYMPCxvjd14Gom2v53kGBvju8Yloh4XFVoy2PsIfE3WL2X5tfhFvETC8eJE+hhiXP4B/PubgG+Z4Pl5GaX2vTufam4j9fPIPqw6GlS4zK0LRqmkx7bt8h/1wz6L7PdLKW5NfRi3ftUOmG0zN9NehQbPs5mOm+Z3KN6lxZ4SmDITNLMjysdEZuTXpIok5aslkWWVe9UCnDCfjgz20HER3WDQ1Yk1/4JS6amnfLwQW9jhC5UsTtHqW+iZED0aNhOSehn7AcOawR/zdwyJp14z2/nZl51tkkUJH7 XLLiXX44 luGGl2XTiQjh2JUULqqcwaMt0Q+VQrkCuomT1WJ6ZIjiuc0C5yinrMNYHu1ljBDGKat32xYsphcXo+kot9g5c9nIfMCt1ZkgK9qpN73mB5BK7zEdbdkfshoV3iw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, 30 Dec 2022 14:52:51 -0700 Yu Zhao wrote: > This patch adds vma_has_recency() to indicate whether a VMA may > exhibit temporal locality that the LRU algorithm relies on. > > This function returns false for VMAs marked by VM_SEQ_READ or > VM_RAND_READ. While the former flag indicates linear access, i.e., a > special case of spatial locality, both flags indicate a lack of > temporal locality, i.e., the reuse of an area within a relatively > small duration. > > "Recency" is chosen over "locality" to avoid confusion between > temporal and spatial localities. > > Before this patch, the active/inactive LRU only ignored the accessed > bit from VMAs marked by VM_SEQ_READ. After this patch, the > active/inactive LRU and MGLRU share the same logic: they both ignore > the accessed bit if vma_has_recency() returns false. > > For the active/inactive LRU, the following fio test showed a [6, 8]% > increase in IOPS when randomly accessing mapped files under memory > pressure. > > kb=$(awk '/MemTotal/ { print $2 }' /proc/meminfo) > kb=$((kb - 8*1024*1024)) > > modprobe brd rd_nr=1 rd_size=$kb > dd if=/dev/zero of=/dev/ram0 bs=1M > > mkfs.ext4 /dev/ram0 > mount /dev/ram0 /mnt/ > swapoff -a > > fio --name=test --directory=/mnt/ --ioengine=mmap --numjobs=8 \ > --size=8G --rw=randrw --time_based --runtime=10m \ > --group_reporting > > The discussion that led to this patch is here [1]. Additional test > results are available in that thread. > > --- a/include/linux/mm_inline.h > +++ b/include/linux/mm_inline.h > @@ -595,4 +595,12 @@ pte_install_uffd_wp_if_needed(struct vm_area_struct *vma, unsigned long addr, > #endif > } > > +static inline bool vma_has_recency(struct vm_area_struct *vma) > +{ > + if (vma->vm_flags & (VM_SEQ_READ | VM_RAND_READ)) > + return false; I guess it's fairly obvious why these hints imply "doesn't have recency". But still, some comments wouldn't hurt! > + return true; > +} > #endif > diff --git a/mm/memory.c b/mm/memory.c > index 4000e9f017e0..ee72badad847 100644 > --- a/mm/memory.c > +++ b/mm/memory.c > @@ -1402,8 +1402,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, > force_flush = 1; > } > } > - if (pte_young(ptent) && > - likely(!(vma->vm_flags & VM_SEQ_READ))) > + if (pte_young(ptent) && likely(vma_has_recency(vma))) So we're newly using VM_RAND_READ for the legacy LRU? Deliberate? If so, what are the effects and why? > mark_page_accessed(page); > } > rss[mm_counter(page)]--; > @@ -5148,8 +5147,8 @@ static inline void mm_account_fault(struct pt_regs *regs, > #ifdef CONFIG_LRU_GEN > static void lru_gen_enter_fault(struct vm_area_struct *vma) > { > - /* the LRU algorithm doesn't apply to sequential or random reads */ > - current->in_lru_fault = !(vma->vm_flags & (VM_SEQ_READ | VM_RAND_READ)); > + /* the LRU algorithm only applies to accesses with recency */ > + current->in_lru_fault = vma_has_recency(vma); > } > > static void lru_gen_exit_fault(void) > diff --git a/mm/rmap.c b/mm/rmap.c > index 8a24b90d9531..9abffdd63a6a 100644 > --- a/mm/rmap.c > +++ b/mm/rmap.c > @@ -823,25 +823,14 @@ static bool folio_referenced_one(struct folio *folio, > } > > if (pvmw.pte) { > - if (lru_gen_enabled() && pte_young(*pvmw.pte) && > - !(vma->vm_flags & (VM_SEQ_READ | VM_RAND_READ))) { > + if (lru_gen_enabled() && pte_young(*pvmw.pte)) { > lru_gen_look_around(&pvmw); > referenced++; > } I'd expect a call to vma_has_recency() here, but I'll trust you ;) > if (ptep_clear_flush_young_notify(vma, address, > - pvmw.pte)) { > - /* > - * Don't treat a reference through > - * a sequentially read mapping as such. > - * If the folio has been used in another mapping, > - * we will catch it; if this other mapping is > - * already gone, the unmap path will have set > - * the referenced flag or activated the folio. > - */ > - if (likely(!(vma->vm_flags & VM_SEQ_READ))) > - referenced++; > - } > + pvmw.pte)) > + referenced++; > } else if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) { > if (pmdp_clear_flush_young_notify(vma, address, > pvmw.pmd)) > ... > The posix_fadvise() manpage will need an update, please. Not now, but if/when these changes are heading into mainline. "merged into mm-stable" would be a good trigger for this activity. The legacy LRU has had used-once drop-behind for a long time (Johannes touched it last). Have you noticed whether that's all working OK?