From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9103CC43334 for ; Thu, 16 Jun 2022 22:33:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0B1D66B0071; Thu, 16 Jun 2022 18:33:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 03BE56B0072; Thu, 16 Jun 2022 18:33:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DF6D96B0073; Thu, 16 Jun 2022 18:33:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id CC25F6B0071 for ; Thu, 16 Jun 2022 18:33:53 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 8DC78805B7 for ; Thu, 16 Jun 2022 22:33:53 +0000 (UTC) X-FDA: 79585552746.09.DDE8AF2 Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) by imf08.hostedemail.com (Postfix) with ESMTP id 3CC2F160083 for ; Thu, 16 Jun 2022 22:33:53 +0000 (UTC) Received: by mail-ej1-f48.google.com with SMTP id v1so5329937ejg.13 for ; Thu, 16 Jun 2022 15:33:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=oz0FXxNhpV77sRT+9uMO5C0T7fXnI8REMLiqL4J+sbc=; b=AXx7yCLvqlVFTzJfX0CnXNayJu51h3/JJ1q83ZC+OUrBcl2lANtsYhGkehIwfAMkNB +xjdRV2zP2hygqaE63pMpcYVT6Kh8TEPKK2h/urGujzN7l/x89h80MzVSQL7IhEB1VhA HgkC5kONftmYNAsF09JlEwwMMEkRHzNwhc4gKmWhiXPKA30CTkmggSFwLGTT/JE0pRvv wfaoH2DkhJI9AQ497Me4QScoda6mO5/XvL+Mqft7SDNc5szig+tfjR7875ZTOdM4q0Yz Ja6IX87j4qtBJeQzuVFNI0PLPrGVWJ36DAyYnEemS640Oox+9QL5Mtk0KuUQVQRBE+Ea b8Ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=oz0FXxNhpV77sRT+9uMO5C0T7fXnI8REMLiqL4J+sbc=; b=xPAOi6S45FRh5mFvSLyUAGGrABWk4in4raSFE62PJkTEU7niwpjXy5DdlXKDgS58ap OAjLrSlnntu81qDQsFbBRFdgq5TdwvFrKX8w5Rf3mbyaarNS9DPtup7e+WJHdenKKy42 D4EmmJdzIL+VFYgQlQnuiXf3KpnL3J8Wmi1orViz8rLa9msbM5yNtyLsy52ScYkpQVP9 4mPldttpJs2CpjKzlrQ/QvQqHO6d/MCVP3f97X2o/0WfhymSf/xrYPuF4aJRCAp7fJcq T5y7wKHvawQ4CtNKZYYgab6ES5gJGPCzrH3Y8AO2X+fbzq7iNwXW9L2k1o8TPZgjRwQ2 /Y1w== X-Gm-Message-State: AJIora+SB6C2iLgkRxyzy+JJhYSPVDA5CYQK72tPPwaG+o8rAfh2yK0M 78dLl0a+CRz62hkCoXchTzOWcNMuZwkWba0fGFU= X-Google-Smtp-Source: AGRyM1vJqcrVJsg8SRNSuUIyrwXDuawh731gePruRGVEfT3bLflMt6W9nC+logCtE0MVFEW719gCA2UmIaLi1u1M8WU= X-Received: by 2002:a17:906:fb07:b0:706:ad5a:db9f with SMTP id lz7-20020a170906fb0700b00706ad5adb9fmr6622722ejb.91.1655418831793; Thu, 16 Jun 2022 15:33:51 -0700 (PDT) MIME-Version: 1.0 References: <20220518014632.922072-1-yuzhao@google.com> <20220518014632.922072-8-yuzhao@google.com> <20220607102135.GA32448@willie-the-truck> <20220607104358.GA32583@willie-the-truck> In-Reply-To: From: Barry Song <21cnbao@gmail.com> Date: Fri, 17 Jun 2022 10:33:40 +1200 Message-ID: Subject: Re: [PATCH v11 07/14] mm: multi-gen LRU: exploit locality in rmap To: Yu Zhao Cc: Linus Torvalds , Will Deacon , Andrew Morton , Linux-MM , Andi Kleen , Aneesh Kumar , Catalin Marinas , Dave Hansen , Hillf Danton , Jens Axboe , Johannes Weiner , Jonathan Corbet , Matthew Wilcox , Mel Gorman , Michael Larabel , Michal Hocko , Mike Rapoport , Peter Zijlstra , Tejun Heo , Vlastimil Babka , LAK , Linux Doc Mailing List , LKML , x86 , Kernel Page Reclaim v2 , Brian Geffon , Jan Alexander Steffens , Oleksandr Natalenko , Steven Barrett , Suleiman Souhlal , Daniel Byrne , Donald Carr , =?UTF-8?Q?Holger_Hoffst=C3=A4tte?= , Konstantin Kharlamov , Shuang Zhai , Sofia Trinh , Vaibhav Jain , huzhanyuan@oppo.com Content-Type: text/plain; charset="UTF-8" ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=AXx7yCLv; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf08.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.218.48 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655418833; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oz0FXxNhpV77sRT+9uMO5C0T7fXnI8REMLiqL4J+sbc=; b=XNor3ylHOm4vdtO9XLiLbIJMSI39X2gUHbKwrp+8EObrVUDa/u9nwRHKyzNbD1eDbqTKtA 4zIh+PXQ4/XEWbJHqvVJX1/4cCvL/c2KE4ZHWakMfdJbwc+jMLslI3v7H4YXy6+0gNccqj Y7nZYgFR3pvB1Z0PdAvd+oJrHwcs9H0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655418833; a=rsa-sha256; cv=none; b=Uqvxx4Dny4hw6iaGYLPV27juVg/bwMQjUxufzUliW0KhoaeC/WNyFo0NANVidNAKg8mlV3 2cuZ4WDnjvRvSnwWQzHF3I+p32JlzhwMjEYvAkytLPq4nUFU234n4FhjKXsMZ+dv3zOR+T 2jj/pJJhcvlWiZVOWYzbhgqr0xfsbBg= X-Stat-Signature: x6u8u7hgeocrqcoi5axh3o94yjcg8nkb X-Rspamd-Queue-Id: 3CC2F160083 X-Rspam-User: Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=AXx7yCLv; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf08.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.218.48 as permitted sender) smtp.mailfrom=21cnbao@gmail.com X-Rspamd-Server: rspam10 X-HE-Tag: 1655418833-504867 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Jun 17, 2022 at 9:56 AM Yu Zhao wrote: > > On Wed, Jun 8, 2022 at 4:46 PM Barry Song <21cnbao@gmail.com> wrote: > > > > On Thu, Jun 9, 2022 at 3:52 AM Linus Torvalds > > wrote: > > > > > > On Tue, Jun 7, 2022 at 5:43 PM Barry Song <21cnbao@gmail.com> wrote: > > > > > > > > Given we used to have a flush for clear pte young in LRU, right now we are > > > > moving to nop in almost all cases for the flush unless the address becomes > > > > young exactly after look_around and before ptep_clear_flush_young_notify. > > > > It means we are actually dropping flush. So the question is, were we > > > > overcautious? we actually don't need the flush at all even without mglru? > > > > > > We stopped flushing the TLB on A bit clears on x86 back in 2014. > > > > > > See commit b13b1d2d8692 ("x86/mm: In the PTE swapout page reclaim case > > > clear the accessed bit instead of flushing the TLB"). > > > > This is true for x86, RISC-V, powerpc and S390. but it is not true for > > most platforms. > > > > There was an attempt to do the same thing in arm64: > > https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1793830.html > > but arm64 still sent a nosync tlbi and depent on a deferred to dsb : > > https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1794484.html > > Barry, you've already answered your own question. > > Without commit 07509e10dcc7 arm64: pgtable: Fix pte_accessible(): > #define pte_accessible(mm, pte) \ > - (mm_tlb_flush_pending(mm) ? pte_present(pte) : pte_valid_young(pte)) > + (mm_tlb_flush_pending(mm) ? pte_present(pte) : pte_valid(pte)) > > You missed all TLB flushes for PTEs that have gone through > ptep_test_and_clear_young() on the reclaim path. But most of the time, > you got away with it, only occasional app crashes: > https://lore.kernel.org/r/CAGsJ_4w6JjuG4rn2P=d974wBOUtXUUnaZKnx+-G6a8_mSROa+Q@mail.gmail.com/ > > Why? Yes. On the arm64 platform, ptep_test_and_clear_young() without flush can cause random App to crash. ptep_test_and_clear_young() + flush won't have this kind of crashes though. But after applying commit 07509e10dcc7 arm64: pgtable: Fix pte_accessible(), on arm64, ptep_test_and_clear_young() without flush won't cause App to crash. ptep_test_and_clear_young(), with flush, without commit 07509e10dcc7: OK ptep_test_and_clear_young(), without flush, with commit 07509e10dcc7: OK ptep_test_and_clear_young(), without flush, without commit 07509e10dcc7: CRASH So is it possible that other platforms have similar problems with arm64 while commit 07509e10dcc7 isn't there? but anyway, we depend on those platforms which can really use mglru to expose this kind of potential bugs. BTW, do you think it is safe to totally remove the flush code even for the original LRU? I don't see fundamental difference between MGLRU and LRU on this "flush" thing. Since MGLRU doesn't need flush, why does LRU need it? flush is very expensive, if we do think this flush is unnecessary, will we remove it for the original LRU as well? BTW, look_around is a great idea. Our experiments also show some great decrease on the cpu consumption of page_referenced(). Thanks Barry