From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93940C07545 for ; Tue, 24 Oct 2023 22:33:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8AFD96B02F1; Tue, 24 Oct 2023 18:33:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 838FF6B02F3; Tue, 24 Oct 2023 18:33:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6DA546B02F4; Tue, 24 Oct 2023 18:33:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5B0D76B02F1 for ; Tue, 24 Oct 2023 18:33:17 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 24F98A027C for ; Tue, 24 Oct 2023 22:33:17 +0000 (UTC) X-FDA: 81381807234.05.AF260C5 Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) by imf25.hostedemail.com (Postfix) with ESMTP id 43B36A0005 for ; Tue, 24 Oct 2023 22:33:15 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="bPe/8xE1"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf25.hostedemail.com: domain of yuzhao@google.com designates 209.85.208.43 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698186795; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4iRTEiIAHqdOTFjTizDZPjyhzdJk6A4pywAK9Lo2CAs=; b=5rKc4mFva3NlDeWQVI9d8kYMw3Lb/HiYVJTrFFvR2uGmYGfcoHL3CDLCS0PhPq3XWPK0Aa p2eEHYg2TGY9cRNQRpAK4V/KFvLnhjFK5DJXE+EHM3LQbqAAl2V1CpkRykVhwfPB7hsH7/ 4VJf/TpqNzPGV1x1ZKnVX9BellAm4UE= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="bPe/8xE1"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf25.hostedemail.com: domain of yuzhao@google.com designates 209.85.208.43 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698186795; a=rsa-sha256; cv=none; b=Q7Eno/9C1y2g7xstXuYaSCcznJ/gsb3tMybt1o7LVxj6e2eQ9ZhvKO2UZjPxaWi+D/3Xch tbbcYeuGdiFZIiIMltD5lkIG4u47krpboRkJCD6FoWV4u61qT2iVhFaUQ0dKRsJBLdaQnT 3eRocAjt7Q4LbD9rNKgQI4GCaQk0XUE= Received: by mail-ed1-f43.google.com with SMTP id 4fb4d7f45d1cf-53eeb28e8e5so1938a12.1 for ; Tue, 24 Oct 2023 15:33:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1698186794; x=1698791594; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=4iRTEiIAHqdOTFjTizDZPjyhzdJk6A4pywAK9Lo2CAs=; b=bPe/8xE1NvWRw1hOYAH84jsE+yENIolLobNJmVbGjHFVQJUmRVOaEswF4DavYcE6YA E+8ycRu+pDCl+NSIDub8xnIE6cD47jf0QKO+6W2pMcOvLbq6xPBK+ixbh/wHrlADiZmZ cRe7GaBZF15eMGvCDAfTgbk5YYK5Qy1H3XNQrVE0nn8FeGBHU0XEP7EO+d9DcLetj54M rKLvenEroQX8q5c2H5Zp/MD/8xn7OAvSmuNhW9AtXjFbVBVUyBs54HXecrlU3Qzh0b6O lSfw0gB4bOxXqn02jSvvyBJVm1kv+zX0szls7m47dEbSdLk/Zm8Y3Qvso0Y15hwvP1yC yHSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698186794; x=1698791594; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4iRTEiIAHqdOTFjTizDZPjyhzdJk6A4pywAK9Lo2CAs=; b=WgG9y0PEjl9NckldZjnzhv12oiPsIwIRbKnDm8SAyiFxdrjI0SdeFqioqMzvR5Aimn ENNPko+zGh/dFDdqy3O9OUKnI4iQxIBikXwvHxmDhpRImoxm0/zlXm4+iVjcGRx8nFGj I3dTLBWV2PWBjlHayu1lNM5tL0JcZHeDyHi/oNyhvmV1P71/tp4zmLzXqXFmXFy2ORXU d0bhsK7+DGzBP11sPeS1E3yKKOIXDhxqL4yu/DnHGCk72jEC7Sj5Sb7faRURQqcTYM2g 1YK2j59ouuK+YfzIIWazwse6xUcX18xyDAg13Yj7/8B94VjnIGr3AboVyNgJznlGSe2I IRRw== X-Gm-Message-State: AOJu0YytSO/IzzxoYS9/pypYi8Juq3pUV0NKSFam5Gg/RSO4pe8WAuG4 jWfAWYFHVbNwNUIVR0hcKs1mWQVAgWov8n2hrS3D2A== X-Google-Smtp-Source: AGHT+IEhKi2VOruYNNnNI/sR21aEEQlZTcHU2b10DJUQA6jXPDIAQq/Xqz+xSDCPlcNssBRl7QiDjS9G4VV5ayQdXYs= X-Received: by 2002:aa7:df94:0:b0:540:e4c3:430 with SMTP id b20-20020aa7df94000000b00540e4c30430mr311edy.6.1698186789616; Tue, 24 Oct 2023 15:33:09 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Yu Zhao Date: Tue, 24 Oct 2023 16:32:30 -0600 Message-ID: Subject: Re: [PATCH] arm64: mm: drop tlb flush operation when clearing the access bit To: Baolin Wang , Minchan Kim Cc: catalin.marinas@arm.com, will@kernel.org, akpm@linux-foundation.org, v-songbaohua@oppo.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 43B36A0005 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: fys7qebastsmq3mo18u5cxo31ktsjfcb X-HE-Tag: 1698186795-932345 X-HE-Meta: U2FsdGVkX18BfrxskrT2RHTN+Zrp7vUSPXNYfCXrroZjOgMzX6Gdj0p/2iL1lt9zQcPjStY2buLzKIJGmUQ3QsjEehx5BAZXgKPOjV7+/T8j8TtJHRqj4TqPGOzCMNy4rHpBZ+XdMlm8y05kYJ+F+Gbbn4+qQBcn2lWjSmlUSG6sQX7XxKpI58qpz4D4NQh5Tgty+bkgynBtxwEwl7XD+LukRC/ks1f65Wb5LfSQfQEO0dHyAkkVHILkq3N1CQ5Yjs+nzEYrdxHSk7edCAz643XUn8ZLyyBgBcH8gXnvKeWPg2987tu62+T6G+Paoj9QsX644xDkk/wAvdE4ZUZfsXuHdTrO2mHf+mN277nGMScuBlI+0SxtJpJOBNzlujhmkATrYvaLy6bQ0hTK+yuN/lGaAxxJEYwE0E8Yl4CxQCiXYPD+7ChyGOsLKQNgMG5M21ziHJzAelV/pm638KH+MUK+BIKi1RTqMqg+EdyjqNxSp6G2BJFajZLX1uCtSxv3U22HOGkTL6cRx9Y/et9PWjSe6TVHnJlnSmX23wMmOR7BB7AScgT6aQCf5QUefviHaiJ7XXOzRSrrjhqdOJnw2+kRVC4Hv5cieIV9qxUCkAUtfraf4xJpTFe8Dm/sCYIAIDgF9Q10zsDNHfOXxHbtLmTWnMm5HErFHjuopxjjWpW/R4ffnLuSh3OGgQRNztuSjkiDN15QIYBhmUBoNYTAcBxRyyTqd9U41d/SfWNIxv09oB8Hw3dT0Krcc8CXYXgAGi0rNyGEHd8loU4UWr9CgCEQkd5nHhDkwS6qRLtRdMYyMnHIwoagUdPsIy6A2/5Tz1XhkANVGSWOSZP6oFHH83vmKSffR4Ux2X4X5fx6wjYJE2kYyAJ+p2hBl4uogKWSqCUVgaePdU9jrZqa2sW8NLCYI+Z+EoNtNK0VFDzwgud/JnYRhQiXvE8WMEuhAzmobOPlQ/Nr3j1qG+XP/Mi Z6z7kUb9 ATYwpxPtQ5incer4nqIksi5i/TuSvgtbwkuDTfwSqnKXEhbNQaWaI4kGbIumXC6d4Td4p6AVtohHMphtfiREtv/L69ukCYdT662Obj1DALcliYTJ03e/oHz2yoPVFvwwVs932uTQMZmYGxqOBam87CwGOfALoELshzwbZ91ciD80KVRbbbnn0hXqy9xmgEA2vBXhzuoJtUn0YMIq8qlf6nDEvidWFmvv4IlubCm3+nq+ZL60LN7DwLKw/uENBFwk+0b5X4ExIoIxosN0wV2YFDsvexLi1p4QRibWpGUlMQt81oajP0iUSd50m1YRoOykDhcBsXl0ueLRE1pQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Oct 24, 2023 at 6:56=E2=80=AFAM Baolin Wang wrote: > > Now ptep_clear_flush_young() is only called by folio_referenced() to > check if the folio was referenced, and now it will call a tlb flush on > ARM64 architecture. However the tlb flush can be expensive on ARM64 > servers, especially for the systems with a large CPU numbers. > > Similar to the x86 architecture, below comments also apply equally to > ARM64 architecture. So we can drop the tlb flush operation in > ptep_clear_flush_young() on ARM64 architecture to improve the performance= . > " > /* Clearing the accessed bit without a TLB flush > * doesn't cause data corruption. [ It could cause incorrect > * page aging and the (mistaken) reclaim of hot pages, but the > * chance of that should be relatively low. ] > * > * So as a performance optimization don't flush the TLB when > * clearing the accessed bit, it will eventually be flushed by > * a context switch or a VM operation anyway. [ In the rare > * event of it not getting flushed for a long time the delay > * shouldn't really matter because there's no real memory > * pressure for swapout to react to. ] > */ > " > Running the thpscale to show some obvious improvements for compaction > latency with this patch: > base patched > Amean fault-both-1 1093.19 ( 0.00%) 1084.57 * 0.79%* > Amean fault-both-3 2566.22 ( 0.00%) 2228.45 * 13.16%* > Amean fault-both-5 3591.22 ( 0.00%) 3146.73 * 12.38%* > Amean fault-both-7 4157.26 ( 0.00%) 4113.67 * 1.05%* > Amean fault-both-12 6184.79 ( 0.00%) 5218.70 * 15.62%* > Amean fault-both-18 9103.70 ( 0.00%) 7739.71 * 14.98%* > Amean fault-both-24 12341.73 ( 0.00%) 10684.23 * 13.43%* > Amean fault-both-30 15519.00 ( 0.00%) 13695.14 * 11.75%* > Amean fault-both-32 16189.15 ( 0.00%) 14365.73 * 11.26%* > base patched > Duration User 167.78 161.03 > Duration System 1836.66 1673.01 > Duration Elapsed 2074.58 2059.75 > > Barry Song submitted a similar patch [1] before, that replaces the > ptep_clear_flush_young_notify() with ptep_clear_young_notify() in > folio_referenced_one(). However, I'm not sure if removing the tlb flush > operation is applicable to every architecture in kernel, so dropping > the tlb flush for ARM64 seems a sensible change. > > Note: I am okay for both approach, if someone can help to ensure that > all architectures do not need the tlb flush when clearing the accessed > bit, then I also think Barry's patch is better (hope Barry can resend > his patch). > > [1] https://lore.kernel.org/lkml/20220617070555.344368-1-21cnbao@gmail.co= m/ > Signed-off-by: Baolin Wang +Minchan Kim Minchan and I discussed this (again) yesterday -- I'm in favor and he can voice his different opinion on this.