From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C96F6C25B74 for ; Tue, 4 Jun 2024 04:44:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4A6B56B0098; Tue, 4 Jun 2024 00:44:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4563F6B0099; Tue, 4 Jun 2024 00:44:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 31EAC6B009A; Tue, 4 Jun 2024 00:44:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 12A446B0098 for ; Tue, 4 Jun 2024 00:44:00 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id A3197412E2 for ; Tue, 4 Jun 2024 04:43:59 +0000 (UTC) X-FDA: 82191963798.27.6281693 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by imf29.hostedemail.com (Postfix) with ESMTP id 0BAAF120003 for ; Tue, 4 Jun 2024 04:43:56 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf29.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717476238; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p2AJZnrv21eHHI1PZu0I9bSb59Gz6xu1KWnMBYTsS8Y=; b=KcN+3vgo44OJj4W3Qt7FWitCrp+1Ti6rIzH2TysqPKorNctUXmYNzysefZxpX5lVcFs0hq k7IB1TtpdD8LDVyUlWPZ4MduwS8yA+r1SNtAVC477WBeWXDkmEv/0MsXnCNUF3NeK8m3w5 L1zOblIf8Nc+0k1t2PPx5T0SL+Z1bbA= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf29.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717476238; a=rsa-sha256; cv=none; b=rSEf6XIi8hPz+ij/Q/qAGRlKXkKo+gtw8ZlARsd/R5r9Sik4C7WEzLb79t6X4HkC76v5EJ 4v4sIR27fMfjMB8hSoBwK1VwUoNljL1sFlXDWjHZjxxgtq9WbUQ1VPHqA6oQAfhlVrhLhT WnuKKfh9N6d0bGnsj4URPQAuZUG6n+8= X-AuditID: a67dfc5b-d6dff70000001748-68-665e9b8ac73f Date: Tue, 4 Jun 2024 13:43:48 +0900 From: Byungchul Park To: Dave Hansen Cc: David Hildenbrand , Byungchul Park , linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, akpm@linux-foundation.org, ying.huang@intel.com, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, rjgolo@gmail.com Subject: Re: [PATCH v11 09/12] mm: implement LUF(Lazy Unmap Flush) defering tlb flush when folios get unmapped Message-ID: <20240604044348.GA40889@system.software.com> References: <20240531092001.30428-1-byungchul@sk.com> <20240531092001.30428-10-byungchul@sk.com> <26dc4594-430b-483c-a26c-7e68bade74b0@redhat.com> <20240603093505.GA12549@system.software.com> <20240604015348.GB26609@system.software.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240604015348.GB26609@system.software.com> User-Agent: Mutt/1.9.4 (2018-02-28) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrKIsWRmVeSWpSXmKPExsXC9ZZnkW7X7Lg0gweXRCzmrF/DZvF5wz82 i08vHzBavNjQzmjxdf0vZounn/pYLC7vmsNmcW/Nf1aLo52bmC3O71rLarFj6T4mi0sHFjBZ HO89wGQx/95nNovNm6YyWxyfMpXR4vcPoI6TsyazOAh5fG/tY/HYOesuu8eCTaUem1doeSze 85LJY9OqTjaPTZ8msXu8O3eO3ePEjN8sHvNOBnq833eVzWPrLzuPxqnX2Dw+b5IL4IvisklJ zcksSy3St0vgypg+qYOp4IF0xcED1xgbGLtEuxg5OSQETCT62yeydzFygNmTZpWDmCwCKhJN nVEgFWwC6hI3bvxkBrFFgOxTK5cDVXNxMAscZ5b48HERI0hCWKBA4tWESWBjeAUsJA4eUAap ERK4wixxdPJXsGZeAUGJkzOfsIDYzAJaEjf+vWQCqWcWkJZY/o8DJMwpYCnxsuELG4gtKqAs cWDbcSaQORICh9glrjz9ywJxsqTEwRU3WCYwCsxCMnYWkrGzEMYuYGRexSiUmVeWm5iZY6KX UZmXWaGXnJ+7iREYmctq/0TvYPx0IfgQowAHoxIPr8Gi2DQh1sSy4srcQ4wSHMxKIrx9ddFp QrwpiZVVqUX58UWlOanFhxilOViUxHmNvpWnCAmkJ5akZqemFqQWwWSZODilGhhjCzfWapQo TK1RNmnZlbP25tn234+t82bN3bCWx0FrA8+X3m22z0pP63MuXtJWNqvlKuPmmuWL0i/4X+V8 XLRQM13rhsTr9VqHC+bdj/QyObvx/6o2nXemy10fBzVdS9t+MlM3w/SBR/8Xo1mnHmh6nZz/ g1/D+lZFTO1ynVvKrUomPWa1vZ+UWIozEg21mIuKEwHap/ENyAIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprDIsWRmVeSWpSXmKPExsXC5WfdrNs1Oy7N4PE2Jos569ewWXze8I/N 4tPLB4wWLza0M1p8Xf+L2eLppz4Wi8NzT7JaXN41h83i3pr/rBZHOzcxW5zftZbVYsfSfUwW lw4sYLI43nuAyWL+vc9sFps3TWW2OD5lKqPF7x9AHSdnTWZxEPb43trH4rFz1l12jwWbSj02 r9DyWLznJZPHplWdbB6bPk1i93h37hy7x4kZv1k85p0M9Hi/7yqbx+IXH5g8tv6y82iceo3N 4/MmuQD+KC6blNSczLLUIn27BK6M6ZM6mAoeSFccPHCNsYGxS7SLkYNDQsBEYtKschCTRUBF oqkzqouRk4NNQF3ixo2fzCC2CJB9auVy9i5GLg5mgePMEh8+LmIESQgLFEi8mjCJHaSXV8BC 4uABZZAaIYErzBJHJ38Fa+YVEJQ4OfMJC4jNLKAlcePfSyaQemYBaYnl/zhAwpwClhIvG76w gdiiAsoSB7YdZ5rAyDsLSfcsJN2zELoXMDKvYhTJzCvLTczMMdUrzs6ozMus0EvOz93ECIyz ZbV/Ju5g/HLZ/RCjAAejEg+vxcrYNCHWxLLiytxDjBIczEoivH110WlCvCmJlVWpRfnxRaU5 qcWHGKU5WJTEeb3CUxOEBNITS1KzU1MLUotgskwcnFINjNcbgl4GrJoVvGG5zM3l9m7tU0Jv W7qv7s/aq6fPKRYfdiL6XcKuhfYntrRE7pi83GVP7IOS5ceEDkd5R59vPXxRLj8oIStoxkwN j9MRsS2GSxdrVS9Mbb2upbiE9+IB7cfMu/u2f5hQc95xe9Gl0/ZL2e4vW6udmvI4vN/la2j+ KseziyPYPiixFGckGmoxFxUnAgDVlhqirwIAAA== X-CFilter-Loop: Reflected X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 0BAAF120003 X-Stat-Signature: m5dy8wecpje7zsbngq7zaa6x83cz5m7p X-Rspam-User: X-HE-Tag: 1717476236-308934 X-HE-Meta: U2FsdGVkX1+7Yil5LC+Z2Nz7N90FarKfcdMqssF1WBu+NuZfnzqaU8zMZD9FzSq5pz0C/8ELAE/tXzhW3b571eZ/Nuyj3WpvOpKCgyie/HteE0TMxLRCYaDzIGNXi/FSGT24NQMfSFoNfuVgcQLNqLIVl5+IIzR3yWSZo17EfQJ2eRgxmlywTSExDvkd6BABwhvMDAB6WY7ltQ2QKFisCrIsQWeB/zEXqOQMVQiwRsbDYqYoDeecXVzSnrKd+9uiI7NkWieaz6dzS/1OBpqp5tIXThZzW162K9vmIHPfS+QxKRfdiV7Nes4g6xdvzAbPq+dTJ4a0JfnVf316X4RG6OuxZCIUOFZ01M6throPJfdEBEZFYhkWcbNF4xIo6o3m2BnBUZSezZcyYSrFu5RJ1YrmsdUb8NF+v/XgeOgdfEqUVJ3dhsUknpOV0xt0RcEVANK3QjS+YaumZZzwYgMgEiq3/BqX2DvbPxmOZyikdQutpBNbILGnyvTv5ZEgiUJwwMD/8OuNbXLgESMfUEz6A3mfnzMJJq+wuyx1ze2qmQafswdP0WNTwI1EZXmqSz9lE4rYyuNJMCvg2f6l+j8clt/OSIPU+tKiTtBLAseV8Cf809rm78vURq+pWT0jmCepIt31yRqWNHu4SnFPzavJxoHgalGTLzDWxz76huhYfNTpBadKqf1sZAz5eeTOd4eeiyZMgigy4u2lI6UDRFUkTJvPZzohygcTKGbHB07FaOre+qWAkObXB6Vls00UdK7Hd6Lkcml9nZW5eKHC9Fwo27+vlP5rxYlYr88WkLklvIl56s7kQN1pWtbgCgRcCPV0Maqh8t7k9oFguD7T7KaszavLCjv0PhH0AiUZrnnzTuluIWeG/vEOzEF5JRVfbAFjMBv7h7c+ZOHLArEv/bx5mhA6gRsrQmXpllhKESfxInDVmqV6HdWdZbgNf+UtH9VW3ta5KYMlHS+uITryVt+ 1TxPuLvd M9bCJlyEsXAWyluI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Jun 04, 2024 at 10:53:48AM +0900, Byungchul Park wrote: > On Mon, Jun 03, 2024 at 06:23:46AM -0700, Dave Hansen wrote: > > On 6/3/24 02:35, Byungchul Park wrote: > > ...> In luf's point of view, the points where the deferred flush should be > > > performed are simply: > > > > > > 1. when changing the vma maps, that might be luf'ed. > > > 2. when updating data of the pages, that might be luf'ed. > > > > It's simple, but the devil is in the details as always. > > Agree with that. > > > > All we need to do is to indentify the points: > > > > > > 1. when changing the vma maps, that might be luf'ed. > > > > > > a) mmap and munmap e.i. fault handler or unmap_region(). > > > b) permission to writable e.i. mprotect or fault handler. > > > c) what I'm missing. > > > > I'd say it even more generally: anything that installs a PTE which is > > inconsistent with the original PTE. That, of course, includes writes. > > But it also includes crazy things that we do like uprobes. Take a look > > at __replace_page(). > > > > I think the page_vma_mapped_walk() checks plus the ptl keep LUF at bay > > there. But it needs some really thorough review. > > > > But the bigger concern is that, if there was a problem, I can't think of > > a systematic way to find it. > > > > > 2. when updating data of the pages, that might be luf'ed. > > > > > > a) updating files through vfs e.g. file_end_write(). > > > b) updating files through writable maps e.i. 1-a) or 1-b). > > > c) what I'm missing. > > > > Filesystems or block devices that change content without a "write" from > > the local system. Network filesystems and block devices come to mind. > > AFAIK, every network filesystem eventully "updates" its connected local > filesystem. It could be still handled at the point where updating the > local file system. > > > I honestly don't know what all the rules are around these, but they > > could certainly be troublesome. > > > > There appear to be some interactions for NFS between file locking and > > page cache flushing. > > > > But, stepping back ... > > > > I'd honestly be a lot more comfortable if there was even a debugging LUF > > I'd better provide a method for better debugging. Lemme know whatever > it is we need. > > > mode that enforced a rule that said: Do you means a debugging mode that can WARN or inform the situation that we don't want? If yes, sure. Now that I get this, I will re-read all you guys' talk. Byungchul > Why "debugging mode"? The following rules should be enforced always. > > > 1. A LUF'd PTE can't be rewritten until after a luf_flush() occurs > > "luf_flush() should be followed when.." is more correct because > "luf_flush() -> another luf -> the pte gets rewritten" can happen. So > it should be "the pte gets rewritten -> another luf by any chance -> > luf_flush()", that is still safe. > > > 2. A LUF'd page's position in the page cache can't be replaced until > > after a luf_flush() > > "luf_flush() should be followed when.." is more correct too. > > These two rules are exactly same as what I described but more specific. > I like your way to describe the rules. > > Byungchul > > > or *some* other independent set of rules that can tell us when something > > goes wrong. That uprobes code, for instance, seems like it will work. > > But I can also imagine writing it ten other ways where it would break > > when combined with LUF.