From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16AD7C072A2 for ; Fri, 17 Nov 2023 13:37:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8027D280017; Fri, 17 Nov 2023 08:37:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7AFD4280008; Fri, 17 Nov 2023 08:37:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 69F5E280017; Fri, 17 Nov 2023 08:37:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5B75F280008 for ; Fri, 17 Nov 2023 08:37:05 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 283B5140DD3 for ; Fri, 17 Nov 2023 13:37:05 +0000 (UTC) X-FDA: 81467547210.14.30FA34E Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf03.hostedemail.com (Postfix) with ESMTP id ED9EC20010 for ; Fri, 17 Nov 2023 13:37:02 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=FJ3q580j; spf=none (imf03.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700228223; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nx/Pv1PYCiC9DG1C8Yjkw4xazpeJGYAfjvQtO3Qtn20=; b=U7WBjy4l0ov9yYNt6w+k5vrIqpW6xndH4jeTl/oiJpRUWUDlOmjzX6eG+CdNnOD2dw7lJ7 xLfLN3WwTcwI8SFy2m20Tq5L5gkNvufl69eoczmC2Gd5EeSBQ01k997LSRrr2XQ3YhNsbi +scOPH3qdp85/UcrXTGvLKVi9+rNErY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700228223; a=rsa-sha256; cv=none; b=Cco3W7aXx26h4Frftj4h4B9lcK/Pq8I0qexK+bcIkqVS3PjIIcIwWCL+IkscMOj0BUPNgv 1qxMRiQFyEAenrvTRFVKb+LWYXf/6vZtyos+kb0lc6BHUoNneHaeo26mhKLBBsatJN+YMH Duf0IJsPMEN4SOdK+x0gL/NtZ4rQ6s4= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=FJ3q580j; spf=none (imf03.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=nx/Pv1PYCiC9DG1C8Yjkw4xazpeJGYAfjvQtO3Qtn20=; b=FJ3q580jGlZInbLHGdhlB9N7Pt F4s+C3nz+JJa8kXLtlM+K+3eW0mA/wJUsjVhbtInuZthow4RkWUn0AJbwxu+dV7Mq8IhHONUMDMN3 dkojxw82g6DS3T0bDLhQWwbrkTvPrbvFvpyO+RtrMjUASW/uYKwzppeBGukUJ91mbq+BlEBFjt166 iWwZgHMMq6EqWBUxVSeTQWl+mapiZQNweeKDIGZC1xJut34LCdpu8RUQMGYKzlp2FrgzpTQ8/G0tC Zp1+1uYo9sxks7F8FIAEVjNYnnVB7+5Bv8spiyOEV/VKeB/8FnqMvKee62WT4upc0il49AAeiMEhB P+5Bqfrw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1r3z1m-009g3V-Ul; Fri, 17 Nov 2023 13:36:59 +0000 Date: Fri, 17 Nov 2023 13:36:58 +0000 From: Matthew Wilcox To: Qu Wenruo Cc: Linux Memory Management List , Linux FS Devel , "linux-btrfs@vger.kernel.org" Subject: Re: Mixed page compact code and (higher order) folios for filemap Message-ID: References: <0e995d32-a984-4b65-b9e3-67fc62cc2596@gmx.com> <9ecbebd3-4dc1-4560-9616-1af861c376e1@gmx.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <9ecbebd3-4dc1-4560-9616-1af861c376e1@gmx.com> X-Stat-Signature: jwf8pyw34yctcstwuzzd14h5cz3dh3wo X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: ED9EC20010 X-Rspam-User: X-HE-Tag: 1700228222-976638 X-HE-Meta: U2FsdGVkX1+c7XygXMVvQ9AUKRfEy3DrAb7w5qVwX1GpAKeFOa1pzs8+h/xq3zR+cdATLsEhiXysRd3XMiBF91vXAwir+pDxhlWLWPL9jL7IVANkH7EEEpEfQsZBOTJoAtwPVsd+4/ZVI2eShbE/GSH3plTaI/PdKX2Nphz55u2tTXVOYgBWRZPRb/rTNzX173I0yVkZ0zFN8OS41+cWmgQc8L4kqJaMlbmT5Y1Tt+nPhwBLVt4GcPFE+G8Jo0zi7++7g9p00ySJjv6OiEfKbVQZITH5flmM89Kv3ViaWJG9SD7sh/1dPTDNRKe/7z8KZAqYxuyUFFeJmNZoMlypKuQNQAktLM8maX5hmtXA3vbFPB3A8wTJ77B9l2G0t/Xb9SsaxFIUPzLWwEbyUoKfdiKkxeury1jqiIj0F6lhhlgFSBliATnesOQkINHfqsLcFsUHJTvHLCjxbvPhthZXJL9m1A5CUeKGdWvQCnqyZrfjFFf9Z59dvthWbn8LzKnlq0Gl4cKRZkOexeOiN54Hw6kj11BqY4eDomhNEXXcmSuxHzHSf/m9B1zZa5qxm2xSLRGIbIsjUilc+VVWFyo95gVH8PnsBvlbyfqpYzJYGimur6w18aGVoeru5XxxBz4FHR9DrtFrK1AoAI2jjqwIXtjWQ6axNqXZqcmV8EEFdnFBCYjfUF6mNr1htJQknPOLF8S+fqkwCsk8Br8rn1KVU/Z+8V+yKRDgcx+h0N0+cokDsWHnX4KM3ygozbekR0Ozcs8H4nyczqW6aV88aq+jaa1tfImv0hPOnqIsrdJ42KZo9/5dZ1xr1DKZYGn872Ka6DyLJgnUc7D4Y8wtQEPaM/bRcBSTF8AYlYvCuRC3NWM6B7kqD9VH4G978m9VBNITbTncsUjOKkSjWfOCU49/p2G0lN4aLVNxRL7gCuu9ehPiXFj7YgDeOPdyoCkQYcsLbSQYJ8b6EcVn/o9GJKS /gVlLT9I SFRmv05ncOWRZNsDAdTvmOWveir6bGYJTb3H/ZgRokEfDaMXbzsoPoXbfdDIK0XFNI1SFcBtqEQvLA3jeDnxzEhx1vkvQsD08YzEGGTAFi+mmJEsy0IhrDgIJX9Fj5fu0IjlBjuIgYMylB4G8gvo5KCtvoB55sHe5bOjptSqiL7e4KnmqtvcRKnPVKg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Nov 17, 2023 at 09:10:10AM +1030, Qu Wenruo wrote: > On 2023/11/17 00:53, Matthew Wilcox wrote: > > On Thu, Nov 16, 2023 at 04:00:40PM +1030, Qu Wenruo wrote: > > > On 2023/11/16 15:35, Matthew Wilcox wrote: > > > > On Thu, Nov 16, 2023 at 02:11:00PM +1030, Qu Wenruo wrote: > > > > > E.g. if I allocated a folio with order 2, attached some private data to > > > > > the folio, then call filemap_add_folio(). > > > > > > > > > > Later some one called find_lock_page() and hit the 2nd page of that folio. > > > > > > > > > > I believe the regular IO is totally fine, but what would happen for the > > > > > page->private of that folio? > > > > > Would them all share the same value of the folio_attach_private()? Or > > > > > some different values? > > > > > > > > Well, there's no magic ... > > > > > > > > If you call find_lock_page(), you get back the precise page. If you > > > > call page_folio() on that page, you get back the folio that you stored. > > > > If you then dereference folio->private, you get the pointer that you > > > > passed to folio_attach_private(). > > > > > > > > If you dereference page->private, *that is a bug*. You might get > > > > NULL, you might get garbage. Just like dereferencing page->index or > > > > page->mapping on tail pages. page_private() will also do the wrong thing > > > > (we could fix that to embed a call to page_folio() ... it hasn't been > > > > necessary before now, but if it'll help convert btrfs, then let's do it). > > > > > > That would be great. The biggest problem I'm hitting so far is the page > > > cache for metadata. > > > > > > We're using __GFP_NOFAIL for the current per-page allocation, but IIRC > > > __GFP_NOFAIL is ignored for higher order (>2 ?) folio allocation. > > > And we may want that per-page allocation as the last resort effort > > > allocation anyway. > > > > > > Thus I'm checking if there is something we can do here. > > > > > > But I guess we can always go folio_private() instead as a workaround for > > > now? > > > > I don't understand enough about what you're doing to offer useful > > advice. Is this for bs>PS or is it arbitrary large folios for better > > performance? If the latter, you can always fall back to order-0 folios. > > If the former, well, we need to adjust a few things anyway to handle > > filesystems with a minimum order ... > > > > In general, you should be using folio_private(). page->private and > > page_private() will be removed eventually. > > Just another question. > > What about flags like PageDirty? Are they synced with folio? Yes. You can SetPageDirty() in one function and then folio_test_dirty() in another. Eventually all the PageFoo() functions will be removed, except PageHWPoison and PageAnonExclusive. > The declaration goes PF_HEAD for policy, thus for order 0 it makes no > difference, but for higher order folios, we should switch to pure folio > based operations other than mixing page and folios? Every function in btrfs should be folio based. There should be nothing in btrfs that deals with pages. Take a look at iomap's buffered I/O paths for hints -- there's a per-block dirty and uptodate bit, but other than that, everything is done with folios.