From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D49A1C4332F for ; Thu, 13 Oct 2022 20:16:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1E88A6B0071; Thu, 13 Oct 2022 16:16:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 198AF6B0073; Thu, 13 Oct 2022 16:16:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 038746B0074; Thu, 13 Oct 2022 16:16:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id DEB5F6B0071 for ; Thu, 13 Oct 2022 16:16:45 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 9CAE6C060F for ; Thu, 13 Oct 2022 20:16:45 +0000 (UTC) X-FDA: 80017034370.03.B9441A1 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) by imf08.hostedemail.com (Postfix) with ESMTP id 349DF160031 for ; Thu, 13 Oct 2022 20:16:44 +0000 (UTC) Received: by mail-pl1-f178.google.com with SMTP id 10so2877622pli.0 for ; Thu, 13 Oct 2022 13:16:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=gOTkI6DiG9zNlMVvwhJN0xPlCDiBJf3npR9m1obgkeo=; b=Lsxl/tLwGNZv9z+oGntC+5LE/pkahStDM6pBWlZuCiAaTuym7A3vMYZz82HZHCOOdy 303PbGf9qgLhPz6gYeVjhsODFOQBDTKwPZ4o384PcBLrNJD6T/386rk3h5oOQ3l9kCkt bn5vpwBS4VoLHflw8Dlu0hy0elVT6i/Db9dfVa3kVwAAyJh+4EXsDSvnBGsrLUoHNH0v 4hIFnZeD7QxnfQL/Cq5pkIBG8QmBlIBemKRie1OAuHDVvYcAGJVhaeiTdnDf+f+ze2IR +dDha5Qt6qamaohKfhX1pCPK/JSTypuHSlgw7FuvY6eWjhccY0d6rzTFRje4k0y/rPMR L80Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=gOTkI6DiG9zNlMVvwhJN0xPlCDiBJf3npR9m1obgkeo=; b=MfLisLlwxWaCF6Ditbl0Q5jtADFImV2cvw+rkzUb8nDIMp475DA46pgb0RGQ6Ib6VQ +b0/ulRQZXzGRa/beoJGc3mh5f9I5ESaM/UtYOZh/acmTDGsmSGx59JHg+Me/vHgMDIg J8mnAvvvI/NLen03OKR1Q3bPZ7r9MWlkuV3JIU2bM5czzmllevdJ9A6ZUWxiG7FtB1C/ 8knZeGqQZPz5imBJkT8IzLmG67vsgMPGCHUBi5aakFtAt+lCKMjb2ZiUbLBGrEx41j8R hwF6P7NsI9KJlKwnDm32+vqM4MsVU+c8a2TPs1exb5joYWp2QJtVM2MKWximW4p+jL5/ xiyg== X-Gm-Message-State: ACrzQf0qxWzwea5duE9Q5hogSdipqzTQfPW5DeC2o8YOAOkfPk1oZU1u zIYeJJQKNFc+bz4ck2tMzXrlQCL6MAGSjzXN5V0= X-Google-Smtp-Source: AMsMyM6oJD/yMaxb7cro3W7Zw+cQjBDGxa+i3q11ifg/KVLAfGAnzcBpn57njRrsLsdEm4UMGMYmjk8ypjuUqiKsQ4M= X-Received: by 2002:a17:903:41cf:b0:183:5a22:c63e with SMTP id u15-20020a17090341cf00b001835a22c63emr1364803ple.61.1665692203956; Thu, 13 Oct 2022 13:16:43 -0700 (PDT) MIME-Version: 1.0 References: <20221005180341.1738796-1-shy828301@gmail.com> <20221005180341.1738796-3-shy828301@gmail.com> <20221013123830.opbulq4qad56kuev@techsingularity.net> In-Reply-To: <20221013123830.opbulq4qad56kuev@techsingularity.net> From: Yang Shi Date: Thu, 13 Oct 2022 13:16:31 -0700 Message-ID: Subject: Re: [PATCH 2/4] mm: mempool: introduce page bulk allocator To: Mel Gorman Cc: agk@redhat.com, snitzer@kernel.org, dm-devel@redhat.com, akpm@linux-foundation.org, linux-mm@kvack.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="Lsxl/tLw"; spf=pass (imf08.hostedemail.com: domain of shy828301@gmail.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1665692205; a=rsa-sha256; cv=none; b=d+x9SFyRTodW1Qn+VN+79+e3wPbycK12VzKduCYWzcfg27Bh85NI6uouHvIP9ljp0CqJth nSxV5Itj2PrCu/mFa6Y6toOuChnSAb7Qp4flOwoxrGZ8+LqchjJU8dFhrvOKhJnM2GNo0m QLe5XhXujjlRCKUXnAP0wJRabaPrLvI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665692205; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gOTkI6DiG9zNlMVvwhJN0xPlCDiBJf3npR9m1obgkeo=; b=RlvjOr9cJc69/Cwk4PgdJmJtSSkqXCn47ypCRMpa6hOGnEuZgw+ELCnGO7S8uwsaEKtpc6 FCnUh2UdspjKJY7LezePKxXK+Q1RJpBs6pHL3LitgbVa905Ry+iZOcVVMCXVx7XWBKPfnf +79XsxjuQOHyxvdGC8+08i6O6kaak0A= X-Rspam-User: Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="Lsxl/tLw"; spf=pass (imf08.hostedemail.com: domain of shy828301@gmail.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 349DF160031 X-Stat-Signature: zbyzj878zpijucw78weyrg91upfh6yap X-HE-Tag: 1665692204-905290 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Oct 13, 2022 at 5:38 AM Mel Gorman wrote: > > On Wed, Oct 05, 2022 at 11:03:39AM -0700, Yang Shi wrote: > > Since v5.13 the page bulk allocator was introduced to allocate order-0 > > pages in bulk. There are a few mempool allocator callers which does > > order-0 page allocation in a loop, for example, dm-crypt, f2fs compress, > > etc. A mempool page bulk allocator seems useful. So introduce the > > mempool page bulk allocator. > > > > It introduces the below APIs: > > - mempool_init_pages_bulk() > > - mempool_create_pages_bulk() > > They initialize the mempool for page bulk allocator. The pool is filled > > by alloc_page() in a loop. > > > > - mempool_alloc_pages_bulk_list() > > - mempool_alloc_pages_bulk_array() > > They do bulk allocation from mempool. > > They do the below conceptually: > > 1. Call bulk page allocator > > 2. If the allocation is fulfilled then return otherwise try to > > allocate the remaining pages from the mempool > > 3. If it is fulfilled then return otherwise retry from #1 with sleepable > > gfp > > 4. If it is still failed, sleep for a while to wait for the mempool is > > refilled, then retry from #1 > > The populated pages will stay on the list or array until the callers > > consume them or free them. > > Since mempool allocator is guaranteed to success in the sleepable context, > > so the two APIs return true for success or false for fail. It is the > > caller's responsibility to handle failure case (partial allocation), just > > like the page bulk allocator. > > > > The mempool typically is an object agnostic allocator, but bulk allocation > > is only supported by pages, so the mempool bulk allocator is for page > > allocation only as well. > > > > Signed-off-by: Yang Shi > > Overall, I think it's an ok approach and certainly a good use case for > the bulk allocator. > > The main concern that I have is that the dm-crypt use case doesn't really > want to use lists as such and it's just a means for collecting pages to pass > to bio_add_page(). bio_add_page() is working with arrays but you cannot > use that array directly as any change to how that array is populated will > then explode. Unfortunately, what you have is adding pages to a list to > take them off the list and put them in an array and that is inefficient. Yeah, I didn't think of a better way to pass the pages to dm-crypt. > > How about this > > 1. Add a callback to __alloc_pages_bulk() that takes a page as a > parameter like bulk_add_page() or whatever. > > 2. For page_list == NULL && page_array == NULL, the callback is used > > 3. Add alloc_pages_bulk_cb() that passes in the name of a callback > function > > 4. In the dm-crypt case, use the callback to pass the page to bio_add_page > for the new page allocated. Thank you so much for the suggestion. But I have a hard time understanding how these work together. Do you mean call bio_add_page() in the callback? But bio_add_page() needs other parameters. Or I misunderstood you? > > It's not free because there will be an additional function call for every > page bulk allocated but I suspect that's cheaper than adding a pile of > pages to a list just to take them off again. It also avoids adding a user > for the bulk allocator list interface that does not even want a list. > > It might mean that there is additional cleanup work for __alloc_pages_bulk > to abstract away whether a list, array or cb is used but nothing > impossible. > > -- > Mel Gorman > SUSE Labs