From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD22CC433FE for ; Tue, 18 Oct 2022 18:01:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 643C56B0072; Tue, 18 Oct 2022 14:01:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5F37A6B0075; Tue, 18 Oct 2022 14:01:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4E40E8E0001; Tue, 18 Oct 2022 14:01:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 3F9AB6B0072 for ; Tue, 18 Oct 2022 14:01:46 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 04178C084D for ; Tue, 18 Oct 2022 18:01:45 +0000 (UTC) X-FDA: 80034838212.22.8439E89 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) by imf22.hostedemail.com (Postfix) with ESMTP id 7CEE4C0035 for ; Tue, 18 Oct 2022 18:01:44 +0000 (UTC) Received: by mail-pl1-f181.google.com with SMTP id z20so14555711plb.10 for ; Tue, 18 Oct 2022 11:01:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=kbVWeBLmoXaL5wT4FGVpRcLjWyd0m50VYE8PAOnnEko=; b=a0iG9+gDGzV0/uFEh+xecDTUMLAjs0avyQ2SwEVTlqQ0OWuxIWVvI7+VGzewqEagT7 5N1mcJO41KbKS7i/17ZW9QV3RoHCd4fgRihwlp6D5/+M0mqFYNoJYrou3tjrqrlTJ8dz XbKOlmlgCSm7t4iqfFRIJOMlef9tuCJ85wiUwPbLRtSwyRA1hArJi9oGz7TdQwhuEumS xDaJQz404PU7P6NsWFLsZENDAml1aJHAJ3OBdTrq6GH4hjxTkfT6Qmn14UOcrFVPZstD 4oqbVikV/Ib0yqaGyN6ev3o2wWLK/uUDQCFdWKkr6BcCUVKSR7lOgRAYhE4EsX7u7G0i 93qQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=kbVWeBLmoXaL5wT4FGVpRcLjWyd0m50VYE8PAOnnEko=; b=J4L8WNR4JHhdY/fyi+qwWTZAmz+XzGC+PSbh+wTCSwx2xKdILsb3fFnxthOCLJnGpq T5D6VOAtHD+7bnWA3jTbMduIysHrdX0x3Ij/PvncsHjk54pqFu5rsJHU7Gr7VY8YHt8X 2f8SDphFkcpXWlB99NJDS4fVQk//2jsVYJUmWCm8uY0tPGNGYZ7RG2UFI+bUfZLF4KmV 84OvrGxoELBgG3G7Q7vJqbDBRTMvdYhx+iz0cs+eQp0aKYUsDlz1PWXw/b53FBsSAVR9 32Z4v36S6/tdWQ4NtVYUzLkiPvJh/H9TtaK0EHdbnqVoSlO+wjXeTFYgn7PmFS4gPESo cNfQ== X-Gm-Message-State: ACrzQf36yht73/eb98cErd5EH9lUKAJREV5hmKI+X2NRKK9yeYcCqi0h y3oYw4HHMqqhJNKwPbcIUJbxEBPjPgziRV1VIj1oqlGZ X-Google-Smtp-Source: AMsMyM4dRCMlLxwxpvWPKeEBQqgqTpkBvzI2y4UmcFNFY7lhRcumit+v8Ysop8DlhVkW1Pg4yIXBLJ/OEQVZxMId9tQ= X-Received: by 2002:a17:902:d512:b0:181:f1f4:fcb4 with SMTP id b18-20020a170902d51200b00181f1f4fcb4mr4248525plg.102.1666116103439; Tue, 18 Oct 2022 11:01:43 -0700 (PDT) MIME-Version: 1.0 References: <20221005180341.1738796-1-shy828301@gmail.com> <20221005180341.1738796-3-shy828301@gmail.com> <20221013123830.opbulq4qad56kuev@techsingularity.net> <20221017094132.vnanndrwa2yn7qcw@techsingularity.net> In-Reply-To: <20221017094132.vnanndrwa2yn7qcw@techsingularity.net> From: Yang Shi Date: Tue, 18 Oct 2022 11:01:31 -0700 Message-ID: Subject: Re: [PATCH 2/4] mm: mempool: introduce page bulk allocator To: Mel Gorman Cc: agk@redhat.com, snitzer@kernel.org, dm-devel@redhat.com, akpm@linux-foundation.org, linux-mm@kvack.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666116104; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kbVWeBLmoXaL5wT4FGVpRcLjWyd0m50VYE8PAOnnEko=; b=q2Fql3CeJVcM2S6kWJTvvnXpHMwCGNLxaUfCOFSX5XDTptg/in3fTy9RDeUn8qT6F5OPko 6Z5QQH7WejaE1HoGYtFf9UpJD37+sHl2K50DWWVLsChFMY80/kmLHuXgsUhy7iRX7VBZTj UsOyGSOiuKwjofuO9augx/nlHFfjb2M= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=a0iG9+gD; spf=pass (imf22.hostedemail.com: domain of shy828301@gmail.com designates 209.85.214.181 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666116104; a=rsa-sha256; cv=none; b=v9L/8vX6CeZEhxi/2FBPw9c64ZOAfNAhhPfvpfYGoz18KQzim2PPy9bXLog6DRNkCLTxOs uuKPonnQ6fYwdFm+0OJ2W87V33wMjHz0UvqcIzH1eVgqLOKrUzXJU9D0iE6Uc4IwDYNCT8 WFeyFtIZl9+XenHgypOu7L+8H7ctYUM= Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=a0iG9+gD; spf=pass (imf22.hostedemail.com: domain of shy828301@gmail.com designates 209.85.214.181 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 7CEE4C0035 X-Stat-Signature: kh8rca1kxe76rytuwogczthmk1nifp67 X-Rspam-User: X-HE-Tag: 1666116104-269452 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Oct 17, 2022 at 2:41 AM Mel Gorman wrote: > > On Thu, Oct 13, 2022 at 01:16:31PM -0700, Yang Shi wrote: > > On Thu, Oct 13, 2022 at 5:38 AM Mel Gorman wrote: > > > > > > On Wed, Oct 05, 2022 at 11:03:39AM -0700, Yang Shi wrote: > > > > Since v5.13 the page bulk allocator was introduced to allocate order-0 > > > > pages in bulk. There are a few mempool allocator callers which does > > > > order-0 page allocation in a loop, for example, dm-crypt, f2fs compress, > > > > etc. A mempool page bulk allocator seems useful. So introduce the > > > > mempool page bulk allocator. > > > > > > > > It introduces the below APIs: > > > > - mempool_init_pages_bulk() > > > > - mempool_create_pages_bulk() > > > > They initialize the mempool for page bulk allocator. The pool is filled > > > > by alloc_page() in a loop. > > > > > > > > - mempool_alloc_pages_bulk_list() > > > > - mempool_alloc_pages_bulk_array() > > > > They do bulk allocation from mempool. > > > > They do the below conceptually: > > > > 1. Call bulk page allocator > > > > 2. If the allocation is fulfilled then return otherwise try to > > > > allocate the remaining pages from the mempool > > > > 3. If it is fulfilled then return otherwise retry from #1 with sleepable > > > > gfp > > > > 4. If it is still failed, sleep for a while to wait for the mempool is > > > > refilled, then retry from #1 > > > > The populated pages will stay on the list or array until the callers > > > > consume them or free them. > > > > Since mempool allocator is guaranteed to success in the sleepable context, > > > > so the two APIs return true for success or false for fail. It is the > > > > caller's responsibility to handle failure case (partial allocation), just > > > > like the page bulk allocator. > > > > > > > > The mempool typically is an object agnostic allocator, but bulk allocation > > > > is only supported by pages, so the mempool bulk allocator is for page > > > > allocation only as well. > > > > > > > > Signed-off-by: Yang Shi > > > > > > Overall, I think it's an ok approach and certainly a good use case for > > > the bulk allocator. > > > > > > The main concern that I have is that the dm-crypt use case doesn't really > > > want to use lists as such and it's just a means for collecting pages to pass > > > to bio_add_page(). bio_add_page() is working with arrays but you cannot > > > use that array directly as any change to how that array is populated will > > > then explode. Unfortunately, what you have is adding pages to a list to > > > take them off the list and put them in an array and that is inefficient. > > > > Yeah, I didn't think of a better way to pass the pages to dm-crypt. > > > > > > > > How about this > > > > > > 1. Add a callback to __alloc_pages_bulk() that takes a page as a > > > parameter like bulk_add_page() or whatever. > > > > > > 2. For page_list == NULL && page_array == NULL, the callback is used > > > > > > 3. Add alloc_pages_bulk_cb() that passes in the name of a callback > > > function > > > > > > 4. In the dm-crypt case, use the callback to pass the page to bio_add_page > > > for the new page allocated. > > > > Thank you so much for the suggestion. But I have a hard time > > understanding how these work together. Do you mean call bio_add_page() > > in the callback? But bio_add_page() needs other parameters. Or I > > misunderstood you? > > > > I expected dm-crypt to define the callback. Using bio_add_page > directly would not work as the bulk allocator has no idea what to pass > bio_add_page. dm-crypt would likely need to create both a callback and an > opaque data structure passed as (void *) to track "clone" and "len" I see. Yeah, we have to pass the "clone" and "len" to the callback via pool_data. It should not be hard since dm-crypt already uses crypt_config to maintain a counter for allocated pages, we should just need to pass the struct to the callback as a parameter. But I'm wondering whether this is worth it or not? Will it make the code harder to follow? > > -- > Mel Gorman > SUSE Labs