From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98C05C4332F for ; Thu, 13 Oct 2022 12:38:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D821E6B0071; Thu, 13 Oct 2022 08:38:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D314E6B0073; Thu, 13 Oct 2022 08:38:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C20076B0074; Thu, 13 Oct 2022 08:38:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id AF26C6B0071 for ; Thu, 13 Oct 2022 08:38:35 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 7C79F16113B for ; Thu, 13 Oct 2022 12:38:35 +0000 (UTC) X-FDA: 80015879790.18.9134557 Received: from outbound-smtp28.blacknight.com (outbound-smtp28.blacknight.com [81.17.249.11]) by imf14.hostedemail.com (Postfix) with ESMTP id E8C8810001A for ; Thu, 13 Oct 2022 12:38:33 +0000 (UTC) Received: from mail.blacknight.com (pemlinmail05.blacknight.ie [81.17.254.26]) by outbound-smtp28.blacknight.com (Postfix) with ESMTPS id 39640CCD93 for ; Thu, 13 Oct 2022 13:38:32 +0100 (IST) Received: (qmail 16548 invoked from network); 13 Oct 2022 12:38:31 -0000 Received: from unknown (HELO techsingularity.net) (mgorman@techsingularity.net@[84.203.198.246]) by 81.17.254.9 with ESMTPSA (AES256-SHA encrypted, authenticated); 13 Oct 2022 12:38:31 -0000 Date: Thu, 13 Oct 2022 13:38:30 +0100 From: Mel Gorman To: Yang Shi Cc: agk@redhat.com, snitzer@kernel.org, dm-devel@redhat.com, akpm@linux-foundation.org, linux-mm@kvack.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 2/4] mm: mempool: introduce page bulk allocator Message-ID: <20221013123830.opbulq4qad56kuev@techsingularity.net> References: <20221005180341.1738796-1-shy828301@gmail.com> <20221005180341.1738796-3-shy828301@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline In-Reply-To: <20221005180341.1738796-3-shy828301@gmail.com> ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665664714; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EaMUDQFT0yF7F2yebklG0LvXLGKMn2nIlSNprvEDWHQ=; b=bclAdrQxMNmSd1GVCb/HfhtzGLrYSV2N8Ik8o9yZLwJjXvfezelDvP+dxAym452muoz+sZ 5RyRn0sbuVoeJraACm1cvt0zLiFWXBoNw9x6tnPQQPctkhRBkpWzOJ8mVyp7QX37Grb0dC jhUMhmYS4P3pisxPkiPb7rnL8eDjdyc= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of mgorman@techsingularity.net designates 81.17.249.11 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1665664714; a=rsa-sha256; cv=none; b=u39yl86B5NGxo/FlmqRCnc242XE54oVMMFO7eFt+I0CXW3AoVYe+raroU6NGLtubhfjKTI pRsgrlQTyrzo07d1grc50AM3LTN59TdCfW+Zeg6ozdoa6gdhEEd0f5+zB+S/AXE+uZ6e+k kd1jvqS03pkmkJqli8/CKirjr3JyW0g= X-Rspam-User: X-Rspamd-Queue-Id: E8C8810001A X-Stat-Signature: mrzsn8fopas8e1mnguh9ng1aie6peobz Authentication-Results: imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of mgorman@techsingularity.net designates 81.17.249.11 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net; dmarc=none X-Rspamd-Server: rspam10 X-HE-Tag: 1665664713-91641 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Oct 05, 2022 at 11:03:39AM -0700, Yang Shi wrote: > Since v5.13 the page bulk allocator was introduced to allocate order-0 > pages in bulk. There are a few mempool allocator callers which does > order-0 page allocation in a loop, for example, dm-crypt, f2fs compress, > etc. A mempool page bulk allocator seems useful. So introduce the > mempool page bulk allocator. > > It introduces the below APIs: > - mempool_init_pages_bulk() > - mempool_create_pages_bulk() > They initialize the mempool for page bulk allocator. The pool is filled > by alloc_page() in a loop. > > - mempool_alloc_pages_bulk_list() > - mempool_alloc_pages_bulk_array() > They do bulk allocation from mempool. > They do the below conceptually: > 1. Call bulk page allocator > 2. If the allocation is fulfilled then return otherwise try to > allocate the remaining pages from the mempool > 3. If it is fulfilled then return otherwise retry from #1 with sleepable > gfp > 4. If it is still failed, sleep for a while to wait for the mempool is > refilled, then retry from #1 > The populated pages will stay on the list or array until the callers > consume them or free them. > Since mempool allocator is guaranteed to success in the sleepable context, > so the two APIs return true for success or false for fail. It is the > caller's responsibility to handle failure case (partial allocation), just > like the page bulk allocator. > > The mempool typically is an object agnostic allocator, but bulk allocation > is only supported by pages, so the mempool bulk allocator is for page > allocation only as well. > > Signed-off-by: Yang Shi Overall, I think it's an ok approach and certainly a good use case for the bulk allocator. The main concern that I have is that the dm-crypt use case doesn't really want to use lists as such and it's just a means for collecting pages to pass to bio_add_page(). bio_add_page() is working with arrays but you cannot use that array directly as any change to how that array is populated will then explode. Unfortunately, what you have is adding pages to a list to take them off the list and put them in an array and that is inefficient. How about this 1. Add a callback to __alloc_pages_bulk() that takes a page as a parameter like bulk_add_page() or whatever. 2. For page_list == NULL && page_array == NULL, the callback is used 3. Add alloc_pages_bulk_cb() that passes in the name of a callback function 4. In the dm-crypt case, use the callback to pass the page to bio_add_page for the new page allocated. It's not free because there will be an additional function call for every page bulk allocated but I suspect that's cheaper than adding a pile of pages to a list just to take them off again. It also avoids adding a user for the bulk allocator list interface that does not even want a list. It might mean that there is additional cleanup work for __alloc_pages_bulk to abstract away whether a list, array or cb is used but nothing impossible. -- Mel Gorman SUSE Labs