From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9EFF2C61DA4 for ; Thu, 16 Feb 2023 19:04:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D09426B0071; Thu, 16 Feb 2023 14:04:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CB8436B0072; Thu, 16 Feb 2023 14:04:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BA7CD6B0073; Thu, 16 Feb 2023 14:04:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A947B6B0071 for ; Thu, 16 Feb 2023 14:04:45 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 69DB41206CF for ; Thu, 16 Feb 2023 19:04:45 +0000 (UTC) X-FDA: 80474081730.30.ED86A46 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf20.hostedemail.com (Postfix) with ESMTP id EE7901C000E for ; Thu, 16 Feb 2023 19:04:40 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=WJ5DhKnM; dmarc=none; spf=none (imf20.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676574282; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Yawla20fuhgFU+ZM3/0+0yPtywAw64xLIG63jDmyTgk=; b=nLksQDLtYMzkXXhxoR6J6ox1LNqD6/PJgQdXYXXzC3ovCrXxkUPC6N6/uoDdQTb++4A3IS tdn9V7B/rtcr6CYFj/Y529YeyowXchbyR1t1j8v1QNfJOvv1i7vSKN0UtuCTyNW6v96Pec zxGnpWsywMQ2A2iabHDXDkU2uqf+2Fw= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=WJ5DhKnM; dmarc=none; spf=none (imf20.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676574282; a=rsa-sha256; cv=none; b=m/kP82jZ17velhcdmHiTp6k5uIIUHPZuRP08wqKdhxVepO1RHwqBnsUVtO9874U0KVUbmd 6CyM/wZDhojIv71VUQzOYqXCpSYZm5IGADBKJjj0qhcdMlxe1DHPgvNhUQy0CASRQTd2SX 7lhHhZktWgQovqEFPjxgajHwDZpGvjY= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=Yawla20fuhgFU+ZM3/0+0yPtywAw64xLIG63jDmyTgk=; b=WJ5DhKnMSiKnsiY5Co43FPaRqa rM3DttQzJiela82Afuc+NxA9x76rhFPUEDEvauyhgRZVOYh/36ZefNavqhAMXeEJLnC8w1LsV9ZJz wnuQpdz3Ox4d6nWawtAb/YGlYdRYrZeLtv8SHw3odjH0WXM1SqTfnXyVx8+ZdAXMB69xSJxE/hWCq tosiooAHynOPpDiriNLCJ32QnxlOF6cYpYT22+DYUMaMjtlJUn4hoLKkn/Q+urGyU18zVThHpN2l7 EJxmwN3IPgEtPxeyhuCmNjVaOTtU9EseAv3bdUV0KMtWtDcgrBRm5IhK8PGsq6o1l/8WhSy41dWt5 pQmdTPsA==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1pSjYO-008e70-Gb; Thu, 16 Feb 2023 19:04:24 +0000 Date: Thu, 16 Feb 2023 19:04:24 +0000 From: Matthew Wilcox To: Mikulas Patocka Cc: snitzer@kernel.org, Yang Shi , mgorman@techsingularity.net, agk@redhat.com, dm-devel@redhat.com, akpm@linux-foundation.org, linux-block@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH] dm-crypt: allocate compound pages if possible Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: EE7901C000E X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: i1ktzitmd8w6re41euxtow8wrnghzahz X-HE-Tag: 1676574280-835371 X-HE-Meta: U2FsdGVkX18jsb0XsJtEqrP28hyjcVwDDPC7F9Wb73rmC1SA+qaEr9sdEMGxxkzzB5URa3bhTBFQzy+dUL3FyO5I+lFqvTfeqd6OEVwfD1Q9MxwDcWnipCNE/w1rsGiKONPS/NvchTRgYy1vKs9cxAyObmi3sW/mH6JEjM1eKdNVJcABgtetJrLB3FJSOiObwKPZeDTvqgDCBNwqhzgZoCysbqoFGSISNh55f3a2gDAJbU+5435tadvZZN9rZYfnb8DpKHZlgYxGDnVxUA8MGHUUUyavjYbFSibP7Kjmbv0Otpkb7jAWyZbWALUf2I64mU6AUHNB1g3c5fGYZmfDPJkLLZZvhHdCBd8xvzgSw23ONTpDawl5/Mybn6m9wxuBapdIxpcUNuUcVH0ErjJozEGKiRduPcZ/YluCzq4vYLGfzrGcadPnE3JJb8BwMYsWwyABxCJJM5wEpODpCK22QSmiopku40xPEPcb1jTmo01va4cybl5gYOsTUlr4a7IHA3zluQkBHaZCow5hGC8lDzhmKZm4Ntk/hlwqvuaUXfGVujVMYMkaL+tu9kx95xPVKS47xMlAoLhWvs1g5XAImTv/1u9tOBlsWbjkvoNrvRqZUFQVOQKbJXVKVxH+b1e255rIJg9Ofr/zd9p2+UpjpVAJ5vAZJt+ydkIAURvlLEew1UFES7xCOKlw0JwP2cAWu+tBAtUYWuXwn3qpjXPCQRR7J19Sk1WB2E9w3E84Zq+Xm74Gxxdbc5vgGG1CPTwMyVlRAE7Bd2wZLQqLhMVw8k4aMuCIuIPH35d1xYwzg6iUNM48WjOyHNEJQJDCKldjsrcvzBxws5RKsuj7zIubrfu7FGE56quoegsyvXcDubqekf1+sx/OjYv1xSKOm+DmfIcvUjAzWqRvi8El0ejq1qGwjeZh5ZyWGfXSXQnOT/HHDJ0p2KINcnoLL2KyA5bwgyX7/OhBSe7ypCW5OvL v4fxqHp6 lnPWujX9tcvqOEGSGBAVhYYDLe4YjcZX+h3bv299wkVLhTEjwz5bIoiGnHAvD4AoPq36DUhwJzi8q0Lb8aYv3pvq4nV/Xa/SJy1fOdZeNFJfD/yDiuVhLuXEyZgFI8+OuMm6vAPrZh9xDoP8psnfdYl0PSg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Feb 16, 2023 at 12:47:08PM -0500, Mikulas Patocka wrote: > + while (order > 0) { > + page = alloc_pages(gfp_mask > + | __GFP_NOMEMALLOC | __GFP_NORETRY | __GFP_NOWARN, order); ... | __GFP_COMP > page = mempool_alloc(&cc->page_pool, gfp_mask); > if (!page) { > crypt_free_buffer_pages(cc, clone); > bio_put(clone); > gfp_mask |= __GFP_DIRECT_RECLAIM; > + order = 0; > goto retry; > } > > - len = (remaining_size > PAGE_SIZE) ? PAGE_SIZE : remaining_size; > - > - bio_add_page(clone, page, len, 0); > +have_pages: > + page->compound_order = order; No. You'll corrupt the next page if page is order-0, which it is if it came from the mempool. Also we've deleted page->compound_order in -next so you can't make this mistake. Using __GFP_COMP will set this field for you, so you can just drop this line. > - remaining_size -= len; > + for (o = 0; o < 1U << order; o++) { > + unsigned len = min((unsigned)PAGE_SIZE, remaining_size); > + bio_add_page(clone, page, len, 0); > + remaining_size -= len; > + page++; You can add multiple pages at once, whether they're compound or not. So replace this entire loop with: bio_add_page(clone, page, remaining_size, 0); > @@ -1711,10 +1732,23 @@ static void crypt_free_buffer_pages(stru > { > struct bio_vec *bv; > struct bvec_iter_all iter_all; > + unsigned skip_entries = 0; > > bio_for_each_segment_all(bv, clone, iter_all) { > - BUG_ON(!bv->bv_page); > - mempool_free(bv->bv_page, &cc->page_pool); > + unsigned order; > + struct page *page = bv->bv_page; > + BUG_ON(!page); > + if (skip_entries) { > + skip_entries--; > + continue; > + } > + order = page->compound_order; > + if (order) { > + __free_pages(page, order); > + skip_entries = (1U << order) - 1; > + } else { > + mempool_free(page, &cc->page_pool); > + } You can simplify this by using the folio code. struct folio_iter fi; bio_for_each_folio_all(fi, bio) { if (folio_test_large(folio)) folio_put(folio); else mempool_free(&folio->page, &cc->page_pool); } (further work would actually convert this driver to use folios instead of pages)