From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0BCA0C77B7D for ; Sun, 7 May 2023 21:07:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 410D96B0078; Sun, 7 May 2023 17:07:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3C0E4900002; Sun, 7 May 2023 17:07:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1EC266B007E; Sun, 7 May 2023 17:07:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 0A9A66B0078 for ; Sun, 7 May 2023 17:07:16 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id B7490807A4 for ; Sun, 7 May 2023 21:07:15 +0000 (UTC) X-FDA: 80764694430.24.C645C9B Received: from p3plwbeout25-02.prod.phx3.secureserver.net (p3plsmtp25-02-2.prod.phx3.secureserver.net [216.69.139.14]) by imf28.hostedemail.com (Postfix) with ESMTP id 4A7F1C0007 for ; Sun, 7 May 2023 21:07:13 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of phillip@squashfs.org.uk designates 216.69.139.14 as permitted sender) smtp.mailfrom=phillip@squashfs.org.uk; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1683493633; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=skFksfcXQjG2yXiSlXcEn+oEQLv01AGomXO3+qzUO+Y=; b=cTK2dxtOett1ASuFL0HJn/j38aAiqlfe75tWi4uHeLwd1USf/F6bbcTh7tveHooyYVl5AO /rK7UKtTHHoYvhNCyy6F0W7V4ep3Tq1xgFWnUJkBfynR3222fOYLFusEJQV7ZNddxZpOkF SYp+1qxTZciAEBCuGw3njkpKQwG6LWI= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of phillip@squashfs.org.uk designates 216.69.139.14 as permitted sender) smtp.mailfrom=phillip@squashfs.org.uk; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1683493633; a=rsa-sha256; cv=none; b=ZMIh1uZmFwJRZ7PZAFrl5aHzAQ1yVJNgerJPxgRm93DMr0FZ5CuoWzMtPEQwByE9BWlSsy OaKxpS0JzsmspP3OzSEjE/Rx3yE0aaNa3HE7NzsKPWZYvOO+jQDsM9G6WHBBDKGBTrc1+d vkEx2G42CeGEqro8mmdb95Vcnb091R8= Received: from mailex.mailcore.me ([94.136.40.145]) by :WBEOUT: with ESMTP id vlb4pB8KpNgfIvlb5pj3O6; Sun, 07 May 2023 14:07:12 -0700 X-SECURESERVER-ACCT: phillip@squashfs.org.uk X-SID: vlb4pB8KpNgfI Received: from 82-69-79-175.dsl.in-addr.zen.co.uk ([82.69.79.175] helo=[192.168.178.87]) by smtp02.mailcore.me with esmtpa (Exim 4.94.2) (envelope-from ) id 1pvlb4-0002Oo-Qe; Sun, 07 May 2023 22:07:13 +0100 Message-ID: Date: Sun, 7 May 2023 22:07:07 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: [PATCH 1/1] mm/oom_kill: trigger the oom killer if oom occurs without __GFP_FS From: Phillip Lougher To: Hui Wang , Michal Hocko Cc: Gao Xiang , linux-mm@kvack.org, akpm@linux-foundation.org, surenb@google.com, colin.i.king@gmail.com, shy828301@gmail.com, hannes@cmpxchg.org, vbabka@suse.cz, hch@infradead.org, mgorman@suse.de References: <20230426051030.112007-1-hui.wang@canonical.com> <20230426051030.112007-2-hui.wang@canonical.com> <68b085fe-3347-507c-d739-0dc9b27ebe05@linux.alibaba.com> <4aa48b6a-362d-de1b-f0ff-9bb8dafbdcc7@canonical.com> <70000460-ace2-3965-084d-34be65a6bd6a@squashfs.org.uk> Content-Language: en-GB In-Reply-To: <70000460-ace2-3965-084d-34be65a6bd6a@squashfs.org.uk> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Mailcore-Auth: 439999529 X-Mailcore-Domain: 1394945 X-123-reg-Authenticated: phillip@squashfs.org.uk X-Originating-IP: 82.69.79.175 X-CMAE-Envelope: MS4xfK329O3SlblMCfY2CvgArauX2NkO5dYtT6hoSMmFKcchA1wDYuGRU+DslvrpvCW6zZNDI0NWb08BLFF9E4hXtqd1g1X8t3+g3VuKQkQ5DADFAIIby7zb uF1XRDJimOsZwT/3Z0M6XqH1muBpOo6KVEAg0zML+vKQkcMehZSnC9I+J7wyZrEvZ5DyYmWU6/rt1tRJRayhU4Fb3aV0/oJ3/v0= X-Stat-Signature: 6r4drgkszftadd7p9pg7o6qetxn785x1 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 4A7F1C0007 X-Rspam-User: X-CMAE-Analysis: v=2.4 cv=IZs/G0ma c=1 sm=1 tr=0 ts=64581301 a=BZCDkKuxV8TCbhJY3vv5nA==:117 a=84ok6UeoqCVsigPHarzEiQ==:17 a=ggZhUymU-5wA:10 a=IkcTkHD0fZMA:10 a=P0xRbXHiH_UA:10 a=FXvPX3liAAAA:8 a=Bt9QMstweItqx_BhPtsA:9 a=QEXdDO2ut3YA:10 a=UObqyxdv-6Yh2QiB9mM_:22 X-HE-Tag: 1683493633-334578 X-HE-Meta: U2FsdGVkX1/Yee4qUdZqtCeQcViP/YPUzxSbJs4YX7IZr1Yk4LTSPQGHoeBKIXGhScZyqGzuQJyzmtw7HAoCbG4AbgfzNsXNkkPwQD+Hmk9OOtsDu8Vjmeu5G/S8AMYtRVFY49ynRbzlhw+EwbvWXREH+4hXAScP9EZJsQMYz2pgIbj9gX/HJdvOW8u3YpDQaiLUcm8+w3Ca+yLPb77c5TuRtJL1OL/NOkghoR6FJo4U8QjS2hyu0/151MS4wdwG8K/yY7SdcMkyNwdnDQiQ+/rMNqMi3XP3PgxqmaD02/JqQgWU8yVnOnsNGXNVpEdKmNjm8Nb9VZSOiNUk4jN7M0ynSMii6TXxBA15kwX/ze+RJdr1pZW3833T9oh0Wp7toeaRZ2OMWw0kfOTxL0KTbRbzHOV/2525i1ysGFzXtu7MZG5abMkGL1QnZ61WoEA1WJSmTEZAxwwhThHzNdL3vUqORaCDrYMTAa2rzYO0he4oIJCOF9BLWMKynWcQmOHNWdGvHwPklEXIWYmLApk3Hd9gOvk9R09qNWODbw5IOFgMsgY70BwPEgNY87eG2VzO+WdrzoAHwxH8kvFKFe7xYoA0FZOdSiS2xI37hFLhFLDpNMT1Q2/mecgBM6V8S441Rn1FMQ9KHJ3FKS37x3vDWm4IWI/lblDeD94dpuFi2DPnjo5Fdt4lwt6Jap1hbtiVjEw9nlsJ2AkNdpAIUVDF4mR6kOtMXoATasGTRZ7g17nKmV3IGuhXCv+unfCo9Yiy3AES0WIi4/Uy9kajDphrGMnRgTwAzk85bj98CYK1a6iAivWyyPv2BJJOOY+3NG6dQX5PM5B9qI4bX544RHjoBL1RVA3n022YkU4/PRwVxjWJy5/pW9xsXzmCcLHTKYpQPU7Bf3ndqMX47fIhpkHTrATTexTlgAMNUXsPDlIFMgRyZto3Hh7qhfeRSXWPht/16mMRPY9Z1FG6vS+p1lM yMkHHtlq 0XBYP++/P5wKPPyKpZxCdqZ16bRZJlAXVGJ/SjtUbz/0Hk1q6wwciBDbGD1FD8N5a5HYAw50jhv1I6yQaBA0xndxvsxEbwDvf0Fcde+6CFFFmWcR/YZrhHnR57mekOU/VvuOHqbrHA9Be9BXZ6OC1NudEXiIUJIp0SI9MRI9ORxxGYIIRabKaAKonsxEfoFAFIQTqQaNuCsDHXs4f2J/RTAPYnIgPcCFzA5OfSP/QPZdw30WQ/uhGtwVwWYYLC2XuX3g2Cb8ITB+vkS5gLvJdTiE7EyDa/9XiTmeqOVklXJlFCDcnAanz8O8b20Rg9qCQXh/V5CSntZtMaXWXld8NO4A+/ta7srjJjyy9fBRRZv/i0vnyQOrd5xqF8Xu/9/Dl10c5N37v9+l28n7LBBINEsE+I3NrUJY3q56oWw0VYL2w6RwSA8qMVuQQkMGCfVa9MSgVMKhOWxgvI/khj96eNVaoQIldOALLfOX/Gru4iLUrso4zBlQ//EXmnZaouO2YAbQMhEFPNIQNqU7UohmGxkt5WJpXSOBW6Kys/SyGrBcbbPcfxFOW//kM0qScRLIefUG43YSnEG+BENb9HAAxpYb7KSTbfAsGzxGiqjIarLpTYgjDhh6WOWo2gUzHqMMTPpOMas/H65vQIbirLN1fFUVsG1jxxaHwZtmK X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 03/05/2023 20:10, Phillip Lougher wrote: > > On 03/05/2023 12:49, Hui Wang wrote: >> >> On 4/29/23 03:53, Michal Hocko wrote: >>> On Thu 27-04-23 11:47:10, Hui Wang wrote: >>> [...] >>>> So Michal, >>>> >>>> Don't know if you read the "[PATCH 0/1] mm/oom_kill: system enters >>>> a state >>>> something like hang when running stress-ng", do you know why >>>> out_of_memory() >>>> will return immediately if there is no __GFP_FS, could we drop >>>> these lines >>>> directly: >>>> >>>>      /* >>>>       * The OOM killer does not compensate for IO-less reclaim. >>>>       * pagefault_out_of_memory lost its gfp context so we have to >>>>       * make sure exclude 0 mask - all other users should have at >>>> least >>>>       * ___GFP_DIRECT_RECLAIM to get here. But mem_cgroup_oom() has to >>>>       * invoke the OOM killer even if it is a GFP_NOFS allocation. >>>>       */ >>>>      if (oc->gfp_mask && !(oc->gfp_mask & __GFP_FS) && >>>> !is_memcg_oom(oc)) >>>>          return true; >>> The comment is rather hard to grasp without an intimate knowledge of >>> the >>> memory reclaim. The primary reason is that the allocation context >>> without __GFP_FS (and also __GFP_IO) cannot perform a full memory >>> reclaim because fs or the storage subsystem might be holding locks >>> required for the memory reclaim. This means that a large amount of >>> reclaimable memory is out of sight of the specific direct reclaim >>> context. If we allowed oom killer to trigger we could invoke the oom >>> killer while there is a lot of otherwise reclaimable memory. As you can >>> imagine not something many users would appreciate as the oom kill is a >>> very disruptive operation. In this case we rely on kswapd or other >>> GFP_KERNEL like allocation context to make forward instead. If there is >>> really nothing reclaimable then the oom killer would eventually hit >>> from >>> elsewhere. >>> >>> HTH >> Hi Michal, >> >> Understand. Thanks for explanation. So we can't remove those 2 lines >> of code. >> >> Here in my patch, letting a kthread allocate a page with GFP_KERNEL, >> It could possibly trigger the reclaim and if nothing reclaimable, >> trigger the oom killer. Do you think it is a safe workaround for the >> issue we are facing currently? >> >> >> And Hi Phillip, >> >> What is your opinion on it, do you have a direction to solve this >> issue from filesystem? >> > > The following patch creates the concept of "squashfs contexts", which > moves all memory dynamically allocated (in a readahead/read_page path) > into a single structure which can be allocated and deleted once.  It > then creates a pool of these at filesystem mount time.  Threads > entering readahead/read_page will take a context from the pool, and > will then perform no dynamic memory allocation. > > The final patch-series will make this a non-default build option for > systems that need this. > > Phillip > > An updated version of the patch. From 24f22dbd7fed8d0a90d11a4ce80545d30a9acfdc Mon Sep 17 00:00:00 2001 From: Phillip Lougher Date: Sun, 7 May 2023 20:16:55 +0100 Subject: [PATCH] read_context v2 Signed-off-by: Phillip Lougher --- fs/squashfs/Makefile | 1 + fs/squashfs/block.c | 67 ++++----- fs/squashfs/cache.c | 66 ++++++--- fs/squashfs/decompressor.c | 14 +- fs/squashfs/decompressor_multi.c | 2 + fs/squashfs/decompressor_multi_percpu.c | 2 + fs/squashfs/decompressor_single.c | 2 + fs/squashfs/file.c | 61 ++++---- fs/squashfs/file_direct.c | 59 ++++---- fs/squashfs/page_actor.c | 37 ++--- fs/squashfs/page_actor.h | 8 +- fs/squashfs/read_context.c | 181 ++++++++++++++++++++++++ fs/squashfs/read_context.h | 31 ++++ fs/squashfs/squashfs.h | 12 +- fs/squashfs/squashfs_fs_sb.h | 3 +- fs/squashfs/super.c | 13 +- fs/squashfs/symlink.c | 3 +- 17 files changed, 390 insertions(+), 172 deletions(-) create mode 100644 fs/squashfs/read_context.c create mode 100644 fs/squashfs/read_context.h diff --git a/fs/squashfs/Makefile b/fs/squashfs/Makefile index 477c89a519ee..93511720e94f 100644 --- a/fs/squashfs/Makefile +++ b/fs/squashfs/Makefile @@ -6,6 +6,7 @@ obj-$(CONFIG_SQUASHFS) += squashfs.o squashfs-y += block.o cache.o dir.o export.o file.o fragment.o id.o inode.o squashfs-y += namei.o super.o symlink.o decompressor.o page_actor.o +squashfs-y += read_context.o squashfs-$(CONFIG_SQUASHFS_FILE_CACHE) += file_cache.o squashfs-$(CONFIG_SQUASHFS_FILE_DIRECT) += file_direct.o squashfs-$(CONFIG_SQUASHFS_DECOMP_SINGLE) += decompressor_single.o diff --git a/fs/squashfs/block.c b/fs/squashfs/block.c index bed3bb8b27fa..110fcfd2a0eb 100644 --- a/fs/squashfs/block.c +++ b/fs/squashfs/block.c @@ -23,9 +23,10 @@ #include "squashfs_fs.h" #include "squashfs_fs_sb.h" +#include "read_context.h" +#include "page_actor.h" #include "squashfs.h" #include "decompressor.h" -#include "page_actor.h" /* * Returns the amount of bytes copied to the page actor. @@ -77,7 +78,7 @@ static int copy_bio_to_actor(struct bio *bio, } static int squashfs_bio_read(struct super_block *sb, u64 index, int length, - struct bio **biop, int *block_offset) + struct squashfs_context *context, int *block_offset) { struct squashfs_sb_info *msblk = sb->s_fs_info; const u64 read_start = round_down(index, msblk->devblksize); @@ -87,44 +88,33 @@ static int squashfs_bio_read(struct super_block *sb, u64 index, int length, int offset = read_start - round_down(index, PAGE_SIZE); int total_len = (block_end - block) << msblk->devblksize_log2; const int page_count = DIV_ROUND_UP(total_len + offset, PAGE_SIZE); - int error, i; - struct bio *bio; + struct bio *bio = context->bio; + int error = -EIO, i; - bio = bio_kmalloc(page_count, GFP_NOIO); - if (!bio) - return -ENOMEM; bio_init(bio, sb->s_bdev, bio->bi_inline_vecs, page_count, REQ_OP_READ); bio->bi_iter.bi_sector = block * (msblk->devblksize >> SECTOR_SHIFT); for (i = 0; i < page_count; ++i) { unsigned int len = min_t(unsigned int, PAGE_SIZE - offset, total_len); - struct page *page = alloc_page(GFP_NOIO); + int res = bio_add_page(bio, context->bio_page[i], len, offset); + + if (!res) + goto out_uninit_bio; - if (!page) { - error = -ENOMEM; - goto out_free_bio; - } - if (!bio_add_page(bio, page, len, offset)) { - error = -EIO; - goto out_free_bio; - } offset = 0; total_len -= len; } error = submit_bio_wait(bio); if (error) - goto out_free_bio; + goto out_uninit_bio; - *biop = bio; *block_offset = index & ((1 << msblk->devblksize_log2) - 1); return 0; -out_free_bio: - bio_free_pages(bio); - bio_uninit(bio); - kfree(bio); +out_uninit_bio: + bio_uninit(context->bio); return error; } @@ -138,10 +128,10 @@ static int squashfs_bio_read(struct super_block *sb, u64 index, int length, * algorithms). */ int squashfs_read_data(struct super_block *sb, u64 index, int length, - u64 *next_index, struct squashfs_page_actor *output) + u64 *next_index, struct squashfs_context *context) { struct squashfs_sb_info *msblk = sb->s_fs_info; - struct bio *bio = NULL; + struct squashfs_page_actor *output = context->actor; int compressed; int res; int offset; @@ -166,13 +156,13 @@ int squashfs_read_data(struct super_block *sb, u64 index, int length, res = -EIO; goto out; } - res = squashfs_bio_read(sb, index, 2, &bio, &offset); + res = squashfs_bio_read(sb, index, 2, context, &offset); if (res) goto out; - if (WARN_ON_ONCE(!bio_next_segment(bio, &iter_all))) { + if (WARN_ON_ONCE(!bio_next_segment(context->bio, &iter_all))) { res = -EIO; - goto out_free_bio; + goto out_uninit_bio; } /* Extract the length of the metadata block */ data = bvec_virt(bvec); @@ -180,16 +170,14 @@ int squashfs_read_data(struct super_block *sb, u64 index, int length, if (offset < bvec->bv_len - 1) { length |= data[offset + 1] << 8; } else { - if (WARN_ON_ONCE(!bio_next_segment(bio, &iter_all))) { + if (WARN_ON_ONCE(!bio_next_segment(context->bio, &iter_all))) { res = -EIO; - goto out_free_bio; + goto out_uninit_bio; } data = bvec_virt(bvec); length |= data[0] << 8; } - bio_free_pages(bio); - bio_uninit(bio); - kfree(bio); + bio_uninit(context->bio); compressed = SQUASHFS_COMPRESSED(length); length = SQUASHFS_COMPRESSED_SIZE(length); @@ -207,24 +195,23 @@ int squashfs_read_data(struct super_block *sb, u64 index, int length, if (next_index) *next_index = index + length; - res = squashfs_bio_read(sb, index, length, &bio, &offset); + res = squashfs_bio_read(sb, index, length, context, &offset); if (res) goto out; if (compressed) { if (!msblk->stream) { res = -EIO; - goto out_free_bio; + goto out_uninit_bio; } - res = msblk->thread_ops->decompress(msblk, bio, offset, length, output); + res = msblk->thread_ops->decompress(msblk, context->bio, + offset, length, output); } else { - res = copy_bio_to_actor(bio, output, offset, length); + res = copy_bio_to_actor(context->bio, output, offset, length); } -out_free_bio: - bio_free_pages(bio); - bio_uninit(bio); - kfree(bio); +out_uninit_bio: + bio_uninit(context->bio); out: if (res < 0) { ERROR("Failed to read block 0x%llx: %d\n", index, res); diff --git a/fs/squashfs/cache.c b/fs/squashfs/cache.c index 5062326d0efb..98af4f696edd 100644 --- a/fs/squashfs/cache.c +++ b/fs/squashfs/cache.c @@ -42,19 +42,24 @@ #include "squashfs_fs.h" #include "squashfs_fs_sb.h" -#include "squashfs.h" +#include "read_context.h" #include "page_actor.h" +#include "squashfs.h" /* * Look-up block in cache, and increment usage count. If not in cache, read * and decompress it from disk. */ struct squashfs_cache_entry *squashfs_cache_get(struct super_block *sb, - struct squashfs_cache *cache, u64 block, int length) + struct squashfs_cache *cache, struct squashfs_contexts *contexts, + struct squashfs_context *context, u64 block, int length) { - int i, n; + int i, n, need_context = context == NULL; struct squashfs_cache_entry *entry; + if (cache->contexts) + contexts = cache->contexts; + spin_lock(&cache->lock); while (1) { @@ -107,8 +112,17 @@ struct squashfs_cache_entry *squashfs_cache_get(struct super_block *sb, entry->error = 0; spin_unlock(&cache->lock); + if (need_context) + context = squashfs_get_context(contexts); + + squashfs_page_actor_init(context->actor, entry->data, + cache->pages, 0); + entry->length = squashfs_read_data(sb, block, length, - &entry->next_index, entry->actor); + &entry->next_index, context); + + if (need_context) + squashfs_put_context(contexts, context); spin_lock(&cache->lock); @@ -207,9 +221,9 @@ void squashfs_cache_delete(struct squashfs_cache *cache) kfree(cache->entry[i].data[j]); kfree(cache->entry[i].data); } - kfree(cache->entry[i].actor); } + squashfs_delete_contexts(cache->contexts); kfree(cache->entry); kfree(cache); } @@ -221,7 +235,7 @@ void squashfs_cache_delete(struct squashfs_cache *cache) * is allocated as a sequence of kmalloced PAGE_SIZE buffers. */ struct squashfs_cache *squashfs_cache_init(char *name, int entries, - int block_size) + int block_size, int alloc_contexts) { int i, j; struct squashfs_cache *cache = kzalloc(sizeof(*cache), GFP_KERNEL); @@ -237,6 +251,15 @@ struct squashfs_cache *squashfs_cache_init(char *name, int entries, goto cleanup; } + if (alloc_contexts) { + cache->contexts = squashfs_create_contexts(block_size, entries, + 0, 0); + if (cache->contexts == NULL) { + ERROR("Failed to allocate %s cache\n", name); + goto cleanup; + } + } + cache->curr_blk = 0; cache->next_blk = 0; cache->unused = entries; @@ -268,13 +291,6 @@ struct squashfs_cache *squashfs_cache_init(char *name, int entries, goto cleanup; } } - - entry->actor = squashfs_page_actor_init(entry->data, - cache->pages, 0); - if (entry->actor == NULL) { - ERROR("Failed to allocate %s cache entry\n", name); - goto cleanup; - } } return cache; @@ -341,7 +357,8 @@ int squashfs_read_metadata(struct super_block *sb, void *buffer, return -EIO; while (length) { - entry = squashfs_cache_get(sb, msblk->block_cache, *block, 0); + entry = squashfs_cache_get(sb, msblk->block_cache, NULL, NULL, + *block, 0); if (entry->error) { res = entry->error; goto error; @@ -377,12 +394,12 @@ int squashfs_read_metadata(struct super_block *sb, void *buffer, * filesystem. If necessary read and decompress it from disk. */ struct squashfs_cache_entry *squashfs_get_fragment(struct super_block *sb, - u64 start_block, int length) + u64 start_block, int length, struct squashfs_context *context) { struct squashfs_sb_info *msblk = sb->s_fs_info; - return squashfs_cache_get(sb, msblk->fragment_cache, start_block, - length); + return squashfs_cache_get(sb, msblk->fragment_cache, msblk->contexts, + context, start_block, length); } @@ -396,7 +413,8 @@ struct squashfs_cache_entry *squashfs_get_datablock(struct super_block *sb, { struct squashfs_sb_info *msblk = sb->s_fs_info; - return squashfs_cache_get(sb, msblk->read_page, start_block, length); + return squashfs_cache_get(sb, msblk->read_page, msblk->contexts, NULL, + start_block, length); } @@ -408,7 +426,7 @@ void *squashfs_read_table(struct super_block *sb, u64 block, int length) int pages = (length + PAGE_SIZE - 1) >> PAGE_SHIFT; int i, res; void *table, *buffer, **data; - struct squashfs_page_actor *actor; + struct squashfs_context *context; table = buffer = kmalloc(length, GFP_KERNEL); if (table == NULL) @@ -420,20 +438,22 @@ void *squashfs_read_table(struct super_block *sb, u64 block, int length) goto failed; } - actor = squashfs_page_actor_init(data, pages, length); - if (actor == NULL) { + context = squashfs_create_context(length, 0, 0); + if (context == NULL) { res = -ENOMEM; goto failed2; } + squashfs_page_actor_init(context->actor, data, pages, length); + for (i = 0; i < pages; i++, buffer += PAGE_SIZE) data[i] = buffer; res = squashfs_read_data(sb, block, length | - SQUASHFS_COMPRESSED_BIT_BLOCK, NULL, actor); + SQUASHFS_COMPRESSED_BIT_BLOCK, NULL, context); + squashfs_delete_context(context); kfree(data); - kfree(actor); if (res < 0) goto failed; diff --git a/fs/squashfs/decompressor.c b/fs/squashfs/decompressor.c index 8893cb9b4198..d12255edb129 100644 --- a/fs/squashfs/decompressor.c +++ b/fs/squashfs/decompressor.c @@ -15,9 +15,10 @@ #include "squashfs_fs.h" #include "squashfs_fs_sb.h" +#include "read_context.h" +#include "page_actor.h" #include "decompressor.h" #include "squashfs.h" -#include "page_actor.h" /* * This file (and decompressor.h) implements a decompressor framework for @@ -89,7 +90,7 @@ static void *get_comp_opts(struct super_block *sb, unsigned short flags) { struct squashfs_sb_info *msblk = sb->s_fs_info; void *buffer = NULL, *comp_opts; - struct squashfs_page_actor *actor = NULL; + struct squashfs_context *context; int length = 0; /* @@ -102,14 +103,16 @@ static void *get_comp_opts(struct super_block *sb, unsigned short flags) goto out; } - actor = squashfs_page_actor_init(&buffer, 1, 0); - if (actor == NULL) { + context = squashfs_create_context(PAGE_SIZE, 0, 0); + if (context == NULL) { comp_opts = ERR_PTR(-ENOMEM); goto out; } + squashfs_page_actor_init(context->actor, &buffer, 1, 0); length = squashfs_read_data(sb, - sizeof(struct squashfs_super_block), 0, NULL, actor); + sizeof(struct squashfs_super_block), 0, NULL, context); + squashfs_delete_context(context); if (length < 0) { comp_opts = ERR_PTR(length); @@ -120,7 +123,6 @@ static void *get_comp_opts(struct super_block *sb, unsigned short flags) comp_opts = squashfs_comp_opts(msblk, buffer, length); out: - kfree(actor); kfree(buffer); return comp_opts; } diff --git a/fs/squashfs/decompressor_multi.c b/fs/squashfs/decompressor_multi.c index 416c53eedbd1..0a31012abec7 100644 --- a/fs/squashfs/decompressor_multi.c +++ b/fs/squashfs/decompressor_multi.c @@ -13,6 +13,8 @@ #include "squashfs_fs.h" #include "squashfs_fs_sb.h" +#include "read_context.h" +#include "page_actor.h" #include "decompressor.h" #include "squashfs.h" diff --git a/fs/squashfs/decompressor_multi_percpu.c b/fs/squashfs/decompressor_multi_percpu.c index 1dfadf76ed9a..12094aaba19f 100644 --- a/fs/squashfs/decompressor_multi_percpu.c +++ b/fs/squashfs/decompressor_multi_percpu.c @@ -12,6 +12,8 @@ #include "squashfs_fs.h" #include "squashfs_fs_sb.h" +#include "read_context.h" +#include "page_actor.h" #include "decompressor.h" #include "squashfs.h" diff --git a/fs/squashfs/decompressor_single.c b/fs/squashfs/decompressor_single.c index 6f161887710b..8d9b73103165 100644 --- a/fs/squashfs/decompressor_single.c +++ b/fs/squashfs/decompressor_single.c @@ -11,6 +11,8 @@ #include "squashfs_fs.h" #include "squashfs_fs_sb.h" +#include "read_context.h" +#include "page_actor.h" #include "decompressor.h" #include "squashfs.h" diff --git a/fs/squashfs/file.c b/fs/squashfs/file.c index 8ba8c4c50770..d8ca63a3eb0e 100644 --- a/fs/squashfs/file.c +++ b/fs/squashfs/file.c @@ -38,8 +38,9 @@ #include "squashfs_fs.h" #include "squashfs_fs_sb.h" #include "squashfs_fs_i.h" -#include "squashfs.h" +#include "read_context.h" #include "page_actor.h" +#include "squashfs.h" /* * Locate cache slot in range [offset, index] for specified inode. If @@ -424,7 +425,7 @@ static int squashfs_readpage_fragment(struct page *page, int expected) struct inode *inode = page->mapping->host; struct squashfs_cache_entry *buffer = squashfs_get_fragment(inode->i_sb, squashfs_i(inode)->fragment_block, - squashfs_i(inode)->fragment_size); + squashfs_i(inode)->fragment_size, NULL); int res = buffer->error; if (res) @@ -497,13 +498,13 @@ static int squashfs_read_folio(struct file *file, struct folio *folio) return res; } -static int squashfs_readahead_fragment(struct page **page, +static int squashfs_readahead_fragment(struct squashfs_context *context, unsigned int pages, unsigned int expected) { - struct inode *inode = page[0]->mapping->host; + struct inode *inode = context->page[0]->mapping->host; struct squashfs_cache_entry *buffer = squashfs_get_fragment(inode->i_sb, squashfs_i(inode)->fragment_block, - squashfs_i(inode)->fragment_size); + squashfs_i(inode)->fragment_size, context); struct squashfs_sb_info *msblk = inode->i_sb->s_fs_info; unsigned int n, mask = (1 << (msblk->block_log - PAGE_SHIFT)) - 1; int error = buffer->error; @@ -514,18 +515,18 @@ static int squashfs_readahead_fragment(struct page **page, expected += squashfs_i(inode)->fragment_offset; for (n = 0; n < pages; n++) { - unsigned int base = (page[n]->index & mask) << PAGE_SHIFT; + unsigned int base = (context->page[n]->index & mask) << PAGE_SHIFT; unsigned int offset = base + squashfs_i(inode)->fragment_offset; if (expected > offset) { unsigned int avail = min_t(unsigned int, expected - offset, PAGE_SIZE); - squashfs_fill_page(page[n], buffer, offset, avail); + squashfs_fill_page(context->page[n], buffer, offset, avail); } - unlock_page(page[n]); - put_page(page[n]); + unlock_page(context->page[n]); + put_page(context->page[n]); } out: @@ -541,16 +542,15 @@ static void squashfs_readahead(struct readahead_control *ractl) unsigned short shift = msblk->block_log - PAGE_SHIFT; loff_t start = readahead_pos(ractl) & ~mask; size_t len = readahead_length(ractl) + readahead_pos(ractl) - start; - struct squashfs_page_actor *actor; + struct squashfs_context *context; unsigned int nr_pages = 0; - struct page **pages; int i, file_end = i_size_read(inode) >> msblk->block_log; unsigned int max_pages = 1UL << shift; readahead_expand(ractl, start, (len | mask) + 1); - pages = kmalloc_array(max_pages, sizeof(void *), GFP_KERNEL); - if (!pages) + context = squashfs_get_context(msblk->contexts); + if (context == NULL) return; for (;;) { @@ -566,21 +566,21 @@ static void squashfs_readahead(struct readahead_control *ractl) max_pages = (expected + PAGE_SIZE - 1) >> PAGE_SHIFT; - nr_pages = __readahead_batch(ractl, pages, max_pages); + nr_pages = __readahead_batch(ractl, context->page, max_pages); if (!nr_pages) break; if (readahead_pos(ractl) >= i_size_read(inode)) goto skip_pages; - index = pages[0]->index >> shift; + index = context->page[0]->index >> shift; - if ((pages[nr_pages - 1]->index >> shift) != index) + if ((context->page[nr_pages - 1]->index >> shift) != index) goto skip_pages; if (index == file_end && squashfs_i(inode)->fragment_block != SQUASHFS_INVALID_BLK) { - res = squashfs_readahead_fragment(pages, nr_pages, + res = squashfs_readahead_fragment(context, nr_pages, expected); if (res) goto skip_pages; @@ -591,14 +591,13 @@ static void squashfs_readahead(struct readahead_control *ractl) if (bsize == 0) goto skip_pages; - actor = squashfs_page_actor_init_special(msblk, pages, nr_pages, - expected); - if (!actor) - goto skip_pages; + squashfs_page_actor_init_special(context, msblk, nr_pages, + expected); - res = squashfs_read_data(inode->i_sb, block, bsize, NULL, actor); + res = squashfs_read_data(inode->i_sb, block, bsize, NULL, + context); - last_page = squashfs_page_actor_free(actor); + last_page = squashfs_page_actor_free(context->actor); if (res == expected) { int bytes; @@ -610,26 +609,26 @@ static void squashfs_readahead(struct readahead_control *ractl) PAGE_SIZE - bytes); for (i = 0; i < nr_pages; i++) { - flush_dcache_page(pages[i]); - SetPageUptodate(pages[i]); + flush_dcache_page(context->page[i]); + SetPageUptodate(context->page[i]); } } for (i = 0; i < nr_pages; i++) { - unlock_page(pages[i]); - put_page(pages[i]); + unlock_page(context->page[i]); + put_page(context->page[i]); } } - kfree(pages); + squashfs_put_context(msblk->contexts, context); return; skip_pages: for (i = 0; i < nr_pages; i++) { - unlock_page(pages[i]); - put_page(pages[i]); + unlock_page(context->page[i]); + put_page(context->page[i]); } - kfree(pages); + squashfs_put_context(msblk->contexts, context); } const struct address_space_operations squashfs_aops = { diff --git a/fs/squashfs/file_direct.c b/fs/squashfs/file_direct.c index f1ccad519e28..ab7b990cf71d 100644 --- a/fs/squashfs/file_direct.c +++ b/fs/squashfs/file_direct.c @@ -15,8 +15,9 @@ #include "squashfs_fs.h" #include "squashfs_fs_sb.h" #include "squashfs_fs_i.h" -#include "squashfs.h" +#include "read_context.h" #include "page_actor.h" +#include "squashfs.h" /* Read separately compressed datablock directly into page cache */ int squashfs_readpage_block(struct page *target_page, u64 block, int bsize, @@ -31,8 +32,7 @@ int squashfs_readpage_block(struct page *target_page, u64 block, int bsize, int start_index = target_page->index & ~mask; int end_index = start_index | mask; int i, n, pages, bytes, res = -ENOMEM; - struct page **page; - struct squashfs_page_actor *actor; + struct squashfs_context *context; void *pageaddr; if (end_index > file_end) @@ -40,21 +40,21 @@ int squashfs_readpage_block(struct page *target_page, u64 block, int bsize, pages = end_index - start_index + 1; - page = kmalloc_array(pages, sizeof(void *), GFP_KERNEL); - if (page == NULL) + context = squashfs_get_context(msblk->contexts); + if (context == NULL) return res; /* Try to grab all the pages covered by the Squashfs block */ for (i = 0, n = start_index; n <= end_index; n++) { - page[i] = (n == target_page->index) ? target_page : + context->page[i] = (n == target_page->index) ? target_page : grab_cache_page_nowait(target_page->mapping, n); - if (page[i] == NULL) + if (context->page[i] == NULL) continue; - if (PageUptodate(page[i])) { - unlock_page(page[i]); - put_page(page[i]); + if (PageUptodate(context->page[i])) { + unlock_page(context->page[i]); + put_page(context->page[i]); continue; } @@ -64,17 +64,15 @@ int squashfs_readpage_block(struct page *target_page, u64 block, int bsize, pages = i; /* - * Create a "page actor" which will kmap and kunmap the + * Initialise "page actor" which will kmap and kunmap the * page cache pages appropriately within the decompressor */ - actor = squashfs_page_actor_init_special(msblk, page, pages, expected); - if (actor == NULL) - goto out; + squashfs_page_actor_init_special(context, msblk, pages, expected); /* Decompress directly into the page cache buffers */ - res = squashfs_read_data(inode->i_sb, block, bsize, NULL, actor); + res = squashfs_read_data(inode->i_sb, block, bsize, NULL, context); - squashfs_page_actor_free(actor); + squashfs_page_actor_free(context->actor); if (res < 0) goto mark_errored; @@ -86,22 +84,22 @@ int squashfs_readpage_block(struct page *target_page, u64 block, int bsize, /* Last page (if present) may have trailing bytes not filled */ bytes = res % PAGE_SIZE; - if (page[pages - 1]->index == end_index && bytes) { - pageaddr = kmap_local_page(page[pages - 1]); + if (context->page[pages - 1]->index == end_index && bytes) { + pageaddr = kmap_local_page(context->page[pages - 1]); memset(pageaddr + bytes, 0, PAGE_SIZE - bytes); kunmap_local(pageaddr); } /* Mark pages as uptodate, unlock and release */ for (i = 0; i < pages; i++) { - flush_dcache_page(page[i]); - SetPageUptodate(page[i]); - unlock_page(page[i]); - if (page[i] != target_page) - put_page(page[i]); + flush_dcache_page(context->page[i]); + SetPageUptodate(context->page[i]); + unlock_page(context->page[i]); + if (context->page[i] != target_page) + put_page(context->page[i]); } - kfree(page); + squashfs_put_context(msblk->contexts, context); return 0; @@ -110,15 +108,14 @@ int squashfs_readpage_block(struct page *target_page, u64 block, int bsize, * dealt with by the caller */ for (i = 0; i < pages; i++) { - if (page[i] == NULL || page[i] == target_page) + if (context->page[i] == NULL || context->page[i] == target_page) continue; - flush_dcache_page(page[i]); - SetPageError(page[i]); - unlock_page(page[i]); - put_page(page[i]); + flush_dcache_page(context->page[i]); + SetPageError(context->page[i]); + unlock_page(context->page[i]); + put_page(context->page[i]); } -out: - kfree(page); + squashfs_put_context(msblk->contexts, context); return res; } diff --git a/fs/squashfs/page_actor.c b/fs/squashfs/page_actor.c index 81af6c4ca115..8ff50034afee 100644 --- a/fs/squashfs/page_actor.c +++ b/fs/squashfs/page_actor.c @@ -8,8 +8,9 @@ #include #include #include "squashfs_fs_sb.h" -#include "decompressor.h" +#include "read_context.h" #include "page_actor.h" +#include "decompressor.h" /* * This file contains implementations of page_actor for decompressing into @@ -40,14 +41,9 @@ static void cache_finish_page(struct squashfs_page_actor *actor) /* empty */ } -struct squashfs_page_actor *squashfs_page_actor_init(void **buffer, +void squashfs_page_actor_init(struct squashfs_page_actor *actor, void **buffer, int pages, int length) { - struct squashfs_page_actor *actor = kmalloc(sizeof(*actor), GFP_KERNEL); - - if (actor == NULL) - return NULL; - actor->length = length ? : pages * PAGE_SIZE; actor->buffer = buffer; actor->pages = pages; @@ -56,7 +52,6 @@ struct squashfs_page_actor *squashfs_page_actor_init(void **buffer, actor->squashfs_first_page = cache_first_page; actor->squashfs_next_page = cache_next_page; actor->squashfs_finish_page = cache_finish_page; - return actor; } /* Implementation of page_actor for decompressing directly into page cache. */ @@ -102,35 +97,23 @@ static void direct_finish_page(struct squashfs_page_actor *actor) kunmap_local(actor->pageaddr); } -struct squashfs_page_actor *squashfs_page_actor_init_special(struct squashfs_sb_info *msblk, - struct page **page, int pages, int length) +void squashfs_page_actor_init_special(struct squashfs_context *context, + struct squashfs_sb_info *msblk, int pages, int length) { - struct squashfs_page_actor *actor = kmalloc(sizeof(*actor), GFP_KERNEL); - - if (actor == NULL) - return NULL; - - if (msblk->decompressor->alloc_buffer) { - actor->tmp_buffer = kmalloc(PAGE_SIZE, GFP_KERNEL); - - if (actor->tmp_buffer == NULL) { - kfree(actor); - return NULL; - } - } else - actor->tmp_buffer = NULL; + struct squashfs_page_actor *actor = context->actor; + actor->tmp_buffer = context->hole; actor->length = length ? : pages * PAGE_SIZE; - actor->page = page; + actor->page = context->page; actor->pages = pages; actor->next_page = 0; actor->returned_pages = 0; - actor->next_index = page[0]->index & ~((1 << (msblk->block_log - PAGE_SHIFT)) - 1); + actor->next_index = context->page[0]->index & ~((1 << + (msblk->block_log - PAGE_SHIFT)) - 1); actor->pageaddr = NULL; actor->last_page = NULL; actor->alloc_buffer = msblk->decompressor->alloc_buffer; actor->squashfs_first_page = direct_first_page; actor->squashfs_next_page = direct_next_page; actor->squashfs_finish_page = direct_finish_page; - return actor; } diff --git a/fs/squashfs/page_actor.h b/fs/squashfs/page_actor.h index 97d4983559b1..36fbc93d8907 100644 --- a/fs/squashfs/page_actor.h +++ b/fs/squashfs/page_actor.h @@ -25,17 +25,15 @@ struct squashfs_page_actor { pgoff_t next_index; }; -extern struct squashfs_page_actor *squashfs_page_actor_init(void **buffer, +extern void squashfs_page_actor_init(struct squashfs_page_actor *actor, void **buffer, int pages, int length); -extern struct squashfs_page_actor *squashfs_page_actor_init_special( +extern void squashfs_page_actor_init_special(struct squashfs_context *context, struct squashfs_sb_info *msblk, - struct page **page, int pages, int length); + int pages, int length); static inline struct page *squashfs_page_actor_free(struct squashfs_page_actor *actor) { struct page *last_page = actor->last_page; - kfree(actor->tmp_buffer); - kfree(actor); return last_page; } static inline void *squashfs_first_page(struct squashfs_page_actor *actor) diff --git a/fs/squashfs/read_context.c b/fs/squashfs/read_context.c new file mode 100644 index 000000000000..35a52ed572d7 --- /dev/null +++ b/fs/squashfs/read_context.c @@ -0,0 +1,181 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Squashfs - a compressed read only filesystem for Linux + * + * Copyright (c) 2023 + * Phillip Lougher + * + * read_context.c + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "squashfs_fs.h" +#include "squashfs_fs_sb.h" +#include "squashfs.h" +#include "page_actor.h" + + +void squashfs_delete_context(struct squashfs_context *context) +{ + int i; + + for (i = 0; i < context->bio_pages; i++) + if (context->bio_page[i]) + __free_page(context->bio_page[i]); + + kfree(context->bio); + kfree(context->page); + kfree(context->actor); + kfree(context->hole); + kfree(context); +} + + +struct squashfs_context *squashfs_create_context(int length, int alloc_hole, + int alloc_array) +{ + int pages = (length + PAGE_SIZE - 1) >> PAGE_SHIFT; + int bio_pages = pages + 1, i; + struct squashfs_context *context; + + context = kzalloc(sizeof(struct squashfs_context), GFP_KERNEL); + if (context == NULL) + return ERR_PTR(-ENOMEM); + + context->bio_page = kcalloc(bio_pages, sizeof(struct bio_pages *), + GFP_KERNEL); + + if (context->bio_page == NULL) + goto cleanup; + + context->bio_pages = bio_pages; + + for (i = 0; i < bio_pages; i++) { + context->bio_page[i] = alloc_page(GFP_KERNEL); + + if (context->bio_page[i] == NULL) + goto cleanup; + } + + context->bio = bio_kmalloc(bio_pages, GFP_KERNEL); + if (context->bio == NULL) + goto cleanup; + + context->actor = kmalloc(sizeof(struct squashfs_page_actor), + GFP_KERNEL); + if (context->actor == NULL) + goto cleanup; + + if (alloc_array) { + context->page = kmalloc_array(pages, sizeof(struct page *), + GFP_KERNEL); + if (context->page == NULL) + goto cleanup; + } + + if (alloc_hole) { + context->hole = kmalloc(PAGE_SIZE, GFP_KERNEL); + if (context->hole == NULL) + goto cleanup; + } + + return context; + +cleanup: + squashfs_delete_context(context); + return ERR_PTR(-ENOMEM); +} + + +struct squashfs_context *squashfs_get_context(struct squashfs_contexts *contexts) +{ + struct squashfs_context *context; + + while (1) { + mutex_lock(&contexts->mutex); + + if (!list_empty(&contexts->list)) { + context = list_entry(contexts->list.prev, + struct squashfs_context, list); + list_del(&context->list); + mutex_unlock(&contexts->mutex); + break; + } + + mutex_unlock(&contexts->mutex); + wait_event(contexts->wait_queue, !list_empty(&contexts->list)); + } + + return context; +} + + +void squashfs_put_context(struct squashfs_contexts *contexts, + struct squashfs_context *context) +{ + + mutex_lock(&contexts->mutex); + list_add(&context->list, &contexts->list); + mutex_unlock(&contexts->mutex); + wake_up(&contexts->wait_queue); +} + + +void squashfs_delete_contexts(struct squashfs_contexts *contexts) +{ + struct squashfs_context *context; + + if (contexts == NULL) + return; + + while (!list_empty(&contexts->list)) { + context = list_entry(contexts->list.prev, + struct squashfs_context, list); + list_del(&context->list); + squashfs_delete_context(context); + } + + kfree(contexts); +} + + +struct squashfs_contexts *squashfs_create_contexts(int block_size, int entries, + int alloc_hole, int alloc_array) +{ + int i; + struct squashfs_contexts *contexts; + + contexts = kzalloc(sizeof(struct squashfs_contexts), GFP_KERNEL); + if (contexts == NULL) + return ERR_PTR(-ENOMEM); + + mutex_init(&contexts->mutex); + INIT_LIST_HEAD(&contexts->list); + init_waitqueue_head(&contexts->wait_queue); + + for (i = 0; i < entries; i++) { + struct squashfs_context *context; + + context = squashfs_create_context(block_size, alloc_hole, + alloc_array); + if (context == NULL) + goto cleanup; + + list_add(&context->list, &contexts->list); + } + + return contexts; + +cleanup: + squashfs_delete_contexts(contexts); + return ERR_PTR(-ENOMEM); +} diff --git a/fs/squashfs/read_context.h b/fs/squashfs/read_context.h new file mode 100644 index 000000000000..a9b93a43893b --- /dev/null +++ b/fs/squashfs/read_context.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef READ_CONTEXT_H +#define READ_CONTEXT_H +/* + * Copyright (c) 2023 + * Phillip Lougher + */ + +struct squashfs_context { + int bio_pages; + struct bio *bio; + struct page **bio_page; + struct squashfs_page_actor *actor; + struct page **page; + void *hole; + struct list_head list; +}; + +struct squashfs_contexts { + struct mutex mutex; + struct list_head list; + wait_queue_head_t wait_queue; +}; + +extern void squashfs_delete_context(struct squashfs_context *context); +extern struct squashfs_context *squashfs_create_context(int length, int alloc_hole, int alloc_array); +extern struct squashfs_context *squashfs_get_context(struct squashfs_contexts *contexts); +extern void squashfs_put_context(struct squashfs_contexts *contexts, struct squashfs_context *context); +extern void squashfs_delete_contexts(struct squashfs_contexts *contexts); +extern struct squashfs_contexts *squashfs_create_contexts(int block_size, int entries, int alloc_hole, int alloc_array); +#endif diff --git a/fs/squashfs/squashfs.h b/fs/squashfs/squashfs.h index a6164fdf9435..c5db13907e9f 100644 --- a/fs/squashfs/squashfs.h +++ b/fs/squashfs/squashfs.h @@ -8,6 +8,9 @@ * squashfs.h */ +#include "read_context.h" +#include "page_actor.h" + #define TRACE(s, args...) pr_debug("SQUASHFS: "s, ## args) #define ERROR(s, args...) pr_err("SQUASHFS error: "s, ## args) @@ -16,19 +19,20 @@ /* block.c */ extern int squashfs_read_data(struct super_block *, u64, int, u64 *, - struct squashfs_page_actor *); + struct squashfs_context *); /* cache.c */ -extern struct squashfs_cache *squashfs_cache_init(char *, int, int); +extern struct squashfs_cache *squashfs_cache_init(char *, int, int, int); extern void squashfs_cache_delete(struct squashfs_cache *); extern struct squashfs_cache_entry *squashfs_cache_get(struct super_block *, - struct squashfs_cache *, u64, int); + struct squashfs_cache *, struct squashfs_contexts *, + struct squashfs_context *, u64, int); extern void squashfs_cache_put(struct squashfs_cache_entry *); extern int squashfs_copy_data(void *, struct squashfs_cache_entry *, int, int); extern int squashfs_read_metadata(struct super_block *, void *, u64 *, int *, int); extern struct squashfs_cache_entry *squashfs_get_fragment(struct super_block *, - u64, int); + u64, int, struct squashfs_context *); extern struct squashfs_cache_entry *squashfs_get_datablock(struct super_block *, u64, int); extern void *squashfs_read_table(struct super_block *, u64, int); diff --git a/fs/squashfs/squashfs_fs_sb.h b/fs/squashfs/squashfs_fs_sb.h index 72f6f4b37863..6cee243be9bb 100644 --- a/fs/squashfs/squashfs_fs_sb.h +++ b/fs/squashfs/squashfs_fs_sb.h @@ -23,6 +23,7 @@ struct squashfs_cache { int pages; spinlock_t lock; wait_queue_head_t wait_queue; + struct squashfs_contexts *contexts; struct squashfs_cache_entry *entry; }; @@ -37,7 +38,6 @@ struct squashfs_cache_entry { wait_queue_head_t wait_queue; struct squashfs_cache *cache; void **data; - struct squashfs_page_actor *actor; }; struct squashfs_sb_info { @@ -47,6 +47,7 @@ struct squashfs_sb_info { struct squashfs_cache *block_cache; struct squashfs_cache *fragment_cache; struct squashfs_cache *read_page; + struct squashfs_contexts *contexts; int next_meta_index; __le64 *id_table; __le64 *fragment_index; diff --git a/fs/squashfs/super.c b/fs/squashfs/super.c index e090fae48e68..3ba5c2b5c9e0 100644 --- a/fs/squashfs/super.c +++ b/fs/squashfs/super.c @@ -33,6 +33,7 @@ #include "squashfs_fs.h" #include "squashfs_fs_sb.h" #include "squashfs_fs_i.h" +#include "read_context.h" #include "squashfs.h" #include "decompressor.h" #include "xattr.h" @@ -317,18 +318,23 @@ static int squashfs_fill_super(struct super_block *sb, struct fs_context *fc) err = -ENOMEM; msblk->block_cache = squashfs_cache_init("metadata", - SQUASHFS_CACHED_BLKS, SQUASHFS_METADATA_SIZE); + SQUASHFS_CACHED_BLKS, SQUASHFS_METADATA_SIZE, 1); if (msblk->block_cache == NULL) goto failed_mount; /* Allocate read_page block */ msblk->read_page = squashfs_cache_init("data", - msblk->max_thread_num, msblk->block_size); + msblk->max_thread_num, msblk->block_size, 0); if (msblk->read_page == NULL) { errorf(fc, "Failed to allocate read_page block"); goto failed_mount; } + msblk->contexts = squashfs_create_contexts(msblk->block_size, + msblk->max_thread_num, msblk->decompressor->alloc_buffer, 1); + if (msblk->contexts == NULL) + goto failed_mount; + msblk->stream = squashfs_decompressor_setup(sb, flags); if (IS_ERR(msblk->stream)) { err = PTR_ERR(msblk->stream); @@ -392,7 +398,7 @@ static int squashfs_fill_super(struct super_block *sb, struct fs_context *fc) goto check_directory_table; msblk->fragment_cache = squashfs_cache_init("fragment", - SQUASHFS_CACHED_FRAGMENTS, msblk->block_size); + SQUASHFS_CACHED_FRAGMENTS, msblk->block_size, 0); if (msblk->fragment_cache == NULL) { err = -ENOMEM; goto failed_mount; @@ -454,6 +460,7 @@ static int squashfs_fill_super(struct super_block *sb, struct fs_context *fc) squashfs_cache_delete(msblk->block_cache); squashfs_cache_delete(msblk->fragment_cache); squashfs_cache_delete(msblk->read_page); + squashfs_delete_contexts(msblk->contexts); msblk->thread_ops->destroy(msblk); kfree(msblk->inode_lookup_table); kfree(msblk->fragment_index); diff --git a/fs/squashfs/symlink.c b/fs/squashfs/symlink.c index 2bf977a52c2c..8471a3a5491b 100644 --- a/fs/squashfs/symlink.c +++ b/fs/squashfs/symlink.c @@ -69,7 +69,8 @@ static int squashfs_symlink_read_folio(struct file *file, struct folio *folio) * blocks, we may need to call squashfs_cache_get multiple times. */ for (bytes = 0; bytes < length; offset = 0, bytes += copied) { - entry = squashfs_cache_get(sb, msblk->block_cache, block, 0); + entry = squashfs_cache_get(sb, msblk->block_cache, NULL, NULL, + block, 0); if (entry->error) { ERROR("Unable to read symlink [%llx:%x]\n", squashfs_i(inode)->start, -- 2.35.1