From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43797C52D6F for ; Tue, 27 Aug 2024 06:58:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D5A966B0095; Tue, 27 Aug 2024 02:58:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D0A6E6B0096; Tue, 27 Aug 2024 02:58:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BD2096B0098; Tue, 27 Aug 2024 02:58:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 9CB606B0095 for ; Tue, 27 Aug 2024 02:58:57 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 4A57F12166F for ; Tue, 27 Aug 2024 06:58:57 +0000 (UTC) X-FDA: 82497123114.04.FA28B56 Received: from out30-131.freemail.mail.aliyun.com (out30-131.freemail.mail.aliyun.com [115.124.30.131]) by imf04.hostedemail.com (Postfix) with ESMTP id 5148A40002 for ; Tue, 27 Aug 2024 06:58:52 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=P6Alwok6; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf04.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724741871; a=rsa-sha256; cv=none; b=m4jcocwoyoLeL159Q0xt8js7GjbOiyRKQ/f3f1eXDQZHfL1XmKX76wv86vRWM+MDXXzmX4 NiIGlxwwBz5jGhZYZ/swJlqJF/U9kn0kzo6qz5Mney6jklsrKSuC3V+FLFQyVQwBOuIqC+ 305mhP9yHEPPLySABGP/jKMh7QESIA0= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=P6Alwok6; dmarc=pass (policy=none) header.from=linux.alibaba.com; spf=pass (imf04.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724741871; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=R2jZq5gOpcCxh3fyRzRIu9IKJVOLCHrHSDqAuNqGBV8=; b=wap65HKorLk8rK7DaHlyXTXJv76QctlmTjRTDndNdQxH9Mj/JiSp0OHKWCntFDL5CSn1qR 4v20re092Dq6GGOl/pg7ZF2GTNNwLgwHhJpdn3KQaEhah5tkc4iWqrJKW1dm3UB0LWs+Dj 6uNb6Sfc0FM2tWsh6DXeS4ijtRbteAw= DKIM-Signature:v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1724741923; h=Message-ID:Date:MIME-Version:Subject:To:From:Content-Type; bh=R2jZq5gOpcCxh3fyRzRIu9IKJVOLCHrHSDqAuNqGBV8=; b=P6Alwok6CvKPFPhGgPP8+uJhPY2B1aDU57KVajbcfZZzNhuQNc9eOVGhF9qL5NBZ7rFeGjYUxCBTcWh13go7YD/4Hu00kOrPLEEu85Ob4mcF086/9XR9nympu0k+XCF5RDqrYveLsbUkO/DvjKoBCD2Qwb1eWxaSeLbsVaCqn1c= Received: from 30.97.56.57(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WDlUe9c_1724741920) by smtp.aliyun-inc.com; Tue, 27 Aug 2024 14:58:41 +0800 Message-ID: <16327571-3031-4758-a401-2b5f19c3196b@linux.alibaba.com> Date: Tue, 27 Aug 2024 14:58:40 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v5 9/9] mm: shmem: support large folio swap out To: Hugh Dickins , Andrew Morton Cc: willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: From: Baolin Wang In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 5148A40002 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 8tjwot9dpdfeuyqe91x118b9uhhqpjn6 X-HE-Tag: 1724741932-401102 X-HE-Meta: U2FsdGVkX1+NkIxXRhRNN5rePyPOKobjEy4/+QwM2iZjij5/B7vvTiD6BcvNl8y2nU+5uCvSnpH3rRSTpQj7UhEOVkK36orAMJDJ9bHfW15/w9wn9o7Yb3ZZK/Wk62h6Fh6nAmf4tpmg7MvvVbv9/Bb+1B6eNNy7uxJI/a55HhMN6HL9C6Q35EpnLJqKB5kJXYDxDg3qlnO+PUymtwtbGVEY7AycXJvIrUDfBcyqJSYZueCZe82JRNxFoTiEvESh8WvwOrZhWMYnl/YXdYVigDOPR3opUlp4/vAiecLmiLqrFeR6pGVbDGDyhsLN45CB3VlONZSc8aCYE+xkUtR4PosMMc0GGwcbqebjZyhpCfKg+R2PsWSI9JAZOV/oCCxG7qbnnFuKjed40Nl8An3RX4V7ENoONxtokGoyy9O1wdm2KcQOcOEk6ywXtTaQqJZX37r8frxfP9hSg8QNSUConuzfxBpNhZwox9bGBRyTd8zdVywEOnplduMYV75GUkFqRzbhJ5yp+l4IRxc6POm96la52IjxYCW4ykK1aouRTEcJX6dmHeCdZ4adcln1gDc5UdLcPNyKJ5eQ/eGzTWMWoNRKRkgEgnnHBB9VA5zsnlHNZ8ftotY9KtIH4q4N0lh3q70c1nEaQKW4cWCxB+iGBz5N27Il8aSKZxwS7OPOX7gaRq0e9iAzhX3eUuCEk18nuo0VuAF8E5CbAvfRB9mFLeQGlj7wrx86RXr5nidK5gBP9IMQXbdAezsjvpKXJ4S+Agky4C8KeGsV63kLAVUJYItF5MGCzksmcHko/UCX2mQHtrjdi7OtIxd69CPTMW+7YZ1fNByK9sp5eZFmDMpCPGVa7W+Utji9OuVACkr4dSu19RXzOTs3YW6y1i2EO2XmaMG+bO2PkWUHejK2CqneKQVopxHFQ7AESixrO0kFgl7qChWKJZStvtjCAtCzM6hoXmbqoG0EHtiPj96ylOC cMpHeoUl rx/1LlP37gfmt5ZHX0J1h80BBpvM0PHroLUhprq6GLyvo0pwWbE7NDRYJjj0CaR4Pmn5UDHSHGkk6GTzR7FPUk7i9OiwmSS0Tku/KHu2+GUCrsb4KhlPmr/xgRuzxcuT1fji+P7Gy45FoGcqL91LRNaTjYk4tRFAuvvbqUHfSmOpgdyAAskr1VMqRm7cKkCYvpNsK4+Px8WoihecHRpugZTpdc455DP618wB3HmKdu92+DsVH+PEmZIuDdB2p0/fxJJdKe6ZjN7NF8DbViHAuPR43kkw52S6D5MtgXOgRDZ6MsKJQtjYzMukjYhJmOQjsU4XLLGNNl+X9APRqMa5MT/6R9HQJCUo+rODx X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/8/26 07:14, Hugh Dickins wrote: > On Mon, 12 Aug 2024, Baolin Wang wrote: > >> Shmem will support large folio allocation [1] [2] to get a better performance, >> however, the memory reclaim still splits the precious large folios when trying >> to swap out shmem, which may lead to the memory fragmentation issue and can not >> take advantage of the large folio for shmeme. >> >> Moreover, the swap code already supports for swapping out large folio without >> split, hence this patch set supports the large folio swap out for shmem. >> >> Note the i915_gem_shmem driver still need to be split when swapping, thus >> add a new flag 'split_large_folio' for writeback_control to indicate spliting >> the large folio. > > Is that last paragraph a misunderstanding? The i915 THP splitting in > shmem_writepage() was to avoid mm VM_BUG_ONs and crashes when shmem.c > did not support huge page swapout: but now you are enabling that support, > and such VM_BUG_ONs and crashes are gone (so far as I can see: and this > is written on a laptop using the i915 driver). Thanks for the history, and I understand. > I cannot think of why i915 itself would care how mm implements swapout > (beyond enjoying faster): I think all the wbc->split_large_folio you > introduce here should be reverted. But you may know better! > > I do need a further change to shmem_writepage() here: see fixup patch > below: that's written to apply on top of this 9/9, but I'd be glad to > see a replacement with wbc->split_large_folio gone, and just one > !IS_ENABLED(CONFIG_THP_SWAP) instead. Sure. After Andrew queuing your fixes, I can send a proper fix patch to remove the 'wbc->split_large_folio'. >> [1] https://lore.kernel.org/all/cover.1717495894.git.baolin.wang@linux.alibaba.com/ >> [2] https://lore.kernel.org/all/20240515055719.32577-1-da.gomez@samsung.com/ > > I get "Not found" for that [2] link. Weird, I can access the link, not sure why. > >> Signed-off-by: Baolin Wang >> --- >> drivers/gpu/drm/i915/gem/i915_gem_shmem.c | 1 + >> include/linux/writeback.h | 4 +++ >> mm/shmem.c | 12 ++++++--- >> mm/vmscan.c | 32 ++++++++++++++++++----- >> 4 files changed, 38 insertions(+), 11 deletions(-) > > [PATCH] mm: shmem: shmem_writepage() split folio at EOF before swapout > > Working in a constrained (size= or nr_blocks=) huge=always tmpfs relies > on swapout to split a large folio at EOF, to trim off its excess before > hitting premature ENOSPC: shmem_unused_huge_shrink() contains no code to > handle splitting huge swap blocks, and nobody would want that to be added. > > Signed-off-by: Hugh Dickins > --- > mm/shmem.c | 15 ++++++++++++++- > 1 file changed, 14 insertions(+), 1 deletion(-) > > diff --git a/mm/shmem.c b/mm/shmem.c > index 37c300f69baf..4dd0570962fa 100644 > --- a/mm/shmem.c > +++ b/mm/shmem.c > @@ -1459,6 +1459,7 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) > swp_entry_t swap; > pgoff_t index; > int nr_pages; > + bool split = false; > > /* > * Our capabilities prevent regular writeback or sync from ever calling > @@ -1480,8 +1481,20 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) > * If /sys/kernel/mm/transparent_hugepage/shmem_enabled is "always" or > * "force", drivers/gpu/drm/i915/gem/i915_gem_shmem.c gets huge pages, > * and its shmem_writeback() needs them to be split when swapping. > + * > + * And shrinkage of pages beyond i_size does not split swap, so > + * swapout of a large folio crossing i_size needs to split too > + * (unless fallocate has been used to preallocate beyond EOF). > */ > - if (wbc->split_large_folio && folio_test_large(folio)) { > + if (folio_test_large(folio)) { > + split = wbc->split_large_folio; > + index = shmem_fallocend(inode, > + DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE)); > + if (index > folio->index && index < folio_next_index(folio)) > + split = true; > + } > + > + if (split) { > try_split: > /* Ensure the subpages are still dirty */ > folio_test_set_dirty(folio); Thanks for the fix, Hugh. Very appreciated for your reviewing.