From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3A54C5478C for ; Fri, 1 Mar 2024 17:14:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 565BB6B00A1; Fri, 1 Mar 2024 12:14:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 514806B00A2; Fri, 1 Mar 2024 12:14:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3DD506B00A3; Fri, 1 Mar 2024 12:14:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 2BDEE6B00A1 for ; Fri, 1 Mar 2024 12:14:07 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 012441404FB for ; Fri, 1 Mar 2024 17:14:06 +0000 (UTC) X-FDA: 81849118134.16.3AC962B Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf24.hostedemail.com (Postfix) with ESMTP id 340DD180002 for ; Fri, 1 Mar 2024 17:14:05 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; spf=pass (imf24.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709313245; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qnj3T/yiDAkGvZCqgg6YuETmE+gd/1HO/cDIruX0/iY=; b=RRpe9qS5UbuViwmtHEpORfyLHcxJa78KHJZpLaREnLcIDTW8t6+ds35Ou4Qlgkkrce50qv c6Q3Xf6ve/okFQQ0FXTz4uK9x2VP+DwAnpHgqZ0zNkdD1MjVhXZBrDRJY3JnSSJ8swyZ1k oe+pEJMKWpw8TyFz00ZOY0sX24u1AHQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709313245; a=rsa-sha256; cv=none; b=dXAUkyLjaCZK5050Bqg+XcDOZK6m7jXSyoHhviQqGrz2UH8Isj8wHD0AVRfSfBX43h/RJl HP+VseQ0OjsRelRjgU8V5zthKyC2grKXO5OuXaTBotSqFo9r1h3wb7+1RhJ1JDvRuyoFj5 Zanf82TH0lUYjxoiPdViO0y+1RamTCg= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; spf=pass (imf24.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 663691FB; Fri, 1 Mar 2024 09:14:42 -0800 (PST) Received: from [10.57.68.58] (unknown [10.57.68.58]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3D3B73F6C4; Fri, 1 Mar 2024 09:14:02 -0800 (PST) Message-ID: <7774edad-a194-4259-a95f-88bcef846f90@arm.com> Date: Fri, 1 Mar 2024 17:14:00 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3 1/4] mm: swap: Remove CLUSTER_FLAG_HUGE from swap_cluster_info:flags Content-Language: en-GB To: David Hildenbrand , Matthew Wilcox Cc: Andrew Morton , Huang Ying , Gao Xiang , Yu Zhao , Yang Shi , Michal Hocko , Kefeng Wang , linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <6541e29b-f25a-48b8-a553-fd8febe85e5a@redhat.com> <2934125a-f2e2-417c-a9f9-3cb1e074a44f@redhat.com> <049818ca-e656-44e4-b336-934992c16028@arm.com> <4a73b16e-9317-477a-ac23-8033004b0637@arm.com> <1195531c-d985-47e2-b7a2-8895fbb49129@redhat.com> <5ebac77a-5c61-481f-8ac1-03bc4f4e2b1d@arm.com> From: Ryan Roberts In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 340DD180002 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: pkh5biyuygnzy36cj4385ck6b4o7o613 X-HE-Tag: 1709313245-826798 X-HE-Meta: U2FsdGVkX1/qg1LTcCSZQWvJI60wDnQVHwEWLz7waLGu5CtFQUUNza0CqGFs/v4uTk8VhILIfmDP4sE8Jm6x/BXUovvfJSBK2eKGKloZUmLcIaW3XG0Xa28FkP2DnZx6NUnshk3xlW3zoq8Vhbjonfoi7A8yZlBBL2OVgMe5V7wAe434YxKD8W+wVJEpGH+qh3ntz9VEbAmt7t+OE5SDvI1NSBgpxLohSS8NWw8mjWE3sNImlXXv9PZ2d89Rfg+LMtQbHRAD26VhcMsjQg5PbjP0uqiqgcN+YEWzMuuCRM+tWuCo0yEHayAzWa9KYAuyewDed6cO7mDKMTam1LcyK5Olw3GfTzQh2D3OOo2YtfnzFJakXGKFQNaw06Tn/+IC/KXcXrfo7MUiR7oT95w1jar5wfp6qPoH3ixdiOS29voLFUY1/4UkVd+dbrA9vHnSCbVdLlt88xNef9P+dG2hRob50mLfFHcr1gMcORtiT68T9kHuUh28cRz1rimAlqCxSP39r8nbI0sW7PBrlORBFrpCh4MBjQTi34gwMPL6zP6SqdZASNN2EVk6Ga2oDYkXIb2Z+DDW9qe5vi7vQsP2u9wd8TSUQnPpQ1OVWMDmihI3DAe+8MBlvWebfng8iPhjQOmW9kQY7zIxBnLaxvkiZTSx9fwrXA5MOPlc6byuTgXs30t7RM7rCRwXTm66vwlU0ZugOchx7dFFaqoRhuqS3qfp3gblKvcgItbfZmdSCaDvfeD2+/mLpspo55oE7MVK2A2orb6Smk74FbkJ+Bl/r9Mvsd8QCfsdFWY8FkBQp1CiQovjqUMll2ArK1xRptRbfJM9MN41tuyG2BsnBduls63nSObh/ddevWjR+4dPeKbdd7krlV6BObYOd/xMD/Nu1dMoMfwBMSg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 01/03/2024 17:00, David Hildenbrand wrote: > On 01.03.24 17:44, Ryan Roberts wrote: >> On 01/03/2024 16:31, Matthew Wilcox wrote: >>> On Fri, Mar 01, 2024 at 04:27:32PM +0000, Ryan Roberts wrote: >>>> I've implemented the batching as David suggested, and I'm pretty confident it's >>>> correct. The only problem is that during testing I can't provoke the code to >>>> take the path. I've been pouring through the code but struggling to figure out >>>> under what situation you would expect the swap entry passed to >>>> free_swap_and_cache() to still have a cached folio? Does anyone have any idea? >>>> >>>> This is the original (unbatched) function, after my change, which caused >>>> David's >>>> concern that we would end up calling __try_to_reclaim_swap() far too much: >>>> >>>> int free_swap_and_cache(swp_entry_t entry) >>>> { >>>>     struct swap_info_struct *p; >>>>     unsigned char count; >>>> >>>>     if (non_swap_entry(entry)) >>>>         return 1; >>>> >>>>     p = _swap_info_get(entry); >>>>     if (p) { >>>>         count = __swap_entry_free(p, entry); >>>>         if (count == SWAP_HAS_CACHE) >>>>             __try_to_reclaim_swap(p, swp_offset(entry), >>>>                           TTRS_UNMAPPED | TTRS_FULL); >>>>     } >>>>     return p != NULL; >>>> } >>>> >>>> The trouble is, whenever its called, count is always 0, so >>>> __try_to_reclaim_swap() never gets called. >>>> >>>> My test case is allocating 1G anon memory, then doing madvise(MADV_PAGEOUT) >>>> over >>>> it. Then doing either a munmap() or madvise(MADV_FREE), both of which cause >>>> this >>>> function to be called for every PTE, but count is always 0 after >>>> __swap_entry_free() so __try_to_reclaim_swap() is never called. I've tried for >>>> order-0 as well as PTE- and PMD-mapped 2M THP. >>> >>> I think you have to page it back in again, then it will have an entry in >>> the swap cache.  Maybe.  I know little about anon memory ;-) >> >> Ahh, I was under the impression that the original folio is put into the swap >> cache at swap out, then (I guess) its removed once the IO is complete? I'm sure >> I'm miles out... what exactly is the lifecycle of a folio going through swap out? > > I thought with most (disk) backends you will add it to the swapcache and leave > it there until there is actual memory pressure. Only then, under memory > pressure, you'd actually reclaim the folio. OK, my problem is that I'm using a VM, whose disk shows up as rotating media, so the swap subsystem refuses to swap out THPs to that and they get split. To solve that, (and to speed up testing) I moved to the block ram disk, which convinces swap to swap-out THPs. But that causes the folios to be removed from the swap cache (I assumed because its syncrhonous, but maybe there is a flag somewhere to affect that behavior?) And I can't convince QEMU to emulate an SSD to the guest under macos. Perhaps the easiest thing is to hack it to ignore the rotating media flag. > > You can fault it back in from the swapcache without having to go to disk. > > That's how you can today end up with a THP in the swapcache: during swapin from > disk (after the folio was reclaimed) you'd currently only get order-0 folios. > > At least that was my assumption with my MADV_PAGEOUT testing so far :) >