From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECB5FC4345F for ; Wed, 1 May 2024 04:27:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4035C6B0082; Wed, 1 May 2024 00:27:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3B42A6B0083; Wed, 1 May 2024 00:27:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 254906B0085; Wed, 1 May 2024 00:27:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 0629E6B0082 for ; Wed, 1 May 2024 00:27:13 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 3A5381404D2 for ; Wed, 1 May 2024 04:27:13 +0000 (UTC) X-FDA: 82068542346.29.A3F22A2 Received: from mail-yw1-f179.google.com (mail-yw1-f179.google.com [209.85.128.179]) by imf04.hostedemail.com (Postfix) with ESMTP id 9F41C40005 for ; Wed, 1 May 2024 04:27:11 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="GBr/Uv8P"; spf=pass (imf04.hostedemail.com: domain of ioworker0@gmail.com designates 209.85.128.179 as permitted sender) smtp.mailfrom=ioworker0@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714537631; a=rsa-sha256; cv=none; b=je6UQ+paax2jzBE+3GFYeQ4HQcwakhxw96oR64KAFudII2ubWl8ZsNYl/H36zJd9DhJsbp l3aOomXXDsVb5xqTWQw74vXInCaTcG6xyEss/ECiQaSBOeiwwbcqBJEaZ8Hk3ic0Dl914d TwRnBNX6EDTNp6CIiCj0XK1Irmss1CM= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="GBr/Uv8P"; spf=pass (imf04.hostedemail.com: domain of ioworker0@gmail.com designates 209.85.128.179 as permitted sender) smtp.mailfrom=ioworker0@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714537631; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=98ec+jViXsbQQ2xh2G7x7x/neBurum8aB5rqKPV5rEE=; b=jLtBAP3tIkhNrQZUHmxd+9kx34JnVgiEjDBQqMYdqltOZwlGgBYeL75D/2SGbsZF8WCPSN KiqIiNmrWkgmfzni3T/XiTZu6PwodDrg35AKwBU3qs+X60PumRyG9jtZPSniaucG9MdP2g TggECJOX0jviGTIVqn2t+z4ld60lnCE= Received: by mail-yw1-f179.google.com with SMTP id 00721157ae682-618874234c9so74288367b3.0 for ; Tue, 30 Apr 2024 21:27:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714537631; x=1715142431; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=98ec+jViXsbQQ2xh2G7x7x/neBurum8aB5rqKPV5rEE=; b=GBr/Uv8PEavlHateGlRCuqwffFOzY9CPpVLNj3acpHYqJXEkHqyRJ9nAoaREioiSEx 1FiuuokwN8SLZ19hnOgf+Y4IKgSRo3eQZCuFVV/Kp5UWZljKrbrpmFLypUtaoPiCLxSY lE8EU6e2o5nSSra5i3tbycVGubSmAKoZKADZ5lAwpCxwXVDQdpderdRsYcrpMsvWuLa7 /Oe+mVOJbMGl1UIo7gv0tLvZaHjq/eotANKCiZT463Jg/Dls2Wh+a/9advH4bI3wrvzV RDF8IAY7moDE8xDzMRQWervsDO3Q4NE9ZBkARZeL1gfygoH+k2jxTFvSjkYEUha6oKFX QPYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714537631; x=1715142431; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=98ec+jViXsbQQ2xh2G7x7x/neBurum8aB5rqKPV5rEE=; b=ojp3tuhrqigvCjDzW0dF4Vikg49WWUENg73GS5mRlS7vVfHyjxGegKG1vOEvAYQlua brQsE36Op6uk/RwGOaTS4Mll7if+D7NhEOA31bL13fTtTXvSCA8hg8NGrivP0NkzfmKT HxGLZDPt+9/rZ3zPmeWgLqUMOW9Mo3cHCtMmGdbRy0572VTrS1cQw9EYKEM+/suS8GUi Sfg+Ds5scelUnH0zS0q05x5+tV54hBRZGwg68p6fkN+4y380RbM3It1nyOmpzwZT4U2X wWqjDBnZnpFpKM6TvS+EkgoYhGC0iYXwcAsGSFavJurRAv7c2lvEkvxsSMoHZ1csE0bx Vzyw== X-Forwarded-Encrypted: i=1; AJvYcCWsOAEiXk0Q0xR9xga477bQPOynKieLBKJzdOfevgkwz9ENs8W6YQOlRuXSZjYSZJa8f4uJqSbtDqaNmj2NuxF7b9Q= X-Gm-Message-State: AOJu0YyPgwAFCVs9mhJJHPllKS98PqA4FwEQTrQAVGiQ5xyCPteexfSn EJkDLry+4agfmfrc41G8WP9IFojsslJSL5WOB/nFWonuKVpXx0n+ X-Google-Smtp-Source: AGHT+IHvowBnNqumv0l7fBeB+9uEso4P+IK8K5UtLCH8iXyLnfehmiGproDsTQ+1wyUaRyOSgENU+w== X-Received: by 2002:a05:690c:7090:b0:61b:33fc:ad8f with SMTP id jg16-20020a05690c709000b0061b33fcad8fmr1363257ywb.45.1714537630732; Tue, 30 Apr 2024 21:27:10 -0700 (PDT) Received: from LancedeMBP.lan ([112.10.225.242]) by smtp.gmail.com with ESMTPSA id m15-20020a656a0f000000b005dc4806ad7dsm19165970pgu.40.2024.04.30.21.27.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Apr 2024 21:27:10 -0700 (PDT) From: Lance Yang To: akpm@linux-foundation.org Cc: willy@infradead.org, sj@kernel.org, maskray@google.com, ziy@nvidia.com, ryan.roberts@arm.com, david@redhat.com, 21cnbao@gmail.com, mhocko@suse.com, fengwei.yin@intel.com, zokeefe@google.com, shy828301@gmail.com, xiehuan09@gmail.com, libang.li@antgroup.com, wangkefeng.wang@huawei.com, songmuchun@bytedance.com, peterx@redhat.com, minchan@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Lance Yang Subject: [PATCH v4 0/3] Reclaim lazyfree THP without splitting Date: Wed, 1 May 2024 12:26:57 +0800 Message-Id: <20240501042700.83974-1-ioworker0@gmail.com> X-Mailer: git-send-email 2.33.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Stat-Signature: pymb8sppuauzrcsbph6cfb4tochrd56w X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 9F41C40005 X-HE-Tag: 1714537631-737835 X-HE-Meta: U2FsdGVkX1/XWRNsOVa4HEdneSgMilRq2Rf3spy+aSAxmDa7GSZc0cFs7PWse7xLOywB5fFOxpGgH+n9D6kENfi2g/EWiQWnugwsg9t3See+rsHopfnmlkUAMGxd1G378MTge/+4McceCwiuaBH8mb8azw/yC4WOe9v7wDySlLJupPlhY4KrvbJkq+PLTEPrYgWxkTSLOQsSX36QAxHBLr3fRZsVEMNK9z2kAtdoBS6fFySOC7Mm2BXDFQQdiO9HO1uj3DR0lz/RCBbFqK5qBBQKW7wP//DezhXDripuvM/KtTs0SzMXco/GijZIB8u3tnT7Z1+F6ZpUSQCTJs9GIoZGnIo4w7Al1PzCoaAFVShyQ3yk17wSvm5fggOCHT4bhJyQIdiK2nJL/LSKehxcN5TtBQBzUtlE0mnRgoXOfi5GaYNh7hejS5irg+m8C0tJqEg7odqXeb+glvhldc5UKO7WqqNTSWam/5RmQNSqH9gSsrWKwgwv09Lcj4NIjsvqfsLsZXwhccjY4AC3MlmfSMR8wHXbjaYaMezrydnGSBlJb5RHS5x/61pztHAIVTAt+NiFxAx8ZWiArChH82B8i44NjZ8KcrWV5B+uPcXZxbg+tId6A/3tqEXICgPmTr7wCpDxYSCB1zDxzr/bAX04AbRpuWAsXs9Dv6jWj6gPqz/oeAltn7BfyMZ/6vAhdV/FIKkvUqnbj1P3TKu2ofLCLRwlb9SqSuLOsmMf3f6RIJHMX2j8YtQ+YHajP8GSlCchsu00aV8B/Rj+49rXLxgEX0eT1FXYz7k0L0CB9M55N9v2tvT7fPz+oCHMW0SmymKxQWbTSsOj7wknyHuDo5861kh1V8PuGmOF+L29tnkpQg7Vhgii3cpNDor8h3DvChmPXB7BLfanRNL/m5yLYucyGHwbSqNBRaYCN2EtwcRHCRdjeeO492geTsJ/7KYQ1iFiXDz+X5SOCvtBZmlBpm3 sXkadaOH XiUsuuQFsUSw0facDWecaXrEC+tBweyi1bzi3Hp0+MzZAn8jisyMz4xnm4HPs0Ic/0YGxvILg0tIc5Hgu7BpOcwNRGCCBgXvcltpZ+TUV4Oe+ABHhdFAQC5JRVg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi all, This series adds support for reclaiming PMD-mapped THP marked as lazyfree without needing to first split the large folio via split_huge_pmd_address(). When the user no longer requires the pages, they would use madvise(MADV_FREE) to mark the pages as lazy free. Subsequently, they typically would not re-write to that memory again. During memory reclaim, if we detect that the large folio and its PMD are both still marked as clean and there are no unexpected references(such as GUP), so we can just discard the memory lazily, improving the efficiency of memory reclamation in this case. Performance Testing =================== On an Intel i5 CPU, reclaiming 1GiB of lazyfree THPs using mem_cgroup_force_empty() results in the following runtimes in seconds (shorter is better): -------------------------------------------- | Old | New | Change | -------------------------------------------- | 0.683426 | 0.049197 | -92.80% | -------------------------------------------- --- Changes since v3 [3] ==================== - mm/rmap: integrate PMD-mapped folio splitting into pagewalk loop - Resolve compilation errors by handling the case where CONFIG_PGTABLE_HAS_HUGE_LEAVES is undefined (thanks to SeongJae Park) - mm/vmscan: avoid split lazyfree THP during shrink_folio_list() - Remove the unnecessary conditional compilation directives (thanks to Barry Song) - Resolve compilation errors due to undefined references to unmap_huge_pmd_locked and split_huge_pmd_locked (thanks to Barry) Changes since v2 [2] ==================== - Update the changelog (thanks to David Hildenbrand) - Support try_to_unmap_one() to unmap PMD-mapped folios (thanks a lot to David Hildenbrand and Zi Yan) Changes since v1 [1] ==================== - Update the changelog - Follow the exact same logic as in try_to_unmap_one() (per David Hildenbrand) - Remove the extra code from rmap.c (per Matthew Wilcox) [1] https://lore.kernel.org/linux-mm/20240417141111.77855-1-ioworker0@gmail.com [2] https://lore.kernel.org/linux-mm/20240422055213.60231-1-ioworker0@gmail.com [3] https://lore.kernel.org/linux-mm/20240429132308.38794-1-ioworker0@gmail.com Lance Yang (3): mm/rmap: remove duplicated exit code in pagewalk loop mm/rmap: integrate PMD-mapped folio splitting into pagewalk loop mm/vmscan: avoid split lazyfree THP during shrink_folio_list() include/linux/huge_mm.h | 29 ++++++++++ mm/huge_memory.c | 115 +++++++++++++++++++++++++++++++++------- mm/rmap.c | 67 ++++++++++++----------- 3 files changed, 160 insertions(+), 51 deletions(-) -- 2.33.1