From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 208D4CD1284 for ; Thu, 11 Apr 2024 10:29:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A1D536B0089; Thu, 11 Apr 2024 06:29:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9CDB56B008A; Thu, 11 Apr 2024 06:29:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 894346B008C; Thu, 11 Apr 2024 06:29:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6A6716B0089 for ; Thu, 11 Apr 2024 06:29:46 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 244CB1C0E40 for ; Thu, 11 Apr 2024 10:29:46 +0000 (UTC) X-FDA: 81996879972.01.25D5CA5 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf06.hostedemail.com (Postfix) with ESMTP id 249B218000A for ; Thu, 11 Apr 2024 10:29:43 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=none; spf=pass (imf06.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712831384; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SebZo1qfIxE1xKec2pRSyqGWVSGy0nS+Vs6KLQ8eP7k=; b=0rnQFA2f4nKZQypWbYYPz5gRGyxDIbg29zRIjaQMfO7sMdSjGzk3mrrCkQoKR3oMkbPCTV 7QXNy6qcRC8Smkz+/nNvCF+Eh4zr8B3eU5taraX4+CXNHGsd+UAdOe31coBOzSbvs0o4GE HzDAAhRX7fT5IpjUAtAfLLAOY6iKocM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712831384; a=rsa-sha256; cv=none; b=brTMHyGaXDWptgHCdICqvGOkUrlTcPv0VV8RXnBE+htxIw1Qvapc+biUfOVQEkRf6Jsvmn Jocx1CmOdH7C7uoIXAkiDUJaRHSvNUeGM4C7yvRejylhsv0CS6xqx1mgLzkK6uOBeriSHr HXyu9zqbzKlTbNpOMh+X0OqA0HewxXE= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=none; spf=pass (imf06.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CF647113E; Thu, 11 Apr 2024 03:30:12 -0700 (PDT) Received: from [10.1.38.151] (XHFQ2J9959.cambridge.arm.com [10.1.38.151]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6A2B93F6C4; Thu, 11 Apr 2024 03:29:41 -0700 (PDT) Message-ID: Date: Thu, 11 Apr 2024 11:29:40 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v5 0/2] mm/madvise: enhance lazyfreeing with mTHP in madvise_free Content-Language: en-GB To: Lance Yang , Andrew Morton Cc: david@redhat.com, 21cnbao@gmail.com, mhocko@suse.com, fengwei.yin@intel.com, zokeefe@google.com, shy828301@gmail.com, xiehuan09@gmail.com, wangkefeng.wang@huawei.com, songmuchun@bytedance.com, peterx@redhat.com, minchan@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20240408042437.10951-1-ioworker0@gmail.com> <20240410145033.5cdb8a41f3a6894a62191f42@linux-foundation.org> From: Ryan Roberts In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 249B218000A X-Rspam-User: X-Stat-Signature: 6jq74t3ob9bu5na4fcpc7jwpqusqsnfk X-Rspamd-Server: rspam03 X-HE-Tag: 1712831383-349426 X-HE-Meta: U2FsdGVkX1/nfIhzOtmYvfbTP19b64/eQlaE+ygYbJbasa7TB4Z6eQqeF4k+o/xN6fi2lI9WFFeEp6eXYKvswULoVnv3k3VvC+7i2YCmJ832MSnKrrrsNu4DQPpITZnvrCgCRthhQ+C9Fqe58VTtoaZFzdz0YT/6ENDYs5BZz+cxK5+ggNGqo7IJO7/HFdzoe39l8b7cLevKYTpX9Qalr5VL1buS2gtNmcNtwMHCkMSiYeP6Af58m04h5MmmOozvZ7XnyEd5wgrPODmkF25FX6H8fKUhr3sQLMXQe6uCkYrSqy1dMvUVoVa/AiEvBBzge1A1ERplMhNmfyqvu4VZ/UK4ZSs44GMmiRmxMr/SiNiVa/xQxsUJ6j7vx5x/r9O08wDIUUgYJxv2Vvg7Pc+qE+MgX0xZR+NyuLnUdynAF4hyMVHU5zWKVWKLJTFmoTayHpAp8/Jpai5/DDl8u6kmleaPyFkDeP/cWVPeCsJLKOhmK2SBhgD0BabC8LkYaDgR4IX8PG7cQM9KbNLaB1eEKeV3LJqt8lBheiNdOlbYE+VOs5LfeHWo7SuB1dKKRnhSf5NuyG9O7x9BxT+aliUGf0s5ihc9qyGIKovrz2bQ1jnym+C8tMx8PVHKL4jLsY17SV46aBWr00Mhu8YE+6hdYdgAPEBQvBZHdCyVjD2Fm12y6/8zi8dbyB51+BBUsfxJztS+aO2VZwaeJSc+m+BzqaF3VSsEEPPlPng4Tps4okt094AAzA6xZERAFwXVeMq778RIFBb4DPxMaNxZLhlSrodf20mD8Psfdsby2gqVla3uoS3jCBZJtUhW/7X2UlxRI/FoFzlYFmTbayB0ixo3uawO91NznzgsmQ07eI4yMIMe2k8hrC1RNTA9xbru28p5sLC9/nk63WM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 11/04/2024 06:01, Lance Yang wrote: > On Thu, Apr 11, 2024 at 5:50 AM Andrew Morton wrote: >> >> On Mon, 8 Apr 2024 12:24:35 +0800 Lance Yang wrote: >> >>> Hi All, >>> >>> This patchset adds support for lazyfreeing multi-size THP (mTHP) without >>> needing to first split the large folio via split_folio(). However, we >>> still need to split a large folio that is not fully mapped within the >>> target range. >>> >>> If a large folio is locked or shared, or if we fail to split it, we just >>> leave it in place and advance to the next PTE in the range. But note that >>> the behavior is changed; previously, any failure of this sort would cause >>> the entire operation to give up. As large folios become more common, >>> sticking to the old way could result in wasted opportunities. >>> >>> Performance Testing >>> =================== >>> >>> On an Intel I5 CPU, lazyfreeing a 1GiB VMA backed by PTE-mapped folios of >>> the same size results in the following runtimes for madvise(MADV_FREE) >>> in seconds (shorter is better): >>> >>> Folio Size | Old | New | Change >>> ------------------------------------------ >>> 4KiB | 0.590251 | 0.590259 | 0% >>> 16KiB | 2.990447 | 0.185655 | -94% >>> 32KiB | 2.547831 | 0.104870 | -95% >>> 64KiB | 2.457796 | 0.052812 | -97% >>> 128KiB | 2.281034 | 0.032777 | -99% >>> 256KiB | 2.230387 | 0.017496 | -99% >>> 512KiB | 2.189106 | 0.010781 | -99% >>> 1024KiB | 2.183949 | 0.007753 | -99% >>> 2048KiB | 0.002799 | 0.002804 | 0% >> >> That looks nice but punting work to another thread can slightly >> increase overall system load and can mess up utilization accounting by >> attributing work to threads which didn't initiate that work. My understanding is that this speedup is all coming from the avoidance of splitting folios synchonously in the context of madvise(MADV_FREE). It's not actually punting anymore work to be done lazily, its just avoiding doing extra uneccessary work up front. In fact, it would result in less work to do at lazyfree time because the folios remain large so there are fewer folios to free. Perhaps I've misunderstood your point? Thanks, Ryan >> >> And there's a corner-case risk where the thread running madvise() has >> realtime policy (SCHED_RR/SCHED_FIFO) on a single-CPU system, >> preventing any other threads from executing, resulting in indefinitely >> deferred freeing resulting in memory squeezes or even OOM conditions. >> >> It would be good if the changelog(s) were to show some consideration of >> such matters and some demonstration that the benefits exceed the risks >> and costs. >> > > Hey Andrew, > > Thanks for bringing up these concerns! > > I completely agree that we need to consider such masters and include > them into the changelog(s). Additionally, I'll do my best to show that the > benefits exceed the risks and costs, and then update the changelog(s) > accordingly. > > Thanks again for your time! > Lance