From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6583FEB64DC for ; Fri, 14 Jul 2023 07:31:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9A9568D0001; Fri, 14 Jul 2023 03:31:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 959986B0072; Fri, 14 Jul 2023 03:31:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7FA288D0001; Fri, 14 Jul 2023 03:31:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 6A6C76B0071 for ; Fri, 14 Jul 2023 03:31:15 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 2FD98A015B for ; Fri, 14 Jul 2023 07:31:15 +0000 (UTC) X-FDA: 81009396510.04.CD69DAE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf11.hostedemail.com (Postfix) with ESMTP id BF5AA4000F for ; Fri, 14 Jul 2023 07:31:12 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=dxgYXCta; spf=pass (imf11.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689319873; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AFS9yCsx5eOJxRc13hfDeSrkj2kufE3IZQlqjqcPfw8=; b=VX+2jWmzkCukf7GsJDWuV6EPpgcA3wR9T7TTmlqzyUfl3AAl40RBnX3vPXdyJTAK0uvnPM OGvfPMACyNxPNgcpTD0CGgjRwrur76MNw4tCB49H3dxz3zsFnLM4nHpNuyQTxAbtz/TUjv 4s/IGfCh5azJDbCK/LEjLfGlGvdND7s= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689319873; a=rsa-sha256; cv=none; b=m0U0TcVj21PuHKejqxos982sUg7Izj41BaWZP1jC8RYHfjcQfie9HRUpUGCgEvOJPFNCad twqDG36CthoUIEQQtefpXC3iml49MEk2HGnnBfC73QJg39PjHtl6wgacbmJs1q2yeV+Pge V/xOuKc7EDwNP2Fh6St9xw1xa7Z3DoE= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=dxgYXCta; spf=pass (imf11.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1689319872; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AFS9yCsx5eOJxRc13hfDeSrkj2kufE3IZQlqjqcPfw8=; b=dxgYXCtatDEYN5VF4lETz8IVsJsojAj8jBAFKt2oPqijuRcJb3HJ21eXVGjF+u/+Whankl wZ1BMYbbd4EzW1vOHATy4/9DN64EM8Xaw2AQohRdWqDoMuC/y1mb2TQ07F3C9vF2AICddd UHx5nHtBgdK0Lo/ai+8XwN3hCCSZQHE= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-22-tC7zbLoNMeeHi2stp3ApKA-1; Fri, 14 Jul 2023 03:31:09 -0400 X-MC-Unique: tC7zbLoNMeeHi2stp3ApKA-1 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-3f41a04a297so8487905e9.3 for ; Fri, 14 Jul 2023 00:31:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689319867; x=1691911867; h=content-transfer-encoding:in-reply-to:subject:organization:from :references:cc:to:content-language:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=AFS9yCsx5eOJxRc13hfDeSrkj2kufE3IZQlqjqcPfw8=; b=GIvN6Qx7uKg88aFduJjGn88/EApgD9t15tpu07VAuOxNlpj79tjzhrDg5Ax3E5SyOR 5waX7nWm0iiGOdohqQZvzxxvw6pj21plEZcWzWRZF/ckwRnFrJZTiTz+Gsp3/fMYfmMq FhYHP9KvAEWxlmMU9u4ZyNwiU10Cnbi6CabhQ0R2F5dFHJZiIr7iQyOo2HQ6WMC9wNW/ HyMCvV4Wsfi79xTnQMXRfBYOQaeBSup7OWm0e3pjN5K1ydFq++0fJyUFoirOTKAQrlyt jrBUbSIciWhFIHUixrZFjEyqHetflsVRc6e8ggTHvXaF/IwWdm3HcPFenr+yvB7cutEp lQWA== X-Gm-Message-State: ABy/qLaH85FXjxGzv8aWSfOqjMw7UbxfjmtKtbDBFMq6vVGB01H1s3xW 13pba44+N639A49u6CvnCDasXFFuJSDyQ2YBwfv5NRvf0JYAtUI0JFwlMRiLfT9Czk37+lxsclh eWZqp9fJaWwY= X-Received: by 2002:a05:600c:205:b0:3f5:878:c0c2 with SMTP id 5-20020a05600c020500b003f50878c0c2mr3291414wmi.3.1689319867339; Fri, 14 Jul 2023 00:31:07 -0700 (PDT) X-Google-Smtp-Source: APBJJlH3WkevYS04U7Q1h5Why0+FtHC0+qv1IsLhd/s9PO3XWXE+lvk/das3PR765u5hXm9MbqPjxA== X-Received: by 2002:a05:600c:205:b0:3f5:878:c0c2 with SMTP id 5-20020a05600c020500b003f50878c0c2mr3291394wmi.3.1689319866865; Fri, 14 Jul 2023 00:31:06 -0700 (PDT) Received: from ?IPV6:2003:cb:c70a:4500:8a9e:a24a:133d:86bb? (p200300cbc70a45008a9ea24a133d86bb.dip0.t-ipconnect.de. [2003:cb:c70a:4500:8a9e:a24a:133d:86bb]) by smtp.gmail.com with ESMTPSA id v3-20020a05600c470300b003f7f475c3bcsm1383010wmo.1.2023.07.14.00.31.05 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 14 Jul 2023 00:31:06 -0700 (PDT) Message-ID: Date: Fri, 14 Jul 2023 09:31:05 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 To: Yu Zhao , "Yin, Fengwei" Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, willy@infradead.org, ryan.roberts@arm.com, shy828301@gmail.com, "Vishal Moola (Oracle)" References: <20230713150558.200545-1-fengwei.yin@intel.com> <8547495c-9051-faab-a47d-1962f2e0b1da@intel.com> From: David Hildenbrand Organization: Red Hat Subject: Re: [RFC PATCH] madvise: make madvise_cold_or_pageout_pte_range() support large folio In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: BF5AA4000F X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: n45ejze5txmjd6ymtdnfczw5i39a8fgw X-HE-Tag: 1689319872-19 X-HE-Meta: U2FsdGVkX1/BD3S63Jxiud86eA0UYh1BWq2jyTEXY4Gfk/Sbesz9x4edCEmq0SuXNzSUa7D95ic7C01ZgkAfNokBVSZmXoyEY9akobe0VFWc4EsUn41i+cpATE2v3qFMyYSU3PRM4IRxYOQ9dObRl3s6bOZz7Yvyx/TnnsC63X0acck3C6dDrCLCDq8HC3ufEpksVp7SqBfD06JAdsviIof7SlUe6oM5vG5N7DHqZBf4bo+SA8CmJX8aywt1dMMkiADEYccSbQV0bBajbsWIXRWC7S2MBlr9Zhw8GwgUxYdvalpgWN3lWUKj1oR0xq9JobUbBHI6/iRMzheQmvEcNoFJuC0gzOKH18KB0fMz+Oyyes0LgMO/tCglJ8VRyk/GaJp/feWU78ZZMQU3i4IV44e91Ve9g68KPQO6lrrXgRSQh1JFpnWHA8wEKlvuxX06bpimoQEseyd4wtVA0D09DK2zl9PzD9k1l9BRg1ItH1RN3v3M2ar9VYPSvjzRCMCT87xP+n1zumTadT8sK9Dn4OgMhx0ztGZHwm8fKvQw4arXMv9Vpgp3tKgSo88hVj8hqBJOmCheFFUm7tpZgJNESPTDCE1JyyllDdLxkLd5szxY44pRCJDfkF2zXJocnDEjYrC0PCbr51dxpxWsvt0fXFLc6iwoskADVudLCWlRWVfDBYJ+SZlbIs7ZqV+aICLB2WNKGaiR9z4KPgBsPF8OjMHsvNqKUb8NTO4pQ5vhjKG0XaoeGukow4AaEG0Zwkv6ubdybKN1rQ5HQF8EzO7nwJ+K0pkk/+as+nvmt0935+r2BNkpbwBIGMWeS1XerMuHGYKWJvtiNg396WrrgAzAm0E8txmRaDTdmGjBs/VlBsoui6v49mpjpK2fPquCqqpmssCnKSpJHqQaocL/NcqhjUJ3hCwwXRyh4Rsuc+QWlpTHpJ35i+hNHnhiT3u82ltPF9aMT4IRABwyyoX6lE7 1zcmk8Fj XCdfV2k0JiMvX37rvb1B/tp/J5TOLQbWAO8N8um7hVbQt4Ummd18fipstZMPHGQLIOLgCzSQfKSqF8Aa0X2qw865sPOWY6DKlWJMmmcQ7i7PXh1GHRz6mJJAvvTwl54pT10kb9f8BxHtbbIjqXyniF6ajO3FFn9HyFo9t9lOfQ2tILvRfFCc15jECKH9nxaJAk4RG1yMiaksxzOuHj3qEvPdxP+GL1Qf1QaE3nWgq8wxSMjX5Qfi5aSVhFzcE0prcUa3AwqaTpYs86umpYEqQkiWNUqwsLHuz570HMLosdQrfC/CNlvnWEq3UzoCwU4ym7r/L/yzcT1rEoGiRgHTBSUSQwVTauLXhYT6GZBO6ye+204qB8H7omgUR/XgV8EQTdKG7bvFm8h/eghzOXOy7CqKtVWOP77ZD5K1lGj9F/10GPWHw3wOc26eFO+vKMUMbZ8pHwtPIK3/0TX9yRWmAxCCDFlYDGafw8w7d98+OZUjovicekMgB9DiV/b9j1OaL8eUscmweExUHKHzEJW6/6CgLIJa+t6czRdPbnbQICjYsUd+V94rQddMJ+uREjgCneah4TEYY06WyF2jY5+6FeNhaGV/tgpg951L+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 14.07.23 05:23, Yu Zhao wrote: > On Thu, Jul 13, 2023 at 9:10 PM Yin, Fengwei wrote: >> >> >> >> On 7/14/2023 10:08 AM, Yu Zhao wrote: >>> On Thu, Jul 13, 2023 at 9:06 AM Yin Fengwei wrote: >>>> >>>> Current madvise_cold_or_pageout_pte_range() has two problems for >>>> large folio support: >>>> - Using folio_mapcount() with large folio prevent large folio from >>>> picking up. >>>> - If large folio is in the range requested, shouldn't split it >>>> in madvise_cold_or_pageout_pte_range(). >>>> >>>> Fix them by: >>>> - Use folio_estimated_sharers() with large folio >>>> - If large folio is in the range requested, don't split it. Leave >>>> to page reclaim phase. >>>> >>>> For large folio cross boundaries of requested range, skip it if it's >>>> page cache. Try to split it if it's anonymous folio. If splitting >>>> fails, skip it. >>> >>> For now, we may not want to change the existing semantic (heuristic). >>> IOW, we may want to stick to the "only owner" condition: >>> >>> - if (folio_mapcount(folio) != 1) >>> + if (folio_entire_mapcount(folio) || >>> + (any_page_within_range_has_mapcount > 1)) >>> >>> +Minchan Kim >> The folio_estimated_sharers() was discussed here: >> https://lore.kernel.org/linux-mm/20230118232219.27038-6-vishal.moola@gmail.com/ >> https://lore.kernel.org/linux-mm/20230124012210.13963-2-vishal.moola@gmail.com/ >> >> Yes. It's accurate to check each page of large folio. But it may be over killed in >> some cases (And I think madvise is one of the cases not necessary to be accurate. >> So folio_estimated_sharers() is enough. Correct me if I am wrong). > > I see. Then it's possible this is also what the original commit wants > to do -- Minchan, could you clarify? > > Regardless, I think we can have the following fix, potentially cc'ing stable: > > - if (folio_mapcount(folio) != 1) > + if (folio_estimated_sharers(folio) != 1) > > Sounds good? Adding to the discussion, currently the COW selftest always skips a PTE-mapped THP. For example: # [INFO] Anonymous memory tests in private mappings # [RUN] Basic COW after fork() ... with base page ok 1 No leak from parent into child # [RUN] Basic COW after fork() ... with swapped out base page ok 2 No leak from parent into child # [RUN] Basic COW after fork() ... with THP ok 3 No leak from parent into child # [RUN] Basic COW after fork() ... with swapped-out THP ok 4 No leak from parent into child # [RUN] Basic COW after fork() ... with PTE-mapped THP ok 5 No leak from parent into child # [RUN] Basic COW after fork() ... with swapped-out, PTE-mapped THP ok 6 # SKIP MADV_PAGEOUT did not work, is swap enabled? ... The commit that introduced that change is: commit 07e8c82b5eff8ef34b74210eacb8d9c4a2886b82 Author: Vishal Moola (Oracle) Date: Wed Dec 21 10:08:46 2022 -0800 madvise: convert madvise_cold_or_pageout_pte_range() to use folios This change removes a number of calls to compound_head(), and saves 1729 bytes of kernel text. folio_mapcount(folio) is wrong, because that never works on a PTE-mapped THP (well, unless only a single subpage is still mapped ...). page_mapcount(folio) was wrong, because it ignored all other subpages, but at least it worked in some cases. folio_estimated_sharers(folio) is similarly wrong like page_mapcount(), as it's essentially a page_mapcount() of the first subpage. (ignoring that a lockless mapcount-based check is always kind-of unreliable, but that's msotly acceptable for these kind of things) So, unfortunately, page_mapcount() / folio_estimated_sharers() is best we can do for now, but they miss to detect some cases of sharing of the folio -- false negatives to detect sharing. Ideally we want something like folio_maybe_mapped_shared(), and get rid of folio_estimated_sharers(), we better to guess the exact number, simply works towards an answer that tells us "yep, this may be mapped by multiple sharers" vs. "no, this is definitely not mapped by multiple sharers". The "mapped" part of it indicates that this does not catch all cases of sharing. But it should handle most of the cases we care about. There, we can then implement something better than what folio_estimated_sharers() currently does: static inline bool folio_maybe_mapped_shared(folio) { if (likely(!folio_test_large(folio))) return atomic_read(&folio->_mapcount) > 0; /* Mapped multiple times via PMD? */ if (folio_test_pmd_mappable(folio) return folio_entire_mapcount() > 1; /* * First subpage is mapped multiple times (especially also via * PMDs)? */ if (page_mapcount(folio_page(folio, 0) > 1) return true; /* TODO: also test last subpage? */ /* Definitely shared if we're mapping a page multiple times. */ return folio_total_mapcount(folio) > folio_nr_pages(folio); } There are some more things we could optimize for. While it's better than what we have right now: (a) It's racy. Well, it's always been racy with concurrent (un)mapping and splitting. But maybe we can do better. (b) folio_total_mapcount() is currently expensive. (c) there are still false negatives even without races. For anon pages, we could scan all subpages and test if they are PageAnonExclusive, but it's also not really what we want to do here. I have some idea to handle anon pages better to avoid any page table walk or subpage scan, improving (a), (b) and (c). It might work for pagecache pages with some more work, but it's a bit more complicated with the scheme I have in mind). First step would be replacing folio->_nr_pages_mapped by folio->_total_mapcount. While we could eventually have folio->_total_mapcount in addition to folio->_nr_pages_mapped, I'm, not sure if we want to go down that path That would make folio_total_mapcount() extremely fast (I'm working on a prototype). The downsides are that (a) We have to make NR_ANON_MAPPED/NR_FILE_MAPPED accounting less precise. Easiest way to handle it: as soon as a single subpage is mapped, account the whole folio as mapped. After all, it's consuming memory, so who cares? (b) We have to find a different way to decide when to put an anonymous folio on the deferred split queue in page_remove_rmap(). Some cases are nasty to handle: PTE-mapped THP that are shared between a parent and a child. I have to do some more thinking about (b), I'd be happy about thoughts on that. -- Cheers, David / dhildenb