From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41A53C001B0 for ; Mon, 24 Jul 2023 11:42:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AA8DD6B0071; Mon, 24 Jul 2023 07:42:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A59198E0002; Mon, 24 Jul 2023 07:42:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 920C36B0075; Mon, 24 Jul 2023 07:42:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 8266C6B0071 for ; Mon, 24 Jul 2023 07:42:19 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 5D14F1409CC for ; Mon, 24 Jul 2023 11:42:19 +0000 (UTC) X-FDA: 81046317198.09.AFB3838 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf13.hostedemail.com (Postfix) with ESMTP id 2423B2001E for ; Mon, 24 Jul 2023 11:42:16 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MAFk3PQs; spf=pass (imf13.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690198937; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GgAI695a4cI34ywj8xkXOcISfqAt3iZAwFHCvpdyFYY=; b=bq59LO6vUKRBfccytPXdN1nis2yzyWG4BcvwAlKb/jrgCMEXn/FNZw1Z7qfXzBPtRlJl8N zUE4yb79SLLPTVgOCsxeQaea0I6pS2kCEhEHh0IYZRrcuR89gohR0S3N3uTIedV4QfupBG 0bHtCFkhf/QdyB0icedeHrk50SKFUUo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690198937; a=rsa-sha256; cv=none; b=Y8vZ0442AIYnLv+kdj0zVTfgEaSoJXEWouiJX1jejTB0uLv4pUapvqMTuCc46M57Y9pnTB UPtOMP9h9P0hTaSjPyhYfrb67WmdDZHV3CA5Mjgq4dQd3Gizw0uufNTebooUzs8Itp1jtT CWxZxOjbxaRpgdb0W+E0JbG/elt2zfs= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MAFk3PQs; spf=pass (imf13.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690198936; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GgAI695a4cI34ywj8xkXOcISfqAt3iZAwFHCvpdyFYY=; b=MAFk3PQs7oeIvHKfGlJEmvRbG++mhqK8vMcbJoVINYbcy3dXOLm4QTQTifKlT1goj2nzgY g+ztCsROYpTjnxpNuVOxhP9MLCgFiJp6t5p1PLfSjLBrlLgYi1Iy03cz4XZIEEbV5de0Z1 xO7TR42GpbV9qWhAnI7V8Dp8CJrB12Q= Received: from mail-lj1-f198.google.com (mail-lj1-f198.google.com [209.85.208.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-150-XYoULazWNGWWg5IoI1xkzQ-1; Mon, 24 Jul 2023 07:42:15 -0400 X-MC-Unique: XYoULazWNGWWg5IoI1xkzQ-1 Received: by mail-lj1-f198.google.com with SMTP id 38308e7fff4ca-2b9939495f8so5414411fa.2 for ; Mon, 24 Jul 2023 04:42:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690198933; x=1690803733; h=content-transfer-encoding:in-reply-to:organization:from:references :cc:to:content-language:subject:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=GgAI695a4cI34ywj8xkXOcISfqAt3iZAwFHCvpdyFYY=; b=XuYyG2e6/vsaQbJ9r5VxTIec5QcafEvDUjXMkNAQRC1Qrf1IptSn3sVNeFyNr5pJpp D4oIdgEZDkBeeOuXfeFvzEcaho1XbO5OsbBeqB5Ik9EcmJPcMoeuHC3NE5kAdc+5uwvq yAC0ELGPPmaPoTuCgMgdebi4wqZpLE8uCscJC+6DFAqehFZf819j3b5j64mjOE3+b04j L1H2B6wWf8OZe8HPhohXfrwT/WBUIuzSFGFAn00R+sj2GSgxhoCMM7Vwxle2iLpQ09ry FpRnz3s+LnEnoeM5yDkd9anT9Zmo8giJ+3M58YINNVBfVn97fYqqbEoQqoy87JBO1GIL 4cPg== X-Gm-Message-State: ABy/qLZ+gkd794KAAjSzxonaekTz8yWBhU5DbhwnBG/Xw8g97PSz18q1 iLpYq3suG8VdypqrIXR3wGTEjW8lgz6Avf5b2Pa657CbKEzk7n43bnYydraNqaN3c46nbdHHsDH OoIlbiQM4ZNE= X-Received: by 2002:a2e:7411:0:b0:2b6:c8ba:90dc with SMTP id p17-20020a2e7411000000b002b6c8ba90dcmr4881020ljc.36.1690198933654; Mon, 24 Jul 2023 04:42:13 -0700 (PDT) X-Google-Smtp-Source: APBJJlEf0ORuvCIwNyyHbrx+3w53+8VWgR5hSNzNFDlY/1IJ/SjJGmghWc9ZGz3LQR+tzaesmazLBw== X-Received: by 2002:a2e:7411:0:b0:2b6:c8ba:90dc with SMTP id p17-20020a2e7411000000b002b6c8ba90dcmr4881002ljc.36.1690198933296; Mon, 24 Jul 2023 04:42:13 -0700 (PDT) Received: from ?IPV6:2003:cb:c73d:bb00:91a5:d1c:3a7e:4c77? (p200300cbc73dbb0091a50d1c3a7e4c77.dip0.t-ipconnect.de. [2003:cb:c73d:bb00:91a5:d1c:3a7e:4c77]) by smtp.gmail.com with ESMTPSA id x12-20020a5d650c000000b003143aa0ca8asm12627449wru.13.2023.07.24.04.42.12 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 24 Jul 2023 04:42:12 -0700 (PDT) Message-ID: <8d68377e-7dab-e117-d47d-06f018a99067@redhat.com> Date: Mon, 24 Jul 2023 13:42:11 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: Prerequisites for Large Anon Folios To: Ryan Roberts , "Yin, Fengwei" , Zi Yan , Matthew Wilcox , Yu Zhao Cc: Linux-MM References: <3393b7c4-2a21-666e-a791-4c1be63b1af3@arm.com> <2c2e9256-2d14-c7c6-2112-69b198cea063@intel.com> <3b08e3d9-29c3-6451-91a3-826f9a28f34d@arm.com> From: David Hildenbrand Organization: Red Hat In-Reply-To: <3b08e3d9-29c3-6451-91a3-826f9a28f34d@arm.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 2423B2001E X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: b9atmbhzkfkxb4dm73b38g4e9fmbg5wb X-HE-Tag: 1690198936-78790 X-HE-Meta: U2FsdGVkX19sM1xdiY8o6xJDyDOGoOTz1SKSwFmoj7sX6uifJAliDlNerx/55VxJv/YvoHpavA/0lOFu+WvTnNJuy0fj9G3ysBjdHMsXH4igiRJIoj0Z6M5BYvU1dXtNy5+yw5c53azGzD1Ew4hAyZKAIQ/BsieGO+ZWXvGuxFZLfMUJKecdy45EwPqUvxmc/1zq3xKfFCXIpqGdESVrBoFOU5Sk3T8+yqRa6oZzGVZeOpQc2x8vBGionV1PXgtGD+NEvQLCpvJ3SxwhBI/5ejfEuxtLxN+ZQ8tHEN7hM7ceUbpaOiD9hJMtOTRbMvikMroiBYMp44gMBNXB8A0rEsYRplM1KESM5eJCWAiDjYpJb2pJlKAIdmpaz5sfo70/YK2hRiT+cK4PuS1qFhRWLMPOrMSEOblJ08EM73eGJPd1599phj4Nh61MflizO8phNOlT/hw76ddX8rPrVLF2bEWp6XcE1JvsiOtpEtIkKpqwHd9jHjqklKliZGzbJiQ1RLsyBT1DQgyeOI3RdJB/ESuaasahzx0FFJjsn0NsmTwA7zfbZfT4dlrv8iDjwKTkvsy1tCHUFqYM3nDsaSOicDiaA8IyOLsc+1BmiLs8bQEW3dx4wc+IpwoNTmBl9WBKI/pyeen7cahoKqFmcwS+WPPTsTHsnf+o0DOMUiuomFPoHylo6cDpIoEeViJkxw1j6rAQGQBgOY/O7CUXEN1RytP3R64gekh9XvQmh93ErMP/GcwgtPD0nOQOWkOUm638OyeB2kcBeap+jsILs6SsHCxbzCBFP8rAEkiWbLBqhwz9FL1NkNwEnXENcb4SApQOqHuGxx9tO0MfshW1pGiP1SAqz927TPBX/oYKpOU6vSrILeK5Q9U2mJzzYSjsTZ91Xz357xcCkR3H1d+v+uVDf9+RDmuiir0lIkQK+IyDTo8eMjTn5eGM4S3hB7z4JWmKSgPlwlDOsgpPZvwpc+m iiKKywdr f+l+2FNVwKvUsSuXBUymaxQTSc+lWloqE0ujlzBHg08/wmdkJI4lNUARfcvhJEUOAByuDTwd7V6LqIHRNzwnUFcCzRn/lVIoBU/nxePrCv7ItWg0u4N+8JUuUhX8epF5Sdim3VIJu/l28b5iIqy8FwSQAw1Z69EwUIlVgQHVZdEEXXY46eRkP4gipd8sn/Gl2IKjH/Fd6DJcvnIl5Nw3okJHs3swcaJarolqMKeUFQ5IHKl8mzOZNhAfLFdKHa2gUBCXGXPgYT6E6EeIMMOFqHTp1DdI21jBjDZf3+L2yYIr3EPk/1r6CGsaTqfMlpLPVJwlI1FgJQHz94CAUs2m2/larCJLNWocQWdx/R6bxRUFX54c1IH1FmcYzwgCd3OlRTX56 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000073, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 24.07.23 11:46, Ryan Roberts wrote: > On 24/07/2023 10:33, Yin, Fengwei wrote: >> >> >> On 7/24/2023 5:04 PM, Ryan Roberts wrote: >>> On 23/07/2023 13:33, Yin, Fengwei wrote: >>>> >>>> >>>> On 7/20/2023 5:41 PM, Ryan Roberts wrote: >>>>> Hi All, >>>>> >>>>> As discussed at Matthew's call yesterday evening, I've put together a list of >>>>> items that need to be done as prerequisites for merging large anonymous folios >>>>> support. >>>>> >>>>> It would be great to get some review and confirmation as to whether anything is >>>>> missing or incorrect. Most items have an assignee - in that case it would be >>>>> good to check that my understanding that you are working on the item is correct. >>>>> >>>>> I think most things are independent, with the exception of "shared vs exclusive >>>>> mappings", which I think becomes a dependency for a couple of things (marked in >>>>> depender description); again would be good to confirm. >>>>> >>>>> Finally, although I'm concentrating on the prerequisites to clear the path for >>>>> merging an MVP Large Anon Folios implementation, I've included one "enhancement" >>>>> item ("large folios in swap cache"), solely because we explicitly discussed it >>>>> last night. My view is that enhancements can come after the initial large anon >>>>> folios merge. Over time, I plan to add other enhancements (e.g. retain large >>>>> folios over COW, etc). >>>>> >>>>> I'm posting the table as yaml as that seemed easiest for email. You can convert >>>>> to csv with something like this in Python: >>>>> >>>>> import yaml >>>>> import pandas as pd >>>>> pd.DataFrame(yaml.safe_load(open('work-items.yml'))).to_csv('work-items.csv') >>>>> >>>>> Thanks, >>>>> Ryan >>>> Should we add the mremap case to the list? Like how to handle the case that mremap >>>> happens in the middle of large anonymous folio and fails to split it. >>> >>> What's the issue that you see here? My opinion is that if we do nothing special >>> for mremap(), it neither breaks correctness nor performance when we enable large >>> anon folios. So on that basis, its not a prerequisite and I'd rather leave it >>> off the list. We might want to do something later as an enhancement though? >> The issue is related with anonymous folio->index. >> >> If mremap happens in the middle of the large folio, current code doesn't split it. >> So the large folio will be split to two parts: one is in original place and another >> is in the new place. These two parts which are in different VMA have same folio->index. >> Can rmap_walk_anon() work with this situation? vma_address() combined with head page. >> Can it work for the pages not in same vma as head page? >> >> I could miss something here. Will try to build test against it. > > Ahh, I see. So the rmap is broken for large anon folios that have pages mapped > non-contiguously in VA? In that case, I agree that this is a big issue for > correctness and therefore a prerequisite! I think existing rmap code should be able to handled that, otherwise that would be severely broken. A simple partial mremap() on an ordinary PMD-mapped THP would already trigger that. In any case, we have to make PTE-mapped THPs a first-class citizen. -- Cheers, David / dhildenb