From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D768E7717D for ; Wed, 11 Dec 2024 11:55:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2E27C6B0116; Wed, 11 Dec 2024 06:55:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 26B188D0013; Wed, 11 Dec 2024 06:55:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0BF876B011B; Wed, 11 Dec 2024 06:55:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D9D5D6B0116 for ; Wed, 11 Dec 2024 06:55:31 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 54C3F120D6C for ; Wed, 11 Dec 2024 11:55:31 +0000 (UTC) X-FDA: 82882522758.05.BEF3BD1 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf13.hostedemail.com (Postfix) with ESMTP id 1384F2000C for ; Wed, 11 Dec 2024 11:55:05 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=VTzQa+r8; spf=pass (imf13.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733918105; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=xaRJD8pfzmVQ0M0bxncnJSLEaj5sVMwVbxcpQWL3v6s=; b=MiSEef3nmNk25M5JYUhuFLpAcoIA/WFo9kZOURf8UhqrolHq6aul3TdYTlwLcv603ksSIm 9qS7xo3VCprzw1h8UgSgumnGYTkbXP6O7pfYyZL0I0/Zufm3p+BDVtRtgVhBUHoQz4Neof X0q3SdhfvNgmNfNaG00ebT3Cj6PZSFI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733918105; a=rsa-sha256; cv=none; b=4hp+xaxas6ZZNeOR9fSgm7Dn/Elfobz0YD6h6OuDDD2QoPoixgS7U9Fo5rgGtMrxxeTqkQ FOABvYajssCxvxsHdFU6OxdpFmI85OPy5KTrmo2BEOwRTY7X8Rn3rxSZ/OW06UqxwxZHgS bSgqn8fbQsNEadxgYprxLWUka9doZhI= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=VTzQa+r8; spf=pass (imf13.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1733918128; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding:autocrypt:autocrypt; bh=xaRJD8pfzmVQ0M0bxncnJSLEaj5sVMwVbxcpQWL3v6s=; b=VTzQa+r8dKl3fxMyid5lybSKnDCuyeqs02vfUK1tC2bCBZObbrli5n8cNHo02ClZoWLdO2 6IL2QZ3i9x0NsRaJJhVhn493AdefT0a8SEasznU024U4TGFwswV6Y2gnhXgGOd0Xk9+TQs qPgefoUkV9btZMyriSYJNXrdTfPm+LU= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-680-C1WYGpEFOlO2Blv9PYeJJQ-1; Wed, 11 Dec 2024 06:55:26 -0500 X-MC-Unique: C1WYGpEFOlO2Blv9PYeJJQ-1 X-Mimecast-MFC-AGG-ID: C1WYGpEFOlO2Blv9PYeJJQ Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-434e8beeed9so37913095e9.1 for ; Wed, 11 Dec 2024 03:55:26 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733918126; x=1734522926; h=content-transfer-encoding:subject:to:organization:autocrypt:cc:from :content-language:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xaRJD8pfzmVQ0M0bxncnJSLEaj5sVMwVbxcpQWL3v6s=; b=IksZ3y7gqvjATbn68nG+xTEEYdK8AkuhYMqcAszNud1DzcS1+SCoPq2hncoGdaIVx2 M6v6xMlXYRiGEkBD0ThdKSUsnmDZx6r19DXzpt4j3EGVH1fO605IobNfuWQkFKAHhU2R NxsW4d5NNRYuGYf4wPGu4R0XIwGP0mmPX9vYZ+DEUuaRcHM2JI0Vh5KkBj9fkLba6dmL X8aQUSQyWXgGYfYMN/V1i/QwdfwpB35ujvHNB1g+m+g294ZTDkuXVgPwYSwyuRH2HFgN wlDNFvrhvpcU1CGWU1AFs/TmGiUs86lHi8LxKtCYugCXu/hrm6282b2Rpnfn4c0g0Neg vd/w== X-Gm-Message-State: AOJu0Ywt5PVuxfKszLQVn8+4fXC6A58VOH54RIpid4W402GoavM4f9Z2 eOe1L48ljpu8ACXV/XEqUxrrjN+jVNnze9pI1ggKPwWytRueoJ8hFCjyhrznbYPpAQ/fO9fVbsK jYEV36M1f1SAbXWOh9YG0KJdmLydyfXpUkJmA/eoNjqzOkeGAfzl4gBcRrisvRI17BYXXkwnuHS 9YPUzFDf1dxFJ5eKFLfsmqHqlDihW8nYY= X-Gm-Gg: ASbGncuqAf3aI/T9ZTgmNivQtlrFRiaw+ZDT8OzepOqpLHLHMngl5DhxqpwXJ8tcn62 2x3K/EJhB6KhTPQamLzD5GM/dXLoJSVdnVLGzZxc03GyNTAVT+a88x56T9jn96kaW15jtdRV+46 FwISYfklHB8G+Ut8fxLtXlGjvNiGtXMJEHmV1dFbbi1yCgV8D/mf6FQWXpegqKpWfGdRWMm7RTG AwOd1FYCEAZhBwHF09KSnI87HBeder3Pim6QnaNR4Frf1MRJMwwhydbgnNeu/25mpTRcbe8cRW7 X-Received: by 2002:a05:600c:4e05:b0:434:f804:a992 with SMTP id 5b1f17b1804b1-4361c44d5e0mr15459765e9.32.1733918125641; Wed, 11 Dec 2024 03:55:25 -0800 (PST) X-Google-Smtp-Source: AGHT+IF5VoSsD1XrUNe3HWRFL4rp0KKtvMz3hl1RhT+U4ECJKhgCHs2oXD15mEq+KTcF/2PWp38Z/Q== X-Received: by 2002:a05:600c:4e05:b0:434:f804:a992 with SMTP id 5b1f17b1804b1-4361c44d5e0mr15459595e9.32.1733918125125; Wed, 11 Dec 2024 03:55:25 -0800 (PST) Received: from [10.32.64.156] (nat-pool-muc-t.redhat.com. [149.14.88.26]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-434f32b8a65sm125135905e9.6.2024.12.11.03.55.22 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 11 Dec 2024 03:55:23 -0800 (PST) Message-ID: <9c2a17af-4df8-42f0-93c8-83133b6104fd@redhat.com> Date: Wed, 11 Dec 2024 12:55:22 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird From: David Hildenbrand Cc: willy@linux.intel.com, Ryan Roberts Autocrypt: addr=david@redhat.com; keydata= xsFNBFXLn5EBEAC+zYvAFJxCBY9Tr1xZgcESmxVNI/0ffzE/ZQOiHJl6mGkmA1R7/uUpiCjJ dBrn+lhhOYjjNefFQou6478faXE6o2AhmebqT4KiQoUQFV4R7y1KMEKoSyy8hQaK1umALTdL QZLQMzNE74ap+GDK0wnacPQFpcG1AE9RMq3aeErY5tujekBS32jfC/7AnH7I0v1v1TbbK3Gp XNeiN4QroO+5qaSr0ID2sz5jtBLRb15RMre27E1ImpaIv2Jw8NJgW0k/D1RyKCwaTsgRdwuK Kx/Y91XuSBdz0uOyU/S8kM1+ag0wvsGlpBVxRR/xw/E8M7TEwuCZQArqqTCmkG6HGcXFT0V9 PXFNNgV5jXMQRwU0O/ztJIQqsE5LsUomE//bLwzj9IVsaQpKDqW6TAPjcdBDPLHvriq7kGjt WhVhdl0qEYB8lkBEU7V2Yb+SYhmhpDrti9Fq1EsmhiHSkxJcGREoMK/63r9WLZYI3+4W2rAc UucZa4OT27U5ZISjNg3Ev0rxU5UH2/pT4wJCfxwocmqaRr6UYmrtZmND89X0KigoFD/XSeVv jwBRNjPAubK9/k5NoRrYqztM9W6sJqrH8+UWZ1Idd/DdmogJh0gNC0+N42Za9yBRURfIdKSb B3JfpUqcWwE7vUaYrHG1nw54pLUoPG6sAA7Mehl3nd4pZUALHwARAQABzSREYXZpZCBIaWxk ZW5icmFuZCA8ZGF2aWRAcmVkaGF0LmNvbT7CwZgEEwEIAEICGwMGCwkIBwMCBhUIAgkKCwQW AgMBAh4BAheAAhkBFiEEG9nKrXNcTDpGDfzKTd4Q9wD/g1oFAl8Ox4kFCRKpKXgACgkQTd4Q 9wD/g1oHcA//a6Tj7SBNjFNM1iNhWUo1lxAja0lpSodSnB2g4FCZ4R61SBR4l/psBL73xktp rDHrx4aSpwkRP6Epu6mLvhlfjmkRG4OynJ5HG1gfv7RJJfnUdUM1z5kdS8JBrOhMJS2c/gPf wv1TGRq2XdMPnfY2o0CxRqpcLkx4vBODvJGl2mQyJF/gPepdDfcT8/PY9BJ7FL6Hrq1gnAo4 3Iv9qV0JiT2wmZciNyYQhmA1V6dyTRiQ4YAc31zOo2IM+xisPzeSHgw3ONY/XhYvfZ9r7W1l pNQdc2G+o4Di9NPFHQQhDw3YTRR1opJaTlRDzxYxzU6ZnUUBghxt9cwUWTpfCktkMZiPSDGd KgQBjnweV2jw9UOTxjb4LXqDjmSNkjDdQUOU69jGMUXgihvo4zhYcMX8F5gWdRtMR7DzW/YE BgVcyxNkMIXoY1aYj6npHYiNQesQlqjU6azjbH70/SXKM5tNRplgW8TNprMDuntdvV9wNkFs 9TyM02V5aWxFfI42+aivc4KEw69SE9KXwC7FSf5wXzuTot97N9Phj/Z3+jx443jo2NR34XgF 89cct7wJMjOF7bBefo0fPPZQuIma0Zym71cP61OP/i11ahNye6HGKfxGCOcs5wW9kRQEk8P9 M/k2wt3mt/fCQnuP/mWutNPt95w9wSsUyATLmtNrwccz63XOwU0EVcufkQEQAOfX3n0g0fZz Bgm/S2zF/kxQKCEKP8ID+Vz8sy2GpDvveBq4H2Y34XWsT1zLJdvqPI4af4ZSMxuerWjXbVWb T6d4odQIG0fKx4F8NccDqbgHeZRNajXeeJ3R7gAzvWvQNLz4piHrO/B4tf8svmRBL0ZB5P5A 2uhdwLU3NZuK22zpNn4is87BPWF8HhY0L5fafgDMOqnf4guJVJPYNPhUFzXUbPqOKOkL8ojk CXxkOFHAbjstSK5Ca3fKquY3rdX3DNo+EL7FvAiw1mUtS+5GeYE+RMnDCsVFm/C7kY8c2d0G NWkB9pJM5+mnIoFNxy7YBcldYATVeOHoY4LyaUWNnAvFYWp08dHWfZo9WCiJMuTfgtH9tc75 7QanMVdPt6fDK8UUXIBLQ2TWr/sQKE9xtFuEmoQGlE1l6bGaDnnMLcYu+Asp3kDT0w4zYGsx 5r6XQVRH4+5N6eHZiaeYtFOujp5n+pjBaQK7wUUjDilPQ5QMzIuCL4YjVoylWiBNknvQWBXS lQCWmavOT9sttGQXdPCC5ynI+1ymZC1ORZKANLnRAb0NH/UCzcsstw2TAkFnMEbo9Zu9w7Kv AxBQXWeXhJI9XQssfrf4Gusdqx8nPEpfOqCtbbwJMATbHyqLt7/oz/5deGuwxgb65pWIzufa N7eop7uh+6bezi+rugUI+w6DABEBAAHCwXwEGAEIACYCGwwWIQQb2cqtc1xMOkYN/MpN3hD3 AP+DWgUCXw7HsgUJEqkpoQAKCRBN3hD3AP+DWrrpD/4qS3dyVRxDcDHIlmguXjC1Q5tZTwNB boaBTPHSy/Nksu0eY7x6HfQJ3xajVH32Ms6t1trDQmPx2iP5+7iDsb7OKAb5eOS8h+BEBDeq 3ecsQDv0fFJOA9ag5O3LLNk+3x3q7e0uo06XMaY7UHS341ozXUUI7wC7iKfoUTv03iO9El5f XpNMx/YrIMduZ2+nd9Di7o5+KIwlb2mAB9sTNHdMrXesX8eBL6T9b+MZJk+mZuPxKNVfEQMQ a5SxUEADIPQTPNvBewdeI80yeOCrN+Zzwy/Mrx9EPeu59Y5vSJOx/z6OUImD/GhX7Xvkt3kq Er5KTrJz3++B6SH9pum9PuoE/k+nntJkNMmQpR4MCBaV/J9gIOPGodDKnjdng+mXliF3Ptu6 3oxc2RCyGzTlxyMwuc2U5Q7KtUNTdDe8T0uE+9b8BLMVQDDfJjqY0VVqSUwImzTDLX9S4g/8 kC4HRcclk8hpyhY2jKGluZO0awwTIMgVEzmTyBphDg/Gx7dZU1Xf8HFuE+UZ5UDHDTnwgv7E th6RC9+WrhDNspZ9fJjKWRbveQgUFCpe1sa77LAw+XFrKmBHXp9ZVIe90RMe2tRL06BGiRZr jPrnvUsUUsjRoRNJjKKA/REq+sAnhkNPPZ/NNMjaZ5b8Tovi8C0tmxiCHaQYqj7G2rgnT0kt WNyWQQ== Organization: Red Hat To: "linux-mm@kvack.org" Subject: The future of PageAnonExclusive X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: yDBVo40blN5E8EMX5nuqzb7nYImRPFIBjvExs-GglzI_1733918126 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 1384F2000C X-Rspam-User: X-Rspamd-Server: rspam07 X-Stat-Signature: etxxsy3sgu9srp33e5wxskb4y7jbt59d X-HE-Tag: 1733918105-509165 X-HE-Meta: U2FsdGVkX18Z7xUZ1TW/h6gKr4icg61Hq0leQqH9Lp1qMzUYEYnDf5sOaxlRN0h//u3x/Fv4bIpaLWNiXCbko1ZlYa8HUvtqJ55rKs2JON5WEkrRfywzWJL/wp+Fpi4hMV2c52ZZO2FUoifB8WKz2yDerm+qH9uJ0tXp57nl2EsEPNjRxfPu6mDdIFHo4iIFlxD3scQW4VjfNdD1ZLq1XiOLVUdlHbiFiB9iZQLJcJeIvQKB5kj1g680MaX6ZWpe63q4FVeq2xt//ap0/ppRnupFDoq1b2UV/ET3wlz7XmknjudXEYDHT5VkqJaBjz1t6dDDkOXgXlJ8HpzHG9fxehcId1DyPO7CKbDxV1r/MDf8Wj5S95EU/r35wISZzpqLrH9hVjToOJXUo7UWCG9rjOdcHUyXf0gNWpwbsrajh2KSgtcfDz07R4b77Ufo1uZP9LkziR55A8ipAgoosC/zG57436m1uHiQG/pqhSHO/cCYzU05k3GqhGkTeBiNVdGS8nJPhQM16WpH8vKsJ1LKy8njFtt2KSUPCdSUAQj0+16HXi35G/RnVPshepLIK550Yguxyl7lE0KYrq6/Di+vlezvZkE+N5l+3IEZ4r1cIn85kM0qNBKZJPppG3Anw42KwUflq9G2gnA7AbnfGvlMDoX01wDN9n3BCNJgjysIFZ0Q0t0UEf4jKvyRGtD6H7YL23tQbMPfVtvSciRpFnaPmrDyAVfJEexVKBnM8kBhIN8myx1qV1lhd5Y2OhBk6ouEzLUnS8sW2+PBsybBD9Cm6l1nHaedJx1YxY3+rxqPXbHXkH8seyzupNRX7y/A1UhhB7OtXP8l0zfeNhz3081B516JudNgdaVWgOKvVVhkuCbM9C2L5fhaX+55OorgYJUcQNlQoEp1Gc6DPBFkBJGTHRa2yxD4FPsQfEV+u3UNqU0rIhb4yybp/J6vlharvUXy+Wfse0AvgAOcdSndF40 JAeDdzag vYTjoqNAF2yQu5gpiTM6gmBY7RZhSEvCX8nbPhtFMImpGmjKwHEWS3+5vYzYazCPAgRhkJ3y3FMVRo2HKxg6stJXfOK808/xLWjF2Y4xUNMcfKqx+sKfvB6etHlINvO/Ba+2cdZwQelTQxBpmXHB3nn22UngOmzuW1hpBn5L84m64yAi5Ll+iIdMozK5/rn0xHWcBj6nuE+uphVETph3CM0vUgpT/dygsG0jAJ4GSsL0MXn5673X92R+s7h0DtT/eKTnnXXd2GLR0v3WJSHhDhftmCKqQprGzpYB4xbhFKStDE0LqbblPNHgpTOOJKayvnxdxLuL6vce7Y725+zHEUX7iu09q2TF2DEALUO9h4CgiVqNGNgXC6W229RavcZzredQ7aEVcO8Ci9AVp8z2ycewlwTvbrncfAf0Ldc44+gB+bJQ7su4FD1fb/gaW7EY+E4e+iEIMzIzTNUI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi, PageAnonExclusive (PAE) is working very reliable at this point. But especially in the context of THPs (large folios) we'd like to do better: (1) For PTE-mapped THP, we have to maintain it per page. We'd like to avoid per-page flags as good as possible (e.g., waste in "struct page", touching many cachelines). (2) We currently have to use atomics to set/clear the flag, even when working on tail pages. While there would be ways to mitigate this when modifying the flags of multiple tail pages (bitlock protecting all tail page flag updates), I'd much rather avoid messing with page tail flags at all. In general, the PAE bit can be considered an extended PTE bit that we currently store in the "struct page" that is mapped by the PTE. Ideally, we'd just store that information in the PTE, or alongside the PTE: A writable PTE implies PAE. A write-protected PTE needs additional information whether it is PAE (-> whether we can just remap it writable, FOLL_FORCE to it, PIN it ...). We are out of PTE bits, especially when having to implement it across *all* architectures. That's one of the reasons we went with PAE back then. As a nice side-effect it allowed for sanity checks when unpinning folios (-> PAE must still be set, which is impossible when the information stored in the PTE). There are 3 main approaches I've been looking into: (A) Make it a per-folio flag. I've spent endless hours trying to get it conceptually right, but it's just a big pain: as soon as we clear the flag, we have to make sure that all PTEs are write-protected, that the folio is not pinned, and that concurrent GUP cannot work. So far the page table lock protected the PAE bit, but with a per- folio flag that is not guaranteed for THPs. fork() with things like VM_DONTCOPY, VM_DONTFORK, early-abort, page migration/swapout that can happen any time during fork etc. make this really though to get right with THPs. My head hurts any time I think about it. While I think fork() itself can be handled, the concurrent page migration / swapout is where it gets extremely tricky. This can be done somehow I'm sure, but the devil is in the corner cases when having multiple PTEs mapping a large folio. We'd still need atomics to set/clear the single folio flag, because of the nature of concurrent folio flag updates. (B) Allocate additional metadata (PAE bitmap) for page tables, protected by the PTL. That is, when we want to *clear* PAE (fork(), KSM), we'd lazily allocate the bitmap and store it for our page table. On x86: 512 PTEs -> 512bits -> 64byte fork() gets a bit more expensive, because we'd have to allocate this bitmap for the parent and the child page table we are working on, so we can mark the pages as "!PAE" in both page tables. This could work, I have not prototyped it. We'd have to support it on the PTE/PMD/PUD-table level. One tricky thing is having multiple pagetables per page, but I assume it can be handled (we should have a single PT lock for all of them IIRC, and only need to address the bitmap at the right offset). Another challenge is how to link to this metadata from ptdesc on all archs.. So far, __page_mapping is unused, and could maybe be used to link to such metadata -- at least page tables can be identified reliably using the page type. (C) Encode it in the PTE. pte_write() -> PAE !pte_write() && pte_dirty() -> PAE !pte_write && !pte_dirty() -> !PAE That implies, that when wrprotecting a PTE, we'd have to move the dirty bit to the folio. When wr-unprotecting it, we could mark the PTE dirty if the folio is dirty. I suspect that most anon folios are dirty most of the time either way, and the common case of having them just writable in the PTE wouldn't change. The main idea is that nobody (including HW) should ever be marking a readonly PTE dirty (so the theory behind it). We have to take good care whenever we modify/query the dirty bit or modify the writable bit. There is quite some code to audit/sanitize. Further, we'd have to decouple softdirty PTE handling from dirty PTE handling (pte_mkdirty sets the pte softdirty), and adjust arm64 cont-pte and similar PTE batching code to respect the per-PTE dirty bit when the PTE is write-protected. This would be the most elegant solution, but requires a bit of care + sanity checks. Any thoughts or other ideas? -- Cheers, David / dhildenb