From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4ABAEC3ABDD for ; Mon, 19 May 2025 18:00:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B69746B00CF; Mon, 19 May 2025 14:00:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B14156B00D0; Mon, 19 May 2025 14:00:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9411B6B00D1; Mon, 19 May 2025 14:00:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 71E5E6B00CF for ; Mon, 19 May 2025 14:00:36 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id E1886160C76 for ; Mon, 19 May 2025 18:00:39 +0000 (UTC) X-FDA: 83460422598.18.BACF3DF Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf24.hostedemail.com (Postfix) with ESMTP id 64E50180002 for ; Mon, 19 May 2025 18:00:37 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hKRwLE2N; spf=pass (imf24.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=quarantine) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1747677637; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UXbC4Dnz7rnSGIvOTw3tnEOtF89rRJMSvQjwW/9D6t8=; b=3Aykcs5ivKYtEvAaDmw5osadA1BM9NMN2rswlEAHVc0t2LfxKBUxaqvYpicLNUcjwRNbsf MDktl21AO59dk8P6obG3/2fWo3S9jqiZDCxM92cuG7sUjejqlmBocyWUa+4BLD7xi/VCXm QJhXT/VIStmn+fUNajeEb9tKiW5J7h0= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hKRwLE2N; spf=pass (imf24.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=quarantine) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1747677637; a=rsa-sha256; cv=none; b=cadJ53c/5LbCeiLy9k5F5R7ba+AyWytItjmg2pTRlJDlyscHt6lRY6LOgNsjK9nMhEm23H N7KgQM09zvvMmqQQR9UPvnR4an3kgAD1zbVzrGE/8nac9rv5+9xWGfOvqvSARSqaHWXp4+ M/CaUcCNumiaaV7nKSRip6jwlgvMFNM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1747677636; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=UXbC4Dnz7rnSGIvOTw3tnEOtF89rRJMSvQjwW/9D6t8=; b=hKRwLE2Nj5r83/UyaxAI032WrAOn8osr4TTskuvT0ETCmBhBXDHF33uM4g8Gs6CeQhvtQ6 JdWjAiRrYXaH3eN+azyW0S79NGjqrY0e5MfV5aDynMNQ07Mci11aQJFh7DFmeeMKbFj7Uz MFrYddbC1lrxMy4zyeY7KEua/vlKEVU= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-346-IrngvoMQPEG6QHgqlai0UQ-1; Mon, 19 May 2025 14:00:33 -0400 X-MC-Unique: IrngvoMQPEG6QHgqlai0UQ-1 X-Mimecast-MFC-AGG-ID: IrngvoMQPEG6QHgqlai0UQ_1747677632 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-3a3703c1fe7so598519f8f.1 for ; Mon, 19 May 2025 11:00:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747677632; x=1748282432; h=content-transfer-encoding:in-reply-to:organization:autocrypt :content-language:from:references:cc:to:subject:user-agent :mime-version:date:message-id:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=UXbC4Dnz7rnSGIvOTw3tnEOtF89rRJMSvQjwW/9D6t8=; b=mUOt8eo1M20dAKye5u86terUh5Vncu6GFKQQVkczB1M1I9GgLQG7kx/suH2FHmNH/B BIhIEHX0fsDbtFR0WhHNbB8gb1y53tGxL15mjsiCRz9mIat0yKCGEsFYqMOSOt1yNvQe R4cR3LGADHdHvSjVChY1YjcC290sk2h42MuFB/WBwGOqYUuNQkADhHqvtDVlZwIbRwj1 N3947UcSpkLqxN4641sG8QxMpYzpQOjFkEJGwgmlYi437ehHSojxiMsc72sSKxRSXlBa ZU0EfX5ptCmn0536sIRV4zqf+rfvImHf4FYsekhLYYpnM+NdibPWbmNw+yRvzK46/BHx 5WoQ== X-Forwarded-Encrypted: i=1; AJvYcCUFKQ5sZrm4W6M1ptRr5zw2G1i8uny45UnVivpRjbKLqyFm/CKAgV1yUNAIWwmlk7EKZtPRgDyImw==@kvack.org X-Gm-Message-State: AOJu0YxYBuHjJGMSzfEILLX5KcXLx81xLuAErvMuXnewn/gYyyoKk38H lVaxcTdeCynEkm9Y1WP7QbnZM+CS7u4KhXJk/e9E9jOBv9mL04oNy44Ho9mE7ayq/u8bfPU7UNo Jh8mgEsB7zk45nX14uGyDJG1or1bWxmmzRQBgbzz1ZooACOYwj8hQ X-Gm-Gg: ASbGnct4JUtiI6iiEGrb5X/5Vh+qC02qKd9vmKkSDuY/JOHEPPofM+0MriJUjVPcA/4 BglsKNMAXtnh1PS5BUcHTvXDUdT5Yb4t1r2bpQe7XX3XzzB5sSDh71X7B8X3rjoT34wLkzAQQwL rU63nWc+svcxVclDMgDoL8HhP586nOGYHWfxE8ZLt3s4jhKEbG7bez+/ihHMQmAj/OzzLctmQtw EUwdADsynogBCYgnE59TGPBmq7PdrCUld+FeRPbD0LRiCezAAvBdtPvCMS9r1L3pk/TqRGFMMHu z1J7olWX2vInuEbcsPXwd9jBIcyhhbLaJvOtitk3xC/OG6IR+waQ/g9393ClcNYaYKbN1Tr8LNh mnS6g3TZOrME1KwqwgJHnMaaXzAe5ZjznSsuM18s= X-Received: by 2002:a05:6000:1881:b0:3a1:f635:1136 with SMTP id ffacd0b85a97d-3a35fead113mr11130049f8f.28.1747677631925; Mon, 19 May 2025 11:00:31 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFNOlabkxqP3Ndi5EnXXXETD47WSduA7TE65M2Mu+MvxkHNO1hvWofRxxLgEY+hHNW2x7LZrA== X-Received: by 2002:a05:6000:1881:b0:3a1:f635:1136 with SMTP id ffacd0b85a97d-3a35fead113mr11130023f8f.28.1747677631473; Mon, 19 May 2025 11:00:31 -0700 (PDT) Received: from ?IPV6:2003:d8:2f3c:3a00:5662:26b3:3e5d:438e? (p200300d82f3c3a00566226b33e5d438e.dip0.t-ipconnect.de. [2003:d8:2f3c:3a00:5662:26b3:3e5d:438e]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3a35ca8874bsm13775350f8f.67.2025.05.19.11.00.30 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 19 May 2025 11:00:31 -0700 (PDT) Message-ID: Date: Mon, 19 May 2025 20:00:29 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 3/4] mm: prevent KSM from completely breaking VMA merging To: Lorenzo Stoakes , Andrew Morton Cc: Alexander Viro , Christian Brauner , Jan Kara , "Liam R . Howlett" , Vlastimil Babka , Jann Horn , Pedro Falcato , Xu Xin , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org References: <418d3edbec3a718a7023f1beed5478f5952fc3df.1747431920.git.lorenzo.stoakes@oracle.com> From: David Hildenbrand Autocrypt: addr=david@redhat.com; keydata= xsFNBFXLn5EBEAC+zYvAFJxCBY9Tr1xZgcESmxVNI/0ffzE/ZQOiHJl6mGkmA1R7/uUpiCjJ dBrn+lhhOYjjNefFQou6478faXE6o2AhmebqT4KiQoUQFV4R7y1KMEKoSyy8hQaK1umALTdL QZLQMzNE74ap+GDK0wnacPQFpcG1AE9RMq3aeErY5tujekBS32jfC/7AnH7I0v1v1TbbK3Gp XNeiN4QroO+5qaSr0ID2sz5jtBLRb15RMre27E1ImpaIv2Jw8NJgW0k/D1RyKCwaTsgRdwuK Kx/Y91XuSBdz0uOyU/S8kM1+ag0wvsGlpBVxRR/xw/E8M7TEwuCZQArqqTCmkG6HGcXFT0V9 PXFNNgV5jXMQRwU0O/ztJIQqsE5LsUomE//bLwzj9IVsaQpKDqW6TAPjcdBDPLHvriq7kGjt WhVhdl0qEYB8lkBEU7V2Yb+SYhmhpDrti9Fq1EsmhiHSkxJcGREoMK/63r9WLZYI3+4W2rAc UucZa4OT27U5ZISjNg3Ev0rxU5UH2/pT4wJCfxwocmqaRr6UYmrtZmND89X0KigoFD/XSeVv jwBRNjPAubK9/k5NoRrYqztM9W6sJqrH8+UWZ1Idd/DdmogJh0gNC0+N42Za9yBRURfIdKSb B3JfpUqcWwE7vUaYrHG1nw54pLUoPG6sAA7Mehl3nd4pZUALHwARAQABzSREYXZpZCBIaWxk ZW5icmFuZCA8ZGF2aWRAcmVkaGF0LmNvbT7CwZgEEwEIAEICGwMGCwkIBwMCBhUIAgkKCwQW AgMBAh4BAheAAhkBFiEEG9nKrXNcTDpGDfzKTd4Q9wD/g1oFAl8Ox4kFCRKpKXgACgkQTd4Q 9wD/g1oHcA//a6Tj7SBNjFNM1iNhWUo1lxAja0lpSodSnB2g4FCZ4R61SBR4l/psBL73xktp rDHrx4aSpwkRP6Epu6mLvhlfjmkRG4OynJ5HG1gfv7RJJfnUdUM1z5kdS8JBrOhMJS2c/gPf wv1TGRq2XdMPnfY2o0CxRqpcLkx4vBODvJGl2mQyJF/gPepdDfcT8/PY9BJ7FL6Hrq1gnAo4 3Iv9qV0JiT2wmZciNyYQhmA1V6dyTRiQ4YAc31zOo2IM+xisPzeSHgw3ONY/XhYvfZ9r7W1l pNQdc2G+o4Di9NPFHQQhDw3YTRR1opJaTlRDzxYxzU6ZnUUBghxt9cwUWTpfCktkMZiPSDGd KgQBjnweV2jw9UOTxjb4LXqDjmSNkjDdQUOU69jGMUXgihvo4zhYcMX8F5gWdRtMR7DzW/YE BgVcyxNkMIXoY1aYj6npHYiNQesQlqjU6azjbH70/SXKM5tNRplgW8TNprMDuntdvV9wNkFs 9TyM02V5aWxFfI42+aivc4KEw69SE9KXwC7FSf5wXzuTot97N9Phj/Z3+jx443jo2NR34XgF 89cct7wJMjOF7bBefo0fPPZQuIma0Zym71cP61OP/i11ahNye6HGKfxGCOcs5wW9kRQEk8P9 M/k2wt3mt/fCQnuP/mWutNPt95w9wSsUyATLmtNrwccz63XOwU0EVcufkQEQAOfX3n0g0fZz Bgm/S2zF/kxQKCEKP8ID+Vz8sy2GpDvveBq4H2Y34XWsT1zLJdvqPI4af4ZSMxuerWjXbVWb T6d4odQIG0fKx4F8NccDqbgHeZRNajXeeJ3R7gAzvWvQNLz4piHrO/B4tf8svmRBL0ZB5P5A 2uhdwLU3NZuK22zpNn4is87BPWF8HhY0L5fafgDMOqnf4guJVJPYNPhUFzXUbPqOKOkL8ojk CXxkOFHAbjstSK5Ca3fKquY3rdX3DNo+EL7FvAiw1mUtS+5GeYE+RMnDCsVFm/C7kY8c2d0G NWkB9pJM5+mnIoFNxy7YBcldYATVeOHoY4LyaUWNnAvFYWp08dHWfZo9WCiJMuTfgtH9tc75 7QanMVdPt6fDK8UUXIBLQ2TWr/sQKE9xtFuEmoQGlE1l6bGaDnnMLcYu+Asp3kDT0w4zYGsx 5r6XQVRH4+5N6eHZiaeYtFOujp5n+pjBaQK7wUUjDilPQ5QMzIuCL4YjVoylWiBNknvQWBXS lQCWmavOT9sttGQXdPCC5ynI+1ymZC1ORZKANLnRAb0NH/UCzcsstw2TAkFnMEbo9Zu9w7Kv AxBQXWeXhJI9XQssfrf4Gusdqx8nPEpfOqCtbbwJMATbHyqLt7/oz/5deGuwxgb65pWIzufa N7eop7uh+6bezi+rugUI+w6DABEBAAHCwXwEGAEIACYCGwwWIQQb2cqtc1xMOkYN/MpN3hD3 AP+DWgUCXw7HsgUJEqkpoQAKCRBN3hD3AP+DWrrpD/4qS3dyVRxDcDHIlmguXjC1Q5tZTwNB boaBTPHSy/Nksu0eY7x6HfQJ3xajVH32Ms6t1trDQmPx2iP5+7iDsb7OKAb5eOS8h+BEBDeq 3ecsQDv0fFJOA9ag5O3LLNk+3x3q7e0uo06XMaY7UHS341ozXUUI7wC7iKfoUTv03iO9El5f XpNMx/YrIMduZ2+nd9Di7o5+KIwlb2mAB9sTNHdMrXesX8eBL6T9b+MZJk+mZuPxKNVfEQMQ a5SxUEADIPQTPNvBewdeI80yeOCrN+Zzwy/Mrx9EPeu59Y5vSJOx/z6OUImD/GhX7Xvkt3kq Er5KTrJz3++B6SH9pum9PuoE/k+nntJkNMmQpR4MCBaV/J9gIOPGodDKnjdng+mXliF3Ptu6 3oxc2RCyGzTlxyMwuc2U5Q7KtUNTdDe8T0uE+9b8BLMVQDDfJjqY0VVqSUwImzTDLX9S4g/8 kC4HRcclk8hpyhY2jKGluZO0awwTIMgVEzmTyBphDg/Gx7dZU1Xf8HFuE+UZ5UDHDTnwgv7E th6RC9+WrhDNspZ9fJjKWRbveQgUFCpe1sa77LAw+XFrKmBHXp9ZVIe90RMe2tRL06BGiRZr jPrnvUsUUsjRoRNJjKKA/REq+sAnhkNPPZ/NNMjaZ5b8Tovi8C0tmxiCHaQYqj7G2rgnT0kt WNyWQQ== Organization: Red Hat In-Reply-To: <418d3edbec3a718a7023f1beed5478f5952fc3df.1747431920.git.lorenzo.stoakes@oracle.com> X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: WZiYKkfeAcruwxyvnmMowmdEs1FHmaz6DYCdTF3GGDo_1747677632 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 64E50180002 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: kk3g74kooeo17k8msaf6hsrh83g87hpo X-HE-Tag: 1747677637-856808 X-HE-Meta: U2FsdGVkX1/7gfjI/vhUrk3Sbr6jciQcdBFgYFJb9ZTOF/SHfWi5AJlmuvNaH/ggi4hZaPYuxRIl6NII96jx5pYS95Jjs3hRgQ89FvVMBKz4Y87jVjzYqHW+IyHENbLKAokA3mWgUGav+z1d4NQo+pBZ8Qcl9dQIBQ80jNHdgucBXakxeukj9gxWE0fw9sTw+cU9QNQAmSJ6stovKEwsZX3uwGYXxEh85icJNh14PsJeD9l/+b/y9eGedGPcwD7HRHu3J7MIvhlOTHtQn/mwavrhJoxfIa2BGne1Yxf0f0JOZgfEVE5fzdVaHTJCekQbJNSI3didxt+whfTGV+9+iHw9hF5V6TppH3kC75tXznLFPqHk1ZiyC0q4ffFHDV9rzcitcgKtLHvyHUqblheLZWCPxVLEJ3fPIHGE3ZnnSFRkb9MGQTT3ao877waLKCyzTPkfeJ9pcSGadpCJuWEN7+Q1628MtoqDDssaCDF1bD9T0mRzDCIfQn4iKNlT+Ox/pTqLpGmqyAtZ5HbkYh6EBu6i6M/AhltmWb577c6KBvhQiWtWYKEIE4cYQU9yEM2j3a64vPsxtp0Qzn+7QLR7B1biwXEizkLrf/21n3Px/35hAPZy17e7H9hwznGaHq/julgWvTPQH7JdXJCabFdGyyfFzFqGZQG4bLbwPDxcdNpXYhDeeBDLtai6VOvee+yaqScyDrtTBDbUumoMTVyNccb6SoEBiPOn4lwWRlcxIV8kdBLsQgCdJzIF/YWn7KqkgYbXdelU/vhNiiqbuIlqMITMu/26EdyW5bIVLHWBQWcJ1Vxc43ZHTF9opEjQTk7bQOh86fkXMcOPs7sPo2sO0XbeTnh6rXPaPj8ruwcTVpBM4kz+cuG4Ws2Ghtpayb7YD27WsoSSdzwx1q8Jqlsd7GcY3ydsRVCDpXACZJu8sMniiepa/L3LvR+/VZ4baSKeWftv68xeVMu19eMmfid gdZ4FrMJ bG0oVWmhOwhRLasJsqXaqH86M/LamF0ooVskAyXQQ5ETSvngOwuBHsJyykKslDTjnhbVKdHZbN+fSDTV7LRNP4Nl1xTeR51pC7nYU1G34gOESUZQF2SFuAxytB18mAh7cdILOaq+VS6Nax0qFj2nF5emBype3R5K9uppMWhGkgYOs6tTYUE98YoTuVtSjBoxW9i79zgVu/i3OjI44q+admN3u/5jGt9DwLnX2yGmzZ5CJua4tbBfHZwLV5QSWqUARlvj0v3Z/84LB4bUQufPxRZSe9iuiV9AMgH/T+TtxkIi+T/aQtWFOqAJLaZ5NpDxDEMbtSj/hvK/iJdPmKbf41w6ndupzF/+AN/HwxXo+ggCuZcU+yneNLYd3zNN9f8jZzd1WmCrAdJc5OtScN6P27TO0NA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 19.05.25 10:51, Lorenzo Stoakes wrote: > If a user wishes to enable KSM mergeability for an entire process and all > fork/exec'd processes that come after it, they use the prctl() > PR_SET_MEMORY_MERGE operation. > > This defaults all newly mapped VMAs to have the VM_MERGEABLE VMA flag set > (in order to indicate they are KSM mergeable), as well as setting this flag > for all existing VMAs. > > However it also entirely and completely breaks VMA merging for the process > and all forked (and fork/exec'd) processes. > > This is because when a new mapping is proposed, the flags specified will > never have VM_MERGEABLE set. However all adjacent VMAs will already have > VM_MERGEABLE set, rendering VMAs unmergeable by default. > > To work around this, we try to set the VM_MERGEABLE flag prior to > attempting a merge. In the case of brk() this can always be done. > > However on mmap() things are more complicated - while KSM is not supported > for file-backed mappings, it is supported for MAP_PRIVATE file-backed > mappings. > > And these mappings may have deprecated .mmap() callbacks specified which > could, in theory, adjust flags and thus KSM merge eligiblity. > > So we check to determine whether this at all possible. If not, we set > VM_MERGEABLE prior to the merge attempt on mmap(), otherwise we retain the > previous behaviour. > > When .mmap_prepare() is more widely used, we can remove this precaution. > > While this doesn't quite cover all cases, it covers a great many (all > anonymous memory, for instance), meaning we should already see a > significant improvement in VMA mergeability. We should add a Fixes: tag. CCing stable is likely not a good idea at this point (and might be rather hairy). [...] > /** > - * ksm_add_vma - Mark vma as mergeable if compatible > + * ksm_vma_flags - Update VMA flags to mark as mergeable if compatible > * > - * @vma: Pointer to vma > + * @mm: Proposed VMA's mm_struct > + * @file: Proposed VMA's file-backed mapping, if any. > + * @vm_flags: Proposed VMA"s flags. > + * > + * Returns: @vm_flags possibly updated to mark mergeable. > */ > -void ksm_add_vma(struct vm_area_struct *vma) > +vm_flags_t ksm_vma_flags(const struct mm_struct *mm, const struct file *file, > + vm_flags_t vm_flags) > { > - struct mm_struct *mm = vma->vm_mm; > + vm_flags_t ret = vm_flags; > > - if (test_bit(MMF_VM_MERGE_ANY, &mm->flags)) > - __ksm_add_vma(vma); > + if (test_bit(MMF_VM_MERGE_ANY, &mm->flags) && > + __ksm_should_add_vma(file, vm_flags)) > + ret |= VM_MERGEABLE; > + > + return ret; > } No need for ret without harming readability. if () vm_flags |= VM_MERGEABLE return vm_flags; [...] > +/* > + * Are we guaranteed no driver can change state such as to preclude KSM merging? > + * If so, let's set the KSM mergeable flag early so we don't break VMA merging. > + * > + * This is applicable when PR_SET_MEMORY_MERGE has been set on the mm_struct via > + * prctl() causing newly mapped VMAs to have the KSM mergeable VMA flag set. > + * > + * If this is not the case, then we set the flag after considering mergeability, > + * which will prevent mergeability as, when PR_SET_MEMORY_MERGE is set, a new > + * VMA will not have the KSM mergeability VMA flag set, but all other VMAs will, > + * preventing any merge. Hmmm, so an ordinary MAP_PRIVATE of any file (executable etc.) will get VM_MERGEABLE set but not be able to merge? Probably these are not often expected to be merged ... Preventing merging should really only happen because of VMA flags that are getting set: VM_PFNMAP, VM_MIXEDMAP, VM_DONTEXPAND, VM_IO. I am not 100% sure why we bail out on special mappings: all we have to do is reliably identify anon pages, and we should be able to do that. GUP does currently refuses any VM_PFNMAP | VM_IO, and KSM uses GUP, which might need a tweak then (maybe the solution could be to ... not use GUP but a folio_walk). So, assuming we could remove the VM_PFNMAP | VM_IO | VM_DONTEXPAND | VM_MIXEDMAP constraint from vma_ksm_compatible(), could we simplify? That is: the other ones must not really be updated during mmap(), right? (in particular: VM_SHARED | VM_MAYSHARE | VM_HUGETLB | VM_DROPPABLE) Have to double-check VM_SAO and VM_SPARC_ADI. > + */ > +static bool can_set_ksm_flags_early(struct mmap_state *map) > +{ > + struct file *file = map->file; > + > + /* Anonymous mappings have no driver which can change them. */ > + if (!file) > + return true; > + > + /* shmem is safe. */ > + if (shmem_file(file)) > + return true; > + > + /* > + * If .mmap_prepare() is specified, then the driver will have already > + * manipulated state prior to updating KSM flags. > + */ > + if (file->f_op->mmap_prepare) > + return true; > + > + return false; > +} So, long-term (mmap_prepare) this function will essentially go away? Nothing jumped at me, this definitely improves the situation. -- Cheers, David / dhildenb