From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 440FCD73606 for ; Sat, 30 Nov 2024 16:24:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B22546B0083; Sat, 30 Nov 2024 11:24:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AD30E6B0088; Sat, 30 Nov 2024 11:24:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 99A676B0089; Sat, 30 Nov 2024 11:24:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 790D46B0083 for ; Sat, 30 Nov 2024 11:24:19 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 381171C7D3E for ; Sat, 30 Nov 2024 16:24:19 +0000 (UTC) X-FDA: 82843283628.11.B606EAE Received: from mail-ed1-f46.google.com (mail-ed1-f46.google.com [209.85.208.46]) by imf01.hostedemail.com (Postfix) with ESMTP id 2371040007 for ; Sat, 30 Nov 2024 16:24:09 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=VF+BBoJb; spf=pass (imf01.hostedemail.com: domain of 9erthalion6@gmail.com designates 209.85.208.46 as permitted sender) smtp.mailfrom=9erthalion6@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732983853; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=hU0JAHisz4s9FmhJYG3LQ2lhubsrCB2eXRCFVL/xedA=; b=VwCzBoVVl0z4N/CQDoKro1znpVOuNjZG1KBqtdoOyyoarwL8mNPq+GxQrg4RbEMOyjfCw9 rVyQoJdbE85adh3JykOpahiHJCPJ6uWgz8e6pjFgiXiqAJAy9RqVD5DQvwCE+2xbeij+VP FuKEndayfup6RkrTsBjbmDFqZ5KCLHE= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=VF+BBoJb; spf=pass (imf01.hostedemail.com: domain of 9erthalion6@gmail.com designates 209.85.208.46 as permitted sender) smtp.mailfrom=9erthalion6@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732983853; a=rsa-sha256; cv=none; b=oCRjU/YtOSmteFqiiw9FTIXnUTfcE2e44oep38+CyLCYCzaDJDRQABGqlFKlbLZzdalD3/ 4p3PD33iCGPqe76gEqeL7E3rpnlVMSQqj4gKLrLH/8pQUjq9iaCJsn8w2Or5be2RUWzQ09 kaz3wAg3WRw5QpwQVn6ZnmhMgPookCc= Received: by mail-ed1-f46.google.com with SMTP id 4fb4d7f45d1cf-5ceca0ec4e7so3496482a12.0 for ; Sat, 30 Nov 2024 08:24:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732983856; x=1733588656; darn=kvack.org; h=content-disposition:mime-version:message-id:subject:to:from:date :from:to:cc:subject:date:message-id:reply-to; bh=hU0JAHisz4s9FmhJYG3LQ2lhubsrCB2eXRCFVL/xedA=; b=VF+BBoJb5SpgBjjL+qosNdxyaV9PRcEdO0UKMoe7n20tka6lgA8IxXIXh0I9P5TMuT pf2g7heHk8BC4sFbI8Xmz/K5hblfLKD34GnGLwRXmhSf9Q5wXEmZY9xk3qvHeGD9EJTi TIBR1jx3gkHBQtASuibYX9pHvcONAkvXTfCmKK4Xqfc48b9V7YGgRD/BVpkk/qPL6TVm vdc3AuX80BClStW2CQ+K5XwbiUljnQuqgbzvaNN65zwdMzK7trRRY/Q3twBnEXBHEW36 VNYyQs5ag0oo165k4jsXDx4saTt19isMlq0xtjSdJ0NXEYtNAD5NYIhzsszUcefqZAZk WnCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732983856; x=1733588656; h=content-disposition:mime-version:message-id:subject:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=hU0JAHisz4s9FmhJYG3LQ2lhubsrCB2eXRCFVL/xedA=; b=L5D6+rA6pkcXhbLnaJgPNWMEDKXK/GJyhGHO/KeB1jAOPjQ4UtQBOggVznrMg/9O8v M/WSuvzdQYWIKSOXgbODV97YdMvsR7awj1IBg0Xx+Kd/E0fOSiruz4OitpUR8jbGYvhQ nvT6VXeRq+YHMz9ORrdCV/JHYswrZYM0GDbJRqfa5NxVlffLHg7SyaWDi/9qQKT0p0id NdbARy5QJhcLtq5xCj9l+b1b08QM6TE37g6VsmttSODN4X6MqKUgLE7KNm0cBKghPp4Z 4xELW8xwXVfIUfpJ8/OIujPOz6QT/fQLQ1RyEad2hOC9IMpwMnVxFZ8pJThmMG7ZSEB8 fAgA== X-Gm-Message-State: AOJu0Yw9T1ofnrIKzJhn00RohUEpNRApY3/mmw659S243XQz1Ffr9TZR TNiRheM+H4cj46vqkMvA29lQqQXnFrsolzNF0j6Y/GDFuD6V/IuH+gqqGw== X-Gm-Gg: ASbGnct3rB84qGguhhYqbsEE7cxLClBCv0+E6ncW2eDWWX7H7ir9789VTh2XZc8rSrQ SZlJMwgEbFa2YYpxVLl1/FC3JKxRh2Ngjejobw5VTz6SJadFmMcO3Ki9IDeKk7bjZKLkQj0Evco 80ENgQpRcl5Lh2sDybA9ZbmS2En7vw95JiX9hbsJbD0EYYgMkn0UHsewgNjVfdqpoFC7aEw1H7m xoKxHB/pS/SWCmGHqFC6qELkou0H5B9zWosdHln2uKeL3n+iQTU7WsMJMng00G8SeNUGsR+S72R YdwnLOlPGFyt0E4y2sD/BCF6w9mlXjHED/sJc74RhHWxuzyH6QIub0c7WpnQJ5c= X-Google-Smtp-Source: AGHT+IGse00WjvxkOoZrvZ/8WprmqD9vudquwvQqMPCpsr2BcDQwiS+0UncJsCmIzuELk76ajLevhw== X-Received: by 2002:a05:6402:13d1:b0:5ce:d43e:fece with SMTP id 4fb4d7f45d1cf-5d080b97dcamr17726486a12.10.1732983855794; Sat, 30 Nov 2024 08:24:15 -0800 (PST) Received: from ddolgov-thinkpadt14sgen1.rmtde.csb (dslb-178-005-232-220.178.005.pools.vodafone-ip.de. [178.5.232.220]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5d0d1de7429sm403469a12.74.2024.11.30.08.24.15 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 30 Nov 2024 08:24:15 -0800 (PST) Date: Sat, 30 Nov 2024 17:24:13 +0100 From: Dmitry Dolgov <9erthalion6@gmail.com> To: linux-mm@kvack.org Subject: [QUESTION] Resizing shared mapping without clashing with others Message-ID: <3kpxpd3dbjgg6epasi2554c4qyils4t3cm2pjnyzer7gkyoaxl@khhdxjiggyhp> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Rspamd-Queue-Id: 2371040007 X-Stat-Signature: octqsmj1m4ycihedbcynqe77jokd661w X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1732983849-890963 X-HE-Meta: U2FsdGVkX1+NohVIgjpylUtTbdAgUvibL0/ZuMjicbwSwYI5lpNfLR8D2BsuhUz0K5+ocYk5/2pcGadej5ZR0bB46NmT+fd16/NzwBck0xKgCN8EmG2xITtpJ2porZFVnJbUcoe0l3/3BDnYWzH9s6C2FJH8hAq5gsNCq2VkfssCaPPXmci4u1PY6grnSy58NV5/CW32hQQ80NjBTNRN5x+gW8PezHo+aAGfD/Mqpi6QLK+KyCp7fUE1KzZ/De3+zL/TnufjlaQT20OZKq/COxm1g/DB/mPsUEPBRewXkhRYmHDw5ZSG8HXv2rxSIJva+e3ufipda8h3nJ8qLKDw4DXn87ImGk8R2BwCgGsENbMjcr9Ub8kLqk4Z2iF7nalf0HV4kXhWZFKxWfqyDpiVSrKMVxqMMvZn3k1eZMPzaM1BP0+4PQZ+sCy54ogNkKe8eASSJhIrpXFZ9GXmnqUfgb1nS5ehHpDwSGg8LqHr47Aq0wFXRthHCPCX44xyghKoTDGJqogiID/5foVe+5dJRpm1EwkG2zufBxks5DIvuEGpHCHKm9wC0bHh8E/CGvX+nLdUZEclAJDdF1qxBUSe45gBruPEw3kIXeO25V8W75QhPngGOmyFmAG0rcORacglTSjBwKv9FypbED9oRGvkwSFvZ+C6XPna2f5k0QDxgf66DgF12hVrPhs0ioW5Dk07riFwAt8xz2zHK9KHvNYTB+apBEcGChcK7R9o+Lf7Y2KcwW5API94k3s90arz8m8Z/safmyR5tvqTSHJcf804mGUdH4Pb+T5/MePeB9t4Q2/XgDMZGXI4s4q9fKc+UTRk/NzvmrisNPL6e+SjE6qWY7k1xZbInjVTq1y2rpRvKiBb37g7DV9JZxQuWpLdWVKpwzHmvDn9tVcaYgmi1qy/hrPtTofqZSuT03GgcnbSiAxM9HNymVKR9yvmKAxh7nfetsuVyuW4jShzsNcmGon QQMGZTMf z6DLmz+wNhB/YIGAIEFxNOMN+yOUksnfuMOCWPA0Cod4VkXpPjVO2Nm9LiBPCPuAw0oIITRW7kZnIQ2s6lUQ+pSXvlPBzuio19lFgbR3L6yURt5HQ3zPWMI/hQpQgxyFetJ1nxC0sR0TeKH7ZRuCXh+c00nr/El+ZoKkxTicXOWoM6DKcfI9Yrau20bftYGRWG9TQQkeKIJ6bu2kr4mCHi9ze6EtB2SiWfAm+DttjeWSqq8UoKBSTcQudIjTPST22fMlcdMXU4ZEZZD/E3KdX9t9WQj6j+kuTePK4LU4blZo2j/NiSoOU0lVRN59JaCUfwI1d/y67P0RkN6G/IGG8fCTiF6TS6AZdY79HLl4TvabAfKyfhty+Zmd7JmubU/9srVPMJdqHrgTTAJ+wuIUxRVHCLDzsvSygKbp1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000042, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi, While working on PostgreSQL [1] we've stumbled upon a question regarding resizing of shared mappings without conflicting with any other possible mappings. Before making any wrong conclusions, I would love to get some consultation from kernel folks on that topic. To put it into a context, PostgreSQL uses anonymous shared memory mapping as a buffer cache for data. The mapping size is configured at the start, and could not be changed without a restart. Now, we would like to make it more flexible and allow to change it at runtime, ideally without changing already used addresses and copying stuff back and forth. The idea is to place the shared mapping at a specified address (with MAP_FIXED if needed) with a gap, then use mremap to resize it into the gap. This approach has an open question -- how to make sure there will be no other mapping created withing the same address space, where we want to expand the shared mapping? E.g. the shared mapping was created, then large memory allocation caused another mapping to be created close to it, so that expanding is not possible. One way to solve it would be to rely on the fact that the kernel, if addr is NULL, always picks up a lowest address that allows for enough space (AFAICT in vma_iter_area_lowest). This makes it possible to leave the gap to lowest address large enough to accumulate any possible allocations, as well as leaving space for resizing, something like: 01339000-0139e000 [heap] 0139e000-014aa000 [heap] 7f2dd72f6000-7f2dfbc9c000 /memfd:strategy (deleted) 7f2e0209c000-7f2e269b0000 /memfd:checkpoint (deleted) 7f2e2cdb0000-7f2e516b4000 /memfd:iocv (deleted) 7f2e57ab4000-7f2e7c478000 /memfd:descriptors (deleted) 7f2ebc478000-7f2ee8d3c000 /memfd:buffers (deleted) ^ note the distance between two mappings, which is intended for resize 7f3168d3c000-7f318d600000 /memfd:main (deleted) ^ here is where the gap starts 7f4194c00000-7f4194e7d000 ^ this one is an anonymous maping created due to large memory allocation after shared mappings were created 7f4195000000-7f419527d000 7f41952dc000-7f4195416000 7f4195416000-7f4195600000 /dev/shm/PostgreSQL.2529797530 7f4195600000-7f41a311d000 /usr/lib/locale/locale-archive 7f41a317f000-7f41a3200000 7f41a3200000-7f41a3201000 /usr/lib64/libicudata.so.74.2 [...] What I'm not sure about is how safe it is to rely on such behavior in long term? It seems like the way how mapping address will be chosen if addr is NULL is not standartized, and considered to be implementation specific. What are the chances this behavior becomes different over time? Another question is whether there are better ways or mechanisms to achieve resizable shared mapping? The topic sounds natural enough, it feels like there should be other use cases that require this as well. [1]: https://www.postgresql.org/message-id/scor5gscd42d4nwszuwvtwss6e22fg3dnvxmqwrcsdkpyyigny%40efjlkj6ccv7u