From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B006D49767 for ; Sun, 1 Dec 2024 11:55:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F301F6B0082; Sun, 1 Dec 2024 06:55:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EDFCB6B0083; Sun, 1 Dec 2024 06:55:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DCF7F6B0085; Sun, 1 Dec 2024 06:55:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C090C6B0082 for ; Sun, 1 Dec 2024 06:55:41 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 6B5EFC081D for ; Sun, 1 Dec 2024 11:55:41 +0000 (UTC) X-FDA: 82846235346.10.22FCAD8 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf21.hostedemail.com (Postfix) with ESMTP id 37A9D1C03E8 for ; Sun, 1 Dec 2024 11:55:18 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="aUnRQ3/M"; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733054127; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZTW6EscwvlanQu0UbTGw3WkVLDOUXD8LWWTUiSfo1FQ=; b=J79I+sJ1IJ/6Kd/2Y7eSd5tVZFzQM2Dr9ppgvNeIh73/k7GUNVwMiUskf+PjgyGKobyHEy Zd5Ksiq8kNBf/chD06byYnCEiqE01t9A4VCRfi6nrXkLPDepNLx7vM1jHjlPwpQ0l0QvXm IypBf7mvzLAI8UJIDAHf9QL0xt5snn4= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="aUnRQ3/M"; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733054127; a=rsa-sha256; cv=none; b=tuM0wnlUPCRPx5D6q3Os6m5texju/5clfu1onPPPIwTXK8K0oB9jSn1fIxw/8Zixs2e6I6 Q2NyCWcvL1cdIomCjF48xmQkO6M4Zy8n7rPGqi6fGMTUMTdHjS9BcASmgOVyGT1dy4tAGn wsxoESemPkOo6P1t9Q215+fkm7My8Mk= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=ZTW6EscwvlanQu0UbTGw3WkVLDOUXD8LWWTUiSfo1FQ=; b=aUnRQ3/Ma1+wowjUMo8DY8ZklL MQEjY4X8JbYqg7+UZk+EdtA8Scsy4viEU17I9WPlMMF6C1gkKnhnjGM2OzfO1xa6a5cla6wy6lXSo l6UIIoBPpibrwqTEqkvCLDQKk1P0TLynjRJGYfgEp0xMp+yyLooBBcXqC4rQPJsjGBPGBvsywsIdb LfOY8yC+M8wOZtHTyZMA47YekkNCj6qTGK+hTvXFmj8rFZakQzWtoCNm56XSAqyqdTn8fGo08gzwJ /cdn4yyp5CSrz4EF5E5TbhVrQ9kxAAdcThvMxBEYU/qg/1LPlURhpuJxiRhpJViUycoeVGfKZLQkK DvsGrIIw==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tHiY5-00000006gPc-31a7; Sun, 01 Dec 2024 11:55:37 +0000 Date: Sun, 1 Dec 2024 11:55:37 +0000 From: Matthew Wilcox To: Dmitry Dolgov <9erthalion6@gmail.com> Cc: linux-mm@kvack.org Subject: Re: [QUESTION] Resizing shared mapping without clashing with others Message-ID: References: <3kpxpd3dbjgg6epasi2554c4qyils4t3cm2pjnyzer7gkyoaxl@khhdxjiggyhp> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3kpxpd3dbjgg6epasi2554c4qyils4t3cm2pjnyzer7gkyoaxl@khhdxjiggyhp> X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 37A9D1C03E8 X-Rspam-User: X-Stat-Signature: dafebqacbwouxbksnt64zojsnrh15cfr X-HE-Tag: 1733054118-31546 X-HE-Meta: U2FsdGVkX1+QEinYAUISyvMqCpYFgiQ/RXE1mIsSekGHUFYnSfQv6fz1o2EaYtPlyQf0WDD2GneaBneNile2mIwrUCvd2U9xJKbi4BTsfwSUN7X4R86CxwimbkbwDeL0UaHlSP4T9ZiPX4yp8Cask1IGLlo6cfyMyoIdUl64Do350VayPVBLlU+JLdcuTDxosWb7o2vYh8Lg9A9f8dgT77DAJLsMSujjxeZid1q/iwxhL7U/dAZCj+QQN3mAnp0GKQhhTXwHk+jYpskzoFX60MAhVswL0TYRj5o5YKxnO/ROKbvk341pOqM1DmamL9BRXEBrUcwdYrgCCOlfqTIomqWfwIVi9O8VsWCXS/9B6PJT4vms9E4taenS6O9iLBN57edwUgd7J8Q0riYJzAriEydV5xVlsdG1q/1dcg7iF5ISZOLaM2qLhxu1yi/7uJni4uHum8lCLFICk/8MCTmN9wF8XoEXaY1ySwoR8rWfE954Vi79tGKeUovtky5M2ju1E2JplWWSipZs4cfJ2ZqErirBm/A9494r7yKRPIX0zTJDZPq35/zFW/6t2FAO7a8Cv8ny+Q/nFDMOeb/3+mYgHQJm8zIIDwmwUikx7CmGKt19tAxJB2uNT8xPwkzCdhUn8e/kwqVR3Np67l3odn3yA4yuPQ2a1Dk1ojKmpEPEyAEcZrkEf71Fz389oVn0ZlSN2yKesEYvSODKWktk92WAbYiEMhYCySKXiM8blL4U0NtaQe6rwIzqDFZ4YfIPpqu0JomPKh0mZ6ZMcN9N2JPsS7LA+YMhVNpEkEHNOtCiSVL2rxA2/N8xh4Sv3DApjPiTjDg0XKlXFptSTL/eS/N5ujM3KJwwflyvy0PCdjRPzODxk22vPU0PNtIBV+i6N0ojchrNP2n29G7aS8uZT9FRM54yf6Rd+YuTmImpUz3EkuKedOXWee5028ber9u9rphDNw2Vs5Q31tWWD+brxMb N/GAnFdB OppS+Sznk7DvahFLObtLGUMDiWHFeN+UwldXAn81uTOMKka2KYJkFtkYn5lIr8qqMfSwzkjVC4tM//F470aAvx/AN7Ui34dRtc6ChBhzCKK3PUXqMnR+FKgSeHPvm2QDBO3lKQuUjFkl+BPqTlv7pIhq/Rx78bsbtwcPGq7EId29fT9ER+nWVF4dC+fNQ4yPDO8Ymnk3ea4tHPe5vvFjiKJgd7pP2723HAC82 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sat, Nov 30, 2024 at 05:24:13PM +0100, Dmitry Dolgov wrote: > Hi, > > While working on PostgreSQL [1] we've stumbled upon a question regarding > resizing of shared mappings without conflicting with any other possible > mappings. Before making any wrong conclusions, I would love to get some > consultation from kernel folks on that topic. > > To put it into a context, PostgreSQL uses anonymous shared memory > mapping as a buffer cache for data. The mapping size is configured at > the start, and could not be changed without a restart. Now, we would > like to make it more flexible and allow to change it at runtime, ideally > without changing already used addresses and copying stuff back and > forth. > > The idea is to place the shared mapping at a specified address (with > MAP_FIXED if needed) with a gap, then use mremap to resize it into the > gap. This approach has an open question -- how to make sure there will > be no other mapping created withing the same address space, where we > want to expand the shared mapping? E.g. the shared mapping was created, > then large memory allocation caused another mapping to be created close > to it, so that expanding is not possible. I think there's a very straightforward answer, which is to mmap() it to the larger size to begin with. If, say, you create a file of 1GB, you can mmap() the first 100GB of that file. If you access the last 99GB of the mapping, you'll get SIGBUS, but you can truncate() the file larger and gain access to the new memory that way. Does that work for you? Or if you're doing MAP_ANON | MAP_SHARED, just don't access the last 99GB until your configuration changes. Memory is allocated on demand, so you won't be charged for it until you use it.