From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98DA1D78333 for ; Mon, 2 Dec 2024 16:14:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B77C26B007B; Mon, 2 Dec 2024 11:14:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B27056B0083; Mon, 2 Dec 2024 11:14:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9EEE96B0085; Mon, 2 Dec 2024 11:14:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 7D6836B007B for ; Mon, 2 Dec 2024 11:14:31 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 2873880120 for ; Mon, 2 Dec 2024 16:14:31 +0000 (UTC) X-FDA: 82850516364.12.4F6A83F Received: from mail-ed1-f52.google.com (mail-ed1-f52.google.com [209.85.208.52]) by imf01.hostedemail.com (Postfix) with ESMTP id 178814001B for ; Mon, 2 Dec 2024 16:14:19 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=bhmqlo4W; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf01.hostedemail.com: domain of 9erthalion6@gmail.com designates 209.85.208.52 as permitted sender) smtp.mailfrom=9erthalion6@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733156063; a=rsa-sha256; cv=none; b=pHt5GCJGrQRw9BhNGiV+xOd7oqs/d6m6XsFOosiMH00WU2IpCFEjVVWyDMwAZ7mGPyd8jG LKGh0VHNxllHWurpsVRI01EFglsPIkGq8RpO8TpUcq0vOWetHb7kZZPz7MSx+RMTus43D2 A8bRmr/BxYQw5X3kH4sm/XrZV4gRILs= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=bhmqlo4W; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf01.hostedemail.com: domain of 9erthalion6@gmail.com designates 209.85.208.52 as permitted sender) smtp.mailfrom=9erthalion6@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733156063; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fjKSvi6xplHOEzMDTIvf9B1t3FEHPozXHizcpdO/xtw=; b=XnIhP396zJ7FUdvyF8Yp2Nftw46xqzs7qQ1NlgBCZlHJJ1TaYOl+CoFWHKQEx3nq0lRES/ s+IRuK1G/dHUzAXIX2GsDd6LuVDMpTOtEJR9Pv4+BnWzBXB3xS92H/oY/SnN61z4yhKfTm Eb9fGBFsC+UYo/l3IRMH3++adDPBB6M= Received: by mail-ed1-f52.google.com with SMTP id 4fb4d7f45d1cf-5d0bdeb0374so3396643a12.0 for ; Mon, 02 Dec 2024 08:14:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733156067; x=1733760867; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=fjKSvi6xplHOEzMDTIvf9B1t3FEHPozXHizcpdO/xtw=; b=bhmqlo4WZZh7Y0A1br+7Dklc3Y2Q7d42rFKCr3x1hxDkW3+3vJjCug47rFlyd7pnOu h6aKXXiRlrBHRureqR5q0J5JZWWTt7yIAeopI9OctwyNfQnkIfR4Med9IoswKdm3HzGh BUP0u1kzEOYbCoNgngmYeVdGC4Q9PlsKVAAcU3mtKPN1cOrI1u9dneZrchw4TQ7JAJSF jRfAyCXJLXIDTMwpDWmk1hDpncP+0E6YL00ObrwpCvJHSbJ6IoAeGOsyWNDsvwYQ5U9Y v6PY9XJxQk5AZ1AEA3uwZzI2Ncp+RWFw2F1ZRZjiJ8lOdDBnLVFEz687gcM2uUO3r3KF X51g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733156067; x=1733760867; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=fjKSvi6xplHOEzMDTIvf9B1t3FEHPozXHizcpdO/xtw=; b=Pfw9/wqpw7thR2fUbj3a9of6G2hI4jOAf2KkKxi75VN3VImFWtweaRzGyzSt014YgM 1hT3qVnH2X83DHkRHrxzkTO16AgUYE0kaguELOxQyZXvTRIKsrdXVz5CRj4dHZMwAjTQ EmKocmcLWBud0BW7FRnbst3N94dOEqbTaEyzIDWdxJWEXaKYmJjeuZgFCV0aN+RbOpoi vZGm+9wBtqz+CxnDQvu/CqwY32HV97JVUUD8waeGSu5BpDxHNJme5P1r0qzypegtkL3I 5DD9RJEvn7BTHVNttlhhF82w56cQw/oKBFJCGDMaDCg8l0VQYw9egcibF53vdiOY4B7o HG8A== X-Gm-Message-State: AOJu0Yxt/bJFn+AV4HqzGxSv6S62Ay+sPvN3n6IvG9NgweffLn2OAYc1 79XJiLGv3kBAaHtqcZk3wIC7GE9ZOBvy37UnEcevM+42wR6EfPsvQBh1EmDn X-Gm-Gg: ASbGncuUMI5TrR1vr5mU3KPYmv/ZZDVzgfwhSPexCFE+XK+yMxEylpCnhtF9EvIJxYF fwzPz3EDS6PtTmMDcLcY78iHo1epreOUjNUGyOyaDU24+ZVMGzF+5c3XbaSxN6LokKbamlRsXR8 lp+K38QofIR4LsalbsdqscxBk1qiK8js8CpG2zGBWfOe1YZVRRSxgsMG4WsyWUfiGwo9RZACgq+ GInzy/UI0X6tFYdwMcQaI39BSc7AAQ+fOhNUYwK9VIqtIkwueltSncbFudxcPDoLQ5XDm3dh4/U hZ17cTx71BdjkU/Dk6V7lcPWG0lpQE29BYUyBiBhyhzx7lOYvhSvGLg6AYBjpwA= X-Google-Smtp-Source: AGHT+IGHp3LRshOD61FWeyk4BRJaAC5+rnNjtunuRZeIzgGMuxbQOsZh2oElztaKCHZGkRtC4bkXFA== X-Received: by 2002:a17:907:6d19:b0:a99:eb94:3e37 with SMTP id a640c23a62f3a-aa5810634f8mr2604022366b.58.1733156067280; Mon, 02 Dec 2024 08:14:27 -0800 (PST) Received: from ddolgov-thinkpadt14sgen1.rmtde.csb (dslb-178-005-232-220.178.005.pools.vodafone-ip.de. [178.5.232.220]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aa5997d574fsm523768866b.58.2024.12.02.08.14.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Dec 2024 08:14:27 -0800 (PST) Date: Mon, 2 Dec 2024 17:14:23 +0100 From: Dmitry Dolgov <9erthalion6@gmail.com> To: David Hildenbrand Cc: linux-mm@kvack.org Subject: Re: [QUESTION] Resizing shared mapping without clashing with others Message-ID: References: <3kpxpd3dbjgg6epasi2554c4qyils4t3cm2pjnyzer7gkyoaxl@khhdxjiggyhp> <795a454b-a432-4200-8524-003e5ad53d03@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <795a454b-a432-4200-8524-003e5ad53d03@redhat.com> X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 178814001B X-Stat-Signature: 4khafob8qjohh3o4woeohy334rt9yqie X-HE-Tag: 1733156059-58189 X-HE-Meta: U2FsdGVkX1/GLSwDyZD36l3yl76+nRIa7yTePlwL8iUgUFqAtF7nl9NZ/jOh2TE5bJDpjjfKJbcdDy0NAXaNv7HeqGsfeNmOIydpWAKJv8x7g7UPOt8qPdeUvV7WHD6MDRpZHr107/nPTK9rR+USa9+ytYRurhbDZHew3busQVNT9sADgZrVJNDdaiIYUQ0eoOM/8UZUT/NFPK3Ms2qI+t7yHDWEYaGZFE5LE+gSOpREY2Hpk4K+k1iMxzxVtbZbU22w/m6EKlW343fD4+FrULTXNPzsHPb2zNVwj7hP00u8YFwMTbKY7bgENxLB2d7ZA17Qxsrhh0xgANJSrcLg6cewbi8vY7fttd+YbrnpAx4HZTcRtvs1MIhkZkBb9ht2loCSKrrcw+mlQfcTtVcf2iaCDLCi1cC3Q+blkMnrDkiuqXzEFtIGUvfoG8fCd8KRQuhztAoXqBmGFkGAITcdQZ7sAkzyicoLz3y/Dw6xMQL5i8DKaaxlPaucbgfugXz0WqCNNyzmMpEIxrWZt8HBTArl2kk6RiEb5jgCG9nGHXw3a08kSP0RyeN/ZRM1soTjpXxhNKcRjXeIHROpgn8GjGrku9omnuOfucwPSLFuP7wIi7YfNzJnXLv5xBHvrYrpKpXEPxZ0HwRXSFmZ8E0Fh2TIbRgx7+YTB4Fv3JD+6WOhBTq35plGUehNRnD2UHpwBMvdbERbLi3mW83V3Y+3vRSjmBCeVww1odNt60rgjcx3ahtq0axfEgXtQhBg8S59UGfNy+Av3cPvL6F2elL1W8twrlEuK49UmvQtH3+e0KYQLt3jUfdct/UTNxaZQ7Ku3Gb5OmaMZaU1hOBl/aSXe5nLsuLIvEvYoPx1vHiJnXEFfQq9mr9YVZgAdiDmdZmaUeAvkuJO9j9x3xic1gvPbCEvSvZJmZ8mysCcpdvUdMl0Fa/bmnk2tP9clmJzVkaNM1vaL1DXne5/vreUJ2a zVYbVpBY jchs87YE7jQYj3jFEEqOuQWzkN070Z7hxueG43PxS9Jp6bVbJZm8Ww2BmbF3eov2LqZ7pltY3XRjXaN0biFRfY+JTF9FCiF4DZ3GkieKRvrDva7udWjF3H/gyuhvSArXKSKLzUkldC09hjE4faVY+44HGGcq0N3h0IbfY+MQ5PSevFSXytAkDTz4bCcE34sJrQaniN5rB6eI2y9F4V2ioTrpy0FSwvMpePwDJZwp/B/FyoMCf9/f/oyKPSpvBxrCAVwHe/DG3xveO4Br6l8jOnww8JrhRsTSBBx6jYmU77ALHWIWlhXNea26kw+FHac1moI+RElGxGyHR/NBavOD2cB9kvP9FHBfYq62yBDRj26ffZY3ko26ocJJ7XN3uG0+UeOXJHCzKGJgUfYA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: > On Mon, Dec 02, 2024 at 04:40:30PM GMT, David Hildenbrand wrote: > On 02.12.24 16:04, Dmitry Dolgov wrote: > > > On Mon, Dec 02, 2024 at 12:07:01PM GMT, David Hildenbrand wrote: > > > > > > Likely it's best to look into reserving a large VMA space using > > > mmap(MAP_PRIVATE|MA_ANON, PROT_NONE); it's reserved but inaccessible, so it > > > cannot get reallocated for different purposes. Then converting pieces of > > > that into actually usable shared anonymous memory (e.g., MAP_FIXED). > > > > Yes, we've considered this option, preparing an initial "reserving" > > mapping, then take pieces out of it for usage. As I've mentioned to > > Matthew, there seems to be arguments against this approach. In a few > > words: > > Note that this approach is extremely common ;) That's why I'm asking your folks, trying to get some wisdom :) Although I have to translated this knowledge it into the database world, is it common for access patterns like we see in databases (large amount of memory used as a buffer cache) as well? > > * In certain cases this will be counted against a reservation limits, > > although we don't use the memory yet (e.g. with hugetlb insige a > > cgroup with hugetlb.2MB.rsvd.limit_in_bytes set). > > > With mmap(MAP_PRIVATE|MAP_ANON, PROT_NONE)? Unlikely. In any case, you > might be able to tame reservations using MAP_NORESERVE. Oh, interesting, I didn't realize this flag controls that as well. This might just work, although there are still annoying parts -- postgres actually needs reservation limits checking at the start to figure out of any huge pages are available at all (there were too many complains about getting SIGBUS, when limitations applied at page fault time). But maybe I can work something out. Thanks!