From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78817CA0FFE for ; Fri, 30 Aug 2024 15:04:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D43776B016A; Fri, 30 Aug 2024 11:04:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CF4326B016B; Fri, 30 Aug 2024 11:04:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BBBCA6B016C; Fri, 30 Aug 2024 11:04:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 9E62B6B016A for ; Fri, 30 Aug 2024 11:04:12 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 3C34C160ADE for ; Fri, 30 Aug 2024 15:04:12 +0000 (UTC) X-FDA: 82509232344.04.3493FE6 Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) by imf04.hostedemail.com (Postfix) with ESMTP id 1A6AD40020 for ; Fri, 30 Aug 2024 15:04:09 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Xp9HpNgu; spf=pass (imf04.hostedemail.com: domain of pedro.falcato@gmail.com designates 209.85.221.53 as permitted sender) smtp.mailfrom=pedro.falcato@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725030204; a=rsa-sha256; cv=none; b=nSqAKeCygr5iT9Mbr8h2QeDrUCldq0ztWl92tx6pnqOEqHuf6mbBo27qUr+cAXcKvUc2DH OmWA++h7NzQpYK8oVtJ8xurTpynB8rwrPSPp4PYbWo80nQiLrgEjeuvLOnZXattx9JgJTX WzNJczdkLHxbiHDGDhtnoonldWrPgF8= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Xp9HpNgu; spf=pass (imf04.hostedemail.com: domain of pedro.falcato@gmail.com designates 209.85.221.53 as permitted sender) smtp.mailfrom=pedro.falcato@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725030204; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5+y3zV4EDrHM8E/Avurk5GQvKXIz1DQBJgb/vXY4IU8=; b=AxHX9V33DZEp4DpzX03wdk5kZdZksFlwNX3m7j51tU0Cwdh+V7CJpvTvgrMmorexNG3eaq OyqTDE+QMGXHUhzzlRW5Oq5ouhV1CE8YQ7sSxZSvRkoZMLMtU9K4Xwavwp/gD4FUckLMfX 3apw6AxcvUWhb/aZSSCGHuEHlUgePjA= Received: by mail-wr1-f53.google.com with SMTP id ffacd0b85a97d-371893dd249so1406279f8f.2 for ; Fri, 30 Aug 2024 08:04:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725030248; x=1725635048; darn=kvack.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=5+y3zV4EDrHM8E/Avurk5GQvKXIz1DQBJgb/vXY4IU8=; b=Xp9HpNguG4ji/rAVvf4t0GQ9lri3myx1A5QH1HFqeZv+i1R7BxcV+t1x6L5Wg8DruI Rxp7fMEPKytZuONW2QQVHYXqudhVutI5fPcJCJvcyHlgVElyqXVvbepMnPYLlKVaTxtO VFirrYSZgK/Xkq/XZPXReWOTZcBnpA14/EpIPK5p1pUuP1JNSyMBMV6M4FxiEj5Z4uph fAVWmHjuteexgX+5qepLtw/s7yGj/kj+p5VLWAfHoaN5/Ban07bPBiPR46Iye0mv0Mzy gye/fuyQDZ50oIIR3vEwn9Y+OgxPLUThUqeUQ9aPC3cMgn+Q3JUHbbluAnnU81ZX5XGs 9dBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725030248; x=1725635048; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=5+y3zV4EDrHM8E/Avurk5GQvKXIz1DQBJgb/vXY4IU8=; b=cQfl4xLnxS3rp4nbdoQfNZokZnHb/EEwNtxJcm+4SsLPKBhql6G5zMhdlXx44Rb5Px iedGDQVp0svaF4rAkAR00ZsQesKCHgRjNhSRZd+7NE8whhgWGGoFLES0t5vdyZbBRXK4 Umh2hen46ZoErLG79Km+ZcAc712k3FZOWA6jJqSJ9QCTb4677wFWXePbDOeDeKgrH75T GBU5tFSbiVhaFDE5P1B1uoidt8d4uAM8ox1VziHKhPT/K5/99+/C8TtQ0y0/hnlh7R7r Wn7W6ffG8uIuyEJqo0/G8PVd/zfmL9ZnYdRWyayh2Jr9dIch+C/7ZZtrUShaQmTgAEt6 ul/w== X-Forwarded-Encrypted: i=1; AJvYcCUOBlqA2/gS6M2sRT+PvGwF1gcfgb5e0dsGXUOxpObCiuLD/+6JrAidRUc43Tqxc8vEqYQw2Q6quQ==@kvack.org X-Gm-Message-State: AOJu0Ywjt96nOyeNI4+5FdQ2TX20KexuiqQHdRCHHBvlr/yjF7h4OxWO dDVVXNzGxR1O4LZNqn2pHucu3veyVbxxGvGUfFMhymjzoZiOJJxe X-Google-Smtp-Source: AGHT+IHmnfdQQ1hfPAXxYQUJZKzBNqemK3VME7LWK8oD3vyz5RbznDMrdodK4EXTUEkVgqcIn3QkAQ== X-Received: by 2002:a05:6000:1811:b0:367:8876:68e6 with SMTP id ffacd0b85a97d-3749b582257mr4258615f8f.48.1725030247744; Fri, 30 Aug 2024 08:04:07 -0700 (PDT) Received: from PC-PEDRO-ARCH ([2001:8a0:7862:ea00:1d36:5f53:3f57:14ad]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3749efb1682sm4227265f8f.101.2024.08.30.08.04.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Aug 2024 08:04:07 -0700 (PDT) Date: Fri, 30 Aug 2024 16:04:05 +0100 From: Pedro Falcato To: Petr =?utf-8?B?xaBwYcSNZWs=?= Cc: Lorenzo Stoakes , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vlastimil Babka , Liam Howlett Subject: Re: [PATCH RFC] mm: mmap: Change DEFAULT_MAX_MAP_COUNT to INT_MAX Message-ID: References: <20240830095636.572947-1-pspacek@isc.org> <90f07fec-3f46-4b38-86fd-07c9f8201904@lucifer.local> <82960a7e-9013-475e-9b80-7b29a5667482@lucifer.local> <5dca8600-0352-4b5b-acb0-0cd4f84733f4@isc.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <5dca8600-0352-4b5b-acb0-0cd4f84733f4@isc.org> X-Stat-Signature: qcehgn9bjcg73a86jyu58ufgetibqw8g X-Rspamd-Queue-Id: 1A6AD40020 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1725030249-190014 X-HE-Meta: U2FsdGVkX1+xzk2uNgBuTsmsaYof4JFrhyoDISILEFnCQt5LgjIi0lvc+H+vGQCKXf7Rq8XntV0QH+T8sFVBZ1bqNd2QsmO5nVXHy3QKfQxN4rWxtWbycT5RQy9o46wa0m8UNr5v5rsO+6op2zj0Ij0gS9BS48G1wmW+44sj8CEY1vLgz0zanC5mRiA7fKWbASZXsBbQQjhX4zhr3+gFephcrvGHGme7coAXXoaDq5VFBcUeBD7ibuyNcgHj8MokfB4Y321xbxRpRa3wdsYEYkwIWRHyr3Ty7rs+bOGApdZkEqByMOUFpodpXwZnj8gyKWu15Y1h4up7SDZfPIzpvM1sqTgaFFsvd8Cl+MJ09fPoLwkiZa1+usjCQMXXobu4jSJyGQtrLvjn4LFSXCR5gAq2Prdh4Ssiv8OE1SnOjlLBfmPPwBtuzCpT8DJmj9HRIfytLq03d+dYVS+LElSPhVZb7HIVdMiINnsLTq87QV/WKD0+GsgBQvikBRtGhFuYE/YmbHu4OxGH5V/w5VVHvEFit9/eF+mbrCR/XNSU3uD1nL4Y4JoST2C9NVcUyLpO1irDCYYz0UVe5uc7AWL3wMcCBrP/1uRHQq+PMKfwFJ1IjJWOm7h3YSdikrkgDR/Om8YeM5tnkv/3hLv3ANzg45kuYjvlNUW3ASeIG2Lx0HGYCQ5bpz0BIM1/dvJHuuHcq9tTOsUkVF/A38lhxVSyi+LXNCO1ul5jYktqzZB0wlXDAK6C7W2nUuxInnk/LX5NEie85yo2uzNYdxM4DSgn7Vbycvm7e+V5XdTZkYwoReZ/X0SejehW19rb1mh3P46CBVE6RhTgIBmZmsek7VgWlj1OF+bPlHFfdxZ2FbRhzhWGfnqs3UT2Hj0Iy3X5i83/azHdQjLd94V2h12cVRPU8IRsZ1wwCD18HtDWlCPdQdrRl74NMkLIcVaycxTfgrqvclBGsA1KjPVaydVBCzE iIc4XT0K xlMtUd/c51sCJ3yXO/71u1usspzTYJnnMJpgJmU1lTgGShKmH2M+y4j/KA8OJBkO3IW2Q2VfeXhAq3xE6GVvyaP4lW7YV/EHQeDO3J06dO5bklHgEMj9NB09pnn13k4Fr8n/3cW/ydHPeHgPAyreIXvFmbF0hiUNmNvl0OjzboBcjyvvjMud2jOIwkrc7J+eiyXA7Kw0TShPDFu9M0ErnSMrjIWfXsTxy2pcGojEvksVOhkAzdiJaghHX9tv1jGU4IaTmh3pHDohqIGgbXBQGE9rCR48K7JHiBk0IZCF4871fasD1NZqjbPhq/cZqPoua5EqkwbizciTM8hKqulhNbxlGAG462ze+LIbtroURcJRxXlwIBRuB0S9EDGQ0y+gTshSpk03xO9TizDstK+XOLTwMl5HfkIcDGfBDjegICN2bEmrzKI4KvZrv0hrOBUlbXW/CaGHKK84Ri8sGE4KVJrHkkmvW33Fp9acZ3RNHAf5c73kEaYRccJdsvniNUqVGW8rOHsBSSWoh1QEDecHwwdjs0zdoMaODrX6C X-Bogosity: Ham, tests=bogofilter, spamicity=0.029135, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Aug 30, 2024 at 04:28:33PM GMT, Petr Špaček wrote: > Now I understand your concern. From the docs and code comments I've seen it > was not clear that the limit serves _another_ purpose than mere > compatibility shim for old ELF tools. > > > It is a NACK, but it's a NACK because of the limit being so high. > > > > With steam I believe it is a product of how it performs allocations, and > > unfortunately this causes it to allocate quite a bit more than you would > > expect. > > FTR select non-game applications: > > ElasticSearch and OpenSearch insist on at least 262144. > DNS server BIND 9.18.28 linked to jemalloc 5.2.1 was observed with usage > around 700000. > OpenJDK GC sometimes weeps about values < 737280. > SAP docs I was able to access use 1000000. > MariaDB is being tested by their QA with 1048576. > Fedora, Ubuntu, NixOS, and Arch distros went with value 1048576. > > Is it worth sending a patch with the default raised to 1048576? > > > > With jemalloc() that seems strange, perhaps buggy behaviour? > > Good question. In case of BIND DNS server, jemalloc handles mmap() and we > keep statistics about bytes requested from malloc(). > > When we hit max_map_count limit the > (sum of not-yet-freed malloc(size)) / (vm.max_map_count) > gives average size of mmaped block ~ 100 k. > > Is 100 k way too low / does it indicate a bug? It does not seem terrible to > me - the application is handling ~ 100-1500 B packets at rate somewhere > between 10-200 k packets per second so it's expected it does lots of small > short lived allocations. > > A complicating factor is that the process itself does not see the current > counter value (unless BPF is involved) so it's hard to monitor this until > the limit is hit. Can you get us a dump of the /proc//maps? It'd be interesting to see how exactly you're hitting this. -- Pedro