From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 283ACC4707B for ; Thu, 18 Jan 2024 13:43:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AF1D26B00A5; Thu, 18 Jan 2024 08:43:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A7A556B00A7; Thu, 18 Jan 2024 08:43:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8F5176B00A8; Thu, 18 Jan 2024 08:43:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 797176B00A5 for ; Thu, 18 Jan 2024 08:43:27 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 49AD4120CBA for ; Thu, 18 Jan 2024 13:43:27 +0000 (UTC) X-FDA: 81692548854.11.983B217 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf22.hostedemail.com (Postfix) with ESMTP id 27C54C0016 for ; Thu, 18 Jan 2024 13:43:24 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=CmN1miDM; dkim=pass header.d=suse.com header.s=susede1 header.b=CmN1miDM; spf=pass (imf22.hostedemail.com: domain of mhocko@suse.com designates 195.135.223.131 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705585405; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NAWBeypBvGFM6Xet4cmHeLDFXwdHP1NsXHe8czK2YMg=; b=6VR+/qNWSoNUnToUTyzYvMraRhYQI/y4W0JDqV8l6hVPWMn0byNCeHszXkwQQlOC/GvxRs 7uCFOuXy2J8IJeb2aZCM+RwRzgLksNzBPkxtW/ZgtGPbzbwke1kTYc14vYhwI/m1nx938h 7hgdImZwjNVEnmarpsUFE7SOCS5yiZc= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=CmN1miDM; dkim=pass header.d=suse.com header.s=susede1 header.b=CmN1miDM; spf=pass (imf22.hostedemail.com: domain of mhocko@suse.com designates 195.135.223.131 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705585405; a=rsa-sha256; cv=none; b=iBZOgD+LJ+zxFIIqFxJ8K8SWB4iEPeCDO/oJ4lGDWk6LEDMgigfA59XWS5r/HNkZA24FNu ujMXiUoAnabmpDHlmue0H9sqU97q0TOSxJkQrF8/7zWn/YjLdh/wltBGjyCQ/QM62xR/zP +Uw2ulS5k77dr2dF4T+zbr8OSja8lf4= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 9F6071FCF4; Thu, 18 Jan 2024 13:43:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1705585403; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=NAWBeypBvGFM6Xet4cmHeLDFXwdHP1NsXHe8czK2YMg=; b=CmN1miDMCWoPZDcJl9YS1hNtPAeMclubrDcIqROVR76CBjuXXcFa4wfjjq0sICX54wZX+6 Fz3a491/y9KDfEH5an44nGnbdES5Gc4UJr+TtZNGb0eDXerVeGho1gqbjjl5BET50a0ojt WZX38fDrUgiiLC096IEsXyEcq/E1pco= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1705585403; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=NAWBeypBvGFM6Xet4cmHeLDFXwdHP1NsXHe8czK2YMg=; b=CmN1miDMCWoPZDcJl9YS1hNtPAeMclubrDcIqROVR76CBjuXXcFa4wfjjq0sICX54wZX+6 Fz3a491/y9KDfEH5an44nGnbdES5Gc4UJr+TtZNGb0eDXerVeGho1gqbjjl5BET50a0ojt WZX38fDrUgiiLC096IEsXyEcq/E1pco= Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 86CB813874; Thu, 18 Jan 2024 13:43:23 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id bfnRHvsqqWUHDAAAD6G6ig (envelope-from ); Thu, 18 Jan 2024 13:43:23 +0000 Date: Thu, 18 Jan 2024 14:43:23 +0100 From: Michal Hocko To: Lance Yang Cc: akpm@linux-foundation.org, zokeefe@google.com, david@redhat.com, songmuchun@bytedance.com, shy828301@gmail.com, peterx@redhat.com, mknyszek@google.com, minchan@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-api@vger.kernel.org Subject: Re: [PATCH v2 1/1] mm/madvise: add MADV_F_COLLAPSE_LIGHT to process_madvise() Message-ID: References: <20240118120347.61817-1-ioworker0@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 27C54C0016 X-Rspam-User: X-Stat-Signature: g14jf4iddbcfp5fic7niq68zm1ypdgpx X-Rspamd-Server: rspam01 X-HE-Tag: 1705585404-189046 X-HE-Meta: U2FsdGVkX1/fySthSOJYZAgUM+3XU/1Rbp4CufRQ8vzlQXrRInEyGQ8KGdbjLinpi0tle2Ovihi9YIC+cTqyku8fJrdbnp1hWo0KgiQGxla3MzKs5HPZSLbbI05dHCV/qYdrNRA2OTOu6jAlu8zFGFHRuVioxHxR5zNQCqRTM4Nngc5R4BkUUiC5KXH0OM9w9b+HR8srJ1DtzSZhgWx7sx0ZDIVl0KjL37qFyJ+hYXFPPGB199rkiENegmlU3Tb+EzUJbUXM8f5tHtu3DAwLQwBUf09qZJsTzJVMMzaEclMCLR/Ki/G2Hkr4Dwt77rOAVpCxecR6kzmo7cleGGj0dorv7LBnqpcGCKPDRLxTqaGSdrG6l3vIQl7Pya2CzWiMiXY4CQa0X9scaP95stsOwMFvIKWJo10Db0ejer0ZgN4vbxA/frD1XEUoKZ4b4O6JyHGgBOKz8wR8UvEgH7iFdFkvpKhH0dmO7sowomAcMYskAaTYvFGi5S04CeICxCZaa+yTyZsZMNf0FDKDeOgALi9VF2g9H0ZIG+N1sJrsvU/0Cb4WhzYAi2ypZ4jPsRGOwrkAFZaCwpoF9ng2Qn/x54GbE+urlhhdeVv2mY0PUshPiCTHCS96kd1k2ArvTj60GAqYxv8q6pH85SF9uzg/DP5xL0oayQB4rEeaXP/FIh5S8O/tlsgaHOVasF6l4CdhpM4pzaiv56Qp9vUg35FveGecmFEyOVbdOUTzufOY5BOeBlkbVT279yaL5Psx8fklNplebGGyEN9IRKUtYnKJczSSX97p18oaUxwHcNkgDVYwkKi6WODFvra5TlLC+YewJ+vdCQdh+4kb8Jk+8cCAW2iQrLBwInBI8vAEx/jjCjx+XQBAloX6WyOQK+oK0/2hgP8PYVWrTw20/amGogn6JLyDNuIodAjbOgvBf1IX5+c9NAt77JoNeXKMlFXEfRoeUJOuBRC/0FGJuHISadk Zd5hEVwX zK7aXoSyPGac4pZgwq7kGf278HhSDORZ0z/EtrtGA1+Q8spOdSvuj1V54G46rTbVdyCK0Lt2oTTlvPW7D9zT60zAXhV+sbF6cy8Lctfb8lqbzND76vWYfj3UTwTxbxF6AOVg9fZ9mBbxSutaYPlLE8wnVfDgORcEvA5ZcPwe2T5/xhdpi3i4VF0w93cK4Tt7+Ge1eh50n9urynglI5YwQCTDVaZIJZBuQg3QJlKjjuarDDeGjGCLw/ppk3Sz1Cp4/SGLz0UqpNeBpe8EXpkM5njCHcej2lyFxBm51JHEXIeiI6z+EUUw4Cgrw9kRg0iLE+GVOasaC4Gdpaj236uPIVTwI41Elytper/AjoFHVUmtiC/92LuxdcBwjnQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Dang, forgot to cc linux-api... On Thu 18-01-24 14:40:19, Michal Hocko wrote: > On Thu 18-01-24 20:03:46, Lance Yang wrote: > [...] > > before we discuss the semantic, let's focus on the usecase. > > > Use Cases > > > > An immediate user of this new functionality is the Go runtime heap allocator > > that manages memory in hugepage-sized chunks. In the past, whether it was a > > newly allocated chunk through mmap() or a reused chunk released by > > madvise(MADV_DONTNEED), the allocator attempted to eagerly back memory with > > huge pages using madvise(MADV_HUGEPAGE)[2] and madvise(MADV_COLLAPSE)[3] > > respectively. However, both approaches resulted in performance issues; for > > both scenarios, there could be entries into direct reclaim and/or compaction, > > leading to unpredictable stalls[4]. Now, the allocator can confidently use > > process_madvise(MADV_F_COLLAPSE_LIGHT) to attempt the allocation of huge pages. > > IIUC the primary reason is the cost of the huge page allocation which > can be really high if the memory is heavily fragmented and it is called > synchronously from the process directly, correct? Can that be worked > around by process_madvise and performing the operation from a different > context? Are there any other reasons to have a different mode? > > I mean I can think of a more relaxed (opportunistic) MADV_COLLAPSE - > e.g. non blocking one to make sure that the caller doesn't really block > on resource contention (be it locks or memory availability) because that > matches our non-blocking interface in other areas but having a LIGHT > operation sounds really vague and the exact semantic would be > implementation specific and might change over time. Non-blocking has a > clear semantic but it is not really clear whether that is what you > really need/want. > > > [1] https://github.com/torvalds/linux/commit/7d8faaf155454f8798ec56404faca29a82689c77 > > [2] https://github.com/golang/go/commit/8fa9e3beee8b0e6baa7333740996181268b60a3a > > [3] https://github.com/golang/go/commit/9f9bb26880388c5bead158e9eca3be4b3a9bd2af > > [4] https://github.com/golang/go/issues/63334 > > > > [v1] https://lore.kernel.org/lkml/20240117050217.43610-1-ioworker0@gmail.com/ > -- > Michal Hocko > SUSE Labs -- Michal Hocko SUSE Labs