From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40F84C5B552 for ; Mon, 9 Jun 2025 07:22:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9FD1A6B007B; Mon, 9 Jun 2025 03:22:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9ADF06B0088; Mon, 9 Jun 2025 03:22:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 89E836B0089; Mon, 9 Jun 2025 03:22:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 6AC4A6B007B for ; Mon, 9 Jun 2025 03:22:10 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 13B641611EF for ; Mon, 9 Jun 2025 07:22:10 +0000 (UTC) X-FDA: 83535018420.27.009C5B0 Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) by imf21.hostedemail.com (Postfix) with ESMTP id 5909D1C000B for ; Mon, 9 Jun 2025 07:22:07 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=E1LHTV5t; spf=pass (imf21.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.216.54 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1749453728; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tlZBYyVcyDUOvgy5ixpGTAUg8PQpkLuJBFcAyQwuqwA=; b=RpJjW+dj7X3GGWrmOvHqnNNFfJifOpcYakLUXFHEm4gXXUFblDcuR8CDDofvSFh93rWZDw +qJnZjL2c+xdyA8XIvNbZ6MfsJezrJgn4svxg7ayUZjDQAYllaTR2BKwIP21KFNcsBOmAX 5vysGq95AFqLyRYL/9/+NvawuzubCf0= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=E1LHTV5t; spf=pass (imf21.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.216.54 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1749453728; a=rsa-sha256; cv=none; b=poy9B/8xtM/7OuZ3A//F11sCwPoQRf7ANJxGWgbI6c51YtroyR2SoENafTO2BYTjcfb2eF Jq77vmEVM5juOfelMNIvB1S2IbEb8CyZiYTNc+iPp1dMRgPoCgLP/LAXsmzQKrtN7aEPMd z+OJYbHFw9BrbUC+U0KBT9Mvqwkax9c= Received: by mail-pj1-f54.google.com with SMTP id 98e67ed59e1d1-312a62055d0so4027256a91.3 for ; Mon, 09 Jun 2025 00:22:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1749453726; x=1750058526; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=tlZBYyVcyDUOvgy5ixpGTAUg8PQpkLuJBFcAyQwuqwA=; b=E1LHTV5tKAaSdVTbLN7oFh/4pONb990Fh4ln5pyy0Q1LPTQUEloq78Z2CAUfUwVD+8 ehJmNCCj2VLgtiOFWXyYO7C3fNyAXbVe6TwCQCTcWUUslwBksC47v85JkJCH6LlAjNFc gPfDMWQ7OiDb21x03PURwPbxedEHlpyWg7wt9QZb9d+PSsyDCQ2au8R/hmUyzABUCUEa y7Y2JxYBRXPs1lpjiCVBYr7AJCqKGtMjy/rXAeH91JrcX2rBIJMbVu3Eq3E/LmDKdeS/ KiP7Q9dFwc2OZ8Uy03hhObFAcXJy3iJsVsf+uRGqlweX/RaeA+uzXO32j4Z9At27NY+D n1eA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749453726; x=1750058526; h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=tlZBYyVcyDUOvgy5ixpGTAUg8PQpkLuJBFcAyQwuqwA=; b=SojA9qDy6XjYvFmaYrNL67duZYlDUGyn9bwLN/wQ9J7Uy6J10rnJ+Ns+Q1TgN93Nh7 wiTiMM+MPxjsUSKB3GWsZN+dBVfEblhy+MpAuNS928aFR1Rz2O2Sm/qt3p65pAxQpq4N ojKSJ3g8PgiBfCSwDAx3G39uOjJdYCr+JF8LJbgv2gjSykHYoDpKxlE4AqedTNWg1jp8 4m6FxHlvYQXfJ4ysn118eom0F1rfMuTRnm+pzhTr9Ql8tb5HCx/+Jd9T6wBXIERAELea 8j0uNjkMvIzWbj1sk/J9L6uld/WZ5nbmHJloUJj/XLFwlbWngfFztS43ZpxQKs/JvkD+ a75A== X-Forwarded-Encrypted: i=1; AJvYcCXevWvFqXVy7yRjy7C0vuIi5yqCjz67oZ9S1UeOrwAqWdW8onmrEpO7PLD/MHE0n+uVL9q4EFIfYw==@kvack.org X-Gm-Message-State: AOJu0YwZ3+GJCpC67xOUa610SoPueDwYRdPFqAUVW7IzYp2G7Byo9MCK vNxFxLCyK+XzDRmzXCSM25yFkCXMDbQwtrYeXb1+86Tshs2pkLo2IrtjU81D0YhB8Go= X-Gm-Gg: ASbGncu8xF2L9t61EP94j6uv8f5JtT4tWG+GvWD3n/oDhajFU0SAbxErz42VtzTr0j2 BkzRznROb0zDBE4XGN/Oj3D5WDPn0ttc8fjumw6oS2KkXhyxDcnEHdjW8cheEHzB2qn0pvf1+B/ /9enNON765UFx1CUTHu2b02WUNMdfD1PlOQOlZBVzBltPuKvgJcIEWQx8utJ7AxFrnQOfy5xuDa CN9rHtg6WWpgoakhJdIjpuBBvGZsoPvh5FPHSOOCmXOrdG8ZcAL1r3hvGLNl8F4AC/2yG4j23yw FU0ctLY+bHS9qcRbUKbfzNHMiKX/U1PqYZOamK/tzGUUHM3MAQvzUv7D5PXz/gxKFBtioPxu91q F37/dIpbNfbQ+71elucY= X-Google-Smtp-Source: AGHT+IEh4LPffN8yuJEsvHfJQbaYgOAOjlnHV104Z4hZYSIoE1PS8agIZzSxAbwcbd3A792x5GiqXA== X-Received: by 2002:a17:90b:58cb:b0:311:f692:6831 with SMTP id 98e67ed59e1d1-3134707730bmr16002660a91.32.1749453725873; Mon, 09 Jun 2025 00:22:05 -0700 (PDT) Received: from [10.254.237.177] ([139.177.225.248]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-31349f44bccsm5041596a91.22.2025.06.09.00.22.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 09 Jun 2025 00:22:05 -0700 (PDT) Message-ID: Date: Mon, 9 Jun 2025 15:21:56 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v4] mm: use per_vma lock for MADV_DONTNEED To: Barry Song <21cnbao@gmail.com>, akpm@linux-foundation.org, linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Barry Song , Lorenzo Stoakes , "Liam R. Howlett" , David Hildenbrand , Vlastimil Babka , Jann Horn , Suren Baghdasaryan , Lokesh Gidra , Tangquan Zheng References: <20250607220150.2980-1-21cnbao@gmail.com> From: Qi Zheng In-Reply-To: <20250607220150.2980-1-21cnbao@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 5909D1C000B X-Stat-Signature: 1cuapsz1gs4y8knr6mxz9nduwqwkqykw X-Rspam-User: X-HE-Tag: 1749453727-147342 X-HE-Meta: U2FsdGVkX19jCMnG3PCRSKDetCseLTIOYQNS1NHBqaxkmvGAcovObAmJP3xnHuzsmKAcgLkCSmtKtQ/KeZHovFMrQPWIgPlMe9W+cNib4aCFg2i09TaAp5WvNlfCB26uVT1O7e4nfc/CmPCORxehCJHEYfBCzt2yoOaD9pqz8aOVKsPKEUIxDokPgUw3/sz4cGDwnjnYelGQ1Ee77TkoNXNmByBgkHP/eGq/mJqrndPDH7qaW55qL5IdBvIh+J+lCJPlM6CN38ZX3dShT3vMLdPzWljfbqNK4wmX4fnRZvFctxvY4LysEfd2Oc1AdYOrXei4lvy38cjeBJWGcbUfVmfY8t1Qi1LeGlmPWGwwOFnQzaYLng889SH0m1PTrnDpDpY1VsRMyRWoQs3bFFrvPlQYKyIcZl3/VnCrr6RPFTJpi6qTFbGnOE079cktRdrYPu1a/PwkljYE20EYT+N2vuyOKYxHuHtccnJOgAO8D6QGUrsW6zEzt8xWCLkQEqRg/gd8XrIkBASlQY6ezA0SfOYoI33+f+kDBmr4xX/vqywQ6SbA6xkRv9svzPssd8gQFlPIGqt1CMLaa15/QAzLRxF9SyqQiC1DXZ0yLhrMtwDPLlXhSQ+wkyQfQqIs3PbMkBjwRW5N4fdxMcsL/A+wPXm7jWuDqgzbw3x3dvFNte9oKb8Vy6HQn0eVULQrniPjuMlfiWbrTFbsC+mpK5p6npUwkX6odobeKAtPG9O+0bykVl23mf4IKtfqoM2wVGHeKvGW/uz/IiyX1p8dvbhCA5KqDcNshAWn3DJ7zGLY6QvPx//6JTdlydAKq2W9RFJzfU+kealv3vXh4eq03DdhT4xcMSO9WHlLwJfpx/jEZ3XuIKop0lzLBTqQWiU/+Ajkwalul9TYJw5MDLPm2nLaBDybXhSWN8JWaoefAjXnR6GJZc5hEHYUa6M3LbVv/V2q28NiooFs5MAFygVrTKs iqNNvTyG xywlSM+q/goDBCm4DyOFtixDvcBhoEWqGxyQFUxg/ykWBy4zCc+pOa3/+vSpmq2pPg2zGrfh2NLknctmkaRwKVlRfN9ZI7xVzVh7NY/Cx3+Dy/FuaGqaeyZ4uaClDacAbHjfyD/LAwfxPozO4J/3lWzbiSwm+diy/XP8SYJeTf6MEov5WYA/yT9aD/6SCsGpYFYM7lKA79Og5uc6TGWdzVuR1X+dO0O0zTFxpoHwpAs3tLgxjPCN1TFVHTYFOVL6MsgbrBiojDWcpLm2xe8ReGOESlvGocCXD/MPO/Arc/HR44ULG3TERS05XYQwsnYcvOWRoOLmpF8QWb7rWETRnxjgeNpbbyPuN4S9s2jbUVpKrMaonl/RI3SyULlbvJB4DR1DCgFS5uKEP8G/SENwu5hRD9Mw5FAG8Kta+XZOjF+O6kya3Sx5tf/dmvbDiHfCkd8fqf1asvxP+Ahdi4BRrp2ataZ9/bMO004w7aSDxIbuTz0NVZ8sqUMg9wrJEhQabQwpgqUe332j81ecnGWMqT1oCB2+N67qI/2qVIrzph64LKz0OBClPVE4T8fOlPDhltXWq X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 6/8/25 6:01 AM, Barry Song wrote: > From: Barry Song > > Certain madvise operations, especially MADV_DONTNEED, occur far more > frequently than other madvise options, particularly in native and Java > heaps for dynamic memory management. > > Currently, the mmap_lock is always held during these operations, even when > unnecessary. This causes lock contention and can lead to severe priority > inversion, where low-priority threads—such as Android's HeapTaskDaemon— > hold the lock and block higher-priority threads. > > This patch enables the use of per-VMA locks when the advised range lies > entirely within a single VMA, avoiding the need for full VMA traversal. In > practice, userspace heaps rarely issue MADV_DONTNEED across multiple VMAs. > > Tangquan’s testing shows that over 99.5% of memory reclaimed by Android > benefits from this per-VMA lock optimization. After extended runtime, > 217,735 madvise calls from HeapTaskDaemon used the per-VMA path, while > only 1,231 fell back to mmap_lock. > > To simplify handling, the implementation falls back to the standard > mmap_lock if userfaultfd is enabled on the VMA, avoiding the complexity of > userfaultfd_remove(). > > Many thanks to Lorenzo's work[1] on: > "Refactor the madvise() code to retain state about the locking mode > utilised for traversing VMAs. > > Then use this mechanism to permit VMA locking to be done later in the > madvise() logic and also to allow altering of the locking mode to permit > falling back to an mmap read lock if required." > > One important point, as pointed out by Jann[2], is that > untagged_addr_remote() requires holding mmap_lock. This is because > address tagging on x86 and RISC-V is quite complex. > > Until untagged_addr_remote() becomes atomic—which seems unlikely in > the near future—we cannot support per-VMA locks for remote processes. > So for now, only local processes are supported. > > Link: https://lore.kernel.org/all/0b96ce61-a52c-4036-b5b6-5c50783db51f@lucifer.local/ [1] > Link: https://lore.kernel.org/all/CAG48ez11zi-1jicHUZtLhyoNPGGVB+ROeAJCUw48bsjk4bbEkA@mail.gmail.com/ [2] > Reviewed-by: Lorenzo Stoakes > Cc: "Liam R. Howlett" > Cc: David Hildenbrand > Cc: Vlastimil Babka > Cc: Jann Horn > Cc: Suren Baghdasaryan > Cc: Lokesh Gidra > Cc: Tangquan Zheng > Cc: Qi Zheng > Signed-off-by: Barry Song > --- > -v4: > * collect Lorenzo's RB; > * use visit() for per-vma path > > mm/madvise.c | 195 ++++++++++++++++++++++++++++++++++++++------------- > 1 file changed, 147 insertions(+), 48 deletions(-) > Acked-by: Qi Zheng Thanks.