From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97B3BC369CB for ; Fri, 18 Apr 2025 22:37:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 979D5280006; Fri, 18 Apr 2025 18:37:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 92774280004; Fri, 18 Apr 2025 18:37:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A123280006; Fri, 18 Apr 2025 18:37:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 5782E280004 for ; Fri, 18 Apr 2025 18:37:21 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 7DE0F140F54 for ; Fri, 18 Apr 2025 22:37:22 +0000 (UTC) X-FDA: 83348627124.22.2A7DB02 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf26.hostedemail.com (Postfix) with ESMTP id B79B1140003 for ; Fri, 18 Apr 2025 22:37:20 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IvC3Kksc; dmarc=pass (policy=quarantine) header.from=redhat.com; spf=pass (imf26.hostedemail.com: domain of bhe@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=bhe@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1745015840; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5CqaNyVqh6Pco3lRKN4a8MRDCkCUASVHhS+U1Kf4kAE=; b=v87SlA4lyTiyDFgcOEO2ltaM5HSHLl6xjNTKxtj/600Bd7jCBuiI7fFw3M3hQCa9e6P3mb Yi5uoUTrcbrhztgEk+J/tMQ3yEqjvlPPi6vn68d/ctvgldqdpOf2lk9eKK2z2LBU3Qs0Pd FM9KI7Bl6kKU0n53sKZy0TmOjahadQI= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IvC3Kksc; dmarc=pass (policy=quarantine) header.from=redhat.com; spf=pass (imf26.hostedemail.com: domain of bhe@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=bhe@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1745015840; a=rsa-sha256; cv=none; b=H/85U9PimGUd2LqakpHGt92Q2/I7uZ8UhqhqOCneNt+2t2we0ygFStOxuomjjUH2fRk2yG GpWYfEzkzUvsDbX5HW2UvDTMG22gW9pNgL8HBxY83Iif7rA9Zs4fde4bMzbDUeQdxKEsYL mUUo9Wxdlee4i7m4zp185Xc5uiYWDaU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1745015840; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5CqaNyVqh6Pco3lRKN4a8MRDCkCUASVHhS+U1Kf4kAE=; b=IvC3KkscpWqxaVHxoJAya82JO0Ag9ynM5ZHDxRDSMxXMb6QICTheaXRJCtT83ldrFr0NwD LVzCoxZeG6aYoRl3PIuNdIjPfP4TfYjb2bQTe76EEZvDQkCWPy2vgkfrWFaa/bF+1Zxz4V JdXS1mmCO2VH7YE3RZ1c0B8S3POEnTs= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-447-SpM8dBtoNYK9z96TxEmhJw-1; Fri, 18 Apr 2025 18:37:17 -0400 X-MC-Unique: SpM8dBtoNYK9z96TxEmhJw-1 X-Mimecast-MFC-AGG-ID: SpM8dBtoNYK9z96TxEmhJw_1745015835 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 998D31800263; Fri, 18 Apr 2025 22:37:15 +0000 (UTC) Received: from MiWiFi-R3L-srv.redhat.com (unknown [10.72.112.18]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id EF78A1801770; Fri, 18 Apr 2025 22:37:11 +0000 (UTC) From: Baoquan He To: linux-mm@kvack.org Cc: akpm@linux-foundation.org, urezki@gmail.com, shivankg@amd.com, vishal.moola@gmail.com, linux-kernel@vger.kernel.org, Baoquan He Subject: [PATCH v2 3/5] mm/vmalloc.c: optimize code in decay_va_pool_node() a little bit Date: Sat, 19 Apr 2025 06:36:51 +0800 Message-ID: <20250418223653.243436-4-bhe@redhat.com> In-Reply-To: <20250418223653.243436-1-bhe@redhat.com> References: <20250418223653.243436-1-bhe@redhat.com> MIME-Version: 1.0 Content-type: text/plain Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 X-Stat-Signature: c8kp7rarndy1bbmmerhzedyks8bjawmu X-Rspamd-Queue-Id: B79B1140003 X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1745015840-347496 X-HE-Meta: U2FsdGVkX19EDeZfOYQ6shb7xG1AvkX1YpTdzUKgFLrUdWDcAgucVJaJZsOMgRAChFG84/b0EwgOWU1ln8R7jsh6A+A2FgKo1sloc5hhW0DBJ/65uID48cKnozi3ST9oeWJP5uuJvY9y2QDjiraDnEamnXX/C6UQKtlCv/4xC66+eUtoVgZt2pduNvyLtwUO3Kd5RmaVOvncR8r2I2HFVw3JYuDQX6FoVYDPfo2GkKqiAvV02EkDqLeONGvyl7Z3NlGzD9S5wz09RVNVFMXVSL6fkmkiPzPKq9KC6tHsTaeqzdVQnsiij6eOyZvwCsX3aYQSJdl0EdFoLFEGl3oLY6t7mg3rPHdhzwdFFrEIJAfkj6sDuCj3adKF6kKa+K2qOcGpztFk+09YSuzJomyq2aaxVF2YmBP2EhK9Ib4MJENiGZlWb9HaGH2OCBibXMmmXZkKtfmtU0DvI/TAY7WO/UaMFnkgNtHhmnY1bwkKttT5IK4v9dr09x/Q4AOKlXzjs2mqkT6PCe82OXghUvShLzK+BtUZ6w0urnOZvKk+PrpQxqCZklSsqCRyJcY2R3MVy7aGJLj+glZhgpWm0oTAtIMyI3TND0wibf9pCFA9hm7hvEWiuPGaxCKzPXC9n+DaDRn1FacrNBafBO+4XYtR6RkXhBhBXqUbIISkaeFiITKAvJQiiiGYb8SO5+IengwN0670GxM2yY/dLfEowekficvh8d0jUp55VdvFYdpp5zFh0zt4l7sx7zrU8SIxEngw+b4RMzUyRhSs5pOEYtvQzs6s1hOb286dlfP95QAkvPMy/qSifOz3lHYJGXtolkyR4x3hFxZKSD3heiO9cwawBB1NwFhyQaF7qlYu/okPKWO5D9s2FOVm5nwzmoaUQMxN2yOVjD+fQUCSw9J2yEu9gH4hBLpydgSrD4jDtsWxR966/SUMc+lLObs9kvANRbtZOZPG4czXe9VkLopWrfR ts/7FRZ2 C7VotxfQ5NAy7vgdHZYlgcnIVszpMdzAfw6LsiXt7mtb8OdYOQpc9CTWXVox5fzPZ9TT1TWx3m+PGe+AWJAiYhrInseJ6Lr19aHlCIzidizsjovNJJp4HGLEopRMuzGa6WXiCrnUN+smx+GnzYzAJZ4EAUZGmFOPb2tUlr2CcWWw1VIV6VlBZMmuthoYLssECOco6+F4O+Jktxv90+lGZVcm3+4Fvxl7V+V7CSmUXLXyu0SxNuRnFLmN/4jlkop1qhrQZQ0pKeXLsqKUBpzL34v61DTDjNCnQLLwO+mjdsCZfxLgX2hddbbbCZuxLrPJ5BT7bsVZARfHyzevmf2RQy5GoNLCUdOX/yJRo686xRAorgwIXDbEP/is3S7n3Y49612iEnJqOjd3y+rTILC8DwWQKe9ui6LrgzKKYMjMQGLv9yPQvKIluhO7kLY8p5pKMyI/1KerOvuVRKpTyIwacIdggJO7xh6A4AsgaFD0Dd0k4Pkg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When purge lazily freed vmap areas, VA stored in vn->pool[] will also be taken away into free vmap tree partially or completely accordingly, that is done in decay_va_pool_node(). When doing that, for each pool of node, the whole list is detached from the pool for handling. At this time, that pool is empty. It's not necessary to update the pool size each time when one VA is removed and addded into free vmap tree. Here change code to update the pool size when attaching the pool back. Signed-off-by: Baoquan He --- mm/vmalloc.c | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 76ab4d3ce616..cd654cc35d2b 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2133,7 +2133,7 @@ decay_va_pool_node(struct vmap_node *vn, bool full_decay) LIST_HEAD(decay_list); struct rb_root decay_root = RB_ROOT; struct vmap_area *va, *nva; - unsigned long n_decay; + unsigned long n_decay, pool_len; int i; for (i = 0; i < MAX_VA_SIZE_PAGES; i++) { @@ -2147,22 +2147,20 @@ decay_va_pool_node(struct vmap_node *vn, bool full_decay) list_replace_init(&vn->pool[i].head, &tmp_list); spin_unlock(&vn->pool_lock); - if (full_decay) - WRITE_ONCE(vn->pool[i].len, 0); + pool_len = n_decay = vn->pool[i].len; + WRITE_ONCE(vn->pool[i].len, 0); /* Decay a pool by ~25% out of left objects. */ - n_decay = vn->pool[i].len >> 2; + if (!full_decay) + n_decay >>= 2; + pool_len -= n_decay; list_for_each_entry_safe(va, nva, &tmp_list, list) { + if (!n_decay--) + break; + list_del_init(&va->list); merge_or_add_vmap_area(va, &decay_root, &decay_list); - - if (!full_decay) { - WRITE_ONCE(vn->pool[i].len, vn->pool[i].len - 1); - - if (!--n_decay) - break; - } } /* @@ -2171,9 +2169,10 @@ decay_va_pool_node(struct vmap_node *vn, bool full_decay) * can populate the pool therefore a simple list replace * operation takes place here. */ - if (!full_decay && !list_empty(&tmp_list)) { + if (!list_empty(&tmp_list)) { spin_lock(&vn->pool_lock); list_replace_init(&tmp_list, &vn->pool[i].head); + WRITE_ONCE(vn->pool[i].len, pool_len); spin_unlock(&vn->pool_lock); } } -- 2.41.0