From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56586E7718E for ; Thu, 26 Dec 2024 17:07:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8F8EE6B009E; Thu, 26 Dec 2024 12:07:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7BF106B009F; Thu, 26 Dec 2024 12:07:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E9A06B00A0; Thu, 26 Dec 2024 12:07:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 3911D6B009E for ; Thu, 26 Dec 2024 12:07:37 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 010E41215F7 for ; Thu, 26 Dec 2024 17:07:36 +0000 (UTC) X-FDA: 82937740830.18.B7D39A3 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf21.hostedemail.com (Postfix) with ESMTP id C81FD1C0019 for ; Thu, 26 Dec 2024 17:06:14 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=tIQiWb1Z; spf=pass (imf21.hostedemail.com: domain of 3VY1tZwYKCGgYaXKTHMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--surenb.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3VY1tZwYKCGgYaXKTHMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735232827; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=itcT003jdnlfYy/z4AAxsZgzhf2fL1y4OTxlsN9l2Ik=; b=EcPMYpDnhcCk9ITKGEyftGU2e+zTbm7ZSChH4FNRWLqq6x1SnuIw6pcFQ26rjo+zzFjRqJ JNCfyGd5kJfn1nZhA1v0SVYw1pKnVDCwDpx+QleJSCy98PBuH14nKH/SgABkwpt5a3Qiam 3fCX1kdg6odLniAlujtLojrK3SOPmEE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735232827; a=rsa-sha256; cv=none; b=VAMGqFy/twVN/hjV+6lQds9bzvWQSzSUeX4H28VDuM4+n1YswQ6TeSnAJcswZ286HQHqu+ XfOgXYNDK2nq197jb9WTbT20ojyB2p1MEmfu/OVRXV3PKewzNU6vNWvHy4FoJuZ5lloeBU 9gNtcOB+eL/AjoBD1OK5xpRyFkVYsVk= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=tIQiWb1Z; spf=pass (imf21.hostedemail.com: domain of 3VY1tZwYKCGgYaXKTHMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--surenb.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3VY1tZwYKCGgYaXKTHMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2163dc0f689so128749245ad.1 for ; Thu, 26 Dec 2024 09:07:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1735232854; x=1735837654; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=itcT003jdnlfYy/z4AAxsZgzhf2fL1y4OTxlsN9l2Ik=; b=tIQiWb1ZV7ZgZmaRB67F+UGMA9eJYjM2AAhw1fpJiU0Ql7kFebDT71Aek+UnbpbCoE bLuDnErN2DbvtgPI/46qhJH5973U3na85YOiBKzf4WNupfbUYXBqkrSDGdwLz/h04Jcr JyGZOtOSZuuc4Bgx1NsMSaqPXKJM1bokeez/87So1iE5WmUNvtqTQdaKwgtq5E8NfChe KcTNSH62SMp2Yr0wl2oWUrXcGF7Eqt4awxTKVWQarvO6GerA0aTygQJzu/73akrBoXfY 3IwROE7MprbP18jakfnQPfRVDdlXcVo2vuPG9n3Gv5tfnMvE4xb/0QZDz/TdwTdC4sPG kKeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735232854; x=1735837654; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=itcT003jdnlfYy/z4AAxsZgzhf2fL1y4OTxlsN9l2Ik=; b=VlADTAgpTbdQwIc1UpEHv0ybWYg1J7CZKQUKHOgZlRmNqOChu9magGSYTH8IT43ysl BB/PegKlin4mcUTY5LhMlNxx7tmAiljCB3au0/SBVVNM5jVT+JGiNRAEvciLqDOpEp4f 5QivPkcZyqyF10yq9gFviDT6Cn4bTA3r3Z1xHBUK5m9MwgAyk4P/lU+RroiMw09gjrMi 3SXEUYEeXy//MdQt50vzW44XbeqaC9YqONLv+Uld29U5liyj2jCfu5dyoS1gdXgMCswP 3t/fUvL/MxJhbnUV/d1wHda4DpLediQhabHUT5Rn6/Jtf2PqXvxxsBWtFmUly8wG0Gp4 nB1g== X-Forwarded-Encrypted: i=1; AJvYcCWgUCY7U2eiEeKbbMrs+8zoqODj2I4dnjjo0CBEZXJK2Zkm5pYt0fNneg5u4z99J9D2IQqEvlzd7A==@kvack.org X-Gm-Message-State: AOJu0YzW8Qvp94GNRr8UDRc21X9BxA17lMWZj4HU7oqI7A/EOq+IHRod K+fkqv/yUJoq99Ekauw6MMVlbtL2n9wff8aZVBnapUXXp3zjfnBVHQyLFWCGK9mW+hqp6pgInkX HQg== X-Google-Smtp-Source: AGHT+IFK/ZcPw9QQ8rKy9Iz7tN9PwEqV+omG9KoBZdHVze2si2OKYWeWR6yvQPLae2+NyznTr7bIa67cCm8= X-Received: from pfaq3.prod.google.com ([2002:a05:6a00:a883:b0:725:e76f:1445]) (user=surenb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:4308:b0:1e0:c8d9:3382 with SMTP id adf61e73a8af0-1e5e0847084mr38655491637.45.1735232853953; Thu, 26 Dec 2024 09:07:33 -0800 (PST) Date: Thu, 26 Dec 2024 09:07:02 -0800 In-Reply-To: <20241226170710.1159679-1-surenb@google.com> Mime-Version: 1.0 References: <20241226170710.1159679-1-surenb@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20241226170710.1159679-11-surenb@google.com> Subject: [PATCH v7 10/17] mm: uninline the main body of vma_start_write() From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: peterz@infradead.org, willy@infradead.org, liam.howlett@oracle.com, lorenzo.stoakes@oracle.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mjguzik@gmail.com, oliver.sang@intel.com, mgorman@techsingularity.net, david@redhat.com, peterx@redhat.com, oleg@redhat.com, dave@stgolabs.net, paulmck@kernel.org, brauner@kernel.org, dhowells@redhat.com, hdanton@sina.com, hughd@google.com, lokeshgidra@google.com, minchan@google.com, jannh@google.com, shakeel.butt@linux.dev, souravpanda@google.com, pasha.tatashin@soleen.com, klarasmodin@gmail.com, corbet@lwn.net, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@android.com, surenb@google.com Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: C81FD1C0019 X-Stat-Signature: jm6koumsmeg419buy45a7qn1n8n9h19j X-Rspam-User: X-HE-Tag: 1735232774-78787 X-HE-Meta: U2FsdGVkX1+Uk4b897b6RtrGxEGnEQuhnMlTonnWqeLbzlq/5huz+HgddC4ZcrU5hJ3mhMjW44Cq4nq2ptixRnaF/7o+mDJ7JC+Y4XB2oep/Qozah6gXPSrxNjzot0XuEWMVJUndYiYpaXscoFVhzDG6vdS1cyRxdrzHJFXsp94OiwEu72CXN2udOtP8DOO0M90M1fMV/iGRRr+t1OYmu/e8Snnjk3byR3wM3+h7Xl2Q6rZsXK4o6EGEPaWtTCK+8+T8bq6Ww1onKAJFgJ4MQGjgv7pRhC9TKyVMuY8zquGiDWDKjQc9JciyCUXus4K5Gjgr8PoO5WuP7jEyybwZPcjFluwkOqLM79i7TiqCcMz3ghMCIDqS7I5nF8UWPjZQpdt1Fu0A2RWT4hRGk3ARHtH45M1s3lejMb7lTyBdFThH/kk04tpH6ixPuXv42HT+cZ6YFjnygLooKnI4TJmXtlRRfKHr3QTKgMsPJlSVJs2L45C3CpPHEKDnDgIFptXLV2MJxClvDNEv6zc4tLZwjnjhNhS3sSgd0fkP6l3yKgU4YKqpZIa4NUHOm/HXeb6e2uDWuQLCzCk12w0s9sDhtudhYXbPR7toDnQBlsJHgKyImtrfPd0sJtVCh/gu57ev620ihA71Nb8pD8nSdWDaflmLXtoW4+T663/O4KSJAfV36Ql+mcmLyq/X4T92RswfzDwlH/TzanJgSIT2vzjcFg0jp0l2NJvUe8NMzFV6FPlxhYBYoIgeLi3IN1qDxtEgdc5CcvzuvkFYanzbcpzmisyxf/FJFbjFxqCDZ/BHRtbQ3/FKn57ZPGONF6vkB5eN4/7CcEc9tDj2Z4ZlAWjPvnxTcliGvutItiJteAve96E205aUhdImoRLCpnHLLx91yMwA4cdy0/KRit3Omfi7vvaQJH8CAGgKVloUNOEVskzfowAowJIpkrV9cnF93Cyj7Tdv8rMMriMkloV2yao 973f94xn 2GiHDJAAisOfGWu5N6fbF4/X0eC+5rWWQL5O8dN0V4+nE83VxltsSHi94zFQUQRrKvjlFJCPnI9le7TjnFGz7tq3FX6E2U6V5cn4uwrhEMDENdEV79g9CJTs8wb/+dXqQaR7gdaBT/q6h0RgVKBWc28/xG0EIbf+zFoAzTOrn4i9hmtVK50EtUhS0W58BxllZicJfXjmpleJKLXKwX8YiXz8doQdbLMnaizNgY7v7UcT3iHSv/Fi2ZNcUXM3GS788BxKVjjxKyXjdA4OJuP9JO0o24BhHOidl1Ysg/AJiUSpSwThMAYb1twrfnJ/yIINJLmTbSp6n2QoIYoeUc8cko9J159bVQL/MVUXcfSzgOlOfRcpVctjBd2guTsMrKnOzkG8vY3OOp82OodM+QgcvVkTyy3bi0EB/0vhwJcRSqrrzhXAqNaHD+tPc5rNXvVYg3sbAMe1g6+4g/5oUezOc8lIVx3jRhMhx1MSscs0O+aYiY5JtLkoNFHfNcvzmgGC9J4znqKo+G8iEW+IvgNR7bLGrmtduviUUeSgPVTo7F5XePKGKGhhHMea3Ultl5AWjqoTjkqIpSIz/jkovQtcTVXPXXg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000075, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: vma_start_write() is used in many places and will grow in size very soon. It is not used in performance critical paths and uninlining it should limit the future code size growth. No functional changes. Signed-off-by: Suren Baghdasaryan --- include/linux/mm.h | 12 +++--------- mm/memory.c | 14 ++++++++++++++ 2 files changed, 17 insertions(+), 9 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index ab27de9729d8..ea4c4228b125 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -787,6 +787,8 @@ static bool __is_vma_write_locked(struct vm_area_struct *vma, unsigned int *mm_l return (vma->vm_lock_seq == *mm_lock_seq); } +void __vma_start_write(struct vm_area_struct *vma, unsigned int mm_lock_seq); + /* * Begin writing to a VMA. * Exclude concurrent readers under the per-VMA lock until the currently @@ -799,15 +801,7 @@ static inline void vma_start_write(struct vm_area_struct *vma) if (__is_vma_write_locked(vma, &mm_lock_seq)) return; - down_write(&vma->vm_lock.lock); - /* - * We should use WRITE_ONCE() here because we can have concurrent reads - * from the early lockless pessimistic check in vma_start_read(). - * We don't really care about the correctness of that early check, but - * we should use WRITE_ONCE() for cleanliness and to keep KCSAN happy. - */ - WRITE_ONCE(vma->vm_lock_seq, mm_lock_seq); - up_write(&vma->vm_lock.lock); + __vma_start_write(vma, mm_lock_seq); } static inline void vma_assert_write_locked(struct vm_area_struct *vma) diff --git a/mm/memory.c b/mm/memory.c index d0dee2282325..236fdecd44d6 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -6328,6 +6328,20 @@ struct vm_area_struct *lock_mm_and_find_vma(struct mm_struct *mm, #endif #ifdef CONFIG_PER_VMA_LOCK +void __vma_start_write(struct vm_area_struct *vma, unsigned int mm_lock_seq) +{ + down_write(&vma->vm_lock.lock); + /* + * We should use WRITE_ONCE() here because we can have concurrent reads + * from the early lockless pessimistic check in vma_start_read(). + * We don't really care about the correctness of that early check, but + * we should use WRITE_ONCE() for cleanliness and to keep KCSAN happy. + */ + WRITE_ONCE(vma->vm_lock_seq, mm_lock_seq); + up_write(&vma->vm_lock.lock); +} +EXPORT_SYMBOL_GPL(__vma_start_write); + /* * Lookup and lock a VMA under RCU protection. Returned VMA is guaranteed to be * stable and not isolated. If the VMA is not found or is being modified the -- 2.47.1.613.gc27f4b7a9f-goog