From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D7ABC54EED for ; Wed, 25 Jan 2023 09:42:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 745246B0072; Wed, 25 Jan 2023 04:42:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 712076B0075; Wed, 25 Jan 2023 04:42:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 516276B0073; Wed, 25 Jan 2023 04:42:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 37D806B0072 for ; Wed, 25 Jan 2023 04:42:35 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 0193840C5C for ; Wed, 25 Jan 2023 09:42:34 +0000 (UTC) X-FDA: 80392831470.29.556DCEF Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf11.hostedemail.com (Postfix) with ESMTP id A184E40002; Wed, 25 Jan 2023 09:42:32 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=clWvQxTx; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf11.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674639753; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wTTa+i+nU+PdO+h/KcQj1hMAey36ZrEColPN1KlfYvE=; b=OPyG8KCAjDeZEkE58becx/LGzAMziUUGuW/Neb4HLg/VMVyHiU4T7crw/kE2hlqvdjYXo7 2zHcXBiYTgSrpVJ5DJqVAL2jwkHvHenuXRZC+Zes+HOi9sz+/43vb0gxA/kU7M9lX8rGSc BJEoiUaag1qE7RKJLXMIhLtz/Uz/HJI= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=clWvQxTx; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf11.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674639753; a=rsa-sha256; cv=none; b=Q/NUD61lYww0cvnAO2nJftYWt5YMMJI0An+KPWBngmpT0zDIsXDvtRUF8URcAz22ANCPYQ dprDJbaZRzcXQj3A30Ane6JcNXIYhA4mDXoo0WehD1SyCp/A3ENRyaCYhpOm+PUsS+FTMq qbzpoiqvMu68FLFvHYuKMTP56KNro48= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 4168D21C82; Wed, 25 Jan 2023 09:42:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1674639751; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=wTTa+i+nU+PdO+h/KcQj1hMAey36ZrEColPN1KlfYvE=; b=clWvQxTxYrcNsHq7lTW0Gmn0i9h9Vid00Q4BuOr2XE/g1EjsCUMp3x1tEVjyMZZDFxjZ7J MRDv0FaWEHjjbLpg/9bXRPSZ0dzlfrRGaR9q7sau82jdMWVZcIKDfvaIB1lskwM2rm1evI GWHFNFcuhlRvYoauPeDnqXLmG4Hflh8= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id EA4FB1358F; Wed, 25 Jan 2023 09:42:30 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id yLixOIb50GN6IgAAMHmgww (envelope-from ); Wed, 25 Jan 2023 09:42:30 +0000 Date: Wed, 25 Jan 2023 10:42:30 +0100 From: Michal Hocko To: Suren Baghdasaryan Cc: akpm@linux-foundation.org, michel@lespinasse.org, jglisse@google.com, vbabka@suse.cz, hannes@cmpxchg.org, mgorman@techsingularity.net, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, peterz@infradead.org, ldufour@linux.ibm.com, paulmck@kernel.org, luto@kernel.org, songliubraving@fb.com, peterx@redhat.com, david@redhat.com, dhowells@redhat.com, hughd@google.com, bigeasy@linutronix.de, kent.overstreet@linux.dev, punit.agrawal@bytedance.com, lstoakes@gmail.com, peterjung1337@gmail.com, rientjes@google.com, axelrasmussen@google.com, joelaf@google.com, minchan@google.com, jannh@google.com, shakeelb@google.com, tatashin@google.com, edumazet@google.com, gthelen@google.com, gurua@google.com, arjunroy@google.com, soheil@google.com, hughlynch@google.com, leewalsh@google.com, posk@google.com, will@kernel.org, aneesh.kumar@linux.ibm.com, npiggin@gmail.com, chenhuacai@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, richard@nod.at, anton.ivanov@cambridgegreys.com, johannes@sipsolutions.net, qianweili@huawei.com, wangzhou1@hisilicon.com, herbert@gondor.apana.org.au, davem@davemloft.net, vkoul@kernel.org, airlied@gmail.com, daniel@ffwll.ch, maarten.lankhorst@linux.intel.com, mripard@kernel.org, tzimmermann@suse.de, l.stach@pengutronix.de, krzysztof.kozlowski@linaro.org, patrik.r.jakobsson@gmail.com, matthias.bgg@gmail.com, robdclark@gmail.com, quic_abhinavk@quicinc.com, dmitry.baryshkov@linaro.org, tomba@kernel.org, hjc@rock-chips.com, heiko@sntech.de, ray.huang@amd.com, kraxel@redhat.com, sre@kernel.org, mcoquelin.stm32@gmail.com, alexandre.torgue@foss.st.com, tfiga@chromium.org, m.szyprowski@samsung.com, mchehab@kernel.org, dimitri.sivanich@hpe.com, zhangfei.gao@linaro.org, jejb@linux.ibm.com, martin.petersen@oracle.com, dgilbert@interlog.com, hdegoede@redhat.com, mst@redhat.com, jasowang@redhat.com, alex.williamson@redhat.com, deller@gmx.de, jayalk@intworks.biz, viro@zeniv.linux.org.uk, nico@fluxnic.net, xiang@kernel.org, chao@kernel.org, tytso@mit.edu, adilger.kernel@dilger.ca, miklos@szeredi.hu, mike.kravetz@oracle.com, muchun.song@linux.dev, bhe@redhat.com, andrii@kernel.org, yoshfuji@linux-ipv6.org, dsahern@kernel.org, kuba@kernel.org, pabeni@redhat.com, perex@perex.cz, tiwai@suse.com, haojian.zhuang@gmail.com, robert.jarzmik@free.fr, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, x86@kernel.org, linux-kernel@vger.kernel.org, linux-graphics-maintainer@vmware.com, linux-ia64@vger.kernel.org, linux-arch@vger.kernel.org, loongarch@lists.linux.dev, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-sgx@vger.kernel.org, linux-um@lists.infradead.org, linux-acpi@vger.kernel.org, linux-crypto@vger.kernel.org, nvdimm@lists.linux.dev, dmaengine@vger.kernel.org, amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, etnaviv@lists.freedesktop.org, linux-samsung-soc@vger.kernel.org, intel-gfx@lists.freedesktop.org, linux-mediatek@lists.infradead.org, linux-arm-msm@vger.kernel.org, freedreno@lists.freedesktop.org, linux-rockchip@lists.infradead.org, linux-tegra@vger.kernel.org, virtualization@lists.linux-foundation.org, xen-devel@lists.xenproject.org, linux-stm32@st-md-mailman.stormreply.com, linux-rdma@vger.kernel.org, linux-media@vger.kernel.org, linux-accelerators@lists.ozlabs.org, sparclinux@vger.kernel.org, linux-scsi@vger.kernel.org, linux-staging@lists.linux.dev, target-devel@vger.kernel.org, linux-usb@vger.kernel.org, netdev@vger.kernel.org, linux-fbdev@vger.kernel.org, linux-aio@kvack.org, linux-fsdevel@vger.kernel.org, linux-erofs@lists.ozlabs.org, linux-ext4@vger.kernel.org, devel@lists.orangefs.org, kexec@lists.infradead.org, linux-xfs@vger.kernel.org, bpf@vger.kernel.org, linux-perf-users@vger.kernel.org, kasan-dev@googlegroups.com, selinux@vger.kernel.org, alsa-devel@alsa-project.org, kernel-team@android.com Subject: Re: [PATCH v2 5/6] mm: introduce mod_vm_flags_nolock and use it in untrack_pfn Message-ID: References: <20230125083851.27759-1-surenb@google.com> <20230125083851.27759-6-surenb@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230125083851.27759-6-surenb@google.com> X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: A184E40002 X-Stat-Signature: w4si1hh9zgckzgr383j6fyedkyyzcn6m X-HE-Tag: 1674639752-156280 X-HE-Meta: U2FsdGVkX19MCHPJYjeX1Yv4sJVtFXfoknUUj20Ztbaox+5Mf0vcGN8x8qIVbCfBhDat/xv8a+gFgpGufHGTlgg8uSqCzhXqoGo1gusN5Nt8yFHx85j1ZIzvF3dkh4+ZGYok6zcdsLPnJj6jhb7L8Fb7NwD+xYaX1QLrH88QK4SAf9jk5FdsBowYAcrR2OpUKKeMFkjz7ZmH2GxhN46qtXdTdMWbbwhjAgqIQpDnhTesCA8mnCWjhbjgF+rwPb3zadeeLawiUhnxZv4zQaLZpthWLCtmZfDE4gP9lMInGKN5zMrNGqb0QOrf3TtOPnLfgWZyaVMQMOVSKcs926idedWvoAGaf/bSF5IHzyoP0zhSJ4Asu3tuDPscaQr1sWhv6HaJt26MiaY9m5usmwoGE6GLzviAYP12PX2y268vMmdaM46dTWT+Lt8J9VWMxzgsDtObqKUinLNWNHzzml5WGKhERfQ80J033Oje3562HaibQ8MXVY9Mz2j8+F3053ZZA+LjMQrE5aonJTgTJXoJjS80jtOFb0MXBCnOA+4koOBWNhC90CVd+S0jU4YUlb2yRXZVMn9pDE2GI2vn0piZrA1HGboViTPOPKD4xaQ+m97EWbutBxpISBWEeVZpswjtBylcfcES66n9y786mrRPEF2euvk0ITGQaz6/9+d9xzwSo7xIcEgeFR5G4asn5wjThaNEzxKTaZgilqVQEGUpZyXswfZBRGEVe88bRA2kLVtfEPjnFj2zwwiQNl6YR2+/+k/uHkbLtn1h/omKqbNX1H4J6uzwpoXbx+M5SIsGh0iz6NYfXc3WdmEx6+EMVPViGX6AuaDFl6ge/uH1mN1FZwbHIRUEFgRl8F5X0LLdDlvNzSCiDVcHpQ3uC186FszSDPzUpKhqbdf0vqlZRPhq93uXPstuf7DzdBciJKpZ6qMB27qbK81Ej1lweZz5YEdyjjspjOLnTz5rXPd0bJ8 qnFNbazp TDXad8VnpWIDpZ1SRhUGcD6B9bHktdjRrfALJPzESV50tkimOpNkws6kvCnVgk9UqzgGoLy0t7Yg3bgTO1ODl2H0mWWSb28brd+pZvaiudqERTGCS5mrc1XeOx3QqNFpAVioP0n0/mrSF6HGz3StKCn1wA90jxuwdeAOL/xWFqlyGb/bXxE5JMtAX6n+E+yEivd80bzqcT+cTYktCuJr65FuJ1CWDFNnTUDDgAvj1ko6uo394hTq9zd7BFauH6cXYKcA+4goufhRz8Ln8M1jybFQF/qS55l3N4epbdvx3n5ZXX3OLp7pYcFx5JuitHBVT/I2e400E2Dj4jxeQ5bAxDN+DaHf2PH8nPXM6n7gMi2dOCUcHmg51dzIfgvvVD+cxZ5Pk2inGQFQEwyvXtd225+8/Bqs7YRhhJYFOkuXazJsx4mc327zbUa8HERzRxIhucc6y1pCpFfEndpNWeNvyRqpgo8KW3mnqdGJ9zpmjmD2eqV9rk9ln42ST4R8w6P41x1Q2wGcug3/Ydl64b9oNrONi9txMk3tdJgvqdjI+qwKke6cl4rVqZygSY/ZYBQ0i0Dv9dYJ19Bej3wkZC1uH3Pmtx7vkjenDn1fVpy2mzxebI0QTIh24JW7DAHc1YzgEtUZLQKpeKkUBJE0iOoeXijko7c9rGFNiQt184cA4voZr7Ed4LCfmMA4+bQ+n60dFWv974TZxA0i7sMoSZHH/LzL0k5H5lGQVV/tFbAIHdg4dHJ+VPFn/mNbPIbX+VYoNhT4XWU1XL7kwm9RYMsNRLU/2A5HKTfFxbx5XoDDHRN5JC9aJGyyafa/KY3np8xSHVkT7CbS59zTPhHZSQx/aE+37VIZL8cH1rooD3FYBVscgWUpnViDYPWUVDKMJUMetIUWehXO70NBd7hWTWQvNHFZtPJAGJA8yz3b5O8uJPe/DUa/pB/4oh9xwwPEp91f7S1NlPE+ZSEBn05tNqR/cje2IHdbV gLbIeGkb 5nRVKBTHkUpy0stsmaG7jk6DrJrx07OBa7/YVS78QlNnztDTQZOM/JAlwPhuBuM+vqdWa8Y+RPSIuKZJiw3dd1sLGP7zsF1Ri3s4oW9DghXLSbSRk52D2p62dw/hKTwFpEYiqFCgy+BxGwaOKUKvha7OczuhqWCa5HdOX3FzF8l/j1+iHLcvdxlN+B6u8YwCyPP5xgXR/2MgLF2MiB8bhkliQ/19teV1/EjUhumBlT+bLP4f2We5JTlVYISRAXFJCGLcIqE6+muN+uUUMcO3lT8ksIWtEBn8dDffMtE3IqPDTpL9R1c9bFzZL5tB0EHROP6P7U0Mh1CIxFMUr3tDhqyvKJOuSEQ9oPEB7kC5ipDVPHrmUmHsntFq6RuTl+DyRjZQB8tRRmutiHkM0O/Ctc4LQYQIre/cEA0GnhCsWTF0uLrVaPUwcOXjYX5o4kdd+4VL1d7Sl3J9+r56zomyMvNI2ozt4AG1C6K/iaaFmMCdII66wCOAX4w09qO6hx4tFqYkimc/eBLcXEnmBpdVB2INCMoX9ZYfGxNmMdb3R11wOmTnJN6HJ6P1iV2i4OZ7i+x+SAt6kjA52/mhXSA0VkEAMlvus19/HdI3QC0ixvLHGw6zlkClDVQmOiYIuscTHhjVw0uXdovjtoXCUfej8UN/xmRCXY6DjQ3BayUun5O0rw/pMrkBf6yuJ3T11uI+8YUAw1y/MeN4PwJWnHpbmbVC7aKhJ4TuM+TDs7fUelLfZok8p/zUsmGUoaz7nBrTRL47eQv+9kpE5ogrhsh5vpWTkBwUze5iMiQ/QAv6oYJlkMoi1XQbG8EkW+iLYNITNyaa3HkeToKrSF9cdJqc6EAZgoklc04cDMe7aLV6NduVkqFWLt+tY6NGRgNNe467wW/fZVO8hUS02ZwvNXcNjUvZ2+vyOyumYKdW43JbCW/djC+DEIUuXabqKLC6SJKOdm1W/3Lqr5sN8nmYvd+j35AzkktWK sz6MPckq PyiQ4tXM3/HzqoK1URU1zN123Ya3ujoo5e2D6TcrE1kZLm0UnrnZd1aQMFC6elks+4VnPZylNMIbzfFPZtTbmREhqDjEUSDaCkdWDBCh6BLXj8Mwn6oGghrKPSiqfFPD6R1wAzMa1MHZN8c0kVNA6JjF1pj9cNvDdzqKyH7If3V5U2Sg8fcgBfiBolhV/NQOKPfT5hYhezDZWJlM5a8NXc+xq2rrpdxsDEFXi8b8/IpuN74fmdBF1IMZjDQ9stV/auKch0Nn7WfO9280nzafacVelv1HddwV+svusJpFDZ4+fqkPpzzJ0A+yA/LkxqlT8yW9YUmujdlu/NHQsvujjZURNXd6XgtKl1pY9fQTo2xspDgXoeLW4ASikC0VkppKgAPS56JamHZjIZeGBsAfzQXbXxS9guJXNlMt1wh3u5QYhcqIkNErV/QC8nbpyHvb22JBex0V2195DF1OtgCZozEkm5SnxKYpa6htXgIQ3XeEbPJ1xCNBNronOX5D0lfEBPSss427v+xACK5YZwpJiiUzL2l7JMIwgg3AOn4q6/h6P3E8nTWaf0uCfQIIi4HijJHAHNLzSeO+DBAT8JxL2+/5Uz/D5RvfIHe1Qf1Cz0DNaLBDnU/9bz7kJxcnknIGYmcQxZiNXKs752+Ntafyv/mbTY4otEzfX8F/7zitkfBW7UlOWZwE9LweHIFA5X0jUyN1Cu6QPI4z6Y2p6M859GaOAP2vt3jBOn3YHcJhxCw46xbIy/UL2Q00GlTr1kziZthiyQHybhjSGXPuhTN9lxZawKTPt4HCtf/RPBEeKnnJdaaV56Yxeo5Vn6CdTjF2qN8Kq1K8tmmBmGH9SG5ovtOzJlm3FnLpp7U9Ekmz5a5o+gNT2dN/0sIuGUslLMAz+fbX5NlUGonOp5zWmeyiGZuTZ/L+uv0YzlML2iP/PTWYfYHWXmCj9xCdUECwx18vaHiRHDAVhpKbZ1GINPfJJCS7tb/Gy 8YP4XBDW l/h1MeOB0wG+M8HsXn16eSs2P9XXAmVLOAH9HT5j9qPk/Mr7+sv0XJ04f9R42DWwZol8eS2MO5rhGesspYlOW5g1l27grdOeJz9V4OkKe3XmScmBwOOkOxTDnEdRs0ev6V+qcRWqctUJSwEN1smgGhrgK9WfnySisQg5M7pq/KO4aQyonDQY6rWJDfO1nYC6Gqh7lxgpOsKoVyN9VJswvTAHYwTMyI+4qEwy3ERbTWmwEBuQDNpt2w3wrmJ8s8/SY4t1wHzIhQjfj61/ZwwtYRalFX8k4IuJTbSXAYplQQxvEAVNbUdmI1WrBIrFxibEWr5bA49FAXXQ9bwrH3LHIgjLabAzj6fkdfg4fHl2E4JPeVp8hJyTkhZYdlSNamAiMdKS68ddntpIy6mBWOiJTb2bGHWXsG/ag= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed 25-01-23 00:38:50, Suren Baghdasaryan wrote: > In cases when VMA flags are modified after VMA was isolated and mmap_lock > was downgraded, flags modifications would result in an assertion because > mmap write lock is not held. > Introduce mod_vm_flags_nolock to be used in such situation. > Pass a hint to untrack_pfn to conditionally use mod_vm_flags_nolock for > flags modification and to avoid assertion. The changelog nor the documentation of mod_vm_flags_nolock really explain when it is safe to use it. This is really important for future potential users. > Signed-off-by: Suren Baghdasaryan > --- > arch/x86/mm/pat/memtype.c | 10 +++++++--- > include/linux/mm.h | 12 +++++++++--- > include/linux/pgtable.h | 5 +++-- > mm/memory.c | 13 +++++++------ > mm/memremap.c | 4 ++-- > mm/mmap.c | 16 ++++++++++------ > 6 files changed, 38 insertions(+), 22 deletions(-) > > diff --git a/arch/x86/mm/pat/memtype.c b/arch/x86/mm/pat/memtype.c > index ae9645c900fa..d8adc0b42cf2 100644 > --- a/arch/x86/mm/pat/memtype.c > +++ b/arch/x86/mm/pat/memtype.c > @@ -1046,7 +1046,7 @@ void track_pfn_insert(struct vm_area_struct *vma, pgprot_t *prot, pfn_t pfn) > * can be for the entire vma (in which case pfn, size are zero). > */ > void untrack_pfn(struct vm_area_struct *vma, unsigned long pfn, > - unsigned long size) > + unsigned long size, bool mm_wr_locked) > { > resource_size_t paddr; > unsigned long prot; > @@ -1065,8 +1065,12 @@ void untrack_pfn(struct vm_area_struct *vma, unsigned long pfn, > size = vma->vm_end - vma->vm_start; > } > free_pfn_range(paddr, size); > - if (vma) > - clear_vm_flags(vma, VM_PAT); > + if (vma) { > + if (mm_wr_locked) > + clear_vm_flags(vma, VM_PAT); > + else > + mod_vm_flags_nolock(vma, 0, VM_PAT); > + } > } > > /* > diff --git a/include/linux/mm.h b/include/linux/mm.h > index 55335edd1373..48d49930c411 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -656,12 +656,18 @@ static inline void clear_vm_flags(struct vm_area_struct *vma, > vma->vm_flags &= ~flags; > } > > +static inline void mod_vm_flags_nolock(struct vm_area_struct *vma, > + unsigned long set, unsigned long clear) > +{ > + vma->vm_flags |= set; > + vma->vm_flags &= ~clear; > +} > + > static inline void mod_vm_flags(struct vm_area_struct *vma, > unsigned long set, unsigned long clear) > { > mmap_assert_write_locked(vma->vm_mm); > - vma->vm_flags |= set; > - vma->vm_flags &= ~clear; > + mod_vm_flags_nolock(vma, set, clear); > } > > static inline void vma_set_anonymous(struct vm_area_struct *vma) > @@ -2087,7 +2093,7 @@ static inline void zap_vma_pages(struct vm_area_struct *vma) > } > void unmap_vmas(struct mmu_gather *tlb, struct maple_tree *mt, > struct vm_area_struct *start_vma, unsigned long start, > - unsigned long end); > + unsigned long end, bool mm_wr_locked); > > struct mmu_notifier_range; > > diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h > index 5fd45454c073..c63cd44777ec 100644 > --- a/include/linux/pgtable.h > +++ b/include/linux/pgtable.h > @@ -1185,7 +1185,8 @@ static inline int track_pfn_copy(struct vm_area_struct *vma) > * can be for the entire vma (in which case pfn, size are zero). > */ > static inline void untrack_pfn(struct vm_area_struct *vma, > - unsigned long pfn, unsigned long size) > + unsigned long pfn, unsigned long size, > + bool mm_wr_locked) > { > } > > @@ -1203,7 +1204,7 @@ extern void track_pfn_insert(struct vm_area_struct *vma, pgprot_t *prot, > pfn_t pfn); > extern int track_pfn_copy(struct vm_area_struct *vma); > extern void untrack_pfn(struct vm_area_struct *vma, unsigned long pfn, > - unsigned long size); > + unsigned long size, bool mm_wr_locked); > extern void untrack_pfn_moved(struct vm_area_struct *vma); > #endif > > diff --git a/mm/memory.c b/mm/memory.c > index d6902065e558..5b11b50e2c4a 100644 > --- a/mm/memory.c > +++ b/mm/memory.c > @@ -1613,7 +1613,7 @@ void unmap_page_range(struct mmu_gather *tlb, > static void unmap_single_vma(struct mmu_gather *tlb, > struct vm_area_struct *vma, unsigned long start_addr, > unsigned long end_addr, > - struct zap_details *details) > + struct zap_details *details, bool mm_wr_locked) > { > unsigned long start = max(vma->vm_start, start_addr); > unsigned long end; > @@ -1628,7 +1628,7 @@ static void unmap_single_vma(struct mmu_gather *tlb, > uprobe_munmap(vma, start, end); > > if (unlikely(vma->vm_flags & VM_PFNMAP)) > - untrack_pfn(vma, 0, 0); > + untrack_pfn(vma, 0, 0, mm_wr_locked); > > if (start != end) { > if (unlikely(is_vm_hugetlb_page(vma))) { > @@ -1675,7 +1675,7 @@ static void unmap_single_vma(struct mmu_gather *tlb, > */ > void unmap_vmas(struct mmu_gather *tlb, struct maple_tree *mt, > struct vm_area_struct *vma, unsigned long start_addr, > - unsigned long end_addr) > + unsigned long end_addr, bool mm_wr_locked) > { > struct mmu_notifier_range range; > struct zap_details details = { > @@ -1689,7 +1689,8 @@ void unmap_vmas(struct mmu_gather *tlb, struct maple_tree *mt, > start_addr, end_addr); > mmu_notifier_invalidate_range_start(&range); > do { > - unmap_single_vma(tlb, vma, start_addr, end_addr, &details); > + unmap_single_vma(tlb, vma, start_addr, end_addr, &details, > + mm_wr_locked); > } while ((vma = mas_find(&mas, end_addr - 1)) != NULL); > mmu_notifier_invalidate_range_end(&range); > } > @@ -1723,7 +1724,7 @@ void zap_page_range_single(struct vm_area_struct *vma, unsigned long address, > * unmap 'address-end' not 'range.start-range.end' as range > * could have been expanded for hugetlb pmd sharing. > */ > - unmap_single_vma(&tlb, vma, address, end, details); > + unmap_single_vma(&tlb, vma, address, end, details, false); > mmu_notifier_invalidate_range_end(&range); > tlb_finish_mmu(&tlb); > } > @@ -2492,7 +2493,7 @@ int remap_pfn_range(struct vm_area_struct *vma, unsigned long addr, > > err = remap_pfn_range_notrack(vma, addr, pfn, size, prot); > if (err) > - untrack_pfn(vma, pfn, PAGE_ALIGN(size)); > + untrack_pfn(vma, pfn, PAGE_ALIGN(size), true); > return err; > } > EXPORT_SYMBOL(remap_pfn_range); > diff --git a/mm/memremap.c b/mm/memremap.c > index 08cbf54fe037..2f88f43d4a01 100644 > --- a/mm/memremap.c > +++ b/mm/memremap.c > @@ -129,7 +129,7 @@ static void pageunmap_range(struct dev_pagemap *pgmap, int range_id) > } > mem_hotplug_done(); > > - untrack_pfn(NULL, PHYS_PFN(range->start), range_len(range)); > + untrack_pfn(NULL, PHYS_PFN(range->start), range_len(range), true); > pgmap_array_delete(range); > } > > @@ -276,7 +276,7 @@ static int pagemap_range(struct dev_pagemap *pgmap, struct mhp_params *params, > if (!is_private) > kasan_remove_zero_shadow(__va(range->start), range_len(range)); > err_kasan: > - untrack_pfn(NULL, PHYS_PFN(range->start), range_len(range)); > + untrack_pfn(NULL, PHYS_PFN(range->start), range_len(range), true); > err_pfn_remap: > pgmap_array_delete(range); > return error; > diff --git a/mm/mmap.c b/mm/mmap.c > index 2c6e9072e6a8..69d440997648 100644 > --- a/mm/mmap.c > +++ b/mm/mmap.c > @@ -78,7 +78,7 @@ core_param(ignore_rlimit_data, ignore_rlimit_data, bool, 0644); > static void unmap_region(struct mm_struct *mm, struct maple_tree *mt, > struct vm_area_struct *vma, struct vm_area_struct *prev, > struct vm_area_struct *next, unsigned long start, > - unsigned long end); > + unsigned long end, bool mm_wr_locked); > > static pgprot_t vm_pgprot_modify(pgprot_t oldprot, unsigned long vm_flags) > { > @@ -2136,14 +2136,14 @@ static inline void remove_mt(struct mm_struct *mm, struct ma_state *mas) > static void unmap_region(struct mm_struct *mm, struct maple_tree *mt, > struct vm_area_struct *vma, struct vm_area_struct *prev, > struct vm_area_struct *next, > - unsigned long start, unsigned long end) > + unsigned long start, unsigned long end, bool mm_wr_locked) > { > struct mmu_gather tlb; > > lru_add_drain(); > tlb_gather_mmu(&tlb, mm); > update_hiwater_rss(mm); > - unmap_vmas(&tlb, mt, vma, start, end); > + unmap_vmas(&tlb, mt, vma, start, end, mm_wr_locked); > free_pgtables(&tlb, mt, vma, prev ? prev->vm_end : FIRST_USER_ADDRESS, > next ? next->vm_start : USER_PGTABLES_CEILING); > tlb_finish_mmu(&tlb); > @@ -2391,7 +2391,11 @@ do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, > mmap_write_downgrade(mm); > } > > - unmap_region(mm, &mt_detach, vma, prev, next, start, end); > + /* > + * We can free page tables without write-locking mmap_lock because VMAs > + * were isolated before we downgraded mmap_lock. > + */ > + unmap_region(mm, &mt_detach, vma, prev, next, start, end, !downgrade); > /* Statistics and freeing VMAs */ > mas_set(&mas_detach, start); > remove_mt(mm, &mas_detach); > @@ -2704,7 +2708,7 @@ unsigned long mmap_region(struct file *file, unsigned long addr, > > /* Undo any partial mapping done by a device driver. */ > unmap_region(mm, &mm->mm_mt, vma, prev, next, vma->vm_start, > - vma->vm_end); > + vma->vm_end, true); > } > if (file && (vm_flags & VM_SHARED)) > mapping_unmap_writable(file->f_mapping); > @@ -3031,7 +3035,7 @@ void exit_mmap(struct mm_struct *mm) > tlb_gather_mmu_fullmm(&tlb, mm); > /* update_hiwater_rss(mm) here? but nobody should be looking */ > /* Use ULONG_MAX here to ensure all VMAs in the mm are unmapped */ > - unmap_vmas(&tlb, &mm->mm_mt, vma, 0, ULONG_MAX); > + unmap_vmas(&tlb, &mm->mm_mt, vma, 0, ULONG_MAX, false); > mmap_read_unlock(mm); > > /* > -- > 2.39.1 -- Michal Hocko SUSE Labs