From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40263C71155 for ; Fri, 20 Jun 2025 12:30:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D85DC6B0093; Fri, 20 Jun 2025 08:30:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D0F7F6B0095; Fri, 20 Jun 2025 08:30:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C24986B0096; Fri, 20 Jun 2025 08:30:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B524E6B0093 for ; Fri, 20 Jun 2025 08:30:43 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 74089120AF7 for ; Fri, 20 Jun 2025 12:30:43 +0000 (UTC) X-FDA: 83575712766.26.331FBCA Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf24.hostedemail.com (Postfix) with ESMTP id 5683D18000A for ; Fri, 20 Jun 2025 12:30:41 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=TDzkiczE; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="VF4FnFI/"; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=TDzkiczE; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="VF4FnFI/"; spf=pass (imf24.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1750422641; a=rsa-sha256; cv=none; b=8lUtDEI06l5KvodM5t68fuLf6H4phrAYKcV70pmWo9FxXKF27H22z0ax8pGfBfKYqTSsJu 3pUVPsTRGKRt1BM2NnGz+aU19FEBOjAiN/ubivfGbP0JG5olVzLac5uHqI6JaZL/cgQBuS +fALl+7wUYMAJXOIy91kOyr30KnI5HY= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=TDzkiczE; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="VF4FnFI/"; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=TDzkiczE; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b="VF4FnFI/"; spf=pass (imf24.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1750422641; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6u2+gI+8TX69PvGl2xXRzwHfYerMxuqT3jpTWbxs4io=; b=ujSlTWgW5Q6HjGZizXR8oBuG2jchrP6vzz8CBjRxnn5lBE2S1TUCDCiKbPGtlPvwnSrdhL Z/xoU9jgkSKgL5Wp/UUWi6t7rPBdfR9HNJI/7y0XyCaoTOFF/RkgdgrJb5NuCsNIeu3d6p zS6e6ZBrB4YJDHylF//HtuQ0vsigha4= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id C911221247; Fri, 20 Jun 2025 12:30:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1750422628; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6u2+gI+8TX69PvGl2xXRzwHfYerMxuqT3jpTWbxs4io=; b=TDzkiczET41mkTCZ8P+9YHuEwGuADw2CzQx/o1hn7ns7w7yMTnNbNVgXif+1m/jDTERJqn G3GiLN9f8mKKauSQ8/BTgjwf6NFK0RWMKCBrqm1hDego/dqtEjUK2Veimpi3+uJmnx6gVh qwVUnB4gui8K52rX8iqbhigQoZanmyY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1750422628; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6u2+gI+8TX69PvGl2xXRzwHfYerMxuqT3jpTWbxs4io=; b=VF4FnFI/duTAeAi8flp4qiCCCYV7DqbpWOX4wEkg1rPGe9TofoTw9rJkCllRMjGsTe0Frd ZTD2YeS7XkEqFrBA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1750422628; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6u2+gI+8TX69PvGl2xXRzwHfYerMxuqT3jpTWbxs4io=; b=TDzkiczET41mkTCZ8P+9YHuEwGuADw2CzQx/o1hn7ns7w7yMTnNbNVgXif+1m/jDTERJqn G3GiLN9f8mKKauSQ8/BTgjwf6NFK0RWMKCBrqm1hDego/dqtEjUK2Veimpi3+uJmnx6gVh qwVUnB4gui8K52rX8iqbhigQoZanmyY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1750422628; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6u2+gI+8TX69PvGl2xXRzwHfYerMxuqT3jpTWbxs4io=; b=VF4FnFI/duTAeAi8flp4qiCCCYV7DqbpWOX4wEkg1rPGe9TofoTw9rJkCllRMjGsTe0Frd ZTD2YeS7XkEqFrBA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 549D4136BA; Fri, 20 Jun 2025 12:30:28 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id GLbyEWRUVWjNKAAAD6G6ig (envelope-from ); Fri, 20 Jun 2025 12:30:28 +0000 From: Oscar Salvador To: Andrew Morton Cc: David Hildenbrand , Muchun Song , Peter Xu , Gavin Guo , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [PATCH v2 2/5] mm,hugetlb: Sort out folio locking in the faulting path Date: Fri, 20 Jun 2025 14:30:11 +0200 Message-ID: <20250620123014.29748-3-osalvador@suse.de> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250620123014.29748-1-osalvador@suse.de> References: <20250620123014.29748-1-osalvador@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 5683D18000A X-Stat-Signature: kdy8czanxjxxusauoiczhec89y9pk5b1 X-Rspam-User: X-HE-Tag: 1750422641-810731 X-HE-Meta: U2FsdGVkX19AOWTNaZszY1DxR5/dbG51e+ChKkcO01UU+b0lzJRZNVdvbeeBLNjJV8aLbCm9iI2MjIuDonMa2HzR9I0B9SEG7eQqtJO6KJtv474plmlKcmqSiPuE/g37GlaeKHOt1MnYZBmoduOqeZJX+NX3gNA1FeALAiUH9pitT3L+VGNsTZx82B3ibbThnUrTUe/sO44luSOTgGgSjQrfwyO1cUXNi7YhxxbXEQSUXduAYWe915W4bR2IOhBAPWDVb+0Qi5eYLrEIlvRWSx8vy3eX1st7DFX/2koy3B/IAo10ey/xjw2zh092p2im4FKAzyBR3ktQKcSAv38vXC1VQ1ePxMfSvig0tuDpbmDlU83Ws+LDZSiS9kBdraPhPStYFNjvpfbpCzxkf7ElQ/8+kv4A5L4oKdVbz6t49t+H7eaG3jlzlX9GFV6YqN4Sk+F6evi66PTixTZFLsybG0+iNDVTFfvT657uYgpawWStsfG7woAYtDhINYyQhx8pxjF0arGaOi4Lf8f4igP2UOwdM/OY13MzcOTWbqDSLVNa28LuA4J1ksWHgXgSgTBDJTZ0AYE2r+a3THTSyySBASMjy+omVex2+1jxNqzM1vMBVFwZ2nEGO63LOgYNPCjvsq0fjGk9FlGmyied0gFle0CX6VED35Au+79zdcRE3KPsDBl4jK89P5lGjKU10aVcstLWBTuQb693unm7Q2xyBDVWVkGkhuJ0HxIZzO9jfLFb1pZ7jD8DIv6/ZgJiAUOE8928QIV1axZUF7DkAeWiSzeQ3dJ6pO7SBL/WcNWy49UuLC0ZqLd6TZYmP4EJAe0SaJf8OGlM3TzK8LFgkq6M7SOgfWNLS4eRTko3VJCvbcyq+e+y7Wd4YSRjxoZ4HNud9Jf6X+q5O/ry/X/mOeRSc771sz8Y5E641VaA1YBBEp1LRvjtcXnJZJ/KZ2X/NekDJUNnImaaKk8kozCHDt+ mNkble7y HFhMXzAcrO8Odl5DUAqibtyz/IKpGVPRdbF55TfSJdn4nOybl/v/FcPUW3zvRcOF2zX3VJYLWVr5O1N248kRzdOLXG4tT5ss3n4xLnF10UStsx086beYgA9jBL+iX3uLuyqTXYHPwN6bRi4GDlsG4mjKh/FjqZE24vLPeQykL0h16OAIc5UpefE1fKvB881jOiBtXT6aIIX1PqdOk2tV0XZhX9LRslZYwK7NXNlBbAAckRHxOEeqDLx3IKtJ8mUj2V7E9OC9eATILWSyZ2uVeO+3LHA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Recent conversations showed that there was a misunderstanding about why we were locking the folio prior to call in hugetlb_wp(). In fact, as soon as we have the folio mapped into the pagetables, we no longer need to hold it locked, because we know that no concurrent truncation could have happened. There is only one case where the folio needs to be locked, and that is when we are handling an anonymous folio, because hugetlb_wp() will check whether it can re-use it exclusively for the process that is faulting it in. So, pass the folio locked to hugetlb_wp() when that is the case. Suggested-by: David Hildenbrand Signed-off-by: Oscar Salvador --- mm/hugetlb.c | 43 +++++++++++++++++++++++++++++++++---------- 1 file changed, 33 insertions(+), 10 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 175edafeec67..1a5f713c1e4c 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6437,6 +6437,7 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping, pte_t new_pte; bool new_folio, new_pagecache_folio = false; u32 hash = hugetlb_fault_mutex_hash(mapping, vmf->pgoff); + bool folio_locked = true; /* * Currently, we are forced to kill the process in the event the @@ -6602,6 +6603,11 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping, hugetlb_count_add(pages_per_huge_page(h), mm); if ((vmf->flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED)) { + /* No need to lock file folios. See comment in hugetlb_fault() */ + if (!anon_rmap) { + folio_locked = false; + folio_unlock(folio); + } /* Optimization, do the COW without a second fault */ ret = hugetlb_wp(vmf); } @@ -6616,7 +6622,8 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping, if (new_folio) folio_set_hugetlb_migratable(folio); - folio_unlock(folio); + if (folio_locked) + folio_unlock(folio); out: hugetlb_vma_unlock_read(vma); @@ -6636,7 +6643,8 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping, if (new_folio && !new_pagecache_folio) restore_reserve_on_error(h, vma, vmf->address, folio); - folio_unlock(folio); + if (folio_locked) + folio_unlock(folio); folio_put(folio); goto out; } @@ -6670,7 +6678,7 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, { vm_fault_t ret; u32 hash; - struct folio *folio; + struct folio *folio = NULL; struct hstate *h = hstate_vma(vma); struct address_space *mapping; struct vm_fault vmf = { @@ -6687,6 +6695,7 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, * be hard to debug if called functions make assumptions */ }; + bool folio_locked = false; /* * Serialize hugepage allocation and instantiation, so that we don't @@ -6801,13 +6810,24 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, /* Fallthrough to CoW */ } - /* hugetlb_wp() requires page locks of pte_page(vmf.orig_pte) */ - folio = page_folio(pte_page(vmf.orig_pte)); - folio_lock(folio); - folio_get(folio); - if (flags & (FAULT_FLAG_WRITE|FAULT_FLAG_UNSHARE)) { if (!huge_pte_write(vmf.orig_pte)) { + /* + * Anonymous folios need to be lock since hugetlb_wp() + * checks whether we can re-use the folio exclusively + * for us in case we are the only user of it. + */ + folio = page_folio(pte_page(vmf.orig_pte)); + folio_get(folio); + if (folio_test_anon(folio)) { + spin_unlock(vmf.ptl); + folio_lock(folio); + folio_locked = true; + spin_lock(vmf.ptl); + if (unlikely(!pte_same(vmf.orig_pte, huge_ptep_get(mm, + vmf.address, vmf.pte)))) + goto out_put_page; + } ret = hugetlb_wp(&vmf); goto out_put_page; } else if (likely(flags & FAULT_FLAG_WRITE)) { @@ -6819,8 +6839,11 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, flags & FAULT_FLAG_WRITE)) update_mmu_cache(vma, vmf.address, vmf.pte); out_put_page: - folio_unlock(folio); - folio_put(folio); + if (folio) { + if (folio_locked) + folio_unlock(folio); + folio_put(folio); + } out_ptl: spin_unlock(vmf.ptl); out_mutex: -- 2.50.0