From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10DD5C77B7F for ; Fri, 27 Jun 2025 10:29:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 135EF6B00C0; Fri, 27 Jun 2025 06:29:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0BE8E6B00C2; Fri, 27 Jun 2025 06:29:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EC7648D0006; Fri, 27 Jun 2025 06:29:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id D889E6B00C0 for ; Fri, 27 Jun 2025 06:29:26 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A234410520B for ; Fri, 27 Jun 2025 10:29:26 +0000 (UTC) X-FDA: 83600808732.13.5A10CEE Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf13.hostedemail.com (Postfix) with ESMTP id 7412820007 for ; Fri, 27 Jun 2025 10:29:23 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=qiwJZJ4s; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=dN15uj3R; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=qiwJZJ4s; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=dN15uj3R; spf=pass (imf13.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1751020163; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=evbYQwrIEyoGBUi3yWxrK/P56VGnTwRekMGyPPYLQ0g=; b=XJ2Vk2bdvBstkQcysUOzryyMEjQM5YhsxdnKalcKSM0EkvTSBli3RfMI7j59gydvhF5T4p m1aVqCPzM/26DuX3m4RT1tDpYJYOPr7Cpm+J+F0NAnRCdSbAeMhQdLgURj+CFiLx4e/ahe A9reUxGRmrnufd6E0mxYUwel/grh50A= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1751020163; a=rsa-sha256; cv=none; b=LGKzmAaJvY5XCCnNJHnOVp+4oE4uCX9OqSal2Kr2w15raobSZ24Wp5TC+9bvLfWNTWevah 33nmQMhKZt9Tr24w3WfbJ3EjygP+j6YlZFOZtJQh8joD/PJi1iOW3DoqkvRGJy9xoLbbFt Z5W/bN6CbFpKEBMVvWKkesdHJYdsBDs= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=qiwJZJ4s; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=dN15uj3R; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=qiwJZJ4s; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=dN15uj3R; spf=pass (imf13.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 9FBCB21179; Fri, 27 Jun 2025 10:29:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1751020156; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=evbYQwrIEyoGBUi3yWxrK/P56VGnTwRekMGyPPYLQ0g=; b=qiwJZJ4sWMmcSOenzHf0FWdaqjM06zZLCXLq/ixq5a5Bd4jTDZoeESblyYJXTcwhVmSBHb zgXXzO2c6DYJqkiCM+moSdUjCILUqsWZxDBNeIuTgeu4wTlja3heogQBeuQyOOb3bNgT3n 0RM/slNyD1GJ0BAtRef9h+9WVTcGXk8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1751020156; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=evbYQwrIEyoGBUi3yWxrK/P56VGnTwRekMGyPPYLQ0g=; b=dN15uj3R3/olVSArz5UrBVfaq59DnLPe9+T6lSTGP+9aCrJF18kYtaQwp8HiwrCfGdFd2O N9x1K9XTZNBtNvCA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1751020156; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=evbYQwrIEyoGBUi3yWxrK/P56VGnTwRekMGyPPYLQ0g=; b=qiwJZJ4sWMmcSOenzHf0FWdaqjM06zZLCXLq/ixq5a5Bd4jTDZoeESblyYJXTcwhVmSBHb zgXXzO2c6DYJqkiCM+moSdUjCILUqsWZxDBNeIuTgeu4wTlja3heogQBeuQyOOb3bNgT3n 0RM/slNyD1GJ0BAtRef9h+9WVTcGXk8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1751020156; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=evbYQwrIEyoGBUi3yWxrK/P56VGnTwRekMGyPPYLQ0g=; b=dN15uj3R3/olVSArz5UrBVfaq59DnLPe9+T6lSTGP+9aCrJF18kYtaQwp8HiwrCfGdFd2O N9x1K9XTZNBtNvCA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 26C3F13786; Fri, 27 Jun 2025 10:29:16 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id YLC2BnxyXmgIAgAAD6G6ig (envelope-from ); Fri, 27 Jun 2025 10:29:16 +0000 From: Oscar Salvador To: Andrew Morton Cc: David Hildenbrand , Muchun Song , Peter Xu , Gavin Guo , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [PATCH v3 2/5] mm,hugetlb: Sort out folio locking in the faulting path Date: Fri, 27 Jun 2025 12:29:01 +0200 Message-ID: <20250627102904.107202-3-osalvador@suse.de> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250627102904.107202-1-osalvador@suse.de> References: <20250627102904.107202-1-osalvador@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Action: no action X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 7412820007 X-Stat-Signature: 769ugd1srnmb398ebx8fd4aqnnsdkoqo X-Rspam-User: X-HE-Tag: 1751020163-489134 X-HE-Meta: U2FsdGVkX18po6rpDGrobkrTD0tyoxaLiowVncncsHVrfYx1hmTyLDWeqkbS/OEnh+bYEcayB1FHT2clcYxoioMzI7YPS4AOqHG93QWeZfiSXDdl6Wbs03StQWnts9u48YJNfXF7wfDn0y7XrhKYf+czwsNlP1fHrFJ2ROHYtd0rISzi0VRIA4+rYbxcF/O1IjE7SloN1D2G/3aNU5uBnwh56a+mINV3D/oX4gR4yObvZseEwPWokSd3FGi95WeXsxCtf5ggr/qacaCErepeuYVBA1SmermtB4wd8H+nFuBEin7dWUz7sL5MN8dXV+I5NqMtiMlMouKWjeNaydAoDzmjozByb/tP2sxG4j5P9ISoJTwlBgZb+fMDNVjxk1LCY4XfvKVf04hIcmFEgavjlOLWfLy5tJWf3xRJasPi+Dh9v5CT0oiJp8uhZXn3OsocoBK7Cl6vphPJBctbwPdJbXfo+d0zMF7VLgXaYpHQml5IJvOtg5wnjlt+xvVSAJmb8OyTWBID6hvbuqVWZ1safebD83jMigrbbS9jrlH5RzvViZfOBKB9R1KfITlGSB1DIzJ7c/As30h97s+OXTGRKlu2sClG7uaBPejUsrCuOg/RVwUqoSOdhtv/EHCgA+xplonhFAt1FiDrWxjgLVSjBYzIcH+8+vpirhFjiNgANq5AWK0MaXjSQ1JB6baMfAqO+V4P5OJDrvuLFH40sLOU4YpD+jXEHvwss9/iFa5BUNKjF7nocTj6MRzWVQ3KxZwDoq6t6Afyz+pzBLK3YM4ZmWicg//bIDHskG20wAjQp3SlISeQxMs6WYC/8YAtHVXPwLImvroDu2927lDydMY6atVOA0bOpGXCe1yMo7elhgSPlXpW2G/peRPvnpJLXZItDAjcEfGlUz9EchrIU5TUcHNr6VPigOcp/zDmUk7H5gLHw9a5OAizzU8dlR4E6MkhJtppwzJDrWnAyCddSqX 1P9Ks0bM lue/N4nKIIVQAVv0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Recent conversations showed that there was a misunderstanding about why we were locking the folio prior to call in hugetlb_wp(). In fact, as soon as we have the folio mapped into the pagetables, we no longer need to hold it locked, because we know that no concurrent truncation could have happened. There is only one case where the folio needs to be locked, and that is when we are handling an anonymous folio, because hugetlb_wp() will check whether it can re-use it exclusively for the process that is faulting it in. So, pass the folio locked to hugetlb_wp() when that is the case. Suggested-by: David Hildenbrand Signed-off-by: Oscar Salvador --- mm/hugetlb.c | 33 +++++++++++++++++++++++++++------ 1 file changed, 27 insertions(+), 6 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 87f2d8acdc8a..15113345babb 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6437,6 +6437,7 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping, pte_t new_pte; bool new_folio, new_pagecache_folio = false; u32 hash = hugetlb_fault_mutex_hash(mapping, vmf->pgoff); + bool folio_locked = true; /* * Currently, we are forced to kill the process in the event the @@ -6602,6 +6603,14 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping, hugetlb_count_add(pages_per_huge_page(h), mm); if ((vmf->flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED)) { + /* + * No need to keep file folios locked. See comment in + * hugetlb_fault(). + */ + if (!anon_rmap) { + folio_locked = false; + folio_unlock(folio); + } /* Optimization, do the COW without a second fault */ ret = hugetlb_wp(vmf); } @@ -6616,7 +6625,8 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping, if (new_folio) folio_set_hugetlb_migratable(folio); - folio_unlock(folio); + if (folio_locked) + folio_unlock(folio); out: hugetlb_vma_unlock_read(vma); @@ -6803,16 +6813,27 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, if (flags & (FAULT_FLAG_WRITE|FAULT_FLAG_UNSHARE)) { if (!huge_pte_write(vmf.orig_pte)) { - /* hugetlb_wp() requires page locks of pte_page(vmf.orig_pte) */ + /* + * Anonymous folios need to be lock since hugetlb_wp() + * checks whether we can re-use the folio exclusively + * for us in case we are the only user of it. + */ folio = page_folio(pte_page(vmf.orig_pte)); folio_get(folio); + if (!folio_test_anon(folio)) + goto lock_unneeded; + spin_unlock(vmf.ptl); folio_lock(folio); spin_lock(vmf.ptl); - if (likely(pte_same(vmf.orig_pte, - huge_ptep_get(mm, vmf.address, vmf.pte)))) - ret = hugetlb_wp(&vmf); - folio_unlock(folio); + if (unlikely(!pte_same(vmf.orig_pte, huge_ptep_get(mm, + vmf.address, vmf.pte)))) + goto unlock_folio; +lock_unneeded: + ret = hugetlb_wp(&vmf); +unlock_folio: + if (folio_test_anon(folio)) + folio_unlock(folio); folio_put(folio); goto out_ptl; } else if (likely(flags & FAULT_FLAG_WRITE)) { -- 2.50.0