From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80D17C77B73 for ; Fri, 14 Apr 2023 18:00:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 031C76B0075; Fri, 14 Apr 2023 14:00:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F23CC900004; Fri, 14 Apr 2023 14:00:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DEBBE900003; Fri, 14 Apr 2023 14:00:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id CC02B6B0075 for ; Fri, 14 Apr 2023 14:00:50 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 91E1A1A0139 for ; Fri, 14 Apr 2023 18:00:50 +0000 (UTC) X-FDA: 80680762260.08.0A922A6 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) by imf12.hostedemail.com (Postfix) with ESMTP id 9D1F040012 for ; Fri, 14 Apr 2023 18:00:48 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=I0MSTygg; spf=pass (imf12.hostedemail.com: domain of 3zpQ5ZAYKCAcz1yluinvvnsl.jvtspu14-ttr2hjr.vyn@flex--surenb.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3zpQ5ZAYKCAcz1yluinvvnsl.jvtspu14-ttr2hjr.vyn@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681495248; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=vrpytJrQkeMnSBoK05n6C3PVwm0AhQVwCbqW6lOv8WI=; b=sTjfYVpdreokIw6R1Rdj4bVx/GkYZ2WW2xRTeWaOsipPQMo/WLMnrZjxdugnUK4R7C0duy fsUvCL2M/MbIIXJlFztZQy6/HKeGfkdqVdaQuEX6REMchoruTmHOWLsx5TNcmOIsWSUP2o qxwmY8PYD0ULcE0uH+RrhQ8bAmF5ZPc= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=I0MSTygg; spf=pass (imf12.hostedemail.com: domain of 3zpQ5ZAYKCAcz1yluinvvnsl.jvtspu14-ttr2hjr.vyn@flex--surenb.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3zpQ5ZAYKCAcz1yluinvvnsl.jvtspu14-ttr2hjr.vyn@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681495248; a=rsa-sha256; cv=none; b=CVG4OToZwXoyCa9hM0Ie3NzY4UAB8a2HCYCcbDzIAoConF1+f9IfyhsWCkmmocHFGQZsJ/ VIIEe6MY1lHMhY6sUeywH9GpBgaCS+5sqk0xzCeGlDzMHUGOGUdV/PaNvER8ziMlRYcfvU o37yMMgO5aaW1npMaUJIw9jiXr3TtUw= Received: by mail-pg1-f201.google.com with SMTP id w184-20020a6382c1000000b0050bed8b0b61so8041707pgd.11 for ; Fri, 14 Apr 2023 11:00:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681495247; x=1684087247; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=vrpytJrQkeMnSBoK05n6C3PVwm0AhQVwCbqW6lOv8WI=; b=I0MSTyggzSSsOoQJjyGrso/7Ybeopii3TjXK3ukhg1ZB/4io6QzG1BwsA1C0yT/loh n6ClZYWNFbLZiRPRvChwZHfYVGAwte3VJpkRziPllFiBNERgJ9QRme+PVgf8DNh9u/yd DNXT8doZ70gPqMtfdeOXYsOFny71YkC8/JWP4JcJYyTxwEFsYDUnqLDTF26pORtCpguw +unNZBR8+bYQqvQCmRg8kRM4L9qEB9Sv06+FZxWl2LXZUxEA6jzQDCuuwRdLTdG6IDWg fIKwzH8ycwC2+9LPQmrvE1OIt53IiTrnddq6TvFhw0FCWMdQQ1D3ErYizr1wYFvTHCNp OzVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681495247; x=1684087247; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=vrpytJrQkeMnSBoK05n6C3PVwm0AhQVwCbqW6lOv8WI=; b=CsL9qLjiMopCqlHThH43IsS8LO+oR3kZTT/XUkF1y3keuwMLhn5SSgTJrElBz2Jsuq e+O+0vWqy9O4jPwqNMmfiBzWIH0Kum8cNd9+EvXBqU+uKWWAyI0KXV8YDT+xnA7mfgJd 7KGKtGZ1cSXS2v53T/fNn+ZaBc3bIxd5lChr1j62Not99PQs5jlg1BdkoCXib8pICUBQ JK0E0jgtGuWLbAjK4BirTACWDrKx4Gdt5d/Dq071TilllpdNXQCww4ArAPzPzHDlfsPC 6Kn7Rx7qUonVtQtpGtr9KdvV/RncNUQO8A7ntO2zOCraI0iohwzEHeaEVN0Hw6FB9Lyn PrFA== X-Gm-Message-State: AAQBX9e30N9hUHKzNbcULvyfnHoBsk+tBOXGjrxtwvmF/WQ5HH1hHRig XUpOTAbLzGhUitXNuJjlD3vqvOBZMmM= X-Google-Smtp-Source: AKy350b7HRLz3uh0vBz0/Uzl8Z+fa6BzF8jLXkvpc+cVek+DZVtKLCgPYkVzSF5sDmnJyeu0FdtqA4SmhFg= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:46c0:7584:f020:e09f]) (user=surenb job=sendgmr) by 2002:a63:4818:0:b0:50b:188d:25bb with SMTP id v24-20020a634818000000b0050b188d25bbmr986965pga.5.1681495246966; Fri, 14 Apr 2023 11:00:46 -0700 (PDT) Date: Fri, 14 Apr 2023 11:00:43 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog Message-ID: <20230414180043.1839745-1-surenb@google.com> Subject: [PATCH 1/1] mm: handle swap page faults if the faulting page can be locked From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: willy@infradead.org, hannes@cmpxchg.org, mhocko@suse.com, josef@toxicpanda.com, jack@suse.cz, ldufour@linux.ibm.com, laurent.dufour@fr.ibm.com, michel@lespinasse.org, liam.howlett@oracle.com, jglisse@google.com, vbabka@suse.cz, minchan@google.com, dave@stgolabs.net, punit.agrawal@bytedance.com, lstoakes@gmail.com, surenb@google.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@android.com Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 9D1F040012 X-Stat-Signature: 8dk7t49qmmsibyq8ntr9zx7e9sbsjia1 X-HE-Tag: 1681495248-572646 X-HE-Meta: U2FsdGVkX1/I54rZ0aM0OQ9jzN0XmWFKfXCpxPcdULU1xQnOumf70/rcy/T/vQOwluEL1y5Xmsbe2J7FRnOW9rC11nE8H/W1UBwnPvG6xumQ9Wt4kiKQbf5V8tLad/izFP9S1t6om7Nn52aCGPWhY+Hh4mhEX5fzbAy2Xif24IZH2iEabI7ZqmqXDwCDfx9blIIzWHfrKYsrOg0w5XbgKufIlctngRxiSh7Atuxp7LkxJWm/IerNWaq7F6pt4v9SrQ3byVKn8gbSBTv+TEL4FIFBXzCzyEo/WXesuA6oaMXYelzhboJOhxddHADdLj792+V8N1HYRdLYlD1GxRIXz+DdGUrWGC4TwoOiXm7fMppmH3Ntlu4pMgOvRmGsSnlOEFZ1adFhJoNgrh4JHG2YkAeC19Gv6wjRx0+jcb4meWgMXPxPzrunSFjTclR0/DGo9bE6JuAiXuk4bjmya1/HN0+yK1cj0F//32IP/2xT5h+dX/vSGUAjKcAc1i3eJYOqHbbpm+T0gcGgwfq5avLPPh6EXoNMPg1FQC/sciSRVINW6MFfR/VXYbtZr0WtHv/iO7C4pS/F+RPgIMAd7gxbHDegrAWZZpSWmtoHIvil3NLaYGFgu2t+uZVtNlLA8vaYRi48qJoBb1yLDR1UGG3dwaU6uuEvkdDKxJvH6zTTHicpkToZ7d5q6g4yfwQul8fLnyLSFWOTNzTeH794MVvnN1ISNmm1SB9D6TiFSmLAnaw8G7o7ADJVcxPJw+dIMgNdxAKn+VbuwogsjWPH8+hbEHO6d/sY4PgPRJBZWaipDHQp+l0WW9jCiPok8mbARKGpSROx/hA1aU5/HbSn9Eo6ZiG+sgEWUWq7EKNWH9EekiudAMKr9iWE/bhDVY4eGtdopkG5Y8g13OYqvkztAgrmQG/1w9gvg2DlkGqiqd63OOMGVk/Od8EDzHv7yGFEhIt2FwMfrocVOdWEnwx/19l LZUfIS5h 6/3LSDC3fLMcpHYbYIg/SnBBHx7tedj0yTgpOQt+IAeOai0WfKU7Wa5iNlfWQjqNze66Pc351rtsse9kOnePLR8PcEdhFW7VJRmlcU9eZIjtLLjQH0ABzV68REV8AsHY/6dPPWU0nW3wWRgUZStWSohzX1xL8oNq+FsHiu72i6rCcodyvvdXWrYJ9hARPIHTKwf1r+ejgkjJlfjAlmOw5pZKMApc1pMjuDJHk973GhRdREo8OcxAexusT0qz6wbixkPIoTYDQE2bp6qa5afrdcmSHZo/EhYRJBV2UdbmN+b6HX0u7auks8crFEmfWeg4mJsOgohahtngjbmBSsIt8UVppJ6huwkoUDq9PhldPpzqmOT9HJZYOvtKQoNkAub005KcZqtElMkkMTHLBNOWzlkdCZKW98wlRifjSLkfgyZBihM+3vKpUHxHm7jBYAiuseydD6/PnDdM3Y1xOo3NOBUWcLDJkQS6coOArDaNOeKx35q6QPL96o5eaB/9BgVq1gw0y+rnp/LeI5QWftnqvtq7fkEbvbaBwGsHp0AV4KlqCrkU21tZYMSbYazrm8XD3EpnmJJnaWsfpkgAFFdM0XZCcqJEI8rpczmHLEET0CqWXlE7lSduUoNKyLQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When page fault is handled under VMA lock protection, all swap page faults are retried with mmap_lock because folio_lock_or_retry implementation has to drop and reacquire mmap_lock if folio could not be immediately locked. Instead of retrying all swapped page faults, retry only when folio locking fails. Signed-off-by: Suren Baghdasaryan --- Patch applies cleanly over linux-next and mm-unstable mm/filemap.c | 6 ++++++ mm/memory.c | 5 ----- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 6f3a7e53fccf..67b937b0f436 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1706,6 +1706,8 @@ static int __folio_lock_async(struct folio *folio, struct wait_page_queue *wait) * mmap_lock has been released (mmap_read_unlock(), unless flags had both * FAULT_FLAG_ALLOW_RETRY and FAULT_FLAG_RETRY_NOWAIT set, in * which case mmap_lock is still held. + * If flags had FAULT_FLAG_VMA_LOCK set, meaning the operation is performed + * with VMA lock only, the VMA lock is still held. * * If neither ALLOW_RETRY nor KILLABLE are set, will always return true * with the folio locked and the mmap_lock unperturbed. @@ -1713,6 +1715,10 @@ static int __folio_lock_async(struct folio *folio, struct wait_page_queue *wait) bool __folio_lock_or_retry(struct folio *folio, struct mm_struct *mm, unsigned int flags) { + /* Can't do this if not holding mmap_lock */ + if (flags & FAULT_FLAG_VMA_LOCK) + return false; + if (fault_flag_allow_retry_first(flags)) { /* * CAUTION! In this case, mmap_lock is not released diff --git a/mm/memory.c b/mm/memory.c index d88f370eacd1..3301a8d01820 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3715,11 +3715,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) if (!pte_unmap_same(vmf)) goto out; - if (vmf->flags & FAULT_FLAG_VMA_LOCK) { - ret = VM_FAULT_RETRY; - goto out; - } - entry = pte_to_swp_entry(vmf->orig_pte); if (unlikely(non_swap_entry(entry))) { if (is_migration_entry(entry)) { -- 2.40.0.634.g4ca3ef3211-goog