From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A277C432C2 for ; Thu, 26 Sep 2019 09:41:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E0B02222C5 for ; Thu, 26 Sep 2019 09:41:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E0B02222C5 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 862A16B026F; Thu, 26 Sep 2019 05:41:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7EB9E6B0270; Thu, 26 Sep 2019 05:41:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6D9B06B0271; Thu, 26 Sep 2019 05:41:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0114.hostedemail.com [216.40.44.114]) by kanga.kvack.org (Postfix) with ESMTP id 45BAE6B026F for ; Thu, 26 Sep 2019 05:41:18 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 04368180AD802 for ; Thu, 26 Sep 2019 09:41:18 +0000 (UTC) X-FDA: 75976578636.19.face70_7fb6439e20139 X-HE-Tag: face70_7fb6439e20139 X-Filterd-Recvd-Size: 6239 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Thu, 26 Sep 2019 09:41:17 +0000 (UTC) Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 673B4C057F31 for ; Thu, 26 Sep 2019 09:41:16 +0000 (UTC) Received: by mail-pf1-f197.google.com with SMTP id w126so1338997pfd.22 for ; Thu, 26 Sep 2019 02:41:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AKkEfjdoHGbVoKbr4xUEMS07lQumUdJTKR+cWS6/cLc=; b=ifdlMv+/QF8BgMYONdlLur3vgEL97iwmDzRZUwp8ZWJQS1/KKQcLEvK1O03pJaEbAn M9KjD0yKlqoTJSZjGlAeK9vHZ3g/k534WjyHlxk5r9iLZXNTuGxglvLX05hCO8R5bbRl e5HIwOzH4LyZQ2U0BtHd4BJjTqgO0pAvNS5GD3pfy7IuYucOCDjzzCndx7t+RciOVKPW TkajeddVl8HQyMjaEdoQ3KKauLUPynCWCC5kIH6HbYMmLJnL9A7ytqfdbw5XcSw//ZhP 4vDKpS9Wf+7Tov31hIcKyRf0F98TbuiBj4riDVLJvMXXkV9eRI5RBQBomuILssIYq6F5 oeOA== X-Gm-Message-State: APjAAAXcZHsGAuClXckYEgWgSQRyHd15LB3wI7RLbvOv4rXfJ/vLoU0z sXViW1kZ5WfdV2H29mNg6Z3mYWlt4Wqv50jMWzLxyulNEEwbFq9GJAP7ctBW6wVlTDaFRrZCBos rynBp4NtvCfg= X-Received: by 2002:a63:550a:: with SMTP id j10mr2421065pgb.369.1569490875459; Thu, 26 Sep 2019 02:41:15 -0700 (PDT) X-Google-Smtp-Source: APXvYqxqpoKkAjxWiSXeTQqhUit0kZdkveOCToqkIiL/LTZ+4SbaywQVnIYZ9dztItKxDR6cACgk0A== X-Received: by 2002:a63:550a:: with SMTP id j10mr2421036pgb.369.1569490875206; Thu, 26 Sep 2019 02:41:15 -0700 (PDT) Received: from xz-x1.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id p68sm3224982pfp.9.2019.09.26.02.41.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Sep 2019 02:41:14 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: David Hildenbrand , Hugh Dickins , Maya Gokhale , Jerome Glisse , Pavel Emelyanov , Johannes Weiner , peterx@redhat.com, Martin Cracauer , Matthew Wilcox , Shaohua Li , Marty McFadden , Andrea Arcangeli , Mike Kravetz , Denis Plotnikov , Mike Rapoport , Linus Torvalds , Mel Gorman , "Kirill A . Shutemov" , "Dr . David Alan Gilbert" Subject: [PATCH v5 14/16] mm/gup: Allow VM_FAULT_RETRY for multiple times Date: Thu, 26 Sep 2019 17:39:02 +0800 Message-Id: <20190926093904.5090-15-peterx@redhat.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190926093904.5090-1-peterx@redhat.com> References: <20190926093904.5090-1-peterx@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is the gup counterpart of the change that allows the VM_FAULT_RETRY to happen for more than once. One thing to mention is that we must check the fatal signal here before retry because the GUP can be interrupted by that, otherwise we can loop forever. Signed-off-by: Peter Xu --- mm/gup.c | 27 +++++++++++++++++++++------ mm/hugetlb.c | 6 ++++-- 2 files changed, 25 insertions(+), 8 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index e60d32f1674d..d2811bb15a25 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -644,7 +644,10 @@ static int faultin_page(struct task_struct *tsk, str= uct vm_area_struct *vma, if (*flags & FOLL_NOWAIT) fault_flags |=3D FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_RETRY_NOWAIT; if (*flags & FOLL_TRIED) { - VM_WARN_ON_ONCE(fault_flags & FAULT_FLAG_ALLOW_RETRY); + /* + * Note: FAULT_FLAG_ALLOW_RETRY and FAULT_FLAG_TRIED + * can co-exist + */ fault_flags |=3D FAULT_FLAG_TRIED; } =20 @@ -994,7 +997,6 @@ int fixup_user_fault(struct task_struct *tsk, struct = mm_struct *mm, down_read(&mm->mmap_sem); if (!(fault_flags & FAULT_FLAG_TRIED)) { *unlocked =3D true; - fault_flags &=3D ~FAULT_FLAG_ALLOW_RETRY; fault_flags |=3D FAULT_FLAG_TRIED; goto retry; } @@ -1069,17 +1071,30 @@ static __always_inline long __get_user_pages_lock= ed(struct task_struct *tsk, if (likely(pages)) pages +=3D ret; start +=3D ret << PAGE_SHIFT; + lock_dropped =3D true; =20 +retry: /* * Repeat on the address that fired VM_FAULT_RETRY - * without FAULT_FLAG_ALLOW_RETRY but with - * FAULT_FLAG_TRIED. + * with both FAULT_FLAG_ALLOW_RETRY and + * FAULT_FLAG_TRIED. Note that GUP can be interrupted + * by fatal signals, so we need to check it before we + * start trying again otherwise it can loop forever. */ + + if (fatal_signal_pending(current)) + break; + *locked =3D 1; - lock_dropped =3D true; down_read(&mm->mmap_sem); + ret =3D __get_user_pages(tsk, mm, start, 1, flags | FOLL_TRIED, - pages, NULL, NULL); + pages, NULL, locked); + if (!*locked) { + /* Continue to retry until we succeeded */ + BUG_ON(ret !=3D 0); + goto retry; + } if (ret !=3D 1) { BUG_ON(ret > 1); if (!pages_done) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 31c2a6275023..d0c98cff5b0f 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -4347,8 +4347,10 @@ long follow_hugetlb_page(struct mm_struct *mm, str= uct vm_area_struct *vma, fault_flags |=3D FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_RETRY_NOWAIT; if (flags & FOLL_TRIED) { - VM_WARN_ON_ONCE(fault_flags & - FAULT_FLAG_ALLOW_RETRY); + /* + * Note: FAULT_FLAG_ALLOW_RETRY and + * FAULT_FLAG_TRIED can co-exist + */ fault_flags |=3D FAULT_FLAG_TRIED; } ret =3D hugetlb_fault(mm, vma, vaddr, fault_flags); --=20 2.21.0