From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DFB81C5DF60 for ; Thu, 7 Nov 2019 21:27:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A097821D79 for ; Thu, 7 Nov 2019 21:27:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bXzIhoTR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A097821D79 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3FDE56B0003; Thu, 7 Nov 2019 16:27:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3ACC16B0006; Thu, 7 Nov 2019 16:27:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 274A96B0007; Thu, 7 Nov 2019 16:27:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0115.hostedemail.com [216.40.44.115]) by kanga.kvack.org (Postfix) with ESMTP id 1397A6B0003 for ; Thu, 7 Nov 2019 16:27:28 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id B860A4DC7 for ; Thu, 7 Nov 2019 21:27:27 +0000 (UTC) X-FDA: 76130767734.21.ring22_6e43005b5c10 X-HE-Tag: ring22_6e43005b5c10 X-Filterd-Recvd-Size: 4527 Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by imf23.hostedemail.com (Postfix) with ESMTP for ; Thu, 7 Nov 2019 21:27:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573162046; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LLb8ODlrE2DcLQV25lHkIdJt8ldk3ozlStqtuMJM5vA=; b=bXzIhoTRqj+a8oDTOACkkPK5QdyKV0oFF+kuM2FSsynNkgJnPSadBtn6OtBnaoDGfHaRYY 4Pm9u4UnqtTpYXEiCoXwCAQtLf8RjDRf+oSfqkSPQr6F6hrsKQQZsj/NC45rdU5njDjOz9 jrLgcUFuenEx4U/gt/qm2yMIrN2h3PQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-19-m8ORxfZJNDW2Vk3fzUHgUg-1; Thu, 07 Nov 2019 16:27:22 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3001D8017DD; Thu, 7 Nov 2019 21:27:21 +0000 (UTC) Received: from llong.remote.csb (dhcp-17-59.bos.redhat.com [10.18.17.59]) by smtp.corp.redhat.com (Postfix) with ESMTP id 21962608B3; Thu, 7 Nov 2019 21:27:18 +0000 (UTC) Subject: Re: [PATCH] hugetlbfs: Take read_lock on i_mmap for PMD sharing To: Matthew Wilcox Cc: Mike Kravetz , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Davidlohr Bueso , Peter Zijlstra , Ingo Molnar , Will Deacon References: <20191107190628.22667-1-longman@redhat.com> <20191107195441.GF11823@bombadil.infradead.org> From: Waiman Long Organization: Red Hat Message-ID: Date: Thu, 7 Nov 2019 16:27:18 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.2 MIME-Version: 1.0 In-Reply-To: <20191107195441.GF11823@bombadil.infradead.org> Content-Language: en-US X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: m8ORxfZJNDW2Vk3fzUHgUg-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 11/7/19 2:54 PM, Matthew Wilcox wrote: > On Thu, Nov 07, 2019 at 02:06:28PM -0500, Waiman Long wrote: >> A customer with large SMP systems (up to 16 sockets) with application >> that uses large amount of static hugepages (~500-1500GB) are experiencin= g >> random multisecond delays. These delays was caused by the long time it >> took to scan the VMA interval tree with mmap_sem held. >> >> The sharing of huge PMD does not require changes to the i_mmap at all. >> As a result, we can just take the read lock and let other threads >> searching for the right VMA to share in parallel. Once the right >> VMA is found, either the PMD lock (2M huge page for x86-64) or the >> mm->page_table_lock will be acquired to perform the actual PMD sharing. >> >> Lock contention, if present, will happen in the spinlock. That is much >> better than contention in the rwsem where the time needed to scan the >> the interval tree is indeterminate. > I don't think this description really explains the contention argument > well. There are _more_ PMD locks than there are i_mmap_sem locks, so > processes accessing different parts of the same file can work in parallel= . I am sorry for not being clear enough. PMD lock contention here means 2 or more tasks that happens to touch the same PMD. Because of the use of PMD lock, modification of the same PMD cannot happen in parallel. If they touch different PMDs, they can do that in parallel. Previously, they are contending the same rwsem write lock and hence have to be done serially. > Are there other current users of the write lock that could use a read loc= k? > At first blush, it would seem that unmap_ref_private() also only needs > a read lock on the i_mmap tree. I don't think hugetlb_change_protection(= ) > needs the write lock either. Nor retract_page_tables(). It is possible that other locking sites can be converted to use read lock, but it is outside the scope of this patch. Cheers, Longman