From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FAAEC43460 for ; Fri, 7 May 2021 14:13:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 10FA66128B for ; Fri, 7 May 2021 14:13:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 10FA66128B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2F78F8D0011; Fri, 7 May 2021 10:13:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 27C956B00BA; Fri, 7 May 2021 10:13:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 791FB8D0011; Fri, 7 May 2021 10:13:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0178.hostedemail.com [216.40.44.178]) by kanga.kvack.org (Postfix) with ESMTP id EFF7E6B00B9 for ; Fri, 7 May 2021 10:13:29 -0400 (EDT) Received: from smtpin32.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 86652824999B for ; Fri, 7 May 2021 14:13:29 +0000 (UTC) X-FDA: 78114627738.32.9961EFF Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf16.hostedemail.com (Postfix) with ESMTP id A84AA8019391 for ; Fri, 7 May 2021 14:13:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620396808; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ZYOO3UAjlF+hfKg9yF+CEPnA0Soqks99ZWxEcE/K28Q=; b=C8SVMzEjBPMQG4ggaeNVt8LHWmiE6tENK6X59mR5ePtiThV5B09BpW35qGo0Mq17HkW0zK A+J4bt5irnpIbPc063jEzn1hmbJjZtO+Ss5P1EvfRIpzKMr9es8+bLpoMmTQEAZ6h8nlDr FOs8GfDZ5hR3h34iucI9lhVZkuhqJMI= Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-42-p7O9mOIyMISX2EaC3lsubg-1; Fri, 07 May 2021 10:13:25 -0400 X-MC-Unique: p7O9mOIyMISX2EaC3lsubg-1 Received: by mail-qk1-f198.google.com with SMTP id g2-20020a37b6020000b02902ea40e87ecbso6201290qkf.14 for ; Fri, 07 May 2021 07:13:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=ZYOO3UAjlF+hfKg9yF+CEPnA0Soqks99ZWxEcE/K28Q=; b=BBQuUME1BMIV8rXeXcTTBHPQWMRA04N9do/G5kNmToH27UtaIMdkgfcCpuaEkFIhgZ 1GJ5GAlXlclPMSdFrQ3ch/lMz1ZxKzxEaM+eqH3aKfPuRXfQmakvykex9eAJRz/3gs2p AOy7CtiAcD6HEsk6qloKAeKecO4QkDoH4NvsebNFV5aF/RxBYxGpxQ0JPqt1qQq5gRiQ 6XRbFyZGsMf8Nbd5fd/cuL+nuZ21izXTto0myqPyOurayZ8UbSch8NGJk2hu5PvYFaUF n1hIEjMzpEw0mczjLhEFTs559kCB6NzzBTISPsxPrMQQtjYSE0WtGjyMwLHhwGMGLETG ab8A== X-Gm-Message-State: AOAM5337dH3wIfKg73gxkPiQ/BoEkbGkUdTtmTtVHPn3qoK3/EUSu/dx sy+aDyYR03ApVppf3M/t2UqIPkfPcwy8x5oQsEPxdkZSy9wr09/HB469j4sXfQpXrZn04y2qsxe O4OjGX3G4/4c= X-Received: by 2002:a37:bbc4:: with SMTP id l187mr9631388qkf.153.1620396804606; Fri, 07 May 2021 07:13:24 -0700 (PDT) X-Google-Smtp-Source: ABdhPJybb6hUAjm6c0snSkcA9xF1jcgUFqAKMSK05z3GVbIOoar26/N2ZrzQutQliHhwe/3DHxuWag== X-Received: by 2002:a37:bbc4:: with SMTP id l187mr9631350qkf.153.1620396804336; Fri, 07 May 2021 07:13:24 -0700 (PDT) Received: from t490s (bras-base-toroon474qw-grc-72-184-145-4-219.dsl.bell.ca. [184.145.4.219]) by smtp.gmail.com with ESMTPSA id q126sm4704696qkd.48.2021.05.07.07.13.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 07 May 2021 07:13:23 -0700 (PDT) Date: Fri, 7 May 2021 10:13:21 -0400 From: Peter Xu To: John Hubbard , Linus Torvalds Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Hugh Dickins , Jan Kara , Kirill Shutemov , Jason Gunthorpe , Andrew Morton , Kirill Tkhai , Michal Hocko , Oleg Nesterov , Jann Horn , Linus Torvalds , Matthew Wilcox , Andrea Arcangeli Subject: Re: [PATCH 2/3] mm: gup: allow FOLL_PIN to scale in SMP Message-ID: References: <20210506232537.165788-1-peterx@redhat.com> <20210506232537.165788-3-peterx@redhat.com> MIME-Version: 1.0 In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=C8SVMzEj; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf16.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com X-Stat-Signature: wojb3hjggieax7q7hmcizj3py51r161x X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: A84AA8019391 Received-SPF: none (redhat.com>: No applicable sender policy available) receiver=imf16; identity=mailfrom; envelope-from=""; helo=us-smtp-delivery-124.mimecast.com; client-ip=170.10.133.124 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1620396799-241098 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, May 06, 2021 at 11:07:32PM -0700, John Hubbard wrote: > On 5/6/21 4:25 PM, Peter Xu wrote: > > From: Andrea Arcangeli > > > > has_pinned cannot be written by each pin-fast or it won't scale in > > SMP. This isn't "false sharing" strictly speaking (it's more like > > "true non-sharing"), but it creates the same SMP scalability > > bottleneck of "false sharing". > > > > To verify the improvement a new "pin_fast.c" program was added to > > the will-it-scale benchmark. > ... > > > > This commits increases the SMP scalability of pin_user_pages_fast() > > executed by different threads of the same process by more than 4000%. > > > > Remarkable! I mean, yes, everyone knows that atomic writes are > "expensive", but this is a fun, dramatic example of just *how* > expensive they can get, once you start doing contended atomic writes. > > > Reviewed-by: John Hubbard > > Other notes, that don't have any effect on the above reviewed-by > tag: > > On the commit log, I will add a "+1" to the idea of deleting the > pin_fast.c contents from the commit log, and just providing a URL > instead. No need to put C programs in the commit log, IMHO, especially > when you have them elsewhere anyway. I guess it's put there only because it does not exist elsewhere. :) I didn't run it, but I think it needs to be put into tests/ of if-it-scale repo and build, something like that. https://github.com/antonblanchard/will-it-scale/tree/master/tests But yeah since we've got the 1st patch which can also reproduce this, I'll reference that instead in the commit mesasge and I'll be able to shrink it. I'll also start to use parentheses as Linus suggested. My thanks to both of you on the quick comments! -- Peter Xu