From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78DCAC11D3D for ; Thu, 27 Feb 2020 17:43:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 40F1A2469F for ; Thu, 27 Feb 2020 17:43:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="h0ccKzey" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 40F1A2469F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C55E36B0005; Thu, 27 Feb 2020 12:43:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C05EB6B0006; Thu, 27 Feb 2020 12:43:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B1B016B0007; Thu, 27 Feb 2020 12:43:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0187.hostedemail.com [216.40.44.187]) by kanga.kvack.org (Postfix) with ESMTP id 98DAF6B0005 for ; Thu, 27 Feb 2020 12:43:13 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 6BA468048DD2 for ; Thu, 27 Feb 2020 17:43:13 +0000 (UTC) X-FDA: 76536628266.02.wire87_39292c4cc8819 X-HE-Tag: wire87_39292c4cc8819 X-Filterd-Recvd-Size: 5290 Received: from mail-qk1-f194.google.com (mail-qk1-f194.google.com [209.85.222.194]) by imf18.hostedemail.com (Postfix) with ESMTP for ; Thu, 27 Feb 2020 17:43:12 +0000 (UTC) Received: by mail-qk1-f194.google.com with SMTP id 145so186294qkl.2 for ; Thu, 27 Feb 2020 09:43:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=oCjJQPxZpDAddCTP2fyq9zy2cQ1xSVL89kL40WfyDFw=; b=h0ccKzeyP3flPLgjKffvX6NEZ8a/uwNV100LizRKN14n3zedm2cahPgJEhxnAi/Eux SoJyZIMb4T7nlpMiSIVzl4rMs2X4TuO582QZ1v3uBlvSIJKL/2eHq1X0Q3RguRMqGiyD jIHlP2hMNaMnlwbC+BJKvF6YRP2s83wQLQnu6HWuvl9gBkHIXEKgRfF2bx2kqxfLtaoi NHPp/iB678wthxbg1SayDlPMTRa2+OyRd0c8slMZFoj0NWbnumSUHITUyCw6JAjfKy0s /ZPnh7hQIi+kffDRK1XYwUv4XEf8RRXfY3Aofqotene7stHVy6OiSQWIReq879CDmrtM 2sgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=oCjJQPxZpDAddCTP2fyq9zy2cQ1xSVL89kL40WfyDFw=; b=V3DEjwvtrzhdXxrfBxbeXpod9pkyiG1igN/h/ftCyxkfmq8H4WLrXb5CPj4MvUMAGC PJnIA004zXrOMpzvlCbLX49yvxqtksSCKenbNHZ2utSTOlJgg6kxmvjGtnBaNBLK9WZa jQdGiqd5iPKMQA0ulTOPdaEu6GEPGhhfdrYi+7Plt08+OGyl/FYdGq8VXzu6e9FJQgUO 0pTed4bGAp63KaV/fmDYccfmhqoeiDad3WEj2YkUbZGrfgZR2B3UrT0vshzniaE02XNW PSsFcJCAraWs7LxsA7acjr1kf68oSdvd8+O2ju+mhf33HZYIug0WL6mVVbRC7Xv1kn83 1r3A== X-Gm-Message-State: APjAAAXa6a4mgJpCb0VE6jKZsnpzsqniPa+g4YwsjYesf3y7OgxKVIeE Q5ZljLLoRqzcNB2w0BWBZV2ISw== X-Google-Smtp-Source: APXvYqz1bAJmZgA8/8aWp40ku++Grc84lutv6gcjfJ4VE4iLl6cxS/fnpq6F9eLVrs8dckDJaWVR0g== X-Received: by 2002:a05:620a:2185:: with SMTP id g5mr445502qka.4.1582825391979; Thu, 27 Feb 2020 09:43:11 -0800 (PST) Received: from ziepe.ca (hlfxns017vw-142-68-57-212.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.57.212]) by smtp.gmail.com with ESMTPSA id k23sm3317124qtq.89.2020.02.27.09.43.11 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 27 Feb 2020 09:43:11 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1j7NBn-0003iZ-2v; Thu, 27 Feb 2020 13:43:11 -0400 Date: Thu, 27 Feb 2020 13:43:11 -0400 From: Jason Gunthorpe To: Logan Gunthorpe Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-ia64@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-mm@kvack.org, Dan Williams , Michal Hocko , David Hildenbrand , Andrew Morton , Christoph Hellwig , Catalin Marinas , Will Deacon , Benjamin Herrenschmidt , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Eric Badger Subject: Re: [PATCH v3 0/7] Allow setting caching mode in arch_add_memory() for P2PDMA Message-ID: <20200227174311.GL31668@ziepe.ca> References: <20200221182503.28317-1-logang@deltatee.com> <20200227171704.GK31668@ziepe.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Feb 27, 2020 at 10:21:50AM -0700, Logan Gunthorpe wrote: > > > On 2020-02-27 10:17 a.m., Jason Gunthorpe wrote: > >> Instead of this, this series proposes a change to arch_add_memory() > >> to take the pgprot required by the mapping which allows us to > >> explicitly set pagetable entries for P2PDMA memory to WC. > > > > Is there a particular reason why WC was selected here? I thought for > > the p2pdma cases there was no kernel user that touched the memory? > > Yes, that's correct. I choose WC here because the existing users are > registering memory blocks without side effects which fit the WC > semantics well. Hm, AFAIK WC memory is not compatible with the spinlocks/mutexs/etc in Linux, so while it is true the memory has no side effects, there would be surprising concurrency risks if anything in the kernel tried to write to it. Not compatible means the locks don't contain stores to WC memory the way you would expect. AFAIK on many CPUs extra barriers are required to keep WC stores ordered, the same way ARM already has extra barriers to keep UC stores ordered with locking.. The spinlocks are defined to contain UC stores though. If there is no actual need today for WC I would suggest using UC as the default. Jason