From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B7D7C433E5 for ; Tue, 21 Jul 2020 14:32:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D844F206E3 for ; Tue, 21 Jul 2020 14:32:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BuKjOr11" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D844F206E3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 761506B0005; Tue, 21 Jul 2020 10:32:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 710636B0006; Tue, 21 Jul 2020 10:32:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5D7BC6B0007; Tue, 21 Jul 2020 10:32:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0101.hostedemail.com [216.40.44.101]) by kanga.kvack.org (Postfix) with ESMTP id 48DBC6B0005 for ; Tue, 21 Jul 2020 10:32:16 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id DEA8B183A3527 for ; Tue, 21 Jul 2020 14:32:15 +0000 (UTC) X-FDA: 77062323030.20.nut61_3f0fa9626f2e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin20.hostedemail.com (Postfix) with ESMTP id D4F2A1802DA4A for ; Tue, 21 Jul 2020 14:30:10 +0000 (UTC) X-HE-Tag: nut61_3f0fa9626f2e X-Filterd-Recvd-Size: 8266 Received: from mail-wm1-f67.google.com (mail-wm1-f67.google.com [209.85.128.67]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Jul 2020 14:30:10 +0000 (UTC) Received: by mail-wm1-f67.google.com with SMTP id 17so3102941wmo.1 for ; Tue, 21 Jul 2020 07:30:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:subject:to:cc:references:in-reply-to:mime-version :message-id:content-transfer-encoding; bh=NtR3vJ1N4iGqoNW7LE2c1cXPrbxowMXHvYixqbulC4A=; b=BuKjOr11JAgWbiAF4Sj32zPuFLiD9iRV+uWXQboiosDmEW5UZ0xy9LEwHdcmln3L/w Z4yi4/arnJ6RZmqym+O1wipOeuqgSE/gQ7NyaQhtWKSDNESyouAirTmB7XBlAZtP7cNK ho/3x9QXKzM/HXaUFbrpHKEpUepG36+GVXUo1FGKXksY4ivDE6wzLClaydvaDAnvBNBa 0rolFj5vLpGYFwcf7tSPNgYnMfQyP74LejqxV5QPQEQ2RvpgrT0wv1m5zzFIrzoGBEc9 ADgk+yYBe4UqtY2SfkETXxoEfUgH1mo4et8iwb1JyQO1sj85KsMTahA3LqaBH2PWR3Mx Mw+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:subject:to:cc:references:in-reply-to :mime-version:message-id:content-transfer-encoding; bh=NtR3vJ1N4iGqoNW7LE2c1cXPrbxowMXHvYixqbulC4A=; b=IEzO93o/cKYYh1ZR3eaQXhcvJ+rF83h5xeQ02q29xWmVbY5J/LGwL12udrKSAdVpWG k9wTfAHJneEZSlxoBwAqA1EDRWToHvdzZJ9+jfz927UCAFRN1xgrIaUlB/FfIKCeVCvz v90BVZfolOahJKUA4PIRkEym8Ub8+QcLDaI7G2JydXQaMusWSohXvOyUerzc3kVS8vyY ouO5tkA9VLx3Okbsoyo086DykEre+ikHIziosJmYwEK+GkepndYbIlJJ4u+TXQwZ8Ssj xwV6vfTR2dY67E/mnqu6SPCLCwTLrCLlJDujoq/53eh6/ekjVqOEVihdMDk2Ygg77nl/ ttSA== X-Gm-Message-State: AOAM532C4Z1nYa1xJnxSh5BCzqmEAmniBdDakytE0RjjfjWTSBykXCRu wPgi9gIkzasNrTO4JibTgSY= X-Google-Smtp-Source: ABdhPJwB9fyGj6S2BjQwyEItOwht6pp2/UOFe0UgTMjlm93I+Hjufyw5bWfBW9fmR+lDu40fsypZXA== X-Received: by 2002:a1c:6809:: with SMTP id d9mr4291206wmc.34.1595341809069; Tue, 21 Jul 2020 07:30:09 -0700 (PDT) Received: from localhost (110-174-173-27.tpgi.com.au. [110.174.173.27]) by smtp.gmail.com with ESMTPSA id a22sm3655822wmj.9.2020.07.21.07.30.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Jul 2020 07:30:08 -0700 (PDT) Date: Wed, 22 Jul 2020 00:30:01 +1000 From: Nicholas Piggin Subject: Re: [RFC PATCH 4/7] x86: use exit_lazy_tlb rather than membarrier_mm_sync_core_before_usermode To: Mathieu Desnoyers Cc: Anton Blanchard , Arnd Bergmann , Jens Axboe , linux-arch , linux-kernel , linux-mm , linuxppc-dev , Andy Lutomirski , Andy Lutomirski , Peter Zijlstra , x86 References: <1594868476.6k5kvx8684.astroid@bobo.none> <1594892300.mxnq3b9a77.astroid@bobo.none> <20200716110038.GA119549@hirez.programming.kicks-ass.net> <1594906688.ikv6r4gznx.astroid@bobo.none> <1314561373.18530.1594993363050.JavaMail.zimbra@efficios.com> <1595213677.kxru89dqy2.astroid@bobo.none> <2055788870.20749.1595263590675.JavaMail.zimbra@efficios.com> <1595324577.x3bf55tpgu.astroid@bobo.none> <470490605.22057.1595337118562.JavaMail.zimbra@efficios.com> In-Reply-To: <470490605.22057.1595337118562.JavaMail.zimbra@efficios.com> MIME-Version: 1.0 Message-Id: <1595341248.r2i8fnhz28.astroid@bobo.none> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: D4F2A1802DA4A X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam01 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Excerpts from Mathieu Desnoyers's message of July 21, 2020 11:11 pm: > ----- On Jul 21, 2020, at 6:04 AM, Nicholas Piggin npiggin@gmail.com wrot= e: >=20 >> Excerpts from Mathieu Desnoyers's message of July 21, 2020 2:46 am: > [...] >>=20 >> Yeah you're probably right in this case I think. Quite likely most kerne= l >> tasks that asynchronously write to user memory would at least have some >> kind of producer-consumer barriers. >>=20 >> But is that restriction of all async modifications documented and enforc= ed >> anywhere? >>=20 >>>> How about other memory accesses via kthread_use_mm? Presumably there i= s >>>> still ordering requirement there for membarrier, >>>=20 >>> Please provide an example case with memory accesses via kthread_use_mm = where >>> ordering matters to support your concern. >>=20 >> I think the concern Andy raised with io_uring was less a specific >> problem he saw and more a general concern that we have these memory >> accesses which are not synchronized with membarrier. >>=20 >>>> so I really think >>>> it's a fragile interface with no real way for the user to know how >>>> kernel threads may use its mm for any particular reason, so membarrier >>>> should synchronize all possible kernel users as well. >>>=20 >>> I strongly doubt so, but perhaps something should be clarified in the >>> documentation >>> if you have that feeling. >>=20 >> I'd rather go the other way and say if you have reasoning or numbers for >> why PF_KTHREAD is an important optimisation above rq->curr =3D=3D rq->id= le >> then we could think about keeping this subtlety with appropriate >> documentation added, otherwise we can just kill it and remove all doubt. >>=20 >> That being said, the x86 sync core gap that I imagined could be fixed >> by changing to rq->curr =3D=3D rq->idle test does not actually exist bec= ause >> the global membarrier does not have a sync core option. So fixing the >> exit_lazy_tlb points that this series does *should* fix that. So >> PF_KTHREAD may be less problematic than I thought from implementation >> point of view, only semantics. >=20 > Today, the membarrier global expedited command explicitly skips kernel th= reads, > but it happens that membarrier private expedited considers those with the > same mm as target for the IPI. >=20 > So we already implement a semantic which differs between private and glob= al > expedited membarriers. Which is not a good thing. > This can be explained in part by the fact that > kthread_use_mm was introduced after 4.16, where the most recent membarrie= r > commands where introduced. It seems that the effect on membarrier was not > considered when kthread_use_mm was introduced. No it was just renamed, it used to be called use_mm and has been in the=20 kernel for ~ever. That you hadn't considered this is actually weight for my point, which=20 is that there's so much subtle behaviour that's easy to miss we're=20 better off with simpler and fewer special cases until it's proven=20 they're needed. Not the other way around. >=20 > Looking at membarrier(2) documentation, it states that IPIs are only sent= to > threads belonging to the same process as the calling thread. If my unders= tanding > of the notion of process is correct, this should rule out sending the IPI= to > kernel threads, given they are not "part" of the same process, only borro= wing > the mm. But I agree that the distinction is moot, and should be clarified= . It does if you read it in a user-hostile legalistic way. The reality is=20 userspace shouldn't and can't know about how the kernel might implement=20 functionality. > Without a clear use-case to justify adding a constraint on membarrier, I = am > tempted to simply clarify documentation of current membarrier commands, > stating clearly that they are not guaranteed to affect kernel threads. Th= en, > if we have a compelling use-case to implement a different behavior which = covers > kthreads, this could be added consistently across membarrier commands wit= h a > flag (or by adding new commands). >=20 > Does this approach make sense ? The other position is without a clear use case for PF_KTHREAD, seeing as=20 async kernel accesses had not been considered before now, we limit the=20 optimision to only skipping the idle thread. I think that makes more=20 sense (unless you have a reason for PF_KTHREAD but it doesn't seem like=20 there is much of one). Thanks, Nick