From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B264C433FE for ; Fri, 4 Dec 2020 08:17:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E2072225A9 for ; Fri, 4 Dec 2020 08:17:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E2072225A9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 32D336B0036; Fri, 4 Dec 2020 03:17:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2DDC98D0001; Fri, 4 Dec 2020 03:17:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1A5C26B0068; Fri, 4 Dec 2020 03:17:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0218.hostedemail.com [216.40.44.218]) by kanga.kvack.org (Postfix) with ESMTP id F3C6A6B0036 for ; Fri, 4 Dec 2020 03:17:34 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id AF5C31DFE for ; Fri, 4 Dec 2020 08:17:34 +0000 (UTC) X-FDA: 77554895628.26.toe65_220bafb273c2 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin26.hostedemail.com (Postfix) with ESMTP id 83E681804B668 for ; Fri, 4 Dec 2020 08:17:34 +0000 (UTC) X-HE-Tag: toe65_220bafb273c2 X-Filterd-Recvd-Size: 6772 Received: from mail-pl1-f196.google.com (mail-pl1-f196.google.com [209.85.214.196]) by imf23.hostedemail.com (Postfix) with ESMTP for ; Fri, 4 Dec 2020 08:17:33 +0000 (UTC) Received: by mail-pl1-f196.google.com with SMTP id p6so2691652plo.6 for ; Fri, 04 Dec 2020 00:17:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=Ln++9Eg71n524JlS9NV8GRrodOh9pkKHZvYyLb9NMN8=; b=tDMVJKyflN4AytQ5BZaaYAnTvD40r+ph9Q8CJrtzq7qK9cpK3KDT6NOzafwsRREUcD cFOOnkH3VCh4P6E9SdpOxq99EPa00kcFmWHnSoHwuuBUU9wzc2BnYLtTa8cnb+wgzfWA cYOKmCVOuNPAnNExrsPEQ05s7tlIMsCdXuWeTR/ULMR768eflA18ANFzJp+tE6n7ZDbI Yi0bGLF9muM5t6o5/PHdMx2RcQylbUroFG3yhwwc92dxPrF0kAFGFslctsribroovhp6 dvfEzYiMCqlGfhspEHuBVSRWoyJ6QkaJU9DwZ3WgttxlG5yI+Ee7qGY1fTD1cuKjoG8m KyxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=Ln++9Eg71n524JlS9NV8GRrodOh9pkKHZvYyLb9NMN8=; b=pvX1Dfe5niB5zhmiZsN0FeRgUH7fNHpoyi7uMiiZUrFwMVCvS8jKM9W93iDjtqorE8 KH6vQzQh9qbyAVmOaiP030aE0RdyGffX1yrI2pSTY1oPD5wJIxxqciWaZM3pkQts9UDs 1Q7vM9xeKiUwkJOpJ7VTwYUSczMFwTZ8ygWly75EohTKUo4FPELknxxpfIS1/VhPH2KJ iQAthAPq8NsBSHk/meRRded7rVMTTJOKVlQp0ZyzD/qoMO7gxKpGMqutN5MyUTqNLmhx rbVVAbl/A9aWwEH8B4zQgbYJpsm+vgr1fL5kgdK0K0SNcBINIQn/l88+THNev1ZMIL7x oVTw== X-Gm-Message-State: AOAM532YuItE7Jn35c7B0t7WSggKoMXKFSV5UbjM9kmNCQmASzQF69kV obQoolF8V44XGzuX7mWYMq8= X-Google-Smtp-Source: ABdhPJx1Ho3dBYScBiLMsFzyMeRifNVoKPOlG/ESpTJs/U21Gcdn1OiMMq9h+u0zR4BCVQgth9i5dQ== X-Received: by 2002:a17:902:b498:b029:da:84a7:be94 with SMTP id y24-20020a170902b498b02900da84a7be94mr2861306plr.52.1607069852915; Fri, 04 Dec 2020 00:17:32 -0800 (PST) Received: from ?IPv6:2601:647:4700:9b2:e462:2bde:2e8b:f9df? ([2601:647:4700:9b2:e462:2bde:2e8b:f9df]) by smtp.gmail.com with ESMTPSA id g16sm3842217pfb.201.2020.12.04.00.17.30 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 04 Dec 2020 00:17:31 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.4\)) Subject: Re: [RFC v2 1/2] [NEEDS HELP] x86/mm: Handle unlazying membarrier core sync in the arch code From: Nadav Amit In-Reply-To: <203d39d11562575fd8bd6a094d97a3a332d8b265.1607059162.git.luto@kernel.org> Date: Fri, 4 Dec 2020 00:17:29 -0800 Cc: Nicholas Piggin , Anton Blanchard , Arnd Bergmann , linux-arch , LKML , Linux-MM , linuxppc-dev , Mathieu Desnoyers , X86 ML , Will Deacon , Catalin Marinas , Rik van Riel , Dave Hansen , Jann Horn Content-Transfer-Encoding: quoted-printable Message-Id: References: <203d39d11562575fd8bd6a094d97a3a332d8b265.1607059162.git.luto@kernel.org> To: Andy Lutomirski X-Mailer: Apple Mail (2.3608.120.23.2.4) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: I am not very familiar with membarrier, but here are my 2 cents while = trying to answer your questions. > On Dec 3, 2020, at 9:26 PM, Andy Lutomirski wrote: > @@ -496,6 +497,8 @@ void switch_mm_irqs_off(struct mm_struct *prev, = struct mm_struct *next, > * from one thread in a process to another thread in the = same > * process. No TLB flush required. > */ > + > + // XXX: why is this okay wrt membarrier? > if (!was_lazy) > return; I am confused. On one hand, it seems that membarrier_private_expedited() would issue an = IPI to that core, as it would find that this core=E2=80=99s = cpu_rq(cpu)->curr->mm is the same as the one that the membarrier applies to. But=E2=80=A6 (see below) > @@ -508,12 +511,24 @@ void switch_mm_irqs_off(struct mm_struct *prev, = struct mm_struct *next, > smp_mb(); > next_tlb_gen =3D atomic64_read(&next->context.tlb_gen); > if (this_cpu_read(cpu_tlbstate.ctxs[prev_asid].tlb_gen) = =3D=3D > - next_tlb_gen) > + next_tlb_gen) { > + /* > + * We're reactivating an mm, and membarrier = might > + * need to serialize. Tell membarrier. > + */ > + > + // XXX: I can't understand the logic in > + // membarrier_mm_sync_core_before_usermode(). = What's > + // the mm check for? > + membarrier_mm_sync_core_before_usermode(next); On the other hand the reason for this mm check that you mention = contradicts my previous understanding as the git log says: commit 2840cf02fae627860156737e83326df354ee4ec6 Author: Mathieu Desnoyers Date: Thu Sep 19 13:37:01 2019 -0400 sched/membarrier: Call sync_core only before usermode for same mm =20 When the prev and next task's mm change, switch_mm() provides the = core serializing guarantees before returning to usermode. The only case where an explicit core serialization is needed is when the scheduler keeps the same mm for prev and next. > /* > * When switching through a kernel thread, the loop in > * membarrier_{private,global}_expedited() may have observed = that > * kernel thread and not issued an IPI. It is therefore possible = to > * schedule between user->kernel->user threads without passing = though > * switch_mm(). Membarrier requires a barrier after storing to > - * rq->curr, before returning to userspace, so provide them = here: > + * rq->curr, before returning to userspace, and mmdrop() = provides > + * this barrier. > * > - * - a full memory barrier for {PRIVATE,GLOBAL}_EXPEDITED, = implicitly > - * provided by mmdrop(), > - * - a sync_core for SYNC_CORE. > + * XXX: I don't think mmdrop() actually does this. There's no > + * smp_mb__before/after_atomic() in there. I presume that since x86 is the only one that needs membarrier_mm_sync_core_before_usermode(), nobody noticed the missing smp_mb__before/after_atomic(). These are anyhow a compiler barrier in = x86, and such a barrier would take place before the return to userspace.