From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 271D0EB64D7 for ; Wed, 21 Jun 2023 15:08:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 52E028D0003; Wed, 21 Jun 2023 11:08:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4DEF08D0002; Wed, 21 Jun 2023 11:08:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3A6DE8D0003; Wed, 21 Jun 2023 11:08:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 287388D0002 for ; Wed, 21 Jun 2023 11:08:35 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id D83DB40913 for ; Wed, 21 Jun 2023 15:08:34 +0000 (UTC) X-FDA: 80927086548.24.6C4FD8F Received: from mail-oi1-f180.google.com (mail-oi1-f180.google.com [209.85.167.180]) by imf06.hostedemail.com (Postfix) with ESMTP id 768161800FB for ; Wed, 21 Jun 2023 15:06:17 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=HeF1k6bj; dmarc=none; spf=pass (imf06.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.167.180 as permitted sender) smtp.mailfrom=jgg@ziepe.ca ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687359977; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=i8mvm0Ua7VheRBgm3SyS2f+CUs+ip4U1HTEOUVIIBZg=; b=OflSi6nKeEPE2BDLiTecNwWBtwZEgMVf7Lp83ZtZGPkXIfNj7gVDSPHHDb6I/qWNbu+Tr8 Y2UczfozeFlBztMQvTUpcjv0I8+QsOEodqim1evLzAHlwBEDkK/eqi6WuBnrLcpNuSLNwu lNTTvHQSq+ihcSFzGOl1F/qadGzPifo= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=HeF1k6bj; dmarc=none; spf=pass (imf06.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.167.180 as permitted sender) smtp.mailfrom=jgg@ziepe.ca ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687359977; a=rsa-sha256; cv=none; b=sMronZHYxjJ8agtQS/XJe82hN4Ijbo0gaKSPaPc0uc/l1Tghilo3DQM1ZRQ+dkLikwMHbT JZBn+yzHBnkP2DZCOVAeozBpHRPVlTPIUpyJCRGgsXM42Cj2u6bIB0nBM1Mb3VapgExzaU 8LHbjBQ1RxDQwtGRAofS+k22ZdU9gMI= Received: by mail-oi1-f180.google.com with SMTP id 5614622812f47-39ea511930eso3943842b6e.1 for ; Wed, 21 Jun 2023 08:06:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1687359976; x=1689951976; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=i8mvm0Ua7VheRBgm3SyS2f+CUs+ip4U1HTEOUVIIBZg=; b=HeF1k6bjWFVW0gS1DrVZ7s5steRsGaEzzCs8cnUdWlatT2bkDGzEeLpNR3LkgWjlpq p8HL6SDTUDdeVvXm/6cwQSYiMFo2ig6zdYWYZYRvcIt2OdonMAxt6whMEOgcmqg7p0BF dPU6Z0XeeHzgXZA77Wy350iWlPXC/mrJ8FHlhhSWLKa6+lBZeHznH//y9z4bAY3LGV0j zqvGppwU1sAIn/juqALMn/DhUwL6ZEkKrFnYtinNlAhJCmQGQR3Y1kWid35u5o7SNQd+ RVdAghyucYBDYSFRI+E/A+VSJknTFyRKCufCH+0LtDTFImNCr4Kq/m4V+SxH4tzdMDdM dZCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687359976; x=1689951976; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=i8mvm0Ua7VheRBgm3SyS2f+CUs+ip4U1HTEOUVIIBZg=; b=aQGxkbIeJ2+bTk96d9vbyV13sYePFPRm64b2O0ziOMsCDziGW+3JqmuHd4dkuv8g6/ A034GFkUeaPn1qt3KbUXKgZ+5gbzZ0lrNCLwunlxlIrjx0evcB4rpPNXy6e8NHT/6BDc HS1AM0a+QQHtHRPUEIc7qUfUjFVFKSOwIDWcAVpcPL5T0gUL1Y+xNMLlQmrUiqlh3e56 ilopgmUQXfaVSi1xJLMzz+kEBFCHXWyo4hZRYq4OhcVjqYm5Kh7z+cRKVroU5tZ6G6zi /stMNRokmwV6kJ3ab7sMacS8+WOOWSB2x6J+zIjX3/zFINHf6qjWorUkO373oFkvX20z Jp9Q== X-Gm-Message-State: AC+VfDxm+GGDMHEa+/LQwBuC9zgaLZSLChCGZf+U/jdo0vS6Uyrx8xz2 aH8A+AG7PpW+8YR5SE4mxarA4g== X-Google-Smtp-Source: ACHHUZ636NWZSijMUe4oI3fgGZIHT/qTk8OtX3ic1/cT2sn7NBvatb98BNnqEYBTRAhrHhPIPOOMUg== X-Received: by 2002:a05:6808:1814:b0:39b:f558:8cab with SMTP id bh20-20020a056808181400b0039bf5588cabmr19969218oib.26.1687359976039; Wed, 21 Jun 2023 08:06:16 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-25-194.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.25.194]) by smtp.gmail.com with ESMTPSA id gl12-20020a17090b120c00b0025e7f7b46c3sm9142110pjb.25.2023.06.21.08.06.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Jun 2023 08:06:15 -0700 (PDT) Received: from jgg by wakko with local (Exim 4.95) (envelope-from ) id 1qBzPR-007dQ6-Os; Wed, 21 Jun 2023 12:06:13 -0300 Date: Wed, 21 Jun 2023 12:06:13 -0300 From: Jason Gunthorpe To: Alistair Popple Cc: Andrew Morton , linux-mm@kvack.org, Robin Murphy , will@kernel.org, catalin.marinas@arm.com, linux-kernel@vger.kernel.org, nicolinc@nvidia.com, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, John Hubbard , zhi.wang.linux@gmail.com, Sean Christopherson Subject: Re: [RFC PATCH 0/2] Invalidate secondary IOMMU TLB on permission upgrade Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 768161800FB X-Stat-Signature: zes8xmi4kwooo6d5f1gphfw6b6tp51mf X-HE-Tag: 1687359977-830931 X-HE-Meta: U2FsdGVkX18f63GzpTTRJF3bWAM8HRyMw3qiIXaK7lwJKd46QQ7gvPGEdL++WcWq+FX++C+vbb+dTRLGIuHQbKMPSf2hbuFxYmWHMeOG6F4zHk8lOLXtzgc2FkjR7UoUcInn1Axap5yuwBWGD439JXF8rob6cJX67pYSuBKrOVhGTGExjfBeRY4CGamMHYh08h04eR2qXB5uQAuG6bx/8DICOYtuLxn353+L3sJe0iymQ5OxH6Gd6KkgTCm6WD5BzSIw2k6LsPsUTIh6VpnwAydacdUpNBThGWjmW1aSn47ofnjafcXS/RE0iDuekzcO1OriTf0OPj374rWBjTVK543RPFlGL6KX0fGaixsLYaJ4I3tds4y9BPDykZvh4+YSsBKS61qNNWlmP7dmhMsERwno2xL+rVKMX0cEEErirvCUlsk9weOeqE+Z0EInH4c3auEh/Mk/2zipw2Exv99bxmnNeTvA3mJGxVjO5t+lYfZI0nhtf8vero3745DO5GXTbpjS+BgLQozCV2EU3qyKY50ryliYiFJXfiNTSORRl0Y9pytCwFQdQ6mxa/hwo3Nuc7vmDN+Xn1tQf5e8ny6glgrvTFi3M+zcXXgyU0bARYo1RZuveVjf1sFFp2Tym3ypqGNa9p6DPiHTXwOVdeColjBYciDWJXWYMk+w14dXxtlhkyX+CBSpLW6LVtUS1WnBhYZJvR8XszGmss5LJj9VdWeCiLIXju/vH6G34YkbJd0o3Xj6Ds3WkjoTOsZalslkGmFmZfA0T+FHlMNyJRXWnCpFrWuFpO6uJaIMw6zuIy3zMOch+1AcbNyKDUq1C2K3xNhfS9ginm4R0RfIfkAfmIdRXYpAvpiPyjcj6uxtKNrjp+2XPdJvU6p/+Wzel91mOIbuvyl1179lSiq27cECptAHi6U63zTGGTFGyz4FyvwIgIb9/wJUfmv/6S0R7wZrp9BqMFnaIFVNtk503GH s45Dt98b 4d7nUWi2v6l+ZHc+Hts/J1AwPsx1/6sRJ8kw4Tc+oaxaAY1PzbA/h9U/PFbaU390iXboR4K7wJRTgzTVr3jTpOc8lgdBoWoLjk+SEH4XP0EwwOpkWTFY4HMe82sylxYBzbyB0d+eHIPJkU2lPOWn4ryc4CaSkpXHYSEo6OYrYMNfjlkSG8nMmp4jH5PUrydu0Wqfcb+NW7VtXxEt/HB4tQBgP0nYfhonHceMD9OYhLkZ0u/SOiA6cixvMI9xxsXcJIXxeuvS23sJrQtmsHs4FZqPXGHNarA4HNbJc2MO78iud6rPeKUyhQoWiNly5Nh5qXBzPDegjfGrmFTmV0PgUSgx4TVg9GH3Q4DHJ+DMnQ+3EvhrcWKxksQciE5eGK4Aei2ZvlKMsF8/DE44= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jun 20, 2023 at 09:18:24PM +1000, Alistair Popple wrote: > 1. Add a call to mmu_notifier_invalidate_secondary_tlbs() to the arm64 > version of ptep_set_access_flags(). I prefer we modify the whole thing to only call mmu_notifier_arch_invalidate_secondary_tlbs() (note the arch in the name) directly beside the arch's existing tlb invalidation, and we only need to define this for x86 and ARM arches that have secondary TLB using drivers - eg it is very much not a generic general purpose mmu notifier that has any meaning outside arch code. > This is what this RFC series does as it is the simplest > solution. Arguably this call should be made by generic kernel code > though to catch other platforms that need it. But that is the whole point, the generic kernel cannot and does not know the rules for TLB invalidation the platform model requires. > It is unclear if mmu_notifier_invalidate_secondary_tlbs() should be > called from mmu_notifier_range_end(). Currently it is, as an analysis > of existing code shows most code doesn't explicitly invalidate > secondary TLBs and relies on it being called as part of the end() > call. If you do the above we don't need to answer this question. Calling it unconditionally at the arches tlbi points is always correct. > To solve that we could add secondary TLB invalidation calls to the TLB > batching code, but that adds complexity so I'm not sure it's worth it > but would appreciate feedback. It sounds like the right direction to me.. Batching is going to be important, we don't need to different verions of batching logic. We already call the notifier from the batch infrastructure anyhow. This still fits with the above as batching goes down to the arch's flush_tlb_range()/etc which can carry the batch to the notifier. We can decide if we leave the notifier call in the core code and let the arch elide it or push it to the arch so it the call is more consistently placed. eg we have arch_tlbbatch_flush() on x86 too that looks kind of suspicious that is already really funky to figure out where the notifier is actually called from. Calling from arch code closer to the actual TLB flush would be alot easier to reason about. Jason