From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9628D149D8 for ; Fri, 25 Oct 2024 17:31:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5F7946B007B; Fri, 25 Oct 2024 13:31:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5A80C6B0083; Fri, 25 Oct 2024 13:31:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 46FB46B0089; Fri, 25 Oct 2024 13:31:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2A8A06B007B for ; Fri, 25 Oct 2024 13:31:42 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id B3DE780B78 for ; Fri, 25 Oct 2024 17:31:24 +0000 (UTC) X-FDA: 82712816634.14.1F6D9B9 Received: from mail-ot1-f45.google.com (mail-ot1-f45.google.com [209.85.210.45]) by imf14.hostedemail.com (Postfix) with ESMTP id 8BDFB100025 for ; Fri, 25 Oct 2024 17:31:17 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Hm0N0zCE; spf=pass (imf14.hostedemail.com: domain of yuzhao@google.com designates 209.85.210.45 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729877346; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8nFesq6fPuvktjrVX6D6kJiVuG2UTKxs3GcGzAOL5j0=; b=CzeLEvqSml3HCSTjpxWhaWohZHNnhMjlTJqLXAqoUMilWaCjXH8ciRPKPKwkseWE1RTd0p vUoqXBy/IDwP+ySMRbuX9n4u+g1L6PiWl5EhNs2Wa6i3m7EvLr6Ur14ZQA1W4YZxLRtjha oo67KNHQlqXTcccPkqMsgI/zzjfC2YY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729877346; a=rsa-sha256; cv=none; b=LeM+5mkS+BvnWBZZaP1r3yLS9RGKST+RPA23tlnu2n32HHJafk+S7bSIksp0XK1fdzuc11 2qoOjg1BUOtpkYOvLMM38X/giCe+aQF6K+qiT8jWlf/4L1lRhElcmIN8ajNi2aCBgOBBGX bP1052GIsVgxdVTXf6zfO762dQ75Qiw= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Hm0N0zCE; spf=pass (imf14.hostedemail.com: domain of yuzhao@google.com designates 209.85.210.45 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-ot1-f45.google.com with SMTP id 46e09a7af769-7180a5ba498so1193882a34.2 for ; Fri, 25 Oct 2024 10:31:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1729877499; x=1730482299; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=8nFesq6fPuvktjrVX6D6kJiVuG2UTKxs3GcGzAOL5j0=; b=Hm0N0zCEH9UL1qXt6z8bGEwul3dJCuPQht8+VG0DxgC6j99yYfUHyWYkj4TYTOcDb1 0AH1PNegcVe+ZUamz1IDvqtyKG9QCySBhcAfpWou+tuOV/051VI2Zqe5+dksaoW6l1dX Yg4huQay6/ahI/vD2tEAJ/B+huEcnSVZSA788zSvOXpFSKwLSmkmIxydpuAy0MIUVdZ+ RuZIyDl/ve4AYHCc9jqcvX33w/yxppcD3JnC4824S1s0XX9570T7GsB9qk0GMHW4eOQL MeJZ8osdZzka0+KVboPIEGnM3rWkmPYCm2JBCXwXIZDLc7xBitF/ey0NtKwCT80pCVjs Lc4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729877499; x=1730482299; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8nFesq6fPuvktjrVX6D6kJiVuG2UTKxs3GcGzAOL5j0=; b=nIPFXlU+MDz9+S6CEebIaJShUMyP5ToSFqQH+IjbZG7a2YvpmSYxf53bstYWzfd/oN zgj/3t6MeAG+d1KnBDM/i1R1ANCv+UCqUCegl8RmVBNRsPfMs5Tsn0Umom2LVOOJVqKm 7xwnJhIIptsARd1kq71UZsJrqutjay5SAbMtd6smkEqjlNOr6EhqXmS7LXwfIHrlqfNK GoNNRN01DgDDRsh43piR2Xh8MJ3ShKsTpWIGKwDQw4k41lR4SiB7SEC5WwLudgTCvzFB WPUTJlLJ/JZUDKkqTRBpXAcZS0F5gcuHu7E+OQ0Vy/WiFZuy6aKsj+t0RBA80mKBOdVh krxw== X-Forwarded-Encrypted: i=1; AJvYcCXtW4VXHjSi1yLtQH4m/oV7BgjCO+o7hJCVQ6IPN7SMLDbGgDQXNwQH5nVvaOpmtJHYXAoglcGG9Q==@kvack.org X-Gm-Message-State: AOJu0Yx3+UGKHKqpXy55UFYJb+RJyOXxPtUDwxE03ZMfd59dgSmP68Bf cXCwjFmFY+njdWB9a6eGO0i9Gayyj2plY6zTrdHCMA36onh59TcRxe1QaoOnzfI/ujLh+ARpe5b t5+MRON7TWA/N9hfZy+AeW5yFYjTz+TlZMkhH X-Google-Smtp-Source: AGHT+IF4WiLhU1Gd/JNblfLAXfS7pNVnnQCGhJ6THzrAm3Qyvdxiau/k0EXOnmxa65gTsETP/x7JeKI91bH/3f56v+M= X-Received: by 2002:a05:6830:2814:b0:718:18d6:a464 with SMTP id 46e09a7af769-71867f11dfemr438530a34.5.1729877498557; Fri, 25 Oct 2024 10:31:38 -0700 (PDT) MIME-Version: 1.0 References: <20241021042218.746659-1-yuzhao@google.com> <20241021042218.746659-4-yuzhao@google.com> <86a5ew41tp.wl-maz@kernel.org> <86h6902m7y.wl-maz@kernel.org> In-Reply-To: <86h6902m7y.wl-maz@kernel.org> From: Yu Zhao Date: Fri, 25 Oct 2024 11:31:01 -0600 Message-ID: Subject: Re: [PATCH v1 3/6] irqchip/gic-v3: support SGI broadcast To: Marc Zyngier Cc: Andrew Morton , Catalin Marinas , Muchun Song , Thomas Gleixner , Will Deacon , Douglas Anderson , Mark Rutland , Nanyong Sun , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 8BDFB100025 X-Stat-Signature: meddcn6qt7djqn9s11dkitsaneks6ukr X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1729877477-934418 X-HE-Meta: U2FsdGVkX1/BBfpvhSsrhK1jNW2G4kXn7AP2vhp1pMYceo9PAAivzmYzPeHHpUGTRYCSPYCga4Yd8Z0SSLVlxujWMtswl/ACzxWuN3DLXRAH3ZNdvYYBwrsQCUHllS6oXVvcLXHFH+1xFtUdsTXaqV8UpdBZruiMMsr1UVWE5h+coMVFQuf2X4IE2F1r4N2eywV7RDP6CdDg9dhf+BbyEYZo5/FDrukikUuyX00PSpXaUe98gk1KdQFqFqvjl4DyitzNM8ASTBVReDr5KTU7Q/do4oE05hgxwxFnS4uzJyoOpz55py5+ZlWWUiKePS4fOXhGCKi4uKJ6J7oZHByFWEZNx9v1PZOhkOE4R2h1K/AL5rb9LB4Q72mM45PFSGJH8j/vRvxH7BOlD0fJoLc7SC9os2u7eM5WJSIlHxPjA+XM5oDefhshUYLMQ6xw6QsiZc0BtOpiptjR49rkZcEB3vYcawQ5vvC8MtOeDBY4uCwyNQC1RfWCsZUkk+GX6tT1RDNQ6VCgpWmW8XRxbnhzBEr3+M46GwAsxbKhweK5Vc+FJM+qmLF6/d1cqIj4yhiK5fC3ifODbHR0Uhq8XHlZcG/rn1BNqgYvHxIR33oUaSnBG2qYgHFUeijT2f5ydrG3ZGewa8Wz7rmF/wavniBNtBBa5iMfGQNBVOpa1+PA/EwxC/tNdg+j+FtgVJe7pnqw37gZmd0COqRRQB11SigxQGytd/jNLi+4sBs6zQ6hNvJSls7CLGuSvVF15DS6QH3YoE8guc3IS7UhjcmKvTbQTZSV6dtPHp06Xer5btQ2N80bxur2nEi4/dswv9vYk2DjF030a/MJlITtquafgvAJb5yI3ed8VWLVy1KZWrMtMR+iXtqVy5yCkZZYaCbqG2YnJ9vFVoqvzzRQ8wHV9if0bA2PYy69pgSzzUVg4dFAbCI1vqY4OH7c/3Ht1p/LsC5pmUpaMiBdU+TW2JWryLC 2V374y/z gouPkzzYEz6/YW0KrNfsyc6IevjDcRuecfkMqf6VhQQnHo31IIlhQfMGKv4bI8ZLgXVbl2rbltzCzzwth2SgaE24OAcCa6g2vbLekrYNQb89KYL3+mo1hzJzhYmkfcozaBPy/yu4D4JCK7yml+RFIAMmw4e86wzcCq1DWvAtxhNG90/OmdNe5L2UFP90JQXo+oyrerz0AGbmZ26jft6FBetKIXFl+eSWW+hkrOvm+TpaQYE9IVR0/VZm4jw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Oct 25, 2024 at 10:15=E2=80=AFAM Marc Zyngier wrot= e: > > On Fri, 25 Oct 2024 06:07:45 +0100, > Yu Zhao wrote: > > > > Hi Marc, > > > > On Tue, Oct 22, 2024 at 9:03=E2=80=AFAM Marc Zyngier w= rote: > > > > > > On Mon, 21 Oct 2024 05:22:15 +0100, > > > Yu Zhao wrote: > > > > > > > > @@ -1407,6 +1418,13 @@ static void gic_ipi_send_mask(struct irq_dat= a *d, const struct cpumask *mask) > > > > */ > > > > dsb(ishst); > > > > > > > > + cpumask_copy(&broadcast, cpu_present_mask); > > > > > > Why cpu_present_mask? I'd expect that cpu_online_mask should be the > > > correct mask to use -- we don't IPI offline CPUs, in general. > > > > This is exactly because "we don't IPI offline CPUs, in general", > > assuming "we" means the kernel, not GIC. > > > > My interpretation of what the GIC spec says ("0b1: Interrupts routed > > to all PEs in the system, excluding self") is that it broadcasts IPIs t= o > > "cpu_present_mask" (minus the local one). So if the kernel uses > > "cpu_online_mask" here, GIC would send IPIs to offline CPUs > > (cpu_present_mask ^ cpu_online_mask), which I don't know whether it's > > a defined behavior. Thanks for clarifying. > Offline CPUs are not known to the kernel. I assume it wouldn't matter to firmware either, correct? IOW, we wouldn't cause firmware any trouble by letting GIC send IPIs to (cpu_present_mask ^ cpu_online_mask), assuming those two masks can be different on arm64 when hotplug is enabled? > Most likely, they are either > powered off, or spending quality time in Secure or Realm mode. Either > way, this is none of our business. > > Your approach would make also the kernel perform pretty inconsistently > depending on whether CPUs are offline and not. > > > > > But if you actually meant GIC doesn't IPI offline CPUs, then yes, here > > the kernel should use "cpu_online_mask". > > > > > > + cpumask_clear_cpu(smp_processor_id(), &broadcast); > > > > + if (cpumask_equal(&broadcast, mask)) { > > > > + gic_broadcast_sgi(d->hwirq); > > > > + goto done; > > > > + } > > > > > > So the (valid) case where you would IPI *everyone* is not handled as = a > > > fast path? That seems a missed opportunity. > > > > You are right: it should handle that case. > > > > > This also seem an like expensive way to do it. How about something > > > like: > > > > > > int mcnt =3D cpumask_weight(mask); > > > int ocnt =3D cpumask_weight(cpu_online_mask); > > > if (mcnt =3D=3D ocnt) { > > > /* Broadcast to all CPUs including self */ > > > > Does the comment mean the following two steps? > > 1. Broadcasting to everyone else. > > 2. Sending to self. > > Correct. > > > My understanding of the "Interrupt Routing Mode" is that it can't > > broadcast to all CPUs including self, and therefore we need the above > > two steps, which still can be a lot faster. Is my understanding > > correct? > > Yes. > > Thanks, > > M. > > -- > Without deviation from the norm, progress is not possible.