From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA55AEE0211 for ; Wed, 11 Sep 2024 07:19:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 437278D00FC; Wed, 11 Sep 2024 03:19:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3E6D98D0056; Wed, 11 Sep 2024 03:19:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2AE538D00FC; Wed, 11 Sep 2024 03:19:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 0DC858D0056 for ; Wed, 11 Sep 2024 03:19:06 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id AC2E21C529E for ; Wed, 11 Sep 2024 07:19:05 +0000 (UTC) X-FDA: 82551605850.19.197C148 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf14.hostedemail.com (Postfix) with ESMTP id 73DFB10000B for ; Wed, 11 Sep 2024 07:19:03 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=T4JBm2uh; spf=pass (imf14.hostedemail.com: domain of leobras@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=leobras@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726039115; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WExmWVFUVT+GeIi/yPIGow20ZicextOrMJ+vbOaBSZo=; b=bVcBo0BEenmCfnlCT+aHfTE8sW/HvDGvscbirvIIUuk6hORSfqNA6il2FK2hLQjkUtsJWj 93mywbxikWh5VDr9FNfQ1oJH8XAWqfQ+1/nDC51C+I1hmk+ykqFVpgeHGlsdh3kA21ogmx gQDT64wN5BOb8jfcd7CxWL5bvtXtBQY= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=T4JBm2uh; spf=pass (imf14.hostedemail.com: domain of leobras@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=leobras@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726039115; a=rsa-sha256; cv=none; b=x1OVLIF8NbD4/Qmoii3SOSSn2Vq3/5sTTxFB2yE4lIw2i4nGDm5icVGJgzxYcEboMBP6QF ikabKO+1ZyInwHtDligy1ab4XIpaPkJYmeDcFc8QowMxkgiv4pCGS8/EHGPTAaXO8BKxMJ gkVs0LQoB4DrPCFiwU6FVo2UGXZA2to= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1726039142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WExmWVFUVT+GeIi/yPIGow20ZicextOrMJ+vbOaBSZo=; b=T4JBm2uh0PRqA3P8TsHzb5VIzlOt/05UGww+uobu1N8WMemFQpuX5Hg68GnaKNMgm8yLBS J5j8cQ1uqpbadIioo7B/wK7dC2+K0Fi+XxuJg47PTsrFgn7r+PedxtBF22FOvoOW2jG1Ps Vb7r1BNlSPfAoffYfs7vbWnpXwYq+es= Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-584-bjvAJ3T-NSCzV5QeJwk_QA-1; Wed, 11 Sep 2024 03:19:01 -0400 X-MC-Unique: bjvAJ3T-NSCzV5QeJwk_QA-1 Received: by mail-pf1-f197.google.com with SMTP id d2e1a72fcca58-7179469744dso2607956b3a.2 for ; Wed, 11 Sep 2024 00:19:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726039140; x=1726643940; h=content-transfer-encoding:content-disposition:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=WExmWVFUVT+GeIi/yPIGow20ZicextOrMJ+vbOaBSZo=; b=It0o0gz2LNqpmJCsX94LNHW8qBGq391Bv3fZGU9K6SHermf3uHtF1QjfqcamGSv6nq 3CgxQ+9vlHjLkifrbbZZhRfrJTErz3EIvBSCJKN+lys9HLGbKPF64xUSql3gONR3H57I d+4WakqTS6JkTE+YtYXCfyJBtr5ovmiD/qe+JR1Ng4E2RyGfWncTRI+/xh1zmZlHkY6y 4WoJk5dD1UVqbfJ6dxXeFEutCi+psrXAGlRHMVynivRFiFxNHg5tb8iQu7kJr26wIbUx u0Qrb7O/8XcIfQoPXJIz7jTeNllPhWMb6Fk44qhgP6eNag7hYQOlc2pMhSL/nsdReJau SsQQ== X-Forwarded-Encrypted: i=1; AJvYcCVVAcPNp+15x7QNwLG5265I/nLl0cJuKqcrnC8qtMsE1iwG3c4INQLbDgCVzWk+8wnZIX2RYOy9gw==@kvack.org X-Gm-Message-State: AOJu0Yy2MhuGysdGzvKGiQi36LZ0H30QevAvbKJw00HFxIcKT6LNrbg2 Yc2xp11YLm1VsB+YsetdLayjQNqie94LKH+IBHKKXjsm0EgPYjI8hQuMeiEwBFuVTjvr0L5u6T0 r68iGxr8CLesifCZEH/Z6+j07V2BLeEjElUupBKWVmSQ5O/37 X-Received: by 2002:a05:6a00:66c8:b0:70a:f576:beeb with SMTP id d2e1a72fcca58-71916df6fdfmr2660121b3a.15.1726039140094; Wed, 11 Sep 2024 00:19:00 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGc3lptb75oVWaJldXqEJfWwtFJcwuDY5ronghp6vVOfZyKrsvLvrHkcR+YcX3R+mxuPDiR2Q== X-Received: by 2002:a05:6a00:66c8:b0:70a:f576:beeb with SMTP id d2e1a72fcca58-71916df6fdfmr2660082b3a.15.1726039139608; Wed, 11 Sep 2024 00:18:59 -0700 (PDT) Received: from localhost.localdomain ([2804:1b3:a800:3c59:c8f1:7d33:571a:fde2]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71908fc8febsm2466665b3a.19.2024.09.11.00.18.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Sep 2024 00:18:59 -0700 (PDT) From: Leonardo Bras To: Waiman Long Cc: Leonardo Bras , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Thomas Gleixner , Marcelo Tosatti , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Subject: Re: [RFC PATCH v1 1/4] Introducing qpw_lock() and per-cpu queue & flush work Date: Wed, 11 Sep 2024 04:18:42 -0300 Message-ID: X-Mailer: git-send-email 2.46.0 In-Reply-To: References: <20240622035815.569665-1-leobras@redhat.com> <20240622035815.569665-2-leobras@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit X-Rspam-User: X-Stat-Signature: n53m5io56dzxbdr69fhfewpxw9upzg6g X-Rspamd-Queue-Id: 73DFB10000B X-Rspamd-Server: rspam11 X-HE-Tag: 1726039143-873454 X-HE-Meta: U2FsdGVkX1+iYQvLFOgO05Vfe2hTzEkmam/xkwj6oKNB59LTfjkATz+lZKLOGJhHRhR9dMWx0TXdpykS47mKZVOuSLK6UwHlw1W88maRz54p39dhu+QBRdLIMTbKfrH8N6IZNA9NVgFDsn2HN+JNzg4k7Fhq12Ox7Jzuwk2hivKS3vJezNVkvqpSxkXKAGZoT8k/w6TaSinllgonpw3N6H2TsVoRIwuTcj/a9oFGWL7u58Zed1fP5y2uweilewIdFmU0CFySWcELC5F9YzFEvU+Nqy8FD8oFVED7k4IlsMj0rbrBJKFu70sMWXmFE1fdRkPsloj761aynEZjyToHrBo7ONNBvdtZgQRLvOKD84ltbmY+fz4+L/b9baN43RDm66/FTLx10R0cHqik+lLD+qMyQXarPuiHIIPHAysF+7WM1+bZkVMfYcZDOmrtM4lHzGErl83AMXY077zuZrzlJeTRvOmx3RzjOHGdX9eO8aaC7+hIVsY1tHNqdCzRV/YIXI0Xj7SyYQvuemU/LR/wiL+O/lBvZ+yYHYFiQcqL7mPmOJ0lkmuW54eAA30iI8Hhp8ldk7F7VhUXapf3PzWap8aZ/lir3azG5Im4tNiDpOA0z2dD2Uexm9y9dID9LQd4sOXuwjY7YiGKyokikzsoIBNBp9XjdH1zsWUltnoFfDXZ9hbM9UngqD7sEWw+5sSbOMbWUrp//oiF+UAITI6ileblfBj0miJZdzTlpRumhemEQeXXJdrYN60WH9J8HFO44vxxIz3k6FVC1PtbJ/r63rtYhYsATk5Yk0JFT6Xd8YvdZHkltlq7h3QfglJ/V7SqnYn4rJaqj2DL/a3cr2HZ1gHD5Lk7Zagz+Z0gEjYuMQc9uZIFM35aw53Y1os5XGt46TAV4yfnK4uMr2EG//WFk6r6DSTH56rMqA8YLvSrtk5JmagVNvj7RXipBzsyNj3WjanaQt2H0nwd34wbJ2t QI0XsyAu FIMd9bNfAaW2kOjPcUdh0spLlZJ4y4kK+ZUhDJkfB41LdtVNWJF4ILaAFS4/cBa9TJF7n2KUPjiJ07Nzbvu2IU7FJMccE7vE4uQqnbLbjBF/Q4j1PFFkUNSCaC3qgDyjqSCMwyIglUUqUmORxpUu++KyF2ySDdkC8Ohxhr5gaFL/2hxBd9cWXlWm0pjmo6e1TdSYYHPG6aq3CKhStX9ZmMqN9B0DT5iLM0WYp6Uc3ccfwQESmQpK8MNuJiGzXllFwTWlWfiOiNY8QDEwi03xo7xFEVOU/4AdtSEqPRohYs6VotWwz8O2DVlAEqL7rucBx+BmeXwgXTJ0ghgAqTCm51+osFcO/iudJXXTefW0tTlZTaoZRvo+GjgUv9zEaK4aZiVD0a88nWcL94SGogTaQp/v5EK0epVRzJamUeWbHjQlVYtX4466qCCRaUtznPVyx8RFuofxqHrO+wNVMBXeR3qtGo44MwmvmJV8Ha+vl6DbAqbI2DrDlQlFn8ObIX3UQ9vmRHdCimFi1ac3CDTE1eoxVilCDakk66LF4DPVR/Zi7wW0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Sep 04, 2024 at 08:08:12PM -0400, Waiman Long wrote: > On 9/4/24 17:39, Waiman Long wrote: > > On 6/21/24 23:58, Leonardo Bras wrote: > > > Some places in the kernel implement a parallel programming strategy > > > consisting on local_locks() for most of the work, and some rare remote > > > operations are scheduled on target cpu. This keeps cache bouncing > > > low since > > > cacheline tends to be mostly local, and avoids the cost of locks in > > > non-RT > > > kernels, even though the very few remote operations will be > > > expensive due > > > to scheduling overhead. > > > > > > On the other hand, for RT workloads this can represent a problem: > > > getting > > > an important workload scheduled out to deal with some unrelated task is > > > sure to introduce unexpected deadline misses. > > > > > > It's interesting, though, that local_lock()s in RT kernels become > > > spinlock(). We can make use of those to avoid scheduling work on a > > > remote > > > cpu by directly updating another cpu's per_cpu structure, while holding > > > it's spinlock(). > > > > > > In order to do that, it's necessary to introduce a new set of > > > functions to > > > make it possible to get another cpu's per-cpu "local" lock > > > (qpw_{un,}lock*) > > > and also the corresponding queue_percpu_work_on() and > > > flush_percpu_work() > > > helpers to run the remote work. > > > > > > On non-RT kernels, no changes are expected, as every one of the > > > introduced > > > helpers work the exactly same as the current implementation: > > > qpw_{un,}lock*()        ->  local_{un,}lock*() (ignores cpu parameter) > > > queue_percpu_work_on()  ->  queue_work_on() > > > flush_percpu_work()     ->  flush_work() > > > > > > For RT kernels, though, qpw_{un,}lock*() will use the extra cpu > > > parameter > > > to select the correct per-cpu structure to work on, and acquire the > > > spinlock for that cpu. > > > > > > queue_percpu_work_on() will just call the requested function in the > > > current > > > cpu, which will operate in another cpu's per-cpu object. Since the > > > local_locks() become spinlock()s in PREEMPT_RT, we are safe doing that. > > > > > > flush_percpu_work() then becomes a no-op since no work is actually > > > scheduled on a remote cpu. > > > > > > Some minimal code rework is needed in order to make this mechanism work: > > > The calls for local_{un,}lock*() on the functions that are currently > > > scheduled on remote cpus need to be replaced by qpw_{un,}lock_n*(), > > > so in > > > RT kernels they can reference a different cpu. It's also necessary > > > to use a > > > qpw_struct instead of a work_struct, but it just contains a work struct > > > and, in PREEMPT_RT, the target cpu. > > > > > > This should have almost no impact on non-RT kernels: few this_cpu_ptr() > > > will become per_cpu_ptr(,smp_processor_id()). > > > > > > On RT kernels, this should improve performance and reduce latency by > > > removing scheduling noise. > > > > > > Signed-off-by: Leonardo Bras > > > --- > > >   include/linux/qpw.h | 88 +++++++++++++++++++++++++++++++++++++++++++++ > > >   1 file changed, 88 insertions(+) > > >   create mode 100644 include/linux/qpw.h > > > > > > diff --git a/include/linux/qpw.h b/include/linux/qpw.h > > > new file mode 100644 > > > index 000000000000..ea2686a01e5e > > > --- /dev/null > > > +++ b/include/linux/qpw.h > > > @@ -0,0 +1,88 @@ > > > +/* SPDX-License-Identifier: GPL-2.0 */ > > > +#ifndef _LINUX_QPW_H > > > +#define _LINUX_QPW_H > > I would suggest adding a comment with a brief description of what > qpw_lock/unlock() are for and their use cases. The "qpw" prefix itself isn't > intuitive enough for a casual reader to understand what they are for. Agree, I am also open to discuss a more intuitive naming for these. > > Cheers, > Longman > Thanks! Leo