From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA965EE0212 for ; Wed, 11 Sep 2024 06:43:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BFD8C90000C; Wed, 11 Sep 2024 02:43:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BAEB38D00E2; Wed, 11 Sep 2024 02:43:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A767190000C; Wed, 11 Sep 2024 02:43:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 846EA8D00E2 for ; Wed, 11 Sep 2024 02:43:15 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 092CA1413C4 for ; Wed, 11 Sep 2024 06:43:15 +0000 (UTC) X-FDA: 82551515550.20.67D401D Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf18.hostedemail.com (Postfix) with ESMTP id C380C1C0004 for ; Wed, 11 Sep 2024 06:43:12 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=CDXg6+Xe; spf=pass (imf18.hostedemail.com: domain of leobras@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=leobras@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1726036888; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GG53hfFZsBWWfq8LdPjycb2UAQG90BS+o/E8pJ1P9Wk=; b=4qE3exYmd+f6TnwJTRi+uD6Ahbftz4c7uE3ZW2xTsMWU621APfEpdPTHbz2FQ1vWUubd3A zvonfW7IfGnEoY7a9ZCM6HJ4VKH7/sOn2RAqnpd1UqwUGBnQbTrhPdQH15mEMqAvJH11U/ YcUc6hru01YxKv8Cj7SyYErc/j4bxD0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1726036888; a=rsa-sha256; cv=none; b=YzbDUCaBOZ3bc9FkCLB/Qggh7fFC46VfS0DbZbBddY/4N7TD4E9X56u7IGe/8dn7EoW8X7 LlKFfq5H7GpYYyfwXAUEm8obRAccmkoh3RM0b5Pi8MHj9aQtXeTfqsvqfnTQRMT8V3YsMn a9Rr6j2IGSmHXMsSBzOSkMguRn0DZl0= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=CDXg6+Xe; spf=pass (imf18.hostedemail.com: domain of leobras@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=leobras@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1726036992; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GG53hfFZsBWWfq8LdPjycb2UAQG90BS+o/E8pJ1P9Wk=; b=CDXg6+XefG2jAx6CxeBl6/jH3d4ONTC6hY2zQJkIFgpZZbwkK/3zwuvIxBxtCQPxLL+2iX OJY2GqSTEOAeNnXF6J7pm6vGPjDGJPc9Jg+HDKxxz8+y1D96pMqAKDP8lfiJVZ5fR+aal7 Jm5Elwu1MH7O64ppaQqXR9rIPuObkM0= Received: from mail-pf1-f199.google.com (mail-pf1-f199.google.com [209.85.210.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-605-M6Iri0JNP5ONs1FGlxjfEw-1; Wed, 11 Sep 2024 02:43:10 -0400 X-MC-Unique: M6Iri0JNP5ONs1FGlxjfEw-1 Received: by mail-pf1-f199.google.com with SMTP id d2e1a72fcca58-718e82769aeso2648419b3a.0 for ; Tue, 10 Sep 2024 23:43:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726036990; x=1726641790; h=content-transfer-encoding:content-disposition:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=GG53hfFZsBWWfq8LdPjycb2UAQG90BS+o/E8pJ1P9Wk=; b=i+Oe2uRwlHEJLmVrqTnFiCS8Ymox9Ww3G9UvxZ269Am0cZmvyp2AWttBKAJ5Oda3Lq nAA/6UrtfZ/qQ2dz0tXV5V/1fz3VGOKqadL1QVHPZa3MDL4bRRgj4n72b1TPr9Fts72K OYrs2eUC2iCU5Zfz55VMM2RoJvhb3XxP3xHPaO/nkRjBu7Yx5PqexbM0iHHehf0vQvGs K5gGYW4sBSQIhcLXs49fDnCDaQxyAUf0ARJL1C45BGGUW+buTC24ad5kUpQZCNyGcJnM 6XBpkupw+KG0mxOvjNv4mJHXDBQ093gA87gIvw9mqUBLJrfobY4hsMxxsESgdEu8Pz/b o99Q== X-Forwarded-Encrypted: i=1; AJvYcCVQsBVMEtn9N/q7gw8GlPbyvfSLFVR+fMrICznNIR3z2QZgN3kW7MHsPKrx+7FRz32IL/PrWNn/IQ==@kvack.org X-Gm-Message-State: AOJu0YwE4eRuq5beKmkRi2PRTxo9M9pvTliVdtXPxLOt0EuBt86XSwo3 EpCxbM0f07pw9GHd+wVciKuur+4hCwytMQrYKjXQuXe905xAf5JFw4qgF38Uc5gw+101YlJVc6a PP6TyML+84Wk5SO3I3Ka0LzQII7587yTo6mZEJXTEj3hvznVf X-Received: by 2002:a05:6a00:b45:b0:719:1df4:9d02 with SMTP id d2e1a72fcca58-7191df49f2cmr979016b3a.25.1726036989579; Tue, 10 Sep 2024 23:43:09 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHwuQeX0ud6nORQ8TR0AXWmS5VlyhnVnzN28k65Xqpf2kX3IMRBU8JsICzb8QWIa+FRdpdzag== X-Received: by 2002:a05:6a00:b45:b0:719:1df4:9d02 with SMTP id d2e1a72fcca58-7191df49f2cmr978995b3a.25.1726036989042; Tue, 10 Sep 2024 23:43:09 -0700 (PDT) Received: from localhost.localdomain ([2804:1b3:a800:3c59:c8f1:7d33:571a:fde2]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71909003573sm2398061b3a.90.2024.09.10.23.43.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Sep 2024 23:43:08 -0700 (PDT) From: Leonardo Bras To: Hillf Danton Cc: Leonardo Bras , Marcelo Tosatti , Michal Hocko , Roman Gushchin , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [RFC PATCH v1 0/4] Introduce QPW for per-cpu operations Date: Wed, 11 Sep 2024 03:42:49 -0300 Message-ID: X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240905221908.1960-1-hdanton@sina.com> References: <20240905221908.1960-1-hdanton@sina.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: 8bit X-Stat-Signature: x4bhwrtwcmzjbafboh4nkqtktfduxa3r X-Rspamd-Queue-Id: C380C1C0004 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1726036992-418086 X-HE-Meta: U2FsdGVkX191B+l58k5Qfvdo+lKNd2RIJKbmyc+3V6+pxtSlEmtb7FJQ5A5pOiPmF97toJeiEXrVDpWGLL3JIhNGB9xNJLIMT2p06QgpzydV8Jtj7KQzVh3a6tQMFUsMXfphIHxwRjqUWoLn52e/0e/cp2XeIqzZsjme4+XRzMH0eZdDy10eEzB99d9KeoT3I3V+xHjh2TTiLnDFGHX/Axtjsh4oZLQ6XAGtLyPPKXnaBOANxBzw1ncWReMJqgtHVKyy2mAcLBGw0i3+F7CO99Mrkd3k8t5ZW35OxfLYNCyyprciilwxEA1b5Hqnr3AxGfxeiIgYWlFcuma/Cq925dXhF8ltcULKdBYfVmKCZgvFoyk6Rng58Ou5UiLlaHFAMZpDvjDl53Znq2xQUXDd773MSze6ikmxnR6GSlwr04DL4b0bV1PULM2tOdO0zYdsFA+xWNjxkDUV9WrRIL8zxq1ip5ZOWHzgFfmG6yWfEuZ0Khptw+hLjW/ds2ZRZHhsxdc44SCNPK4H3G/1QW5ew8/tv9SDOI3gJRmV0kPvbbbuXURx/3ckBY2ls38kNkYJiB9v6EMH9wdIeQ2KweThfLtAHUuGA1tdDuqAtTXsfYCuhG8GHver7I+lFWNPOH5KYBCJw1Q5KcrphTGlrhL0YsgRSMgZoRcfgg/2uaxVhlcgMKLERoSgVNLrp2gBFrSVN/9DotoPue1FhmwbbwPtIuQIowYT13AyTqQZYCKYcjY69zuTfSEJKFzVJG/Aw4pmy12w4h9a8LNPzc10So6dfVP4WJTcsE0lPTKxmWJJWYULQ3Qh3mdUaZKcx/kFloVdjKRTCw5lLu3tvA2stnN75fZCesgsIOSkS6Fj4JIBW5Tf97dXpD484CZy7Xhih1VZ2crYktt6WK/QPtC4DAdh+S0ur8B7MLgpGpVbkS6oPUyTCEM/pyDeHRzaEL8j3KT0TjuGh5ce6/pHmW6qx/V bkLiMMX1 hy4+QaYMsEWBCzHoFgJYoikEbc4HdzPew/phYY+Xv7GYG5Omo+CQjQKOkWmE5HB2W4LjXEmcRY35s5br7EWmOqVef7GBufhBGXGTdr7sbZT2QQyBmrOmJIVIWs++iAAKtgJioCFPMfeahjLO0DZaj9ryK1SO8TcNxMBHmKl3dviAdeR+2krMTCggdXTje0TAtsY/wj/CVzm5UA/TQxfVyboSaBx6LrQAC4jrHwIFQymlhDPL5vzfOX2c7kvrZWDwqDcvJiTxSy1GOrs112KX6/aXaYxkWGO5TEUlNvjAJv6kMCcOOI4mfpgvOWcb52e/+2K5CEDKXXxplX7/IWfyhiYSsFwp6T/YJnWbeF6E1PcLRdAtWnlelL3fRnGIOSgk/kQj1nU3TNeRFHlze+45FDnF0hvxa4LTpjGNY X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Sep 06, 2024 at 06:19:08AM +0800, Hillf Danton wrote: > On Tue, 23 Jul 2024 14:14:34 -0300 Marcelo Tosatti > > On Sat, Jun 22, 2024 at 12:58:08AM -0300, Leonardo Bras wrote: > > > The problem: > > > Some places in the kernel implement a parallel programming strategy > > > consisting on local_locks() for most of the work, and some rare remote > > > operations are scheduled on target cpu. This keeps cache bouncing low since > > > cacheline tends to be mostly local, and avoids the cost of locks in non-RT > > > kernels, even though the very few remote operations will be expensive due > > > to scheduling overhead. > > > > > > On the other hand, for RT workloads this can represent a problem: getting > > > an important workload scheduled out to deal with remote requests is > > > sure to introduce unexpected deadline misses. > > > > Another hang with a busy polling workload (kernel update hangs on > > grub2-probe): > > > > [342431.665417] INFO: task grub2-probe:24484 blocked for more than 622 seconds. > > [342431.665458] Tainted: G W X ------- --- 5.14.0-438.el9s.x86_64+rt #1 > > [342431.665488] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. > > [342431.665515] task:grub2-probe state:D stack:0 pid:24484 ppid:24455 flags:0x00004002 > > [342431.665523] Call Trace: > > [342431.665525] > > [342431.665527] __schedule+0x22a/0x580 > > [342431.665537] schedule+0x30/0x80 > > [342431.665539] schedule_timeout+0x153/0x190 > > [342431.665543] ? preempt_schedule_thunk+0x16/0x30 > > [342431.665548] ? preempt_count_add+0x70/0xa0 > > [342431.665554] __wait_for_common+0x8b/0x1c0 > > [342431.665557] ? __pfx_schedule_timeout+0x10/0x10 > > [342431.665560] __flush_work.isra.0+0x15b/0x220 > > The fresh new flush_percpu_work() is nop with CONFIG_PREEMPT_RT enabled, why > are you testing it with 5.14.0-438.el9s.x86_64+rt instead of mainline? Or what > are you testing? > > BTW the hang fails to show the unexpected deadline misses. I think he is showing a client case in which my patchset would be helpful, and avoid those stalls in RT=y. > > > [342431.665565] ? __pfx_wq_barrier_func+0x10/0x10 > > [342431.665570] __lru_add_drain_all+0x17d/0x220 > > [342431.665576] invalidate_bdev+0x28/0x40 > > [342431.665583] blkdev_common_ioctl+0x714/0xa30 > > [342431.665588] ? bucket_table_alloc.isra.0+0x1/0x150 > > [342431.665593] ? cp_new_stat+0xbb/0x180 > > [342431.665599] blkdev_ioctl+0x112/0x270 > > [342431.665603] ? security_file_ioctl+0x2f/0x50 > > [342431.665609] __x64_sys_ioctl+0x87/0xc0 >