From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1AF75C77B7C for ; Wed, 19 Apr 2023 12:24:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 986048E0003; Wed, 19 Apr 2023 08:24:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 936638E0001; Wed, 19 Apr 2023 08:24:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 84BA68E0003; Wed, 19 Apr 2023 08:24:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 76D418E0001 for ; Wed, 19 Apr 2023 08:24:10 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 3A7C9A0140 for ; Wed, 19 Apr 2023 12:24:10 +0000 (UTC) X-FDA: 80698057860.28.2293BCE Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf16.hostedemail.com (Postfix) with ESMTP id 8040418000A for ; Wed, 19 Apr 2023 12:24:07 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=db5e0DGY; spf=pass (imf16.hostedemail.com: domain of frederic@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=frederic@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681907047; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=X1rmpqgLs2UzX+8PgjSOEUnjIblZl6W3ebdf/zStw8o=; b=ctnFmpgz6oufNzKE8iLTGthnbdOut7R/l5LP/od0i38q5fsnpH97ymz7ilzCAu7b0dbh3V NmKWT+aKcQfKZP/zlHK7+oaa3MSXtzuK2vVEG8KH2g6Z8x7sVGsH1TbKzhqRpFCxE3XEhu oaANZB3oaKYyebuIQNfg0djLEjOOK+M= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=db5e0DGY; spf=pass (imf16.hostedemail.com: domain of frederic@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=frederic@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681907047; a=rsa-sha256; cv=none; b=B98iT2XSjbuZFfxdJtf1OKwJNnBvk/HkXhYg6WjZk/SSHcy0gsn7mqCw+GCR3jiV/YG3Uh ZTFEurmdncHvRg7aKWAXk0HaKVl4z4PwvqxATojveZ5YJhjjS71avPFf6VBm2PbkuRn1+w TezPlFijvuVvA40/yewINVeHC/ivtXU= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7CB4961032; Wed, 19 Apr 2023 12:24:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 424FAC433D2; Wed, 19 Apr 2023 12:24:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1681907045; bh=VsH3Rfa+SSrVUfZObRgg0ssJsk3aX025Sp4TIkSmH8w=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=db5e0DGYh0jJ8BygAdHfCfab+ls8rvWYDyYVvgTvm9MM5M09vP9656kQmqGEw0OMO VJIkQKaySBTrmOukHlFifXV9pQ3xdOABwdO9fuJ57XWzB8xJBeaxeB4OKGhwBaqAZS yfQebVqtZeqA55RnjMfLUlNSt6T3j+5Ib6mG8GAm91GA+AvCtMpyg3vloewq4S1Z9R IyM8XFzo13QmNVSgZpJ0OLkUQMysgR8CNZxqnBgwo641fbpZIyxwWy19uRzxP5wFTY +ySd4eXktPJLeA4+6M1f5fWWtxu5/QSq7nBPm8+9wN21czPUSyb2C1A3J1sTHXwqsZ 8ojYFLq2026Tw== Date: Wed, 19 Apr 2023 14:24:01 +0200 From: Frederic Weisbecker To: Marcelo Tosatti Cc: Andrew Morton , Christoph Lameter , Aaron Tomlin , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Russell King , Huacai Chen , Heiko Carstens , x86@kernel.org, Vlastimil Babka , Michal Hocko Subject: Re: [PATCH v7 00/13] fold per-CPU vmstats remotely Message-ID: References: <20230320180332.102837832@redhat.com> <20230418150200.027528c155853fea8e4f58b2@linux-foundation.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: 7x3fa8io51tgnwdoj6g593eadkjubs8f X-Rspamd-Queue-Id: 8040418000A X-HE-Tag: 1681907047-419732 X-HE-Meta: U2FsdGVkX19UcbxSb4GVanWJT1fQ6p4hFAIqdBVYMMgi0y/ko/CmY3yc116XJfl5chhbrz9njDFiWeH1PzGqTD3PFWCkhC+XabzhujnJamsMUsH0u2wB3BvKjoCPQoWYnQihXduTQEWckqL6fOnbs4ieHKx33qjENqakDBWyIxgearVImhdEHHWev6sTsUSTERmYaecfAI7C4ZPJvaJL1HOYmlxSEkvnxKqRF3wJrLA14LqMeoXiRZ549MruCqEZU+HyQr5Eel8eN6BuFMpJkgyf3FlW7yuDxxKcY39oPa1B7hI1N0HSYKKgB4OVWX7tIwdEQSjlzMfwZakfrqe1EY3yuMBMOGEJs3R+jNOk7zXYQW3Wx6wB96rJMS71GzFjB0JNISOwaLOlDkITzlD+l0yT7GnbcPcuSgXJ3lUNZ02yk4Hhncaq6gUozNjirQbJNCPwRVVMPDd02wFGi4rt3hRbLc8bF2yvyHRqHR46JMRPDjsS8P69be/n21EBbbnXi2j1GVI7rdN/m+o/K0b8yUWhJUx9/U/EM10n/hoVPPcCIhTFwmP+VIFjPnTAxyM8JZlnZd2ILm+s5r2qB9HTGuR/wX/fVQY1ptQB0HYH5wHDSd67NLVvYGHhW3SYjs27Q1inwwoiHzIvZ72VM3/hiEirMi9z/zcOkFIj8sQwfkbaKhq/Qm8+gh1LvV+TfUGHfagL17XhphCY4e4Khk9H3gCXWh90yqgCE6Qsle3qFs9TGb8xw3uCVej86f5hJdrMlQ9bAFlWSo9BTFb/2oOGqyMycW+yDEdwN66bchkbDxA3naPlVGWO9S9fwlIt1Cs5lxvqD2U+dzddbMtSp+lnSaA01jhkjt18j0HaH20tr0y8t/YOdif1MgPqLhNAxH1cHn5yxYttMJnQPYc2CANsjWxSY/7trSQIhzQmY1nRECvt6f96T4YC/Qj3RbK2uQ7zQistOAvj+P1mk84bRB2 lvH0ZglU J6nI1jD0D3AgSFrV07fpzw0p6mKvgIr+JpSO1Tm53Jh2bu6qvTAyXRRWm4PzOL0AtVXslzzn+BUxAvPl+f0bHGcucy1ChTGRA1SWoXUoyjpcuWVGCYI4uPHSNjJX8VzMtqb12KS/vh1R7zU3lJF/DIVUOI31enDzBSBUYZ4wqX1WIJL6Q2jwuVbjW0la5/v9VYvqCwjkHQCjKhJwpQrOlho7jkDRqEjTXYyFJFClCJtpnDfkKaOxFhEQFV3KNwt0XYQuYL4W+iUVIktr3Z1nGUcHD+EFAixSTDzAkGjc0JR9Ixl36KzdqE0PI4q+cQJ79pWbbNaEY44XrO6o= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Le Wed, Apr 19, 2023 at 08:59:28AM -0300, Marcelo Tosatti a écrit : > On Wed, Apr 19, 2023 at 08:29:47AM -0300, Marcelo Tosatti wrote: > > On Wed, Apr 19, 2023 at 08:14:09AM -0300, Marcelo Tosatti wrote: > > > This was tried before: > > > https://lore.kernel.org/lkml/20220127173037.318440631@fedora.localdomain/ > > > > > > My conclusion from that discussion (and work) is that a special system > > > call: > > > > > > 1) Does not allow the benefits to be widely applied (only modified > > > applications will benefit). Is not portable across different operating systems. > > > > > > Removing the vmstat_work interruption is a benefit for HPC workloads, > > > for example (in fact, it is a benefit for any kind of application, > > > since the interruption causes cache misses). > > > > > > 2) Increases the system call cost for applications which would use > > > the interface. > > > > > > So avoiding the vmstat_update update interruption, without userspace > > > knowledge and modifications, is a better than solution than a modified > > > userspace. > > > > Another important point is this: if an application dirties > > its own per-CPU vmstat cache, while performing a system call, > > Or while handling a VM-exit from a vCPU. > > This are, in my mind, sufficient reasons to discard the "flush per-cpu > caches" idea. This is also why i chose to abandon the prctrl interface > patchset. If you're running your isolated workloads on guests, which sounds quite challenging but I guess you guys managed, I'd expect that VMEXITs are absolutely out of question while the task runs critical code, so I'm not sure why you would care. I guess not only your guests but also your hosts run nohz_full, right? I can't tell if the prctl solution which quiesces everything is the solution for you, I don't know well enough your workloads, but I would expect that the pattern is as follows: 1) Arrange for full isolation (no more interrupts/exceptions/VMEXITs) 2) Run critical code 3) Optionally do something once you're done If vmstat is going to be the only thing to wait for on 1), then the remote solution looks good enough (although I leave that to -mm guys as I'm too clueless about those matters), if there is more to be expected, I guess the quiescing prctl (or whatever syscall) is something to consider. Thanks.