From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D29FDECAAD5 for ; Fri, 2 Sep 2022 19:54:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6C34280125; Fri, 2 Sep 2022 15:54:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6734480120; Fri, 2 Sep 2022 15:54:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5132C80125; Fri, 2 Sep 2022 15:54:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 3EDC580120 for ; Fri, 2 Sep 2022 15:54:00 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 077C61212C9 for ; Fri, 2 Sep 2022 19:54:00 +0000 (UTC) X-FDA: 79868196240.20.393245A Received: from mail-pj1-f49.google.com (mail-pj1-f49.google.com [209.85.216.49]) by imf31.hostedemail.com (Postfix) with ESMTP id 7968C20061 for ; Fri, 2 Sep 2022 19:53:59 +0000 (UTC) Received: by mail-pj1-f49.google.com with SMTP id p8-20020a17090ad30800b001fdfc8c7567so8243937pju.1 for ; Fri, 02 Sep 2022 12:53:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date; bh=BgSQRPeP2vU+fzfPkC5136xUIsrQV+zIPc0zX8WqYvQ=; b=4q1ObG8WZYb+BFWPgZh7N7aITtmlNIGPzQtvKE2aYuRrU3oMZQ76tQz2zqxy31cTLt d8rIjpGYxzTxBhay1NPxlOw7vaD/UOVOLVL9kTH97EOmLP9jqiD95DNmZFsZ1Ptc8XmN SpoCT9nh9ECoUVn4HyZ7Qsn6ksQUfbOmmaLZNC8YNvNZJ0mYksYatinKwvYndzY1XvJm nky6nJQ50OCGfppKmlv+KompjFnK/xiaX17J7WZx+2ulxNlRcgnWzxkBWjasiiHjJ4Jl WgqrWN7XLOnHrvW9WLg3mCJ+/TAuDw9lCpZSCR3GLv/KhadF9BUUSKQnQ8/gjIX8jzxs EhUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date; bh=BgSQRPeP2vU+fzfPkC5136xUIsrQV+zIPc0zX8WqYvQ=; b=6HchSClB5hdcYbHU/m62rmbUlxllxa8Nhv5jGg7T+tvspB2BExDN2M2hptJmNs97IX QJJ/euJtf0orck0jsIPnFCt8MS8GeXFUjV/b6cFO+oOjl0PhO4xlS+9/bXylntHYSh78 +KsrpzYtpKR63qaRgtKCZ1YemjPMJZ26jNnGVwYSn8Y3O5RQc+AY3VCl/Vie8zYEwLwr fNHpSrdI6hA0LWKq4GKTPtwuPmb342HOxNVtN4fTL1bIwc3NzGxIEeg9Ot7WMFwnLNVv TBM33ht0QG/Ck3wX0AZ3JQ/Sv7TKK0VlD8LlNOiDmJLiI8QEZc1nNsXLrsFReqksVlnV 9suA== X-Gm-Message-State: ACgBeo1d6e6s8JSugU5ldszx6QDdOhoTgU+1dc77PWvloO5iRVcJ5mkB op3ZwQ8mMsAbYhQnvPYeIY8rRQ== X-Google-Smtp-Source: AA6agR5BylTPZ3Gegvy/NzTiavEg8hhtbTLj8aGfOurXCnjCbkfIWop802MX5aLKWWzJpm5rxypzOA== X-Received: by 2002:a17:90b:b16:b0:1fd:b47c:6ab with SMTP id bf22-20020a17090b0b1600b001fdb47c06abmr6698292pjb.203.1662148438176; Fri, 02 Sep 2022 12:53:58 -0700 (PDT) Received: from [192.168.1.136] ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id c6-20020a170902c1c600b00172ccb3f4ebsm2008369plc.160.2022.09.02.12.53.54 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 02 Sep 2022 12:53:57 -0700 (PDT) Message-ID: Date: Fri, 2 Sep 2022 13:53:53 -0600 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux aarch64; rv:102.0) Gecko/20100101 Thunderbird/102.1.2 Subject: Re: [RFC PATCH 00/30] Code tagging framework and applications Content-Language: en-US To: Kent Overstreet Cc: Roman Gushchin , Yosry Ahmed , Michal Hocko , Mel Gorman , Peter Zijlstra , Suren Baghdasaryan , Andrew Morton , Vlastimil Babka , Johannes Weiner , dave@stgolabs.net, Matthew Wilcox , liam.howlett@oracle.com, void@manifault.com, juri.lelli@redhat.com, ldufour@linux.ibm.com, Peter Xu , David Hildenbrand , mcgrof@kernel.org, masahiroy@kernel.org, nathan@kernel.org, changbin.du@intel.com, ytcoode@gmail.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, Steven Rostedt , bsegall@google.com, bristot@redhat.com, vschneid@redhat.com, Christoph Lameter , Pekka Enberg , Joonsoo Kim , 42.hyeyoo@gmail.com, glider@google.com, elver@google.com, dvyukov@google.com, Shakeel Butt , Muchun Song , arnd@arndb.de, jbaron@akamai.com, David Rientjes , minchan@google.com, kaleshsingh@google.com, kernel-team@android.com, Linux-MM , iommu@lists.linux.dev, kasan-dev@googlegroups.com, io-uring@vger.kernel.org, linux-arch@vger.kernel.org, xen-devel@lists.xenproject.org, linux-bcache@vger.kernel.org, linux-modules@vger.kernel.org, Linux Kernel Mailing List References: <20220831101948.f3etturccmp5ovkl@suse.de> <20220831190154.qdlsxfamans3ya5j@moria.home.lan> <20220901223720.e4gudprscjtwltif@moria.home.lan> <20220902001747.qqsv2lzkuycffuqe@moria.home.lan> <3a41b9fc-05f1-3f56-ecd0-70b9a2912a31@kernel.dk> <20220902194839.xqzgsoowous72jkz@moria.home.lan> From: Jens Axboe In-Reply-To: <20220902194839.xqzgsoowous72jkz@moria.home.lan> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1662148439; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BgSQRPeP2vU+fzfPkC5136xUIsrQV+zIPc0zX8WqYvQ=; b=xq4eZs8lIqXv+tmF/GpFK7TmZJ+1NEAJUSLpi0OFXtbWukhT1POWg65YJSBbPzf0gPVYoK 6YbATm3pY9BoFTgEO0+7jzffIYmjwKZjbEs2HemRLyXYVoPEO/uf/kcUsTizGsHi8U/EA7 +mI6WaAo8z9fW8P8dLQH9wx+43+BtMw= ARC-Authentication-Results: i=1; imf31.hostedemail.com; dkim=pass header.d=kernel-dk.20210112.gappssmtp.com header.s=20210112 header.b=4q1ObG8W; spf=pass (imf31.hostedemail.com: domain of axboe@kernel.dk designates 209.85.216.49 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1662148439; a=rsa-sha256; cv=none; b=oUcSRH4jm/z8jDG4aDnl7lP7f1r9Jd/Z9owG7Ukbw/aft/m72+ZsPYBxzcTy3p4tlfdKtl yGN7suDRpyN+yP41WJyH9JJfZEUJdH3iNphaNtTbhkxX7+iXNjueSlWOvfSSJFxZGRBkuc XJWHFptX9I3PDtjU+LEtnYGHk2l6L88= Authentication-Results: imf31.hostedemail.com; dkim=pass header.d=kernel-dk.20210112.gappssmtp.com header.s=20210112 header.b=4q1ObG8W; spf=pass (imf31.hostedemail.com: domain of axboe@kernel.dk designates 209.85.216.49 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none X-Rspamd-Server: rspam08 X-Stat-Signature: pj5icqewmqe8fz56mgxf7tbyjn9hhpdf X-Rspamd-Queue-Id: 7968C20061 X-Rspam-User: X-HE-Tag: 1662148439-553357 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 9/2/22 1:48 PM, Kent Overstreet wrote: > On Fri, Sep 02, 2022 at 06:02:12AM -0600, Jens Axboe wrote: >> On 9/1/22 7:04 PM, Roman Gushchin wrote: >>> On Thu, Sep 01, 2022 at 08:17:47PM -0400, Kent Overstreet wrote: >>>> On Thu, Sep 01, 2022 at 03:53:57PM -0700, Roman Gushchin wrote: >>>>> I'd suggest to run something like iperf on a fast hardware. And maybe some >>>>> io_uring stuff too. These are two places which were historically most sensitive >>>>> to the (kernel) memory accounting speed. >>>> >>>> I'm getting wildly inconsistent results with iperf. >>>> >>>> io_uring-echo-server and rust_echo_bench gets me: >>>> Benchmarking: 127.0.0.1:12345 >>>> 50 clients, running 512 bytes, 60 sec. >>>> >>>> Without alloc tagging: 120547 request/sec >>>> With: 116748 request/sec >>>> >>>> https://github.com/frevib/io_uring-echo-server >>>> https://github.com/haraldh/rust_echo_bench >>>> >>>> How's that look to you? Close enough? :) >>> >>> Yes, this looks good (a bit too good). >>> >>> I'm not that familiar with io_uring, Jens and Pavel should have a better idea >>> what and how to run (I know they've workarounded the kernel memory accounting >>> because of the performance in the past, this is why I suspect it might be an >>> issue here as well). >> >> io_uring isn't alloc+free intensive on a per request basis anymore, it >> would not be a good benchmark if the goal is to check for regressions in >> that area. > > Good to know. The benchmark is still a TCP benchmark though, so still useful. > > Matthew suggested > while true; do echo 1 >/tmp/foo; rm /tmp/foo; done > > I ran that on tmpfs, and the numbers with and without alloc tagging were > statistically equal - there was a fair amount of variation, it wasn't a super > controlled test, anywhere from 38-41 seconds with 100000 iterations (and alloc > tagging was some of the faster runs). > > But with memcg off, it ran in 32-33 seconds. We're piggybacking on the same > mechanism memcg uses for stashing per-object pointers, so it looks like that's > the bigger cost. I've complained about memcg accounting before, the slowness of it is why io_uring works around it by caching. Anything we account we try NOT do in the fast path because of it, the slowdown is considerable. You care about efficiency now? I thought that was relegated to irrelevant 10M IOPS cases. -- Jens Axboe