From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B0C2C2D0C5 for ; Thu, 12 Dec 2019 01:29:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C4513208C3 for ; Thu, 12 Dec 2019 01:29:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20150623.gappssmtp.com header.i=@kernel-dk.20150623.gappssmtp.com header.b="tBcfwgxJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C4513208C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4CC196B34FA; Wed, 11 Dec 2019 20:29:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 47CE36B34FB; Wed, 11 Dec 2019 20:29:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 393756B34FC; Wed, 11 Dec 2019 20:29:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0174.hostedemail.com [216.40.44.174]) by kanga.kvack.org (Postfix) with ESMTP id 245896B34FA for ; Wed, 11 Dec 2019 20:29:53 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id CCA85181AEF15 for ; Thu, 12 Dec 2019 01:29:52 +0000 (UTC) X-FDA: 76254757824.10.queen04_1c0225e683d33 X-HE-Tag: queen04_1c0225e683d33 X-Filterd-Recvd-Size: 5954 Received: from mail-pf1-f196.google.com (mail-pf1-f196.google.com [209.85.210.196]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Dec 2019 01:29:52 +0000 (UTC) Received: by mail-pf1-f196.google.com with SMTP id q8so216077pfh.7 for ; Wed, 11 Dec 2019 17:29:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=DhyX9yKmk3Bjq16+cH6CFIH0oBue170bHYeXiMOQ8dA=; b=tBcfwgxJPBhRRcAGhCQgQbyfTfpQsyiGY9DGe/XHH2/qbpQARxKq7yWjGdvqo1IllS dEWH4g1714qg29h4GEZigEfbqrKvxMzRE9LNVpEFK2NyrdjbNtkxs6UWbtUszECnOQ0D VEoKTKtnfhk/vEZGeDpWMdwNOGh8La+bbktv7kvota82pgoXLzk3sL28vntZlDhqgkLX +EyfDp8OYP1xKcWrZTib+Ng+VSUqbKeEhhTBmOwpw+Ojk7X5nKqyrRMXwloZOmKkzPiZ +9yGgbU006bu6ndm0cPeWABAqJMpqOiOI+vxaISk+h/Wium9aaoWX5gz51q7TBIt5Pl3 er3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=DhyX9yKmk3Bjq16+cH6CFIH0oBue170bHYeXiMOQ8dA=; b=hThCFUZOMpg9YBa5HO/gvqqc+B/oneGYKzxVTRKiu38EEu3v4taZLOv5oDjEy+UAqO iRzwREE6wfAOyHqzi3QQi2jf+8s59IdTanZAZfioHD+dqEhhyCJWVA78yQxrvpinGfFl ZFDmk0ZsWeNx0AeBX0ehQmTrXVfAQniQQyMpjLeyfyH8ZaPPy2CXbQiXtxpLmKUraJiU 286qNq49tQQay+ch+znwLl1iDJd4Rh0nzMqAKhLZv5oNAZG8UYI0+BmEQuS88kqA2Oav pG3Tz7FQHtFBa1C7XAqhxeHNuMWzBhEr9HOMkIIUrgp60gh+esIO+4d2ICj4KCigfWqA GEgQ== X-Gm-Message-State: APjAAAUbp/yY+2xIUuZuiGIRtE/RwV/DrszQHDfpKvZm6bw0deMuZ9pa hE9LcDFXEjGk5MVxxGFgobb1EQ== X-Google-Smtp-Source: APXvYqwIZT80qb6KK6M7DWfi9Y0s3aK7ho5OWU9hmanRrqTeBJgLbXHTk13enGqUZ5WC+TJcO9HU0Q== X-Received: by 2002:a63:fd0a:: with SMTP id d10mr7322984pgh.197.1576114191223; Wed, 11 Dec 2019 17:29:51 -0800 (PST) Received: from [192.168.1.188] ([66.219.217.145]) by smtp.gmail.com with ESMTPSA id z14sm2580858pfg.57.2019.12.11.17.29.48 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 11 Dec 2019 17:29:50 -0800 (PST) Subject: Re: [PATCHSET v3 0/5] Support for RWF_UNCACHED To: Linus Torvalds Cc: Linux-MM , linux-fsdevel , linux-block , Matthew Wilcox , Chris Mason , Dave Chinner , Johannes Weiner References: <20191211152943.2933-1-axboe@kernel.dk> <0d4e3954-c467-30a7-5a8e-7c4180275533@kernel.dk> <1c93194a-ed91-c3aa-deb5-a3394805defb@kernel.dk> From: Jens Axboe Message-ID: Date: Wed, 11 Dec 2019 18:29:46 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 12/11/19 6:22 PM, Linus Torvalds wrote: > On Wed, Dec 11, 2019 at 5:11 PM Jens Axboe wrote: >> >> 15K is likely too slow to really show an issue, I'm afraid. The 970 >> is no slouch, but your crypt setup will likely hamper it a lot. You >> don't have a non-encrypted partition on it? > > No. I normally don't need all that much disk, so I've never upgraded > my ssd from the 512G size. > > Which means that it's actually half full or so, and I never felt like > "I should keep an unencrypted partition for IO testing", since I don't > generally _do_ any IO testing. > > I can get my load up with "numjobs=8" and get my iops up to the 100k > range, though. > > But kswapd doesn't much seem to care, the CPU percentage actually does > _down_ to 0.39% when I try that. Probably simply because now my CPU's > are busy, so they are running at 4.7Ghz instead of the 800Mhz "mostly > idle" state ... > > I guess I should be happy. It does mean that the situation you see > isn't exactly the normal case. I understand why you want to do the > non-cached case, but the case I think it the worrisome one is the > regular buffered one, so that's what I'm testing (not even trying the > noaccess patches). > > So from your report I went "uhhuh, that sounds like a bug". And it > appears that it largely isn't - you're seeing it because of pushing > the IO subsystem by another order of magnitude (and then I agree that > "under those kinds of IO loads, caching just won't help") I'd very much argue that it IS a bug, maybe just doesn't show on your system. My test box is a pretty standard 2 socket system, 24 cores / 48 threads, 2 nodes. The last numbers I sent were 100K IOPS, so nothing crazy, and granted that's only 10% kswapd cpu time, but that still seems very high for those kinds of rates. I'm surprised you see essentially no kswapd time for the same data rate. We'll keep poking here, I know Johannes is spending some time looking into the reclaim side. -- Jens Axboe