From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD6BAC43603 for ; Thu, 12 Dec 2019 02:47:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 75AA02054F for ; Thu, 12 Dec 2019 02:47:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b="HQtj4NOC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 75AA02054F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F3F766B3542; Wed, 11 Dec 2019 21:47:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EEFE86B3543; Wed, 11 Dec 2019 21:47:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E05FE6B3544; Wed, 11 Dec 2019 21:47:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0050.hostedemail.com [216.40.44.50]) by kanga.kvack.org (Postfix) with ESMTP id C71C26B3542 for ; Wed, 11 Dec 2019 21:47:45 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 746ED2477 for ; Thu, 12 Dec 2019 02:47:45 +0000 (UTC) X-FDA: 76254954090.11.drop68_7dda86f666434 X-HE-Tag: drop68_7dda86f666434 X-Filterd-Recvd-Size: 5478 Received: from mail-lj1-f194.google.com (mail-lj1-f194.google.com [209.85.208.194]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Dec 2019 02:47:44 +0000 (UTC) Received: by mail-lj1-f194.google.com with SMTP id c19so446286lji.11 for ; Wed, 11 Dec 2019 18:47:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=mHahO8LZCdU8zaS7EVF3B7S7ImGonfhaVfJ1tDELB24=; b=HQtj4NOC4GpXxt0IO2bvTeez5WRIJN4iBxbju4l+SfUr9aGQjQQVxEEqv0iSvAH9Cf 9oCv9CE4ffHGIPUk2jakyqwdkFRFRmmhpkingwTTi0fiZ9fYFLy2E+IJnHkXNmeji7HD t9R7bmUbOJQIYUbypH6AG8tdoLgT7EDiX9sBc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=mHahO8LZCdU8zaS7EVF3B7S7ImGonfhaVfJ1tDELB24=; b=VMVMTdRpNQ5AaGufcP2+z88IZRowz9f4xu9gDCvaYEODRN0LGnSHdvXoX5FriAX6jn QpNO86v0dMFANjDAXHfF/ZcW9gf5bsrA+rK0zRJXE0DBwxLkcEE5Z92t4JniWG9Tv2BP 4YLiYFTQ75uD1fdTREhV84izxITfTVUtus0CXoG38bAUIa7y/48ur2Au3cJDAtnFdaTD sD8DgUSvXS4T0+Zz68AZQ7Jzbf1mW9uQ/qIUEnFfQkFWoIJqHpb+TM+RZZ4V3HE5+Rs5 5oKhaswpv2mZCKJjFpwNGy/Q5hh/ol71fUoeeypro4n5YlNftV9/QoW7C8BcR5RQdshl Errw== X-Gm-Message-State: APjAAAXpyc1ObIZ7sVu1bAS43ClV/haBJSRIzJCa9RhZd6P4cPPNzXOh aBEW0Unptt5YGx2hz8r64WnTnriiHAg= X-Google-Smtp-Source: APXvYqxuJz3F+q/KhLTIebAxWT70rXXV+7L5d2NweGFcXqFqEDuEs9hE14M2BE7Lq2KomHPq0KCBnQ== X-Received: by 2002:a2e:9694:: with SMTP id q20mr4225342lji.248.1576118862976; Wed, 11 Dec 2019 18:47:42 -0800 (PST) Received: from mail-lf1-f43.google.com (mail-lf1-f43.google.com. [209.85.167.43]) by smtp.gmail.com with ESMTPSA id d4sm2035282lfn.42.2019.12.11.18.47.41 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 11 Dec 2019 18:47:42 -0800 (PST) Received: by mail-lf1-f43.google.com with SMTP id f15so430778lfl.13 for ; Wed, 11 Dec 2019 18:47:41 -0800 (PST) X-Received: by 2002:a19:4351:: with SMTP id m17mr4341762lfj.61.1576118861439; Wed, 11 Dec 2019 18:47:41 -0800 (PST) MIME-Version: 1.0 References: <0d4e3954-c467-30a7-5a8e-7c4180275533@kernel.dk> <1c93194a-ed91-c3aa-deb5-a3394805defb@kernel.dk> <20191212015612.GP32169@bombadil.infradead.org> In-Reply-To: <20191212015612.GP32169@bombadil.infradead.org> From: Linus Torvalds Date: Wed, 11 Dec 2019 18:47:25 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCHSET v3 0/5] Support for RWF_UNCACHED To: Matthew Wilcox Cc: Jens Axboe , Linux-MM , linux-fsdevel , linux-block , Chris Mason , Dave Chinner , Johannes Weiner Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Dec 11, 2019 at 5:56 PM Matthew Wilcox wrote: > > It _should_ be the same order of complexity. Since there's already > a page in the page cache, xas_create() just walks its way down to the > right node calling xas_descend() and then xas_store() does the equivalent > of __radix_tree_replace(). I don't see a bug that would make it more > expensive than the old code ... a 10GB file is going to have four levels > of radix tree node, so it shouldn't even spend that long looking for > the right node. The profile seems to say that 85% of the cost of xas_create() is two instructions: # lib/xarray.c:143: return (index >> node->shift) & XA_CHUNK_MASK; movzbl (%rsi), %ecx # MEM[(unsigned char *)node_13], MEM[(unsigned char *)node_13] ... # ./include/linux/xarray.h:1145: return rcu_dereference_check(node->slots[offset], # ./include/linux/compiler.h:199: __READ_ONCE_SIZE; movq (%rax), %rax # MEM[(volatile __u64 *)_80], where that first load is "node->shift", and the second load seems to be just the node dereference. I think both of them are basically just xas_descend(), but it's a bit hard to read the several levels of inlining. I suspect the real issue is that going through kswapd means we've lost almost all locality, and with the random walking the LRU list is basically entirely unordered so you don't get any advantage of the xarray having (otherwise) possibly good cache patterns. So we're just missing in the caches all the time, making it expensive. Linus