From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8975C3F2CD for ; Tue, 3 Mar 2020 14:13:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 99437208C3 for ; Tue, 3 Mar 2020 14:13:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="I57wy1bZ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 99437208C3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2F9426B0005; Tue, 3 Mar 2020 09:13:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2AADC6B0006; Tue, 3 Mar 2020 09:13:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1E6726B0007; Tue, 3 Mar 2020 09:13:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0239.hostedemail.com [216.40.44.239]) by kanga.kvack.org (Postfix) with ESMTP id 034A16B0005 for ; Tue, 3 Mar 2020 09:13:13 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id CEFD8824805A for ; Tue, 3 Mar 2020 14:13:13 +0000 (UTC) X-FDA: 76554243066.20.girls71_73713514c1f62 X-HE-Tag: girls71_73713514c1f62 X-Filterd-Recvd-Size: 4502 Received: from mail-qk1-f170.google.com (mail-qk1-f170.google.com [209.85.222.170]) by imf31.hostedemail.com (Postfix) with ESMTP for ; Tue, 3 Mar 2020 14:13:13 +0000 (UTC) Received: by mail-qk1-f170.google.com with SMTP id m2so3467025qka.7 for ; Tue, 03 Mar 2020 06:13:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to; bh=KehBbwY8xejXFJnURHhNkCBl//IgAXF4mBmFJJ9y78g=; b=I57wy1bZR4jYUlZnVg2GT2efRrhLWQiwWuiYb3xeHtgIW1IFDYWk3Wk79E1Puj0IG3 BvDLnNhqpNbQTYGj4UM42fa/+ggZEbp9ZnSE0+zoTIEMrOKfBkRaUlMO3EFN6HpalKPO h4jZk8ZOiUmp6R+iDROM1JCx3HHcSQDyk6oFqpXLVVsqPF0ScxcBIVGBifzIgaA+u5tW GmMofp6lzdgXN1SntQgd2bVOJDm8EhJm2juMnWCZyIUhZB52Z3j4qwecB4JyoB1OHq0P GKXZcLayPwECEwkfKKbyPyqmLnmLz2Faf2I7c8NWbW4WZve4WBtHu6MlKqM9dDqQpLnP Q+IQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition :content-transfer-encoding:in-reply-to; bh=KehBbwY8xejXFJnURHhNkCBl//IgAXF4mBmFJJ9y78g=; b=GjN73ScxQZ0cgIuV2wW+gqNiN5/RYkrR1rOAmA3oFRUvMtmfma1MbBmNElGj/OmwPw nqn08HrxjkMtPyNcvBt1AJ4Bj0GcvNuIF5+xusUlDms76HOSa7paxFi3B2h+PArwpo7u 0/8LYXq78cncqVZjCFN+hOYKNINo8bEmKFIp/j3CTF48vy4j3c7WznpVh7Fhqwe1jJRd 2QnHc/9eemo3S9Fw93v5KwmyM8xYoY6z7Aa7mgupeQ7gNHDdtCHjmmt1E8AwMr+qT4Yd ppreD6I9KzdaVeCabJ0PDeE3+6YHbdK9jLOBQGuWfK9lmCyq6AuoTmUPEynQi3fSMwBJ rERw== X-Gm-Message-State: ANhLgQ3KJUMygSGjGPxorQtM4SjaINHbLKW3OCsd7BsmCFcnbygRCRaW JsQVrC9yI1OtAvQLSe6NzuM= X-Google-Smtp-Source: ADFU+vtUO9HsiXNFIE6BWCyk8HbS746XX+UIKswbThDXVPeHj+co9Q88CKTqRIDHNQcWqX6bC4HMTA== X-Received: by 2002:a05:620a:22cc:: with SMTP id o12mr4364385qki.331.1583244792301; Tue, 03 Mar 2020 06:13:12 -0800 (PST) Received: from localhost ([2620:10d:c091:500::7f70]) by smtp.gmail.com with ESMTPSA id v6sm3677701qkg.102.2020.03.03.06.13.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Mar 2020 06:13:11 -0800 (PST) Date: Tue, 3 Mar 2020 09:13:11 -0500 From: Tejun Heo To: Michael Stapelberg Cc: Miklos Szeredi , Jack Smith , fuse-devel , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [fuse-devel] Writing to FUSE via mmap extremely slow (sometimes) on some machines? Message-ID: <20200303141311.GA189690@mtj.thefacebook.com> References: <20200303130421.GA5186@mtj.thefacebook.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hello, On Tue, Mar 03, 2020 at 03:03:58PM +0100, Michael Stapelberg wrote: > Here=E2=80=99s a /proc//stack from when the issue is happening: >=20 > [<0>] balance_dirty_pages_ratelimited+0x2ca/0x3b0 > [<0>] __handle_mm_fault+0xe6e/0x1280 > [<0>] handle_mm_fault+0xbe/0x1d0 > [<0>] __do_page_fault+0x249/0x4f0 > [<0>] page_fault+0x1e/0x30 >=20 > How can I obtain the numbers for the next step? Yes, that's dirty throttling alright. Hopefully, the balance_dirty_pages tracepoint which can be enabled from under /sys/kernel/debug/tracing/events/writeback/balance_dirty_pages/ should tell us why bdp thinks it needs throttling and then we can go from there. Unfortunately, I'm rather preoccupied and afraid I don't have a lot of bandwidth to work on it myself for the coming weeks. Thanks. --=20 tejun