From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D667C3DA4A for ; Thu, 22 Aug 2024 21:01:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0DCEF80064; Thu, 22 Aug 2024 17:01:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 065768005A; Thu, 22 Aug 2024 17:01:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E47B180064; Thu, 22 Aug 2024 17:01:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C27B78005A for ; Thu, 22 Aug 2024 17:01:17 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id E951A1619F4 for ; Thu, 22 Aug 2024 21:01:16 +0000 (UTC) X-FDA: 82481101752.20.C81AE5B Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) by imf03.hostedemail.com (Postfix) with ESMTP id 234D12001F for ; Thu, 22 Aug 2024 21:01:13 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=SK+6OPZz; spf=pass (imf03.hostedemail.com: domain of joannelkoong@gmail.com designates 209.85.160.176 as permitted sender) smtp.mailfrom=joannelkoong@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724360410; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vw/JKAnymYOmboHlkwK/DnRu8KqLmYWyOecBkJ6vDRQ=; b=pn1jm+1oeF60KGb6C4sVF2OvkhWpwWj+KCbuARix9W1Gm2Dw69kiOMbUcwIifwDejwLvW3 GO/HsId67/HDPLqKf1HE1DP+uNNu6xV79qw4rtKgQylYjeANZkJbejeK5aMEjFoUC218w7 hoTqwXx/8qAB3Er/ENiSd2pNnzlTXGY= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=SK+6OPZz; spf=pass (imf03.hostedemail.com: domain of joannelkoong@gmail.com designates 209.85.160.176 as permitted sender) smtp.mailfrom=joannelkoong@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724360410; a=rsa-sha256; cv=none; b=aMouF6dwEZHwY4DcqLvoJ2Q1LKgvxYrDHYhnUbewfBSVlzDQ5RnVZZ/mA4FENMqTGkYNWQ v3l9jgIssLEF3eEP7JIq0N9UeqLrYBS2ekOm9Gu8gtnViEmouotOZisDeOVsaFSIQsA79H L2T8VNVexcs/QrUEvyhRZxMpZPB6qw0= Received: by mail-qt1-f176.google.com with SMTP id d75a77b69052e-44fe6672297so7599781cf.1 for ; Thu, 22 Aug 2024 14:01:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724360473; x=1724965273; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=vw/JKAnymYOmboHlkwK/DnRu8KqLmYWyOecBkJ6vDRQ=; b=SK+6OPZzcs74i1o01+4bU7dKm/O3XRw3G/7lBsGmDnwInwZchG+Vz93weaThPTeEAW P3Xzg3Cd/RKIf2CoubEc4oeD/DXzwcxr3ty090REsY+Gi4bxt4axFqjxC/z8kc/DiEun XVL4Lssh6oC8rGUMnqRmsdW8UpJGe5/xYeuZGa3cVYewDXU2hotx+SHFoueBXravSbp0 JRO1HAdjJzLkvi/30HNKPS5m44oOxP45ZYIWP4/FL+kybrk4FezCZlw6fY5v897oCbqK soLVBR835nNxOrIV+I6CkuMmDL21/gmdOjrJxCIyAcNSHBciNYtHznYITdV35701f2kS MRbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724360473; x=1724965273; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vw/JKAnymYOmboHlkwK/DnRu8KqLmYWyOecBkJ6vDRQ=; b=SZETKcmJMauEHsmCsCY40bZiLVghrn0oMTwnhmHN448SE8USbOreKeDvCSvnON83Ao XNF3ivyVq+NJMWfGIav9zNHmjpkZOMA16Q6ckog0oGl6O4NWwl4AVoHohG0SNS4Enlt2 Ch6bsh5JVrbBuI/N0b6JUzXuJ00hdDbpmPpuJELBq3zgiVlAG8gfNu2OEbHMTAYiX/EC eg+Qn4TDkE4EadTX4oHocp1AdP3cESp2jwiCABZHQMRU1BI0Fq8Ys3mXFLD5llwGPngL QOGqDF80Q9/I740DnEWZEmdYb/OZm1mk59zrZiNxj4JDKRf+Gth4xil6y4Ff2H5Uj6pO I7pA== X-Forwarded-Encrypted: i=1; AJvYcCVdDK7hse/66s4KjcbOJE7buXhabd3D8UQemTM+25icz0/sqbrjz4bQVWjle6XQr7Ayt+qEn0wJHQ==@kvack.org X-Gm-Message-State: AOJu0YwHGAiHgaMPruDXYW6o8nRuktkaNy3Zx7Gxxq0nO3qQbCpulGOn mP0eOeOT+//BmARaEBFFzrAXGCrPk+1TQVwpGSj8WRA6K9XYv/q5gOqDtktf9tIBjljElpaUoLV RKbJCt2UMIBu8XHk9/hdFt+E+t2c= X-Google-Smtp-Source: AGHT+IGHek6NPXh4EgdjVCiMwCyd89tfaOc5FG3NKZkwUAaypH6vmbvAblaiWLzo/FOWCYW0fzGjIWQT8UZpLBnIVrI= X-Received: by 2002:a05:622a:5588:b0:440:572f:391c with SMTP id d75a77b69052e-455096454bbmr3131991cf.24.1724360472847; Thu, 22 Aug 2024 14:01:12 -0700 (PDT) MIME-Version: 1.0 References: <495d2400-1d96-4924-99d3-8b2952e05fc3@linux.alibaba.com> <67771830-977f-4fca-9d0b-0126abf120a5@fastmail.fm> <2f834b5c-d591-43c5-86ba-18509d77a865@fastmail.fm> In-Reply-To: From: Joanne Koong Date: Thu, 22 Aug 2024 14:01:02 -0700 Message-ID: Subject: Re: [HELP] FUSE writeback performance bottleneck To: Miklos Szeredi Cc: Bernd Schubert , Jingbo Xu , "linux-fsdevel@vger.kernel.org" , "linux-kernel@vger.kernel.org" , lege.wang@jaguarmicro.com, "Matthew Wilcox (Oracle)" , "linux-mm@kvack.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: yeujuao8at81thb4q17j9azmjo8j8u8t X-Rspam-User: X-Rspamd-Queue-Id: 234D12001F X-Rspamd-Server: rspam02 X-HE-Tag: 1724360473-311842 X-HE-Meta: U2FsdGVkX1+xQmTOC+MVI5PuTU6w99140a/rAlpGmuJN2Ke/XRBQOg9BWq1oFX9+hpFpwkhNhpT1D2aaSWHgClFInx+m8QwlaR+t54YCGcEz4dVTszcCsJrUiRbZurEqYTgf9TRcpxLu1ult0K+HUGw72CgVXVEIy+s+5qm502pfU5cXkWyGndWbhIBcil0AI3DHcSZrunPb4h0GoMSSm4vVRGWAdE3JFm7e3HpYshnTdsuB69jEDQ/p7HLp9Hw6keBQXcYKtnqDnEz+cipypmUSo5fzFOC71BJOWN9y+WiM9qe6pNHYPc+Fac6xpkS9iWgVpb2+vGCyPlEI/F5bsERDiG0sM81CfONYuG0cnnbPAjtakkts6rcTIeXEc3AqZBXZuTwU/fjM9r1n9nS+N8S6kNaQh1akcBdj1Ky2cQowVfsaM5R42TsSbN0znfC+N1w9IToSmDytnIbHpoP3eugNQSW3JWeU/gserDzS4J6s4Vwn5KCQOLlFdg46ewogDRRXDdaGSq037VefitrL0o5BUxnb1ECrCbsuOT37sRnp7KlHF9S8uomCe5bK/xEV0qFn9Z/eg8CShcItX4RdGqs/KFA/Kgx6KffMVEwhGgQ61OAoRHu8klWdJ3fFmwxMLWins5covAwt8+tqe9HeNM0EFd2kAt+VxGpcZrc+xfU6yJrHzyhG+BVoDmlvJR79e2pOCQbX3GUnO6DWUmQzY7yAGrowezdSX3G+rl28ZHR7e5HwUfNT2Um0OjofknRku3yyEntR8P+ulJGIPcbOpRVe0mUezlC6H6kJFOwBLGRQdIz3cRL2Dw8Cnbg18C3IRJDsoj/giO6pm2dBsK30UtSGBp/hMhKIltg7tv1OdzmZT9yPK5qYQ1QXiMRtLhZE8bo5Vt0X3KTM+zLnPdpLbNi2fI4fxAnByM5Lu6dWBCytUue7IJG89mcoKvylnymIyOrr7Gp3lylFIy/n/vM dceMQIaR 4BxjLHqOiuEOT2NOLuR+G8PJ9Nr4p8zB+6i+M9W/GXZ8g20CbgkPX0afLnjQrZiHy/vKNUAEB1+hCcVSQNEf8imEQjXBUHw44asYD6bVoJc72Pn3YXiCL5vPNkwERR7eu2gv/wRRezhgKV2PW1+4J+72sSyH6FZp7L+6LRLpsiQc4YVnGo5TGzGi+VqmCX4TIwGqtYZolqSA7beDNEozKgb6W/+6bKFQlGI7m4XC+LSlB9zg/2Tk9q8fsIqISNYkG4LdsHHvPsDR2w4qzj/iTzGWhVGFspkZFd1VXfGD2D5e0etWLqB2pOJdg4AB/+EkX7bOCRo5mTHTcqRT2nxHLe2VNnDiRvvDzMXuQ8DyAK3F3vxs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Aug 22, 2024 at 10:00=E2=80=AFAM Joanne Koong wrote: > > On Tue, Jun 4, 2024 at 3:02=E2=80=AFAM Miklos Szeredi = wrote: > > > > On Tue, 4 Jun 2024 at 11:32, Bernd Schubert wrote: > > > > > Back to the background for the copy, so it copies pages to avoid > > > blocking on memory reclaim. With that allocation it in fact increases > > > memory pressure even more. Isn't the right solution to mark those pag= es > > > as not reclaimable and to avoid blocking on it? Which is what the tmp > > > pages do, just not in beautiful way. > > > > Copying to the tmp page is the same as marking the pages as > > non-reclaimable and non-syncable. > > > > Conceptually it would be nice to only copy when there's something > > actually waiting for writeback on the page. > > > > Note: normally the WRITE request would be copied to userspace along > > with the contents of the pages very soon after starting writeback. > > After this the contents of the page no longer matter, and we can just > > clear writeback without doing the copy. > > > > But if the request gets stuck in the input queue before being copied > > to userspace, then deadlock can still happen if the server blocks on > > direct reclaim and won't continue with processing the queue. And > > sync(2) will also block in that case. > > Why doesn't it suffice to just check if the page is being reclaimed > and do the tmp page allocation only if it's under reclaim? Never mind, Josef explained it to me. I misunderstood what the PG_reclaim flag does. > > > > > So we'd somehow need to handle stuck WRITE requests. I don't see an > > easy way to do this "on demand", when something actually starts > > waiting on PG_writeback. Alternatively the page copy could be done > > after a timeout, which is ugly, but much easier to implement. > > > > Also splice from the fuse dev would need to copy those pages, but that > > shouldn't be a problem, since it's just moving the copy from one place > > to another. > > > > Thanks, > > Miklos