From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC1AAD70DFA for ; Thu, 28 Nov 2024 23:31:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2C0CA6B0083; Thu, 28 Nov 2024 18:31:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 271036B0085; Thu, 28 Nov 2024 18:31:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 138C06B0088; Thu, 28 Nov 2024 18:31:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E36876B0083 for ; Thu, 28 Nov 2024 18:31:41 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 8F0901C77A3 for ; Thu, 28 Nov 2024 23:31:41 +0000 (UTC) X-FDA: 82837103034.05.1587BF6 Received: from mail-ed1-f52.google.com (mail-ed1-f52.google.com [209.85.208.52]) by imf23.hostedemail.com (Postfix) with ESMTP id ACAB5140017 for ; Thu, 28 Nov 2024 23:31:34 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=i9cV+THJ; spf=pass (imf23.hostedemail.com: domain of mjguzik@gmail.com designates 209.85.208.52 as permitted sender) smtp.mailfrom=mjguzik@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732836691; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JMmQ+euJeUI1bvgZvShMNChaRMxgihGmPY37bUmkejQ=; b=ho23SVDFtwdQEf+LZRO9y+PW2L3/AJW5D7MuPY9tOhsfXuaGhGhBTdxXh7X59KAbEbXs7S ol6TzoPLqpJ24/fjMIccxZgrBOF8sc0+i++LW0C9WYM/6OZEtSKt21rhOOGjtz4Adudlo8 8uRhZPT2BiLkBxQjlgn+gCT/xMHnYvE= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=i9cV+THJ; spf=pass (imf23.hostedemail.com: domain of mjguzik@gmail.com designates 209.85.208.52 as permitted sender) smtp.mailfrom=mjguzik@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732836691; a=rsa-sha256; cv=none; b=C0Zg42k1rBztV9r9sQIao7r6y9zZXj3skh2wscG054hdu8YQLWsc5ybJUxgQw4MXNX56IM sME5q2GFjceARHQ5hKXSge7WwaxbZvOAzKCQNkmUoK4BVuRUsg1nalBUSTS/x6VjHHWVBR ienUtl7aZsA/x4ZhQ3NACkwN1PfIoP8= Received: by mail-ed1-f52.google.com with SMTP id 4fb4d7f45d1cf-5cfa9979cd1so1522315a12.1 for ; Thu, 28 Nov 2024 15:31:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732836698; x=1733441498; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=JMmQ+euJeUI1bvgZvShMNChaRMxgihGmPY37bUmkejQ=; b=i9cV+THJAKwBoyTYHa82of0cB2b6Rnwsji+YwCWsLxjkhMBsxTXfS/UCy+guxMjdLm tQSyw4HR1VS3Z3a7RwN+vlracyRij09S7wc3+ca1Ovo5Mok2tucTKb4OfEtBUicPb1Ww Ze9fWxFKhvzcvpy3CXtOLbtc0BbURGOSW/+F2Tgow5cEPi4Y8n/t77it2S90g5tVbJRb io98ay2HE59lfPjJupi3GJnoyo7j6LJ8Rzsq8UEUpmY15w9jaaS5a6rcYjuRCMncEmR2 ON/Do5KoWaerJudY1Mu0bSwTLsdNos/5R1FNED7EDgs7dtWbH4HJYqU9R0pBekHMTnwo IO8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732836698; x=1733441498; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JMmQ+euJeUI1bvgZvShMNChaRMxgihGmPY37bUmkejQ=; b=lx7pYR1CwYXv2PuDdbqV1dh21swzZPpRFG5JUeMbqVXBFEXGrsKz3c9j7WnqDEKUzk 0740sGJHSW4hsOI8OLrJ/ImByr2HIvja+5aTNsFqAnYMNvuFae6kOqAZauyYvikk4coF X2aM/95y0LQ3mJeb8pbTNkDGzt3LdMLToXWvZwPkafqb6FV/avu44+pLF2oPbBBECSZG aJ0efRyF0X7BrWwi0VHTI4SlL71mpFB9/jUyVFda+m9FX0feD2/JF9p1laamxdLMIpwv STj7t9ZMG3Qjzo+bgzExjYfuNVoB4vJ9roHunfL8ndWIgfzlsvsMEmIvkON6aMSw6LNw NG5g== X-Forwarded-Encrypted: i=1; AJvYcCXRdNGZgsvCwejqxMXymkvNUoDT0qjifb2a0C0+Wka5f0aaY1IBN5VMtQXNIUVywmA606qe4iRrBg==@kvack.org X-Gm-Message-State: AOJu0YxnncGr4V3glA7PxFa5/KBtFEJ/CDVFAu6AFTRMnmwllviyzu1B QknHhhSIyTRs25lJOFgk8S+Q3hL4hrfH+MIjooKtfb8vxGczk+bSPGD0EkiRuMMRhJGxGGjEYgJ UJUPDCcDxeXfyf3p29tFo+RjGXVE= X-Gm-Gg: ASbGncvLmKk6Y3jr8tsAdkJpZIgT+E+MG070eBdGYDZ4evctpu12UAPYF7z3PAyH4m9 TYFafjk/VhZJgX8gA00H1P0iliC8zF8k= X-Google-Smtp-Source: AGHT+IEugrJLH9v+xP61oVG1eZv0KhzLh9Qaq3n+mLo5AN/FqfweRFAco/52Brjfc3g4jpsXZ8JRgfpACIvxebaJg1Q= X-Received: by 2002:a05:6402:1ed4:b0:5d0:975:b1c0 with SMTP id 4fb4d7f45d1cf-5d09516035amr5686024a12.11.1732836696288; Thu, 28 Nov 2024 15:31:36 -0800 (PST) MIME-Version: 1.0 References: <20241127054737.33351-1-bharata@amd.com> <3947869f-90d4-4912-a42f-197147fe64f0@amd.com> <5a517b3a-51b2-45d6-bea3-4a64b75dfd30@amd.com> In-Reply-To: From: Mateusz Guzik Date: Fri, 29 Nov 2024 00:31:24 +0100 Message-ID: Subject: Re: [RFC PATCH 0/1] Large folios in block buffered IO path To: Bharata B Rao Cc: Matthew Wilcox , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, nikunj@amd.com, vbabka@suse.cz, david@redhat.com, akpm@linux-foundation.org, yuzhao@google.com, axboe@kernel.dk, viro@zeniv.linux.org.uk, brauner@kernel.org, jack@suse.cz, joshdon@google.com, clm@meta.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: ACAB5140017 X-Rspam-User: X-Stat-Signature: at1gzucwazn7aw1et759wjoxa7oaqmkt X-HE-Tag: 1732836694-944579 X-HE-Meta: U2FsdGVkX1/JJlKKolUm+RfeiVSl9rUFRLQilKd2oaTNijF2sHNxfxRwk1SzZNijb+EQDlAMtk6w/g2bAp6INi5r8U8dO/4m3njh4WVQU3LPC/nPk2QmbbvvtI84SGVE2unLfCY5pQ0IMvm1zUy+LjBu85Zu2uC0S7hHweHyGX1SoirDQ/8e3OoqBo1DmctpI6dZSRXXl76FcPdUQL0Pp+VmqS7emRZ8+0JKwnwDBMhm2RygQQu/9U27yK2XiqxdHl6ubgSZeBMoBH8rvdPWJ0d4pz9m7/26BbhZgdUW/GGdgOY+r4mc0+bX+cJZ9QgvbMZYvfa39dG7D9OpVI4bWFGnonlWOXbwpWHRpOqRfe2c1r2KZYuODFGy3LjPUh/smHxi4b2ojZI2bxm8CH5YC//7P+LjHEftwSwFHOl+vKIth3c57LQdFE7y5UWF00o5q+JfHjqTgOIXRjZXrxqedVadvIxsPjUuWHXVPaXCyqjEtDB6jaK6xigqvaSLsCVHKA7iSospDKD6/bQGNLfRk7ruxCeTuDx/8sCaUPmGVpm1OdZ9Lgy2pJNdjETYUKiwSrPx4E9LRkIWrPL/xkXhplkpmuyCGdzWtWOD/wxQp1i6QaoBFf8p+pkHdWnPci48j6rhxkMfhE4S0RNTl0ruumuSeyARj5M6LQ9mMBA7nl9ka00Sh5QnSDvHdOQZP4Ab7HbgZrAY00BeAh8dwj5VLyt2GokBD5JoskXbh3maoaJaIgTuMa9XG4+NS2hFO0FD/G5d3sjtgjJ1KCDXxZorOHIS3bs5zHHZRLyuFM+njSaMxs5MRGmQj4G7ILA9QzGP/1hjHNys171EkMtr7aP7i8Uk705aJIbIiQXV2X5lEQmStkKcEFH8ZGaQKEGBmR02YcRN/3UfAkUXgrfWmrpEzPhs+dYZqsks0XwpufUaU877yo9BRS98DTYvBSmvxDhHlJPlWGeNUQJETNOFS0G cFIwWhF8 A71IRQtEkBzrQ60G0HbGlL1znUgbXD8wH2Qj6vcNHylTHfTF5noSzzK5NhYx9/0Lhh8Yn4hAhasfOcYDOfQZGn0uL5ageLIYKdNVzv/h0aLbwqi+z9QvP7pi5x8syneyHMWtdbI474uXQ3my30aHBEHZfDcf9/xRbh1y5SmcVV7p3jDxcnLW3fMvwlwdiRVDJX+8ffv5xt0C8QVpZf/pIYGL2+1o//2wE9p/RCPNpKmVxd+cXFHiCd7e0o9kAqOcafTdPAd1Batsp6DuNRritJL9EADaIQgVGkdo8FXOgj2n0sx7Mi1lckff1Pwk8958axFffUyyJHaASuuY1P05M4iQEODjRYla7y5m8ORuaQj4zSu3HNl5MbrQP7w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Nov 28, 2024 at 12:24=E2=80=AFPM Bharata B Rao wr= ote: > > On 28-Nov-24 10:07 AM, Bharata B Rao wrote: > > On 28-Nov-24 9:52 AM, Matthew Wilcox wrote: > >> On Thu, Nov 28, 2024 at 09:31:50AM +0530, Bharata B Rao wrote: > >>> However a point of concern is that FIO bandwidth comes down drastical= ly > >>> after the change. > >>> > >>> default inode_lock-fix > >>> rw=3D30% > >>> Instance 1 r=3D55.7GiB/s,w=3D23.9GiB/s r=3D9616MiB/s,w=3D41= 21MiB/s > >>> Instance 2 r=3D38.5GiB/s,w=3D16.5GiB/s r=3D8482MiB/s,w=3D36= 35MiB/s > >>> Instance 3 r=3D37.5GiB/s,w=3D16.1GiB/s r=3D8609MiB/s,w=3D36= 90MiB/s > >>> Instance 4 r=3D37.4GiB/s,w=3D16.0GiB/s r=3D8486MiB/s,w=3D36= 37MiB/s > >> > >> Something this dramatic usually only happens when you enable a debuggi= ng > >> option. Can you recheck that you're running both A and B with the sam= e > >> debugging options both compiled in, and enabled? > > > > It is the same kernel tree with and w/o Mateusz's inode_lock changes to > > block/fops.c. I see the config remains same for both the builds. > > > > Let me get a run for both base and patched case w/o running perf lock > > contention to check if that makes a difference. > > Without perf lock contention > > default inode_lock-fix > rw=3D30% > Instance 1 r=3D54.6GiB/s,w=3D23.4GiB/s r=3D11.4GiB/s,w=3D499= 2MiB/s > Instance 2 r=3D52.7GiB/s,w=3D22.6GiB/s r=3D11.4GiB/s,w=3D498= 1MiB/s > Instance 3 r=3D53.3GiB/s,w=3D22.8GiB/s r=3D12.7GiB/s,w=3D557= 5MiB/s > Instance 4 r=3D37.7GiB/s,w=3D16.2GiB/s r=3D10.4GiB/s,w=3D458= 1MiB/s > per my other e-mail can you follow willy's suggestion and increase the hash= ? best case scenario this takes care of it and then some heuristic can be added how to autosize the thing. If someone feels like microoptimizing I also note there is magic infra to have the size hotpatchable into generated asm instead of it being read (see dentry cache as an example user). --=20 Mateusz Guzik