From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C71E4C36000 for ; Fri, 21 Mar 2025 18:38:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C9549280002; Fri, 21 Mar 2025 14:38:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C44B7280001; Fri, 21 Mar 2025 14:38:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B0CA2280002; Fri, 21 Mar 2025 14:38:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 92483280001 for ; Fri, 21 Mar 2025 14:38:40 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 3EB761A0587 for ; Fri, 21 Mar 2025 18:38:41 +0000 (UTC) X-FDA: 83246419242.17.47D4C79 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf11.hostedemail.com (Postfix) with ESMTP id 6B9A94000B for ; Fri, 21 Mar 2025 18:38:39 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=IYFtS8ms; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf11.hostedemail.com: domain of ritesh.list@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=ritesh.list@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742582319; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references:dkim-signature; bh=WLg29Du4d9hxLW5NIOZlGqy6bsHbWf+jMBwTEsqW+BA=; b=ohIQ298RBazehV8RhDurpxfGiW1EUu4kmJD4h3V2r7B8dpZ/yOViXl9SoH8I48+HLwi5Ye vLCFvSSSQRL88wPQyCrSdxpaQAt4mJsIGlquaxaZXzQMc1tmvOY0GvvSobrhaDpwEY2Ehz EFa309uchLrXwbLw+rmyvL643yXmqXw= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=IYFtS8ms; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf11.hostedemail.com: domain of ritesh.list@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=ritesh.list@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742582319; a=rsa-sha256; cv=none; b=ZRm49d5+PyadLHBDKFHKLk5Oo4NLMoLBN8/e3HNxrMWBT4VaSXj/szqMxysmbF1BoNqf+D QOa6P6srsCWi3BaMTpz14wlK1bkiV/Hmuade8e+gHcRP2w/17+VMYDOwbYU4nMpiBFQHr8 bT9/pVMx3pQF3azva2bPRkADzTVhMkY= Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-226185948ffso48892345ad.0 for ; Fri, 21 Mar 2025 11:38:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1742582318; x=1743187118; darn=kvack.org; h=references:message-id:date:in-reply-to:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=WLg29Du4d9hxLW5NIOZlGqy6bsHbWf+jMBwTEsqW+BA=; b=IYFtS8msWA6pb/FTitDBvg5P8dfzPdBkLEKkAJHg6IpyEabp5mS4sfHi34Ed7IEJ9S hYxwNczwQJ7WsOCZU7oR0XRezMdCdN9G618avBiwN9X9R4CrmPG0zRsuDwe0sPQ2dvTE qVvVqRdeswgfqqIh40KfhyOGNMCJosKetmunQa8yZBI19H9Nl1aBq3gVKlc1c9r4F8Cv l9A90SmCBH1eVgI2HGEthikfKOsIFqZK7L+aRwEFaDSA9F9hBlBrFFcD33+wULDp10ok Mm/G8ALkR6s73ybE0BtSHNCIVf4DRNj0s/bpBHUmIrlbybp8bsdavBumwAsEis8v2tVe XGcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742582318; x=1743187118; h=references:message-id:date:in-reply-to:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=WLg29Du4d9hxLW5NIOZlGqy6bsHbWf+jMBwTEsqW+BA=; b=JkTZdYqM2frEQjAKpvf89oR9rHQqZw3Yjn4quDZ4Az2AWTtItBMLB7tG1hItPBBwF1 0tLLNXgNul5dxetisaaRLElfEQr2EjEOzuYxP3ie12tCdM8JS4LqqGiI7ZQXFMV1YRIC JAsqZso7erdTQZPxYioP/HnKas73SsI6tB4gC0rNRTkL8/lYXeEAoIjzl4F6KNXCc0mI kPuHXZ49EsIDpvXnXON6lzFOKC3qEU5TkOvDP1Lnr4iqBxuq9cQ4DB7xRTRlB5EW0Xdv DvasAXn+O0tLEMf0hom40zha9spkKiAz1T8v/7Aa3DFH+68C55NXQn5cQ9RqjgO/q4M8 7WKw== X-Forwarded-Encrypted: i=1; AJvYcCVXOjydHWFhcrXDmO6heKDa38vWlzwbT0rOv84kVbSxXqqp6D0Mom1AetVSuglQl0AdfYEdUbkS3A==@kvack.org X-Gm-Message-State: AOJu0YxgRW2DQQmZCf8SACsPavO6I9DpwwAf0GwLoczMorzHvaQWm9Id o04HqDrHeOKNnFWhVYZslyHfQUmzpiqhTAlJ25CtJktp7f7mTZuS X-Gm-Gg: ASbGncsY1vjjnEldv1WUhyWLKMs4gHChK7Z/G1tXENCbS2Uaq1+bQvCBtUiy+lJIrwo uR2x+uMCQpH7VVMgVmxNU0pXlo3Yq4C2ZdPhLvz1tHLb4sxWNPEFswh7521ire9lJ0OioDpd+Vl NurjXIq1vhtsZm2glxPxjmtvbu9jq5zMKuO4YjyYY8vkJj4dtBzheGVGlCh+bwSWc5E3G/GDY2D TDf7HNwDzaU5DbIofafrB8221/B3joQZhSyQtfYp1xvbNmLQkdj/t6JFkmZH2PdNxXec6psPip2 6Kg75S21SRotpaoc3hwM7aCW0OkIx7D4yMQ2Jg== X-Google-Smtp-Source: AGHT+IHWkYu5R5KHVsYmHJNCVf/nynTskt0Y86WId2DrBJREIzTZC0pMVzC1eXdRjZpNgDwkmpUIoA== X-Received: by 2002:a17:903:98d:b0:223:5e76:637a with SMTP id d9443c01a7336-22780d980fcmr73987485ad.23.1742582318033; Fri, 21 Mar 2025 11:38:38 -0700 (PDT) Received: from dw-tp ([171.76.82.198]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-22780f397d9sm20752535ad.49.2025.03.21.11.38.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Mar 2025 11:38:37 -0700 (PDT) From: Ritesh Harjani (IBM) To: Keith Busch Cc: "Darrick J. Wong" , Luis Chamberlain , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-block@vger.kernel.org, lsf-pc@lists.linux-foundation.org, david@fromorbit.com, leon@kernel.org, hch@lst.de, sagi@grimberg.me, axboe@kernel.dk, joro@8bytes.org, brauner@kernel.org, hare@suse.de, willy@infradead.org, john.g.garry@oracle.com, p.raghav@samsung.com, gost.dev@samsung.com, da.gomez@samsung.com Subject: Re: [LSF/MM/BPF TOPIC] breaking the 512 KiB IO boundary on x86_64 In-Reply-To: Date: Fri, 21 Mar 2025 22:51:42 +0530 Message-ID: <87frj6s3ix.fsf@gmail.com> References: <87o6xvsfp7.fsf@gmail.com> <20250320213034.GG2803730@frogsfrogsfrogs> <87jz8jrv0q.fsf@gmail.com> X-Rspamd-Server: rspam01 X-Stat-Signature: 1gwiiksmk8748c9ff8g8s3xgxeg3m4d4 X-Rspam-User: X-Rspamd-Queue-Id: 6B9A94000B X-HE-Tag: 1742582319-929249 X-HE-Meta: U2FsdGVkX19lsQHjXYOL+oFs+2utA7sOwnbSr6HCKj65XpnEb2TcgbwiEqMSSFocQzFYCoWgIYzJqXn+xjZrGj5M5PB3ZbcJ/hHJKxCvc5xTQ9NyosAq2JQuyJM+v0yU9KLz6tN3N7wNRrknqIpsw9b+XUcwcyOyf2EpmgVmLsCHlvBYU4RoTfJUX5pDHCNvP6IFKX+/3MHfhyUGC72akDujNH4d7N49+chK0m5PVDo6JGo1BI/wbell/rrRF/33OnSkYlqGgVS8y2N/R0fIpltWbr6XiQ+Bd7jnyZdsQrxJuebxIocN4tB33NbfriRrsjttDlN5/Hzzhxp9OYQp1cv+maD+QubN6h/5KPCqZgtGEQJ/5b33PK1QW9ZdqwkguDHpMPgRq1WbCyPD7RWOfCYeAu+ru5ZEYHTOuby2O5WNhgeQUkEp6AxbXlFzGJgR83K6t7F6IuTgEmFmh4OmNIrwYrO0VCzGdMdLrzXDI89AeTrHdwgSj+iuN3ODjgJtJK772lD57ots7g+5pJh7attvX3Pk2cF0rPM3rgdEgXx3dsBbzoyQj+Jxbg0DjUCfspg0B4HAJqQZLY2koT7SvHmtW9RJ+2vBvKM5xjuLw+xrSjIRTTEIeGUJVfojk24d+sp/u9FiZybXaneNOd+oSj9J7Zk+DjvSI53jG6cPBjTfVqDGCEJ/M1kjx/QJy+FSGs3gzS1RNkjNVAVU3j3Tki+hXXysZS8wKdbV5xBjHOkghXtazevtHb24hU0PNmg9RcP3l5NFdUG0JvshEBIWyIlUGhBpYXMMy4nrfmnyYbGNk0clJhx52qWqH4V81HYw5ZYNX2ahcPNQYpqw2BGfKV17sRf6aaB5nSHC1fGfmYgNSnzuUev7MmnTcoB4kN8YaGmuuruYZmQjcROp4yIQB44mRY9UqaXKdbPf4cjmN1bm3AOuydQw0yMycMu+nGAkV81X9NUP1DjtbInzn2n +j9eXLtm 1zcbuVTTRWpvhmBbaNJ0O/w5VkeFGs5rf2YMyUnmmUWu3RoFkbWTvBb83qVckc+edvqY94oz3bSxcLHA0ZrwKweM+F1okxAV82KV26UqDtkkU+/iLGnQOkuDU9EHCuenr92jtmwuWrGdrvo25nasGfVtuoGSXqGn4pFaCf80hiVmjLn9rT42354A8CnlSH7+YBcwN67MbTaOTLLZt8NJ7WF93h2TzwKOLxSkLq4S18qbtxuc625xlaM7Z7ERNHEcfsc9T6pk92MC2PFgwr6H6xyGph/W/TfGvGxTRM3QiDD4w+0VuZ3hRAnB70eozEe5/DB+nZcHTYxZK8W5SuOWO7ZZRTtArSRGu1k8VUZgdPAvEBMPzZIoGS3aIHIq29vxTRXWLI34+rPonu7jiWcjggU0wEWEz/UbaZlQqdvrGCELi+BXOoBp24S1Jdw+qbiwaDe6B X-Bogosity: Ham, tests=bogofilter, spamicity=0.000589, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Keith Busch writes: > On Fri, Mar 21, 2025 at 07:43:09AM +0530, Ritesh Harjani wrote: >> i.e. w/o large folios in block devices one could do direct-io & >> buffered-io in parallel even just next to each other (assuming 4k pagesize). >> >> |4k-direct-io | 4k-buffered-io | >> >> >> However with large folios now supported in buffered-io path for block >> devices, the application cannot submit such direct-io + buffered-io >> pattern in parallel. Since direct-io can end up invalidating the folio >> spanning over it's 4k range, on which buffered-io is in progress. > > Why would buffered io span more than the 4k range here? You're talking > to the raw block device in both cases, so they have the exact same > logical block size alignment. Why is buffered io allocating beyond > the logical size granularity? This can happen in following 2 cases - 1. System's page size is 64k. Then even though the logical block size granularity for buffered-io is set to 4k (blockdev --setbsz 4k /dev/sdc), it still will instantiate a 64k page in the page cache. 2. Second is the recent case where (correct me if I am wrong) we now have large folio support for block devices. So here again we can instantiate a large folio in the page cache where buffered-io is in progress correct? (say a previous read causes a readahead and installs a large folio in that region). Or even iomap_write_iter() these days tries to first allocate a chunk of size mapping_max_folio_size(). However with large folio support now in block devices, I am not sure whether an application can retain much benefit of doing buffered-io (if they happen to mix buffered-io and direct-io carefully over a logical boundary). Because the direct-io can end up invalidating the entire large folio, if there is one, in the region where the direct-io operation is taking place. However this may still be useful if only buffered-io is being performed on the block device. -ritesh