From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AE09C4829B for ; Mon, 12 Feb 2024 09:16:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CBDC26B007B; Mon, 12 Feb 2024 04:16:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C6E146B007E; Mon, 12 Feb 2024 04:16:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B83376B0080; Mon, 12 Feb 2024 04:16:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id AA7D26B007B for ; Mon, 12 Feb 2024 04:16:25 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7EED480687 for ; Mon, 12 Feb 2024 09:16:25 +0000 (UTC) X-FDA: 81782595930.13.7583F5E Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by imf14.hostedemail.com (Postfix) with ESMTP id B9837100012 for ; Mon, 12 Feb 2024 09:16:23 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=UltJMtTr; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf14.hostedemail.com: domain of ritesh.list@gmail.com designates 209.85.210.170 as permitted sender) smtp.mailfrom=ritesh.list@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707729383; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to:references:dkim-signature; bh=g6IO++R9WeYb4qAE84/3bcX8ZgMaoHDBddIgy6fgRYw=; b=yosX9ufltv1RKKATDdsQZ4zjp2v414et2iLlCJjSU+wTJj08/f50e2mihxXka6ZLhhBLsY 0XnPqiXD/xEQZwi9IVf8TIIaLNiaKGRdaYQCMvfcwtBhU2yk2EdBFIOUmA9K36wVZLsu+o Cs2GCDkzb0NkCZg/tCJsqdw+d9AtvQ8= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=UltJMtTr; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf14.hostedemail.com: domain of ritesh.list@gmail.com designates 209.85.210.170 as permitted sender) smtp.mailfrom=ritesh.list@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707729383; a=rsa-sha256; cv=none; b=ZLqIhVi05kw//TgTNP2RPXbYpURAmTKZabBdNRB+I+O5uQ4KzPa7b8PwK+MJ91ey8olPlD cxBh+SihEvHT3WzpZu+hFIWYnyuRGzl1915moB/saFF9vyIaCAdTph9DMstoqNgg5GXkc5 7yh4/RxXIqk7yx+uoFEQl20OjsVRe2E= Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-6d9f94b9186so2493377b3a.0 for ; Mon, 12 Feb 2024 01:16:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707729382; x=1708334182; darn=kvack.org; h=in-reply-to:subject:cc:to:from:message-id:date:from:to:cc:subject :date:message-id:reply-to; bh=g6IO++R9WeYb4qAE84/3bcX8ZgMaoHDBddIgy6fgRYw=; b=UltJMtTrmR5cbh2JRrlCmHwbG43xDcPXZDvyOX91l+XA/epvgjpU88kqigIiOgIuCV lAtAkrkEQwON81Y1GdiZVp+PUoMWxKL7+wsrg7F9eHkbb+sG+JujrfCTAjgd++r/swh/ uQjQ0QStnHT0Wz/BdlLnNpCsS5fZKnwkldBJhyLnddpccY+PqchF5r0koIeNGHbpwwwg cboi6B1oBLLp9eJKfWtkySKxSEOuT/BZLfCb8QxKCBgWibe3zt8DNQRhzCf0Re+ju/ub 8qstU8/2vnDex9Ku9QTxc2xo7pW1oBQildkVuLz2erO6KzmsA0bojIU2Q+e/k0rMrjry ftpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707729382; x=1708334182; h=in-reply-to:subject:cc:to:from:message-id:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=g6IO++R9WeYb4qAE84/3bcX8ZgMaoHDBddIgy6fgRYw=; b=n+WwDzYnXIKZPQ1dbJtC79E/FTHNeryYLa5sx1U1NJ5Q6i6t5bgMgXEKfmIdJkz03v +VjPiwqRxYI5zeinwaS95gJ30GrksrBraRSeVPxowRzumpZJ+uqvKWVg7eACN0GmPgpQ YvXOqXn7tYNw/TM/9bVf5yeCc4f4i4zAhi3PEIeq3gu1IsyqhwPXUW4+OZT/NSiygk11 BIVfE+KRZ3jw/K+dOQ2sPd6S+MtDhnECuPyP+A/ROrRlGPGPupduVTp5WRR7QIa3eeXF 3rERvQoJ1AyzsYSg+0dqETNegx7OKcfuDey6vx/VUfQye3i10UuSDCj9i9eXGFc5Uzj+ XRIg== X-Forwarded-Encrypted: i=1; AJvYcCX/KDKdeoqvhA0Dho1QzavEOwIVFdgJRtqm2abuyy/NGwbJRPJlZuezP3KRx/0pQGqeTiEG757kYils/lyJggzI5Mk= X-Gm-Message-State: AOJu0Yxpz6OsB4cEbMOJ+zvwaYM6F9x5j7VE0/ri9dnmdkFqRVfSh6Il uFLBvt9v4EOq/ZdfEFKkmlorBRnxT+FOmhIyfMXWGu7XwdCJKFkQ X-Google-Smtp-Source: AGHT+IFvWWdoMQM6RNYDapxbsM4h4P3YcXw3SecLY3N7HCNZ2L6CMlzm0f7SFQlfUpgy0Dd9wL6DMg== X-Received: by 2002:a05:6a00:3d4e:b0:6e0:e5d2:5f01 with SMTP id lp14-20020a056a003d4e00b006e0e5d25f01mr1314349pfb.24.1707729382365; Mon, 12 Feb 2024 01:16:22 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCU/RlqtvHvacAp6YKw4f2dp/IVrLfxbfR0vs8PmplbdlO2NqWyb8hnHrUszD8mYwHgpJwpNulu2y3QYAxNPTnBoDeqw9dYOvukIOcrO+O/aCIbIFfJ7H2nIVK0WuRKEjlVSJH5W6RViZu+q6U7S/1uM67vFLzwRDKkTGwmI+8iMcSLHVyVqNmyXaA6w4VJfWZ9kQaAEW7kxBzaqLTW4OhJEco12e8DP/kLK9ZOXViV7yeysijtXfYuSMK/pffRJLgfgFdDuMb2BdysCiteM8/SSN4inr6BGDvYoJeTaLSZJo+6FDbr7GbCEuE8KkqmXBrbemdoUyY48QUP48tVUnjwzHaC9OJBhUi3xANSV8F68tZtYZHA8ZumjGOufjMBqIR/WRrR8uREEpAaOsv2J2yU89MG3moIK6UlnQntEmVA8O2joCfnlIrfDKjAO6Y/gEid2BQ3HcP8R/1NrUzH5iZyXkulsh1LQEd5lVg== Received: from dw-tp ([129.41.58.7]) by smtp.gmail.com with ESMTPSA id lo17-20020a056a003d1100b006dbda7bcf3csm5042750pfb.83.2024.02.12.01.16.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Feb 2024 01:16:21 -0800 (PST) Date: Mon, 12 Feb 2024 14:46:10 +0530 Message-Id: <87ttmef3fp.fsf@doe.com> From: Ritesh Harjani (IBM) To: "Darrick J. Wong" , Zhang Yi Cc: linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, tytso@mit.edu, adilger.kernel@dilger.ca, jack@suse.cz, hch@infradead.org, willy@infradead.org, zokeefe@google.com, yi.zhang@huawei.com, chengzhihao1@huawei.com, yukuai3@huawei.com, wangkefeng.wang@huawei.com Subject: Re: [RFC PATCH v3 00/26] ext4: use iomap for regular file's buffered IO path and enable large foilo In-Reply-To: <20240212061842.GB6180@frogsfrogsfrogs> X-Rspamd-Queue-Id: B9837100012 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: brszw3o6uqroppbjso4g8axxpemkian4 X-HE-Tag: 1707729383-624295 X-HE-Meta: U2FsdGVkX1+6kD/RfGcuhsm9NZysyznLQ2edo6LS8ifpOfv65Umh54SpmNbRlkGPxMHvbSqF20RzfnlHkmEyDJ6E4UhzECBchji3YortN/rs2z8w/GBJpRpw0wl3vyShODfiDRfsEuYKxw4MgiZ1OTYe5bN1wwlMftG9J/UhEkNadz0uarId5kxCeMY7vnzr+NMV79fhaaFkNhSWoxezKq0vw6RHEBcy528bIibIkHdqJKVrWYacqP4jWi0OdxRI/1420+8Sy5f1y02Yv607znbWkL+hPirRWmr+4SA2XNHVo03pHue7AnY+/YU5dXctS6Y32yhxerXBHj1fPDmV2tGt1EAZKQgm9lLvyVQAqrN9n7iAeGcKAWFjENuuA9MjOr4xE5sCVnsZ4MyS8J/Yv05RX9rMrYfGEDVP9CN/ICfTpG/PWgX2vistGNb4TF8jw2zvYT4eNeoN7wdVOY3mH45G7a44ZQ09zRu3VwQYk6qEtBsc0DtBrFcvhGzvS1QmjNDtHHeGPuNcTJ9RObfX9ILHsv8cSDcTB11J5pcTaGgVXCZCnEUBz70uOUFEebThyTxe0C2hM2nJWBjdK7p5Wtt6nHk2LOBdkSI2g03qIBD0AScP8Kuh3jgBX+p0S8iDsbr502V95MTUs6kaFN29pPrLE4gg7yIYH2ceQlwmELOFX92LzAKKV8DcSOQl30ySB126W5TpBcM8tmrzJmYN/KFQ+UQTCnNpGt8lglXuckEEMl/xqVYnx3N9p2+4YKOEBhlL6A6bSXpn3X9E8aCfcJdPkZGqint+WPhfJsciSgo/T3QlKdjJv0K3OLrDf7ZPjRp0dlTTl0WzyKoG82kDJC4edWvOaNWdq84u7A+IiIasXs2jt9ElZN0OAekLS+DThergp7FeacocoNx0QzFBUJROSmkiz/m2lu4D5/LQWa9N6TsdoURg+GQCelwxIUITDvqq4MfvBi37C3qt6AC iEJuY2ol J9f5z7EM5PDbX2wnQ7Vvds+WESu94NzoNCdSnwJT2/5ygeIUo2RMCFoZpPlVTP/LHC7HHAVJDfTiKGpjROiySiD1D/B9LJjJ/ltauKRnbFc9X3aGb7cAQ9GVvtAOl/Q4AH4HtVX5fIJJ6jGONtrPHU4WPxWvWQAEdKl2nCg2hp11yvJ8kev1cGrzE47ShBgaRK2z2odVICOKqkp4MQlyPOVvgKT7HbuE0b1z/18SHSueXSoevcE6H71ry+S6BIIC/2FB7TOSKRja19k1uYVMNNN1l1+uE1Mwk4rn5Y5CDIipD/JdRD8JOh9yrRoevTMsi7yMrO9nJrzNA6+pCRppic0TteIkezIc5BOY8Id+uU7FFyvaLIkkCztXF5JU5rIIXqBgFTkv5uKBrJkKMRnjd6FnCrUKVm0d2efK++lOoa2nMdy0L6l8YuPE0BdRP9ayn3nKmdQYUN+K+1NMnzAH7QhMJXv/UqNxZOimnBq8WT8FGRQWm5mpv/qa833TlPkGZ12uP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: "Darrick J. Wong" writes: > On Sat, Jan 27, 2024 at 09:57:59AM +0800, Zhang Yi wrote: >> From: Zhang Yi >> >> Hello, >> >> This is the third version of RFC patch series that convert ext4 regular >> file's buffered IO path to iomap and enable large folio. It's rebased on >> 6.7 and Christoph's "map multiple blocks per ->map_blocks in iomap >> writeback" series [1]. I've fixed all issues found in the last about 3 >> weeks of stress tests and fault injection tests in v2. I hope I've >> covered most of the corner cases, and any comments are welcome. :) >> >> Changes since v2: >> - Update patch 1-6 to v3 [2]. >> - iomap_zero and iomap_unshare don't need to update i_size and call >> iomap_write_failed(), introduce a new helper iomap_write_end_simple() >> to avoid doing that. >> - Factor out ext4_[ext|ind]_map_blocks() parts from ext4_map_blocks(), >> introduce a new helper ext4_iomap_map_one_extent() to allocate >> delalloc blocks in writeback, which is always under i_data_sem in >> write mode. This is done to prevent the writing back delalloc >> extents become stale if it raced by truncate. >> - Add a lock detection in mapping_clear_large_folios(). >> Changes since v1: >> - Introduce seq count for iomap buffered write and writeback to protect >> races from extents changes, e.g. truncate, mwrite. >> - Always allocate unwritten extents for new blocks, drop dioread_lock >> mode, and make no distinctions between dioread_lock and >> dioread_nolock. >> - Don't add ditry data range to jinode, drop data=ordered mode, and >> make no distinctions between data=ordered and data=writeback mode. >> - Postpone updating i_disksize to endio. >> - Allow splitting extents and use reserved space in endio. >> - Instead of reimplement a new delayed mapping helper >> ext4_iomap_da_map_blocks() for buffer write, try to reuse >> ext4_da_map_blocks(). >> - Add support for disabling large folio on active inodes. >> - Support online defragmentation, make file fall back to buffer_head >> and disable large folio in ext4_move_extents(). >> - Move ext4_nonda_switch() in advance to prevent deadlock in mwrite. >> - Add dirty_len and pos trace info to trace_iomap_writepage_map(). >> - Update patch 1-6 to v2. >> >> This series only support ext4 with the default features and mount >> options, doesn't support inline_data, bigalloc, dax, fs_verity, fs_crypt >> and data=journal mode, ext4 would fall back to buffer_head path > > Do you plan to add bigalloc or !extents support as a part 2 patchset? > Hi Darrick, > An ext2 port to iomap has been (vaguely) in the works for a while, yes, we have [1][2]. I am in the process of rebasing that work on the latest upstream. It's been a while since my last post since I have been pulled into some other internal work, sorry about that. > though iirc willy never got the performance to match because iomap Ohh, can you help me provide details on what performance benchmark was run? I can try and run them when I rebase. > didn't have a mechanism for the caller to tell it "run the IO now even > though you don't have a complete page, because the indirect block is the > next block after the 11th block". Do you mean this for a large folio? I still didn't get the problem you are referring here. Can you please help me explain why could that be a problem? [1]: https://lore.kernel.org/linux-ext4/9cdd449fc1d63cf2dba17cfa2fa7fb29b8f96a46.1700506526.git.ritesh.list@gmail.com/ [2]: https://lore.kernel.org/linux-ext4/8734wnj53k.fsf@doe.com/ -ritesh