From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECFFBC02196 for ; Thu, 6 Feb 2025 08:00:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6E7A9280003; Thu, 6 Feb 2025 03:00:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 69860280002; Thu, 6 Feb 2025 03:00:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 53820280003; Thu, 6 Feb 2025 03:00:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 34E49280002 for ; Thu, 6 Feb 2025 03:00:25 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id D21BC1A07E7 for ; Thu, 6 Feb 2025 08:00:24 +0000 (UTC) X-FDA: 83088772368.20.CE41D73 Received: from mail-ed1-f51.google.com (mail-ed1-f51.google.com [209.85.208.51]) by imf27.hostedemail.com (Postfix) with ESMTP id A6A7640005 for ; Thu, 6 Feb 2025 08:00:22 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=owltronix-com.20230601.gappssmtp.com header.s=20230601 header.b=GkdZsjwX; spf=none (imf27.hostedemail.com: domain of hans@owltronix.com has no SPF policy when checking 209.85.208.51) smtp.mailfrom=hans@owltronix.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738828822; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hbgZDF5c0CDtttVg1c0iGpgYnifwm2dl79coVeP664g=; b=2bhRimEZbllNZATGfDeGUh+vWrZlhJgfu8dn5nC//cIFjsV1SxjnQ3fQmV9iTbHVVdz7F4 UcsNyGRzjpDlzNzbD036v9FzrPsxCUCkMLtgUeuMSZjlWQtgK6guLWqh6a0VRIFtEfypHt aPhvWJrtROOnvuEvBo50b9lpbRSBO4Q= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=owltronix-com.20230601.gappssmtp.com header.s=20230601 header.b=GkdZsjwX; spf=none (imf27.hostedemail.com: domain of hans@owltronix.com has no SPF policy when checking 209.85.208.51) smtp.mailfrom=hans@owltronix.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738828822; a=rsa-sha256; cv=none; b=EbMvRziz3NmvN0p9Bmw64mZ40P+m/JViGK/OWw5AQXEMOt7XrqPX4mhxy3JsOFmNUjYpIb +gr5K0sX1WryT6ty3RFQh+e4YZPihoRCjPpERizRUdQxYxTejgjzYzcI6olkXpmksu8VdM 7dbUERy5STv84po/qExbvDQOKzrds9E= Received: by mail-ed1-f51.google.com with SMTP id 4fb4d7f45d1cf-5d9837f201aso3363417a12.0 for ; Thu, 06 Feb 2025 00:00:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=owltronix-com.20230601.gappssmtp.com; s=20230601; t=1738828821; x=1739433621; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=hbgZDF5c0CDtttVg1c0iGpgYnifwm2dl79coVeP664g=; b=GkdZsjwXzdvP9GhAxwDcJH8+pmmC4RUKiglWJK3pSy9pl6Y/ITpdXSCV5rQfFq8xNK ANLv0fpRuEKHIh9q6+aQZeswnjM2IpS7hEDg3ASOdD5MvJgdVjrl/qaVCxthMnV7ui/o L5KQWF8XKzdrygU/QAP1siLCV8a4a4YfBu8RjeV/ftMfCrgViw0btIMJEyoTJqz/gRBM WC9EN1xnAoSNaBxTOl1uXdMMfOcrx3KQ4rraC4KHgLrixFvSHbGGoxJEGbLVBHBjyyiA rvHGvJ/Snx0fWvapfFhyhLyHMohAxwQ4TgG5xg/WZ0lSlBnswwnSrh4sBMbkT2piDYOP NQTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738828821; x=1739433621; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hbgZDF5c0CDtttVg1c0iGpgYnifwm2dl79coVeP664g=; b=vWak+PW0OvboMxQpblMR1hdFUCY10awvcBvHu7dq8lYCDcWY5NblBeLfUXiDZlWd67 ReVhbXJecbQeaTbJsIdQE1hxW+fZ0mAURspvawZU1V3ORl41/yT4PkjVYJh0zob3DFOw tezGq6sOM4/Xz6LVuS6+NulJypSrM4SWTIShSEyLVbNtMyToxCqVmIVkTnBdI1wu+ptT 8GD66GSBQVBPh4YFVTqqchhAuUbb5DyzSjdpaP6kGGujwZnF2iJMEr2aagsyQPRN7Hvu 3t7D5H1wOfmIuFid7tzOpjR7HfjiD1htlu9/8m+5+KsJB/ZRKHCEAHHS4jHs0jjEB5x4 CwIA== X-Forwarded-Encrypted: i=1; AJvYcCWXcGZYnEhQBT5z/+tSpY+0EZvxkGTiAsajCm/BVm9ej62LzPeewxmCBs08NpVSM1ZJvqk2Y4OyoQ==@kvack.org X-Gm-Message-State: AOJu0YyDBPdgRVLGbu9dkORrvNDh4QzSUk/f24xfXmhEklP6x8uQpoir lRshaVpZmDAc3TR8sv9++YGt2+VtOo6GltQyBRFWrGKnKuYTMxWyjfNjjsr63EgSbGp1mUtoRp+ jcAl1Cjg1cQBlDVvDonYRdCjwqpX2Z4MaIXNP5g== X-Gm-Gg: ASbGnctaHmiqPuRGI5yFKbk7rBnQmASl4qSJTYqU9YVVt6SBXruqlrbWvPkfZDJWBmD 4jWnc08GpHDzAmckaBUPKo4OTEK76gSnq7JN0W9x+jRVTDfvH8JwnhPjBXs/ZEjSPd1GKstHWdA == X-Google-Smtp-Source: AGHT+IElRzdXU9ihnCWisqkyOvOPlw1yUKPT3RVc8XB0QIPGKptq6RLAqxQ6e7F7OtfcIdOIWTr8PicD7kCqwOpbH3A= X-Received: by 2002:a05:6402:3491:b0:5d9:f3fb:fa45 with SMTP id 4fb4d7f45d1cf-5dcecd03156mr2299095a12.16.1738828820703; Thu, 06 Feb 2025 00:00:20 -0800 (PST) MIME-Version: 1.0 References: <20250123202455.11338-1-slava@dubeyko.com> <063856b9c67289b1dd979a12c8cfe8d203786acc.camel@ibm.com> <4907d1ff5cd5a846188b2c9d77d110d926a37ac7.camel@ibm.com> In-Reply-To: <4907d1ff5cd5a846188b2c9d77d110d926a37ac7.camel@ibm.com> From: Hans Holmberg Date: Thu, 6 Feb 2025 09:00:09 +0100 X-Gm-Features: AWEUYZk1oWymAO0tivzc4PDnt_uFIZ6c8QzmNk589SrSt0_xIRlit5PPTit-q4Y Message-ID: Subject: Re: [RFC PATCH] Introduce generalized data temperature estimation framework To: Viacheslav Dubeyko Cc: "Johannes.Thumshirn@wdc.com" , "linux-mm@kvack.org" , "slava@dubeyko.com" , "linux-fsdevel@vger.kernel.org" , "linux-block@vger.kernel.org" , "javier.gonz@samsung.com" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: A6A7640005 X-Stat-Signature: 753zaqa6p5rebsmof4zdyryn1ncd9eiq X-Rspam-User: X-HE-Tag: 1738828822-93646 X-HE-Meta: U2FsdGVkX18/v8X0QnJNxnk8OoDEhLT5LbFJoLGD0LtwBn/OtG6HK6gleAl1jFY23+YYH+zY6BDDiTSJnF+u5/YIQBkrv+NGVyZvv3eNPEZkKVcPklssOVA9Wyyq8neMCaxRQhztAP9k6XX3Dwbd4V+nLXafoccS1gkg9jf0V7/h1wnYxo0ffsvgoAYUEVajC5WupvhWWucCFhivKttlzr6xMSUx397nYpZ3/8ymgOSATM4H/QS1NW7ATwufRGrZldlHoTtOZP5tFdKgi002N5+n5zVgAIgS+ExkGuZWnWwq6sQaa7ZCbPxZrYevObh6uXNFKUoGQo+127nCN2d9hbV4ZhRYbP0pXGoiofShNSzRKuPktwEGAbP5VRRqMk5K8gpmdtLyuoPFqIx/KB8upk6j3u5LDnLaWiNfJLAap9jIKkTZj9RtJOtzexC8XlQrpfcqG3PFp4HMbYeiMwG4gVWeR4QQW1Fe8/MM0P5mYldlwo5C3PiHK3/CC30QGbHG+NuPudPLMmcgTp8eYnxl8pSDy0bi3xJt48l75m7xgOAjyZozuvQgZ9sUlUpPCVlQztU93AAzNV2++fQhOcJIMMlGjEHfTZ7Ju7KkoBBxCdr0ECLHGWwDFLL0/4fa9bCaoolCjtRd570IVrOt2qtqDSJ2/xzx/URKu6Omed2ZDJSDB07EJ4TrD749QDC38vcHi78raPRPlQCAte8qmQc8TbACl4t/IZ+Y11NBFXcODEZvT+SykGLAv4ynID7Zz8k4LVkxUKLgIaJG3u3+XRkupz7sxRwvRAgy07H1SPXO2pvhXGP2/MPcl3Ax3ly9eveUHtHfY/StDHTkwLgfsdYe1SA6HFiFyyotvKc700qkwCILFQ4JQay2WRFMx6b6qmRuIr9AS054l6NLv2UyBxn0UVTBg5fmA9YJeg1gh09JtOAPm3fjfPnNpS3SUKegJmRzMhPtecXF5SCRGF5LRJP nJjT+vDp cQSYV3HgXtFtS/U9EH1TCVq32A/OsP0t0hNFAdvrtHD+IeWANboxFm4j4UtdRKfRjfqKMQ05rHFAeGi7dEnmqwgXiLozAwg5ZblPdoW+Ksi28TdcEiHdWSKq7BfL3IrE5ob6f+FivsRMyf1xumldfKgvv+s+HkWYfZ0FJnAPauh3ydIT0xIrY/SWOSzB7o/oLJdLCfXRJPZCWdQouJslw/z1aIbOY/LVnKeMBmubmQJS8/plUa2SRHHk9nVe41Fddoq2vUvmyStsbIuWrx7yS4DMcU1adyeO5Gxgi X-Bogosity: Unsure, tests=bogofilter, spamicity=0.490697, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jan 30, 2025 at 7:31=E2=80=AFPM Viacheslav Dubeyko wrote: > > On Wed, 2025-01-29 at 11:23 +0100, Hans Holmberg wrote: > > On Tue, Jan 28, 2025 at 11:31=E2=80=AFPM Viacheslav Dubeyko > > wrote: > > > > > > > > > > > > > > > > > > > Another trouble here. What is the way to measure write amplification,= from your > > > point of view? Which benchmarking tool or framework do you suggest fo= r write > > > amplification estimation? > > > > FDP drives expose this information. You can retrieve the stats using > > the nvme cli. > > Do you mean that FDP drives has some additional info in S.M.A.R.T subsyst= em? > Does it some special subsystem in FDP drives? Is it regular statistics or= some > debug feature of the device? It's mandatory for FDP drives. The "nvme fdp stats" nvme-cli command will report host and media bytes writ= ten, and you can calculate write amplification based on that. > > > If you are using zoned storage, you can add write amp metrics inside > > the file system > > or just measure the amount of blocks written to the device using iostat= . > > > > I see the point with iostat or blktrace, for example. But what do you imp= ly by > adding write amp metric inside the file system? Especially, if you are > mentioning zoned storage. What is the difference here between conventiona= l and > zoned storage devices? Since garbage collection happens on the host side for zoned storage, you ha= ve to measure the write amp there. It would be convenient to have the write amp statistics in the file system for this case (otherwise you have to count user writes and device writes through some other method). It adds some overhead though. Cheers, Hans