From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF714C25B74 for ; Thu, 16 May 2024 20:06:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 472406B007B; Thu, 16 May 2024 16:06:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 422CD6B0082; Thu, 16 May 2024 16:06:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2EA9C6B0083; Thu, 16 May 2024 16:06:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 0B5F36B007B for ; Thu, 16 May 2024 16:06:08 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 5A263403D0 for ; Thu, 16 May 2024 20:06:07 +0000 (UTC) X-FDA: 82125340374.07.48C10DE Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) by imf18.hostedemail.com (Postfix) with ESMTP id 79C4D1C000A for ; Thu, 16 May 2024 20:06:04 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Nl+ioPvF; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf18.hostedemail.com: domain of yuzhao@google.com designates 209.85.128.43 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715889964; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3Kby8W2TT/HmRlIiLBv5TmONCRAIY5/HsHszAZhYnec=; b=m/6Tk1OwGlRg9w+xP2/xi+NeuY6xd/s0IFZEoA+gDP4RsNRcwVCCIbH+UWyM9kki5c0B4I hk0ItIBPQZxm8dpV7EtYg0U4AvRLD0PDTkY2TQqDrvTRSYOLXjbscxoQbxV+CJNsEWTThB L9xWhGFs6riHUbsmcrpnN4+KDKjrrX8= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Nl+ioPvF; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf18.hostedemail.com: domain of yuzhao@google.com designates 209.85.128.43 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715889964; a=rsa-sha256; cv=none; b=smycYdQ/jRsSy73PXZdY2AVU4qEhiU/oxLP+wPREnQf7L5Q7iBPVACM8ivUclpLNxfA8gD 1TgRSbgJppTQObhLmXx0iwWfJt/UGUjTwc9HTpNJt+sUyZDGurl/1mlje3aEW9zQFC9KsL TXnfAuBYNmLGChU9KzzXf9u7ETNuCdM= Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-41ff3a5af40so3475e9.1 for ; Thu, 16 May 2024 13:06:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715889963; x=1716494763; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=3Kby8W2TT/HmRlIiLBv5TmONCRAIY5/HsHszAZhYnec=; b=Nl+ioPvFBQ9WnTifjY5+9BM+YfO3zS3lWIlrqkHEl/fLNZyQGqFFy00yR35ICITL3+ XC4k6Gj8BgeJLFSu8qzFHDNlag/bIognGQvYsFc4Bn7gGH4IOOOQ2N386K8UdvdZvKcE XoVgTERyHZl43UyCn7VzoHWyXMxEG2HmXSrCqXRC8DZLkOsaxOSY9yp/HfXa96Iu3d1p vWXZaxkG41t/osR/pPbnVU0cVtwAfvMMRy7+vy4s2jJVkad2sIYPPOc/EZw3t2szs5xl FkmlrR4I/ym+QK7ntAzztvxNCEor2fZZz/Ff+hoKtiV6+66qk+owuHZLdOQZKCNexEAw MVlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715889963; x=1716494763; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3Kby8W2TT/HmRlIiLBv5TmONCRAIY5/HsHszAZhYnec=; b=qfqaxq3UDZRGOXLAivEHkXfGGiUv5KauzAkdS3+5LWWl3Jyc1wMsUR/xSYZAjlyThZ XZwlfLbi2ftzFivk09nHDC5qg4U5p/oawJKIwsMHFfeNz1QfeIX42fWLFuFkpSD5waPU HHtxXv/u2Eogl58E8HE4ZZ7/cgpAgr2emeaBJqifTE4HFscmsqnZaD+l0w3wRtJjK63g i4m3QzWjhknK15qgTxDphMXXKknBBv72iTUhZ/g3G2Iq5lvjhBZGv5Ka5rp0+1ZdIhKn u+/A1QUdF3LP7YN/HSMWd2BvNRs4UolC+/LpIIb2oRk7eg+xAiqdWF14Zk7ArEUbaP0T Ggog== X-Forwarded-Encrypted: i=1; AJvYcCV1g8MMj70I4nw/0V5f9j/o6tXjhvsJTy3AS93syis8DwT8syBzEs6aEOfGI4eSahTYgt0Nw5DBnrRnBI5I9UhiW2U= X-Gm-Message-State: AOJu0YyC9s/+qftB+uUdUWu428rQUu9H4uuqptYIghZeYMaiPKO5G7XU EC+YwrVI6mJrumAckIwClqbeX1D/Y8R1tkTCCRs64NWl8kr7XVKmw1hfYsdA8J+VIEOU6nrff8B HWYVrQiKCOhz6XT8fuRUmZc6Wz7/rBlX2IU90 X-Google-Smtp-Source: AGHT+IEQ/Z7iUdqFlT6pFqvHwkKxdUpkqMGz40XxIAEc3OoRAMPnV+kD6BmUoSxTTqD34DZIeumWNDNjz4qnt31WdBA= X-Received: by 2002:a05:600c:b96:b0:41f:9dd0:7168 with SMTP id 5b1f17b1804b1-4202fc1d082mr354605e9.2.1715889962034; Thu, 16 May 2024 13:06:02 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Yu Zhao Date: Thu, 16 May 2024 14:05:24 -0600 Message-ID: Subject: Re: [LSFMM] automating measuring memory fragmentation To: Luis Chamberlain Cc: David Bueso , Michal Hocko , Dan Williams , John Hubbard , Daniel Gomez , linux-mm , lsf-pc@lists.linux-foundation.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 79C4D1C000A X-Stat-Signature: s9zxbof6yd8g66hbabmeu7cuf5gz6xny X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1715889964-655420 X-HE-Meta: U2FsdGVkX1+1tu300a30f314AhfmeDaDHms7vIzDUaR7fOuxLcOOUJ8cHnHZCOK+a8TeI/nkcs/0wBWCjEPUggyNyQCFwjM6jo2qtCvQAD3grlYvqpzK+IQPbkY1l+hnQUxo6sZpBNxlqixR3nig3BeEbew5zud2tWJXYzRPGWYI9C2juPClw/jC76EIpEAK2Hqbz/4CIoKdLl3OghEJwxAO59D1eCQXMYuQGaquoQ+7sge6FKAN3FYmESnPZGCdfaPLdLEDyUkolAeeLvI9T4Adl/og+8eK69lc692fHFDRDrIsbK9EebPaX4L4YNl8YZDFXo+qfFsoTf9tuXWHUWl2w0kvd5PiXrDyRcAjFn2Et5ECtpnB7k8/j8dQ3YMcNbzVCiUQYRQRZO4hdebGmPgVHFNHxCsO1A0cwM7nRGoAZzR2mRS48ATaQESQPa3Nx+lTkBLX7NA0YGNu+V10Nn8aAg+mWVLRureh+L+p0pfcVOqV7E+HZlQzd3Sfsfcq0vWIaH0vq2Xyjc5ORLsQ0rR/Jet+4aEZ9izfcXG5fAfm+xFrbD71omxwM8mvj3oGT2Jz7he+5FIMlY7ET2a+txmuZSekPpSDhFKh6V0IqPH0lJSsDQStShpbWe5vbhcWQiYpap8XjnNk2AfNWKKOvHo96xS18bBRjDg4sK1tiZscov3QcwQqipmh7uvDJ64VtINCb0Be/lZLXuverByMeAFfGlXY7d139KfKQwygPeMCr0h/NxwS4kpJuh178myuXrQv7552f4looYTAGeNpau8iQnJzO3cWolkirLp7k947jFq39Ts8ER4PvI+XzrwtgNBMwCIYZOqvwi8W39yXOMptUKW46XRrK50ObKRgwZx0pKAFSQQmG3lt0gpvPoLsDAhw43q9YdzOds+RJ+rYJu3kz0ys2LaE0rQHAwn2HU/1+rp1/0oiB+Y8wk+ksD61gDxwPtays4RXC0HFchc Y/lxVM0J aCFGrQxQ+2GZROMNeMntD7i2+0FisPIlCPW6wQxXfkd/lRInksNDdChqsWTaqUH3GJ9QzOCutwxZfx3nk41vnBJAEMtb18TuJgnwr0NYuWxEKH/yrf8q77NCTO5Lj5emuJOWZq20wq/du/wWyFBl0104cT2dptruGZQ8WT2ja4mlFVTwELiuBL30lTSOZczCUcj9K61mHClIJf+ZufjElkrXLHFmxSHpLGCyIu8r4/rr3k6fPkSgKPQpUjYXlTMVvahmhBmVsFyAKmMASc6S6tXbb08f3ngOuMl5+2FtJx0FMWS0t7lMtkm5eLQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000135, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, May 16, 2024 at 12:23=E2=80=AFAM Luis Chamberlain wrote: > > On Wed, May 15, 2024 at 11:15:58PM -0600, Yu Zhao wrote: > > On Wed, May 15, 2024 at 1:34=E2=80=AFPM Luis Chamberlain wrote: > > > > > > RFC to see if we have a breakout session today at LSFMM. > > > > > > After the TAO talk today it occurred to me that it might make sense > > > to review how we're measuring memory fragmentation today. We're looki= ng > > > to add automation support into kdevops for this to help compare and > > > contrast memory fragmentation behaviour with one kernel against anoth= er. > > > A while ago, while mTHP was being evaluated I asked genearlly how we > > > could measure fragmentation with a simple one value, and John Hubbard > > > had one recommendation [0], working that proved we could simplify thi= ngs > > > [1] but we also could just use the existing fragmentation index and o= nly > > > consider the values where this is concerned for fragmentation and not > > > lack of memory. It begs the question of how folks are measuring memor= y > > > fragmentation today in production, and if they have any desirable > > > changes. The first approach being considered is to reproduce the > > > workloads Mel Gorman had written and used for mmtests and leverage th= ose > > > on kdevops, perhaps modernize them, but before we do so it seems > > > reviewing how we measure fragmentation today might be useful to other= s > > > too. > > > > > > As for mmtests integration into kdevops, first order of business are > > > just a few distro-friendly updates [2], for the next steps after that > > > though it would be great to review the above. > > > > > > [0] https://lore.kernel.org/all/5ac6a387-0ca7-45ca-bebc-c3bdd48452cb@= nvidia.com/T/#u > > > [1] https://lkml.kernel.org/r/20240314005710.2964798-1-mcgrof@kernel.= org > > > [2] https://lore.kernel.org/kdevops/20240319044621.2682968-1-mcgrof@k= ernel.org/ > > > > Please correct me if I'm wrong -- I don't think we can use a single > > measure to describe fragmentation in an actionable way. > ^^^^^^^^^^ ^^^ > Two key words: actionable way. > > Even in that sense, to say that you need more would suggest that either > compaction does not suffice to address memory fragmentation, or that we > can improve memory fragmentation through other means. Both are possible, > and only measurements can prove that. > > But my point was not about taking measures in an *actionable way* to > address memory fragmentation though, but simply measuring memory > fragmentation in environment A and evironment B, to address the > question, under which environment is memory fragmentation worse. That > said, I am *also* interested in solutions to address memory > fragmentation, but that's a secondary step, first I'd like to measure, > not take action. > > It does not mean that evaluating measurements to consider memory > fragmentation to evaluate actionable measures are not useful to the > single snapshot of memory fragmentation. In fact if more information is > better, or we're lacking other sources of measurement of memory > fragmentation it'd be great to improve it. > > As noted in the above URL John Hubbard provided a simple metric > recommendation, and I tried to implement it but as the patch in [1] > notes the missing semantic would be used folios per order and to add > this I thought it would be expensive today from, as per my last review > (perhaps I am wrong). Hence my approach to only seek one value and see > if its positive, and if so how high. Thanks. IIUC, the metric(s) you have in mind would be able to compare over time or across different systems. I still don't think a single measurement can do that because different orders are not on the same footing (or so to speak), unless we are only interested in one non-zero order. For example, if we have two systems, one has lower fragmentation for some orders but higher fragmentation for the rest, and the other is the opposite. How would we be able to use a single measure to describe this? IOW, I don't think a single measurement can describe all orders in a comparable way, which would be the weakest requirement we would have to impose. > > IMO, we would need at least multiple values, e.g., fragmentation index > > for each non-zero order, to describe how fragmented the memory is with > > respect to the order of interest. > > Here you seem to accept you can measure how memory is fragmented with the > existing fragmentation index for each order, is that right? Correct. Fragmentation indices for all orders are what we have now. > Or is it that this is the only tool we have today, but likely we could > improve the metric? With them we can compare fragmentation in a system over time, or fragmentation between systems. In addition to comparison, we also can tell whether reclaim+compaction would be able to make an allocation of a specific order possible, as I mentioned earlier. > > Of course we could encode multiple > > fragmentation indices into a single value, but that's not really one > > measure. > > If I am not looking for an actionable measure, but just get a single > quantifiable metric of "how badly fragmented is this system", is > a single value not useful for that purpose? As I (badly) explained earlier, a single value can't do that because different orders are not on the same footing (or so to speak), unless we are only interested in one non-zero order. So we would need fragmentation_index[NR_non_zero_orders]. > For my purpose, it was about evaluating if the general situation is > worse in environment A Vs B, in that world, would a single metric work? No, for example, A can allocate 4 order-1 but 0 order-2, and B can allocate 2 order-1 *or* 1 order-2, which one would you say is better or worse? This, IMO, depends on which order you are trying to allocate. Does it make sense? > > Fragmentation index of an order can tell whether reclaim+compaction > > can theoretically result in a free area of that order. > > Indeed, for my interest it's the positive values, about when a system > has memory fragmented. > > > As an average, > > fragmentation index can't tell which actionable unit area, > ^^^^^^^^^^ > In the A Vs B simple measurement introspection situation one is not > taking into consideration an action but just being a silly memory > fragmentation voyeur. > > > e.g., > > pageblock, would be the best candidate for reclaim and/or compaction. > > That would require a ranking model, e.g., a cost function and weights > > for reclaim and compaction operations, and calculations of the cost to > > produce a free area of a requested order for each pageblock, i.e., a > > 2-dimensional measure > > costs_to_produce_free_area[NR_non_zero_orders][NR_pageblocks]. > > This all makse sense! Thank you!