From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0A6BC47DAF for ; Mon, 22 Jan 2024 13:34:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 751086B0083; Mon, 22 Jan 2024 08:34:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6B9646B0085; Mon, 22 Jan 2024 08:34:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5A7AC6B0088; Mon, 22 Jan 2024 08:34:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 4811A6B0083 for ; Mon, 22 Jan 2024 08:34:20 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 507E51209A4 for ; Mon, 22 Jan 2024 13:34:19 +0000 (UTC) X-FDA: 81707041038.04.98808F6 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) by imf12.hostedemail.com (Postfix) with ESMTP id B2B0C40003 for ; Mon, 22 Jan 2024 13:34:16 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=WJcK3X6n; spf=none (imf12.hostedemail.com: domain of ak@linux.intel.com has no SPF policy when checking 198.175.65.15) smtp.mailfrom=ak@linux.intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705930457; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=137V3UCCAqsz5EqolFilfMmgQNJ17iibSNyThvyIHwk=; b=P4bAoTCNWNw46GUzXePbxbD9mrfDJMjIQXqzRmjUgKA9c92wUbxsuldmvzOeJumyFowUD2 NpTmMom/xSaX6mhgWAoE0geLtA17Hfw+s/c4DD2+usjZ5BHLO7i62aozizGxS3GA/SYyYj LUPzsVXTEtGgEBA3QcWtmAv7BYNTXjk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705930457; a=rsa-sha256; cv=none; b=EE8bA1hoMiOaTm0T4IVeA5aM/s0KWtIqbs0F2d8xXWOSC3vLhqwVvZka7mhXuUAJ3+z0ja VyZZy8ysYWtuAtixswHJZAAERHY3WyEfGjA0R7j8ykBbhHsIx/t9UCZPy0iU+4X6rKmnes A33Dbu0Cw4fuPKNrOHTArzod9M8UDd4= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=WJcK3X6n; spf=none (imf12.hostedemail.com: domain of ak@linux.intel.com has no SPF policy when checking 198.175.65.15) smtp.mailfrom=ak@linux.intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1705930456; x=1737466456; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=58hk3EGgivAqnjEyYxsUglKu29GyzAljTRDcz4+mJzs=; b=WJcK3X6n0EnjK2uv4lsdXzs37aYIbs4uQOZlU71LAadFKmnkd2sqCAAu wa/+zX8vgwXZsLBdH++bUSFsagdH8NHPUETqPXo3m8Ri+Zk39Otn+4Gtz hqz2ZyVYQzF23CanIraMYWSUfQooU7uom1zOVc1GVaeE4wDjX7a1jvwI3 47rBFLMnbh7UZqP/kH+KcQs5Pjrj2/XcaPDg0+xqOaVotlhIBeH6bqQue jHqz3Ypmo5+BQC+EDWd8LQr6rIHMKbcJd4/c5xC64/cBH3jh/1r4m7i1q 9Ks9Upd6HUno77BYI/59Djh0/h/xao89y3aT9lMmvxTD1DqgXTer7GYSH g==; X-IronPort-AV: E=McAfee;i="6600,9927,10960"; a="1091139" X-IronPort-AV: E=Sophos;i="6.05,211,1701158400"; d="scan'208";a="1091139" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jan 2024 05:34:15 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10960"; a="928993791" X-IronPort-AV: E=Sophos;i="6.05,211,1701158400"; d="scan'208";a="928993791" Received: from tassilo.jf.intel.com (HELO tassilo) ([10.54.38.190]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jan 2024 05:34:14 -0800 Date: Mon, 22 Jan 2024 05:34:12 -0800 From: Andi Kleen To: Dave Chinner Cc: linux-xfs@vger.kernel.org, linux-mm@kvack.org Subject: Re: Using Folios for XFS metadata Message-ID: References: <20240118222216.4131379-1-david@fromorbit.com> <87zfwxk75o.fsf@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: B2B0C40003 X-Rspam-User: X-Stat-Signature: qbmf9wam7acjo75ym58czqoj48u3krhb X-Rspamd-Server: rspam03 X-HE-Tag: 1705930456-656026 X-HE-Meta: U2FsdGVkX1+07F0UwfYTfsi8HF1lyGLQQbubW4/QBaVYxeKjPUnUykuoFVJ1QuXt3cShZjf0SpoanL0uS5P7rD99BZcnEiwN/4DlEqStlSpbJnDBwjp7JSL0QCUHPLSxwS1zf0qirW0/g1LlUhau+E3brD2jtp1oh0jItvQphWlbKob+76XkgXjtqDn5XKmuwFcm4LuVsQ0ZX55EKv8WK+asWYNaVuUhcZYBBAVpkl/Py64WcZdz7XDvdkm7zrEqebELMbW5GmYTBZ2aiFCgTL3Vk2Z3ShGYNn/hJNSPwoQaZHPM7ujB+uCLXf5CGC38NvwDUVOrr+mKL+tTBVTOrtXxUiKjaSmAMvAwZyxiZITi82mS9CS5sRQDfwHmS8GU2hzQwNY4Hw7cpG9bFvML7SwXgqEBoUowCQBZU0er5PCBvOMP8W3Nwb2GoC3N12V3jQ2I53ruJT2a0OCrtFK1W3NclBltLKh9HEZLnWndQXOw34CnAevb3siXzEYU6MOfBvdSyk3QQXPIrCYwAOTslDH1xIr5m6S/Y5hA6Hj/+s7tpVxNKidb4IvDpRpYte2jDQdEVb4d+0kJ6ML7dX+JIiF1wBq2vti7AY/q4OLwH/q+ZFQEC9LzzlSNbVe+TuwxbwSXNIRyjFSom7yICKLdahGZ0p5EeQ4C/L/MhwAftOz6VmY29bw98DSSlm8dBfuXNqMjU0hXPQBnKdlx1hEl+ejITzbnKcXLaIOcEha6TQBa24/+hTuTmCBhr/yyM59lxtsvBniT8ZPuKKjVXnS+Hv+i+hXqdhUa4PeFzkPDCJIqiPBcBcJydGxIkCX3u0cVKIQdmrs6jzDhwiqNzKCbfV7yo3W8rQC8eoRfltCUzpUxQYCt5pUliRmtPIoY1ADF4TdrCj9RFyLrJ8i0FpsU6hp8hiBu64gj9bZt8j3dRMLxwGCIIdD33BgEf8ddXoW/np2JZVcyU8G7N5ymoBZ kTbGHWzM 5sRBTZI0ucqHvJOMSCb5yePgceu6GJiu/96EwsoHH7f5YcBSm/WpEOM4pmmj+a24X80MfXMw/JvNb2zoDDtubO3+U0J6i/L6I8xB/gWGxMn4bqNjowZuH0iLZJdM+uWZWP8NC X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: [fixed the subject, not sure what happened there] FWIW I'm not sure fail-fail is always the right strategy here, in many cases even with some reclaim, compaction may win. Just not if you're on a tight budget for the latencies. > I stress test and measure XFS metadata performance under sustained > memory pressure all the time. This change has not caused any > obvious regressions in the short time I've been testing it. Did you test for tail latencies? There are some relatively simple ways to trigger memory fragmentation, the standard way is to allocate a very large THP backed file and then punch a lot of holes. > > I still need to do perf testing on large directory block sizes. That > is where high-order allocations will get stressed - that's where > xlog_kvmalloc() starts dominating the profiles as it trips over > vmalloc scalability issues... Yes that's true. vmalloc has many issues, although with the recent patches to split the rbtrees with separate locks it may now look quite different than before. > > > I would in any case add a tunable for it in case people run into this. > > No tunables. It either works or it doesn't. If we can't make > it work reliably by default, we throw it in the dumpster, light it > on fire and walk away. I'm not sure there is a single definition of "reliably" here -- for many workloads tail latencies don't matter, so it's always reliable, as long as you have good aggregate throughput. Others have very high expectations for them. Forcing the high expectations on everyone is probably not a good general strategy though, as there are general trade offs. I could see that having lots of small tunables for every use might not be a good idea. Perhaps there would be a case for a single general tunable that controls higher order folios for everyone. > > > Tail latencies are a common concern on many IO workloads. > > Yes, for user data operations it's a common concern. For metadata, > not so much - there's so many far worse long tail latencies in > metadata operations (like waiting for journal space) that memory > allocation latencies in the metadata IO path are largely noise.... I've seen pretty long stalls in the past. The difference to the journal is also that it is local the file system, while the memory is normally shared with everyone on the node or system. So the scope of noisy neighbour impact can be quite different, especially on a large machine. -Andi