From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DEC3DCA0ED3 for ; Mon, 2 Sep 2024 09:36:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 363566B01FC; Mon, 2 Sep 2024 05:36:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2EC8F6B01FF; Mon, 2 Sep 2024 05:36:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1702C6B01FC; Mon, 2 Sep 2024 05:36:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E7D496B01FC for ; Mon, 2 Sep 2024 05:36:45 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 78E4CA1B6B for ; Mon, 2 Sep 2024 09:36:45 +0000 (UTC) X-FDA: 82519293570.18.F974063 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) by imf03.hostedemail.com (Postfix) with ESMTP id 701202001D for ; Mon, 2 Sep 2024 09:36:42 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=ivRxZIKP; spf=pass (imf03.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725269781; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=g9V99G/fQuvajLco/VpGaPuxaTwxa+LXYCFJSoYupVA=; b=bIog6t8gt3E/XnHZJtDUPkFh4o/HYQKHiA4oukvFPELcKPCVGvqSUXJ1zjGPCnsfofgFU9 dvGJ1k87qeWd9I4gdZiWCnx8/9JSH8GjgncM2CeppL0iAQJnw4iB2Cn/OgZbe9hP3k/ARA 5mqfadS/ePEz1OJHtnjqg2HNGuEJgpw= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=linux.alibaba.com header.s=default header.b=ivRxZIKP; spf=pass (imf03.hostedemail.com: domain of baolin.wang@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=baolin.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725269781; a=rsa-sha256; cv=none; b=h/XUnR+zZhubKPkCYCRCF2+tumicumNArg2nCbH1wBp8jk9wTT1Fg/PbHiHdnqcAbTC2H3 ETqQ1PvWszo/ot1d2yDSURAj+D5DaijLZLeqJczmgEDTTSSVdmKjY3rorHzlurMWNUb2bG K/+bFTeM7JmEDn7Jn1DUCX6px5TD+iM= DKIM-Signature:v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1725269799; h=Message-ID:Date:MIME-Version:Subject:To:From:Content-Type; bh=g9V99G/fQuvajLco/VpGaPuxaTwxa+LXYCFJSoYupVA=; b=ivRxZIKPmUaf5a+IZxbJ4JIFIfydV7bFr7MFRm8Y9rUISZTHR2RhY0DjjL7aNORB3iru1S5URECneKTfhJDQV6u5GkeUyyH4eXuApHhUPt00nc7thmU7zSvSCvOIFeFQH53hluJCFkScDNd3wiZrK5I+WKFbSTCBCXXdq1+W/+A= Received: from 30.74.144.122(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0WE74Bmk_1725269797) by smtp.aliyun-inc.com; Mon, 02 Sep 2024 17:36:38 +0800 Message-ID: <2cfb4e1a-d9be-47ab-b92d-94cd65bfec43@linux.alibaba.com> Date: Mon, 2 Sep 2024 17:36:36 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH] mm: control mthp per process/cgroup To: Nanyong Sun , Matthew Wilcox Cc: hughd@google.com, akpm@linux-foundation.org, david@redhat.com, ryan.roberts@arm.com, baohua@kernel.org, ioworker0@gmail.com, peterx@redhat.com, ziy@nvidia.com, wangkefeng.wang@huawei.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20240816091327.54183-1-sunnanyong@huawei.com> <3ac1e404-a531-a380-7a2f-6adae4640da6@huawei.com> From: Baolin Wang In-Reply-To: <3ac1e404-a531-a380-7a2f-6adae4640da6@huawei.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspam-User: X-Stat-Signature: if7yswxhk6faok1454p4jzmz7njuap9a X-Rspamd-Queue-Id: 701202001D X-Rspamd-Server: rspam11 X-HE-Tag: 1725269802-868598 X-HE-Meta: U2FsdGVkX1/Xxo8+GDVl9Iaxr4GO+n/nNZRnZNEH7ArnXs9B71lMOD/C92+RnGcs9gQRbzGFcnft8Xgxv1laUJUrSj66oLHqE9/80WQ1T/0KO/OyjBu+YKc1gGZhm1jcFWbcplxmkyeyeCo83kpiZuDVWS1qW0KHtvfuAl6Wxh2dnFzlarqSo6RpQHfBYSoHaOkGkZ6o0cOHGRBraoMZXOn/5sFRM2nJF7CkSYoMPWHsBqGDAAQjobEvLOplQIlZhSP6DCfxD3oYlwa0dwzOIjclKYtPIiNPc5tQwp+yxMq4J096CPOur6+XASF9XG6HwhjiRLcuGpGlPyfYqHNHjI+3yU5Q2jX0noi4+YBmL3tyJHGQRTGDrEYNuuf9hXKNsthO1Nlbv/AX1fuSTlnNDfrvGsziF+Hv5sNGbqUxeT+tfkvFYPn//G1Y0S3U1n0/lqLrQ2dj5VIeHZVYFqMh/Bo49xLYstMbmtp7n48Kpv222FRjes8i9Muc1R4rKg6+Hy9m7DYJMmy1xXDWzocYvRF3z64P0doLo+Kri5lTSLkfipTH1nrlKqb2ju8I6JQRS4P4jCWfGwcXrZZCZgJhyOvscnXkeTf6nrqU4jnApwtiu+iFp1HKAgZMywNK/iWBTLK1KegfrMa4u+fCwe79Huufo0kiu1AgRroZYhcg3aDJy+WJt+6nj2TcmtiH4jowOTqBNOB+QNqspyAE/fyJdafkeF1zOtwBOEts9v9RDn85sChg7S7yiH8Q4AQIsP9Z3rT6WP/8fCAx4ywToxU4Fa3atXtYIHKI/3QZXk4mCeUprqVMCrIK1XaJRQ9PEr9dhI52+/LppFIBuwp6PfPbbtNHEB8nFLZL3O5TwovWolACAlWebRvDPEYf+ct3oQlneYvoZ2o7pDC+5zb4HqHOPDqtEo/iNoGvfRR6eYiKmXhhqJEXvDogX/Gqiv9cuD4wZCL2pVkB7uVvLEJCp6/ TRXmq1gM OCwMWc7cdmzpjP82uEvxSMvXno/vucex9orlBYldRSYnByNuN9TbhQNBygRK3VsHSOGVI/4hVxeyQfvlsVFSDbYowDVRWLlUm8iZWQK06bO0e0VTOVPaYeBKPkIrTLKVbE1OMYrycIcxOjDv6tq/TYXSQKBkVTBrP65VUvBHPS12L8YYNCexg4RCGQA1BPWxcNuahx50sqbbo9MqTBMKACnXG3lfX2ExBtlvSfXFy+E5u7QthjzSijaBcsiTilYKJnoZ9uVYWpJyM+Dx95fBnF2nQrydCmuyO3GGkZftOYHV2ZwR0yGl5044iFgnTZ1HXyUnB X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/8/19 13:58, Nanyong Sun wrote: > On 2024/8/17 2:15, Matthew Wilcox wrote: > >> On Fri, Aug 16, 2024 at 05:13:27PM +0800, Nanyong Sun wrote: >>> Now the large folio control interfaces is system wide and tend to be >>> default on: file systems use large folio by default if supported, >>> mTHP is tend to default enable when boot [1]. >>> When large folio enabled, some workloads have performance benefit, >>> but some may not and some side effects can happen: the memory usage >>> may increase, direct reclaim maybe more frequently because of more >>> large order allocations, result in cpu usage also increases. We observed >>> this on a product environment which run nginx, the pgscan_direct count >>> increased a lot than before, can reach to 3000 times per second, and >>> disable file large folio can fix this. >> Can you share any details of your nginx workload that shows a regression? >> The heuristics for allocating large folios are completely untuned, so >> having data for a workload which performs better with small folios is >> very valuable. >> >> . > The RPS(/Requests per second/) which is the performance metric of nginx > workload has no > regression(also no improvement)，we just observed that  pgscan_direct > rate is much higher > with large folio. > So far, we have tested some workloads' benchmark, some did not have > performance improvement > but also did not have regression. > In a production environment, different workloads may be deployed on a > machine. Therefore, > do we need to add a process/cgroup level control to prevent workloads > that will not have > performance improvement from using mTHP? In this way, the memory > overhead and direct reclaim > caused by mTHP can be avoided for those process/cgroup. OK. So no regression with mTHP, seems just some theoretical analysis. IMHO, it would be better to evaluate your 'per-cgroup mTHP control' idea on some real workloads, and gather some data to evaluation, which can be more convincing. Just my 2 cents:)