From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B7CBC5320E for ; Mon, 26 Aug 2024 02:26:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EC2548D0040; Sun, 25 Aug 2024 22:26:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E4ACF8D001C; Sun, 25 Aug 2024 22:26:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CEDFF8D0040; Sun, 25 Aug 2024 22:26:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id AF7ED8D001C for ; Sun, 25 Aug 2024 22:26:29 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 5659E40AFF for ; Mon, 26 Aug 2024 02:26:29 +0000 (UTC) X-FDA: 82492807698.13.30FAF05 Received: from szxga07-in.huawei.com (szxga07-in.huawei.com [45.249.212.35]) by imf10.hostedemail.com (Postfix) with ESMTP id 15528C0015 for ; Mon, 26 Aug 2024 02:26:25 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf10.hostedemail.com: domain of sunnanyong@huawei.com designates 45.249.212.35 as permitted sender) smtp.mailfrom=sunnanyong@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724639124; a=rsa-sha256; cv=none; b=C+mUjq7t+l8JM7RH1UQ74pS1DGAHt+MkWWstiXIjrXZ2dCQzEyMCooAAg2DmLhXzp/QkZq pD/AOX3rHpbNPJ25jKc9+3kmn134AIdgouzsaVwzoxcoZEVKYt1myULP8/jqQIAAcMXE73 /uV2EhaV+YvAGeRzQYjScn8mcdDhF1k= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf10.hostedemail.com: domain of sunnanyong@huawei.com designates 45.249.212.35 as permitted sender) smtp.mailfrom=sunnanyong@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724639124; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YmelZDqAisSh9VThnmp6r2SCQZPLjo0mRriij2QDDdw=; b=l67jZHkyIQ/LSjjyovn57wQMYbdE+d+jXqH52jx5enVL9/voE1B2U60lInspjTkiqH1ZWr nFBGfHs0tFEzvoh1Kb2P1aBuEqp4hK7bvoxfvbd0wDo/uIbcbsoo9yI8QFuiAlvRxy/zcF +7tbQNoREC7Ur1qokZooDiGkO89HSCE= Received: from mail.maildlp.com (unknown [172.19.163.44]) by szxga07-in.huawei.com (SkyGuard) with ESMTP id 4WsZKw589zz1S8tM; Mon, 26 Aug 2024 10:26:12 +0800 (CST) Received: from kwepemm600003.china.huawei.com (unknown [7.193.23.202]) by mail.maildlp.com (Postfix) with ESMTPS id 91CD4140135; Mon, 26 Aug 2024 10:26:21 +0800 (CST) Received: from [10.174.179.79] (10.174.179.79) by kwepemm600003.china.huawei.com (7.193.23.202) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 26 Aug 2024 10:26:20 +0800 Subject: Re: [RFC PATCH] mm: control mthp per process/cgroup From: Nanyong Sun To: Matthew Wilcox CC: , , , , , , , , , , , References: <20240816091327.54183-1-sunnanyong@huawei.com> <3ac1e404-a531-a380-7a2f-6adae4640da6@huawei.com> Message-ID: <0a7fc0ab-cc45-8829-dd7f-9265e5d2752c@huawei.com> Date: Mon, 26 Aug 2024 10:26:19 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: <3ac1e404-a531-a380-7a2f-6adae4640da6@huawei.com> Content-Type: multipart/alternative; boundary="------------0D81FA3994DD395CF7D520EA" X-Originating-IP: [10.174.179.79] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To kwepemm600003.china.huawei.com (7.193.23.202) X-Rspamd-Queue-Id: 15528C0015 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: p94pa49bicf6x1bucaonnhrtptjurcf9 X-HE-Tag: 1724639185-906622 X-HE-Meta: U2FsdGVkX19Po06jIIVbImvT0h0b8kRDCuugsuUXnpUQgIybjCx8kKHmQh39qe+NaE+2fJzuePRNTVefdkLPdF42Qh/vHzBWIP9tPTgA1uwubHT0WGbC2uQh5v5r7aD0tEc4IWq+5BhuUGO5ksxQHCr2C3f0OmlWjxGYUxMYiZR6ZSTi4LC/uFI1OkgxZyQxMMqELSMiwL4wwsjlg7YDYR2hC+x3PwDu3lnrwd5dHnNcrH8Gj41gqWaeVsFylOL8QMlqhHszNj+2T+7+SRJ6aj56wP/r4rFrHFx+UvWtIbsvuYoYgSEJrpCgK8dtXmRfZssizZvwzsWoAxBXCX96fbsOPQkvMjrw+xXXyQ675P0LfoPIHzhrJLr9rkayxfwkksw3ZM7pFpdkdR3WMKnx/ES5IqN16BD07AOhSpgKBKbchTwZp3zzwA28fo7JjVGlNSEWGTRk2bKPa9rWVGQgM/PIA2JOBsccLe6uwnXoeJ0xO8dQlI9yVknY5JQJ+3ZymyvazZeNfGDLC2eknaGIehuBC9lckTftbz/WF3ji6e7m22ajtphmXsGYPSWM/Jr6tH2saZXnrdO4LB/O0yA4wuPuAjRhfIgps+oG6xZE2stLFUboE/oT9ASQf/ktvpjpGJnJxKXfFVLbwBPUhgSFEIb9hE5HNTur7vyQYEaqrfPP5M/0SGqUNhmHPZt25+HdDtqjtGFfjwkIQNnniyx8POqh2ub6d3i5hNTpHCUYDOL8Scm/NoXmw8ZBOnUXZXdF7Dmkc1AFetIvTeoO67A35wsAoDpOg9ArzL+AjJ+LF/T6SIrXKAsCl2MdHcuH6r8Biw0kIesDiYYErtxbJ5iute6/EkteGlpBAIRz9AQgtJY4QO7332pTKqsSpNUzD6sHetZ0PQEvVZ8pj1IfnzaWI2FXY9RItpNfd7ee/zjwSWTb8J5pq7+H6smm1kLZ75y9x88DqMxUKTK1IV1Z8PF 1cEnazVW jC1qi6T4JJcn2JPUpYtn6AvE8R7AETnt/J7vcsDGMc6Yokk6IdK5LAWa1Om6+v3W4mvDtFApBmWWywr2pYygaPNqMSNpmqdzYonObE81Tj7XzbGkM9324g3pd3qmT5vtQJJgB5tLHQix5e42gif285RiesT2uTJ9z2jbAzZinG/63HIAjub9ADpfrxEG6dE+04oUkZNYE0w2RgmKt7TIjjlNlgNXT++Oghs0tPGC1TmvjaTx/YUT1wHDit3y2T/Otrgqo588cdKz5XZDcY+qr3dz21wrbXtsuztVhi5bUQphFnVc7tYQyi8QgyyMdXboptU/a6IBaQYRqDPM2fcUD9vS2YA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: --------------0D81FA3994DD395CF7D520EA Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 8bit On 2024/8/19 13:58, Nanyong Sun wrote: > On 2024/8/17 2:15, Matthew Wilcox wrote: >> On Fri, Aug 16, 2024 at 05:13:27PM +0800, Nanyong Sun wrote: >>> Now the large folio control interfaces is system wide and tend to be >>> default on: file systems use large folio by default if supported, >>> mTHP is tend to default enable when boot [1]. >>> When large folio enabled, some workloads have performance benefit, >>> but some may not and some side effects can happen: the memory usage >>> may increase, direct reclaim maybe more frequently because of more >>> large order allocations, result in cpu usage also increases. We observed >>> this on a product environment which run nginx, the pgscan_direct count >>> increased a lot than before, can reach to 3000 times per second, and >>> disable file large folio can fix this. >> Can you share any details of your nginx workload that shows a regression? >> The heuristics for allocating large folios are completely untuned, so >> having data for a workload which performs better with small folios is >> very valuable. >> >> . > The RPS//(Requests per second) which is the performance metric of > nginx workload has no > regression(also no improvement)，we just observed that pgscan_direct > rate is much higher > with large folio. > So far, we have tested some workloads' benchmark, some did not have > performance improvement > but also did not have regression. > In a production environment, different workloads may be deployed on a > machine. Therefore, > do we need to add a process/cgroup level control to prevent workloads > that will not have > performance improvement from using mTHP? In this way, the memory > overhead and direct reclaim > caused by mTHP can be avoided for those process/cgroup. Sorry to disturb, just a friendly ping : ) --------------0D81FA3994DD395CF7D520EA Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: 8bit On 2024/8/19 13:58, Nanyong Sun wrote:
On 2024/8/17 2:15, Matthew Wilcox wrote:
On Fri, Aug 16, 2024 at 05:13:27PM +0800, Nanyong Sun wrote:

Now the large folio control interfaces is system wide and tend to be
default on: file systems use large folio by default if supported,
mTHP is tend to default enable when boot [1].
When large folio enabled, some workloads have performance benefit,
but some may not and some side effects can happen: the memory usage
may increase, direct reclaim maybe more frequently because of more
large order allocations, result in cpu usage also increases. We observed
this on a product environment which run nginx, the pgscan_direct count
increased a lot than before, can reach to 3000 times per second, and
disable file large folio can fix this.

Can you share any details of your nginx workload that shows a regression?
The heuristics for allocating large folios are completely untuned, so
having data for a workload which performs better with small folios is
very valuable.

.
The RPS(Requests per second) which is the performance metric of nginx workload has no
regression(also no improvement)，we just observed that  pgscan_direct rate is much higher
with large folio. 
So far, we have tested some workloads' benchmark, some did not have performance improvement
but also did not have regression.
In a production environment, different workloads may be deployed on a machine. Therefore,
do we need to add a process/cgroup level control to prevent workloads that will not have
performance improvement from using mTHP? In this way, the memory overhead and direct reclaim
caused by mTHP can be avoided for those process/cgroup.
Sorry to disturb, just a friendly ping : ) --------------0D81FA3994DD395CF7D520EA--