From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A36C4C369C2 for ; Mon, 28 Apr 2025 03:44:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 326656B0030; Sun, 27 Apr 2025 23:43:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2AF246B0031; Sun, 27 Apr 2025 23:43:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 14FA66B0032; Sun, 27 Apr 2025 23:43:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E53786B0030 for ; Sun, 27 Apr 2025 23:43:58 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 5E94B1D01BF for ; Mon, 28 Apr 2025 03:43:59 +0000 (UTC) X-FDA: 83382058998.13.47485E4 Received: from mail-lj1-f174.google.com (mail-lj1-f174.google.com [209.85.208.174]) by imf27.hostedemail.com (Postfix) with ESMTP id 40E8240005 for ; Mon, 28 Apr 2025 03:43:57 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Qv+Ypt3C; spf=pass (imf27.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.208.174 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1745811837; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zM04meY/cNVobywryZ3QNUIjX4LroMojaucg0JZUKCM=; b=Z+OI8XptBZlI1naInVz6yF+9HtvN7Fn4/hkit9kXZ526ee4GwcUwFHYojuLvyQy8Ad+Kc/ w65FdUQF2lw4mCMFkRrVvHC1n/eVQp3UK4qhx81B5iSH/sWBV5ktDVp9jdGYQzyP/mL8Sn u56e39eZBlArIn53fSlZ8QKAkclku1g= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Qv+Ypt3C; spf=pass (imf27.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.208.174 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1745811837; a=rsa-sha256; cv=none; b=kThBxtypuZ4LMaYeKN6K99UtTloxQogzsqxWZFL0Ej50D4CZZaxVNKHJz/vbiHxiiz+KZi CbnAw5Xrb373y4lA7IpahhRsMsCAlTgtGsjPNlQUBxT7JeBZGQWunXWj+vZXwLOziEvgdf zkvjfTDw9tM/ithGhoohmz46sRbwnBo= Received: by mail-lj1-f174.google.com with SMTP id 38308e7fff4ca-30ddad694c1so47583601fa.2 for ; Sun, 27 Apr 2025 20:43:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1745811835; x=1746416635; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=zM04meY/cNVobywryZ3QNUIjX4LroMojaucg0JZUKCM=; b=Qv+Ypt3CcqSRobXBOkuy+MYhw27Lxv+DywmgzdKbcurRUtr2YQMU4gD3smQG9OkIII hrZG76m6SVjBjybs/PhA3xo26CW94SYZ9R8SIcciH0QIm5G+EhHd/7neqM/Az/tcBgcN aMzNi6Tmzs12ph5+WiC21OK3KSQGWFBdcQ+7nAg7NmyRipUcnNoHdaQjUOOj4dqlCLH9 CuMVqa89BIORoVB5w1JGmLz7U64qstuS6m3D7hqRa4XCUwB57WN9t6rVFbyNabuRsPPb TmL5TNxw2++4pJGG1jHM1/7Mrmfu2IGzhHHGmChsfWiaCH5w8n6AVFHJ6LYsJdn3cSrs TNDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745811835; x=1746416635; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zM04meY/cNVobywryZ3QNUIjX4LroMojaucg0JZUKCM=; b=i8yca3RK5c4YjkApSjE/J9kMZaCXw3TgU0FHS2ehK3+Xt3QUfNhqEbksKzn1zkhPt3 JzE87RPKor5uFYxt7k5w9uuUGbmC02j+oHOUFacC2SACMcHK+83IR1jTcEs1HkMPUWAm YAe5EI60KzO8tsSzifnzW3ZzllqzbWG8FKQTBrz4CuU2XHt5EpiEopkcgG74SxuYI8C7 4MvYSQ7LJXTEH6GOPiR6NFOc2jy10JgQVdlMUO8GqaDQ4UFHX5SVaomp+TS5YOej45Fo N1prtu5HJpaEj0J1AC8uzCPXz+x8KJzq/9zw7OQlFlYCgUO4iHj6pfj4DD4I5gbp69tt RqtQ== X-Forwarded-Encrypted: i=1; AJvYcCWOO0+KmauSKcjW/4MTje4mRh382x+B/p5ECf/33dWusAlwkdEZRV62ZOY/n54lA/d1l4/2Yr5e7w==@kvack.org X-Gm-Message-State: AOJu0Yw1vydnm9otxfJbwikqjLdzTFB7kUnFvjHSf01AKZOd4WJDAuJy I/vx7pzh3fdcHvXHEm6cpD8amSqVfu6AO6rKOeAagdmJYpt/7zAVx1OIrncdUXBRiJ4mHCeRm7g riC5/5PLS/COndZhExjdJ3yFk3qw= X-Gm-Gg: ASbGncsbfL01du/uWng4s1v95doZD6mqDsrCkNkdFHU2g2NHboMBN+wRVOkRiH4SEXw iO5AMG10I1HUmiVHnN6IaCR/w0XS414pt57tL7SFHYn41acWbYYkOzNecVp3rxv09wAbMYy4aNW Z8Bl59SJUc91hnDQcy5MKIC2wnSAad+y5F X-Google-Smtp-Source: AGHT+IEH70hVVW8rOsOgQxJZC6NCsSrX4mNRBQgHnfMcJhGj2rh1j/QWmLKtV8gyZHlAIJARxIxzR5Q7v1nitZy2bb4= X-Received: by 2002:a2e:a802:0:b0:30b:ee78:79d2 with SMTP id 38308e7fff4ca-319088ac327mr36378271fa.36.1745811835253; Sun, 27 Apr 2025 20:43:55 -0700 (PDT) MIME-Version: 1.0 References: <20250415024532.26632-1-songmuchun@bytedance.com> In-Reply-To: From: Kairui Song Date: Mon, 28 Apr 2025 11:43:38 +0800 X-Gm-Features: ATxdqUH374CfcqPtJrjSLJjOFu2q-YT7JNnIJJVXMirRn2dBGdwtM4zrlTltdZI Message-ID: Subject: Re: [PATCH RFC 00/28] Eliminate Dying Memory Cgroup To: Roman Gushchin Cc: Muchun Song , Muchun Song , hannes@cmpxchg.org, mhocko@kernel.org, shakeel.butt@linux.dev, akpm@linux-foundation.org, david@fromorbit.com, zhengqi.arch@bytedance.com, yosry.ahmed@linux.dev, nphamcs@gmail.com, chengming.zhou@linux.dev, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, hamzamahfooz@linux.microsoft.com, apais@linux.microsoft.com, yuzhao@google.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: 4wfhfqunijchk5ye7inai6okjqfq4sh9 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 40E8240005 X-Rspam-User: X-HE-Tag: 1745811837-688657 X-HE-Meta: U2FsdGVkX18T6HNz7wVKTJSDyX7mTzh1Wr9KhMkRA8FEHOU7orll+37NHIIzxeiVucSau9T9htDgZLmNWlA5RIOwH5bN2IDZW0ToKY7tPOldxt9GS3eSoBv41NoO1sZTMfpDfjS/YSg1JQrmOmWOa4DTOWNsdLlggrSv+yCg1Rf6WgdERBo/vOOEv35mwrfutE7PTsXCQHGoaBaE9k252gXyO263lsNaH6RC7b505FvvxW9r//k/vdwEp4y8E89g6Giw6V6Kz+cjMTCsyUfrzay1s8IpE8e38/D5vboGHsAeAULs4l2CEnUd5h/3QDCu2/1E9KPe9/bYUJx1WXPHOaos2+3YZCjQExc00X5eThX8gHxH6KQc/X4UdbSk9PQelrakDmfZ7sh6QnJjGAfr67DcT6NZIM4AQnzKreJmmttAjZSZALgsLh8BXzra9BwcxUv3EnVij84ie7PuSmpKkROoC/3vRbIj7kaYgHmExccPedGcIffJl/blav3qQy2sPe5mMlTtqPacB/ZZ5i+GgLbhrB6S0sP3fUqnh/mh+310CSGKvQglU9LNGdU35n09aDDCF8OCYKYv55GcuzPSyk6uj8ujxEcS7+qj7ri3xLWr9YBdEXBwrMH7BSOyOapUb+AyzHtvqKex11YOfuFic6v7fPhpDqeZX4Hwp+ssEcYfqXNjH7W1Lk3wybTBGNzHVVEyr+8Vpz3yt+HlWHZFB3Yyo/nmcD5zE1S6eClMaIMQdOK/3elgBUxRD/4jLQtHZVqilLk2ASM/CiOGwhxPF/kzsVdzVZPJxLqdVibuJaFD3sfBrDOLBJds4zDrgYpZPXgqXtzE/8CSZMOVyz8xzQvg5qhPcCr4SCzcrYaZ/lsiYYlCjP9H8G80CxDyfmUvXG6UtM7uLxAHO1mRpUku/la1zuesfoPIHLpcuR3FBjw9yGvbJClobfMJcmW6tOhbdqzVWiBJK/4gGJJJ4Xq dpJ4Zfep JGi0fQgAau/KtKnOwcn7p6mMLKsPwnC2lBDKWGLBKkAa4dmVOoMcCvb5F4a7QQko3JSJMJicG5mstsmPYfdzBIoYYRtj/GdlN4V6iHFyL6tWb+Y0jchCboL8e9u4PIRUkmPHARumvd07RaNuqeMAnv5df3TSrcWo0/Y16mvKiZC1RO7jRj2oESKsrZue9JNvNQhPfEaUQPQt/t3ejGo9Ov8fIDyXU7eY3kPMEZIPiFL6RcByP990eaYSTb6yyZ72e6I2WtMJ9qEm1ImnFeGamf0m7tT9J5lJ8xTxhSQvyYvpDmHnRuW9fYA9TP7VMcP5rOZ73 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Apr 18, 2025 at 5:45=E2=80=AFAM Roman Gushchin wrote: > > On Fri, Apr 18, 2025 at 02:22:12AM +0800, Kairui Song wrote: > > > > We currently have some workloads running with `nokmem` due to objcg > > performance issues. I know there are efforts to improve them, but so > > far it's still not painless to have. So I'm a bit worried about > > this... > > Do you mind sharing more details here? > > Thanks! Hi, Sorry for the late response, I was busy with another series and other works= . It's not hard to observe such slow down, for example a simple redis test can expose it: Without nokmem: redis-benchmark -h 127.0.0.1 -q -t set,get -n 80000 -c 1 SET: 16393.44 requests per second, p50=3D0.055 msec GET: 16956.34 requests per second, p50=3D0.055 msec With nokmem: redis-benchmark -h 127.0.0.1 -q -t set,get -n 80000 -c 1 SET: 17263.70 requests per second, p50=3D0.055 msec GET: 17410.23 requests per second, p50=3D0.055 msec And I'm testing with latest kernel: uname -a Linux localhost 6.15.0-rc2+ #1594 SMP PREEMPT_DYNAMIC Sun Apr 27 15:13:27 CST 2025 x86_64 GNU/Linux This is just an example. For redis, it can be a workaround by using things like redis pipeline, but not all workloads can be adjusted that flexibly. And the slowdown could be amplified in some cases.