From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39C26E7C4E9 for ; Wed, 4 Oct 2023 19:03:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F06ED8D0002; Wed, 4 Oct 2023 15:03:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D90758D00B0; Wed, 4 Oct 2023 15:03:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA4B48D00AE; Wed, 4 Oct 2023 15:03:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id AB6678D00AE for ; Wed, 4 Oct 2023 15:03:19 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 81176B4124 for ; Wed, 4 Oct 2023 19:03:19 +0000 (UTC) X-FDA: 81308702118.23.4DEB003 Received: from 66-220-144-179.mail-mxout.facebook.com (66-220-144-179.mail-mxout.facebook.com [66.220.144.179]) by imf14.hostedemail.com (Postfix) with ESMTP id A6D15100037 for ; Wed, 4 Oct 2023 19:03:17 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; spf=neutral (imf14.hostedemail.com: 66.220.144.179 is neither permitted nor denied by domain of shr@devkernel.io) smtp.mailfrom=shr@devkernel.io; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696446197; a=rsa-sha256; cv=none; b=0VpzNKXxGGBQk9oSJ0UokLQuT4G5pFF17H+yj4Npofu5TpUGWJmuFUKRY/J8V6cKA3Xq+3 nLI+ie3IRleNiiokM4VK5W8TkhgvF3/L4slAElADqlAMATaOoCXehrGPrWjG6D07R5vIyB X6UXK3rhalMwW+dW6UEbZGkoJ4eRV7Q= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; spf=neutral (imf14.hostedemail.com: 66.220.144.179 is neither permitted nor denied by domain of shr@devkernel.io) smtp.mailfrom=shr@devkernel.io; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696446197; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references; bh=UdhRP+56g7b5b4WlCDk1RDAsZ9u51QRKI0TldpcL+tA=; b=OM8ShoMY1Qe0F3pqKmVyhowUF5FEsqefJ/7CwYfhFhrSW74uR5ECXCvb3L9JonFvIcZyyi ImcIwTymavHtQVtSn/NV2gNj+K/y32y+9Dpi3XbBI697OeccuTGjtJ+Rqp9TjuitgvRZni eFHRTmjOTX5x+xcEY0kUBVljxcABVRo= Received: by devbig1114.prn1.facebook.com (Postfix, from userid 425415) id 5BCF4D08BFF9; Wed, 4 Oct 2023 12:03:02 -0700 (PDT) From: Stefan Roesch To: kernel-team@fb.com Cc: shr@devkernel.io, akpm@linux-foundation.org, david@redhat.com, hannes@cmpxchg.org, riel@surriel.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v1 0/4] mm/ksm: Add ksm advisor Date: Wed, 4 Oct 2023 12:02:45 -0700 Message-Id: <20231004190249.829015-1-shr@devkernel.io> X-Mailer: git-send-email 2.39.3 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: A6D15100037 X-Stat-Signature: gof7asg96uc9dqnaht7wssgfd9j8gmks X-HE-Tag: 1696446197-968136 X-HE-Meta: U2FsdGVkX187v3cmq+m56wilJuL00xdiJMJfzRJYt9RaHAX2bW5KcmtzQJ985SnY2q8sYCDVaWyCv74f2ntFKSiGKsVFq0tm3CH/EYemL/tD9Q9Vp5B1DmdNnDbjkaDjF89r3I1LtOqQv3wDCU8nJBa612cdNcT1oSLRUMMNfiKlROCw1TRxC8pZ6DwoaEtAuOiWJe8ur8buv2qMnhiY3nlXD/vRZsJpPB+Lm03vEPYq4OA+MfkR9SgEZcFBCfRT6v35vOtsy7LxeIoMG+J820Cg5Kj0x3b0ZGTzRSqJnzpe6mzRR3g1uxGc21FluHQS1atesZs6GEt4M4eMCJmgdGPLcdf5da3FnF0Hzlpee241o2h3aOGAQwcxAQ0e5Gu+RVTaknB/MGe3JPsBcsQvv8YjWg21y8z7484XMTYh+wFbFnutgw2Rg/VyV3M4WS74ec+73XJ5WZIndCy5YWijDxH/8y0DZdR47rJorWFRgE/fy2u/5y+jp4mTJqrjlOLCPw+lsxhw/oAH35c3GjSFbk4aQAXIwPBl2Xz8kV0iZ9AokBbB4hW0YlNjvYgpdMi08czMnQPsYNtf40vTiUFpYN/CwlluuyPtCQ3UPVCTmfxynOJzrhg/QHWq2yRV2+QAibtxR2CO1dwd6KvytYtLRmDAltADOr3xo6zUwnivNQEgeyfSCDBtP9YIFkJO0B1nwqcXjBjT4OgLyW0UqK1u6wFw6Jr045LXytMVHPcSLfMLvDm0KcOPHYCx/G6sLp8vCLwbor3GivgbFPfgwxNtOZP3+/JswBAYchFnFianB4T32Eilc76/SiHa/Of2WWalEfRaK8/GFR57gfRdGBr+lfAc4Dc/hZrS7JccFNI5spjq+xb30/spmzr5dAzcV1l2AmA6ap66fryibbbUVD+Wh5O/jK7oO422efJPPIzaQ8tgXqTSq+bf5VZguxBWziv4USVlbQu1mbVWPcNufE0 3DMKFgTi vO5Gypqz1MaJtCqxs2rpsZItoz6Vuj3+zN1bLVHz8/NWvVc98WyN4I4CXxFrnUp9RTWaDmNec7PLhAxu7m7D6f6WPXR6QVGQ0feUg X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: What is the KSM advisor? =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D The ksm advisor automatically manages the pages_to_scan setting to achieve a target scan time. The target scan time defines how many seconds it should take to scan all the candidate KSM pages. In other words the pages_to_scan rate is changed by the advisor to achieve the target scan time. Why do we need a KSM advisor? =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D The number of candidate pages for KSM is dynamic. It can often be observe= d that during the startup of an application more candidate pages need to be processed. Without an advisor the pages_to_scan parameter needs to be sized for the maximum number of candidate pages. With the scan time advisor the pages_to_scan parameter based can be changed based on demand. Algorithm =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D The algorithm calculates the change value based on the target scan time and the previous scan time. To avoid pertubations an exponentially weighted moving average is applied. The algorithm has a max and min value to: - guarantee responsiveness to changes - to avoid to spend too much CPU Parameters to influence the KSM scan advisor =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D The respective parameters are: - ksm_advisor_mode 0: None (default), 1: scan time advisor - ksm_advisor_target_scan_time how many seconds a scan should of all candidate pages take - ksm_advisor_min_pages minimum value for pages_to_scan per batch - ksm_advisor_max_pages maximum value for pages_to_scan per batch The parameters are exposed as knobs in /sys/kernel/mm/ksm. By default the scan time advisor is disabled. Currently there are two advisors: - none and - scan time. Resource savings =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Tests with various workloads have shown considerable CPU savings. Most of the workloads I have investigated have more candidate pages during startup, once the workload is stable in terms of memory, the number of candidate pages is reduced. Without the advisor, the pages_to_scan needs to be sized for the maximum number of candidate pages. So having this advisor definitely helps in reducing CPU consumption. For the instagram workload, the advisor achieves a 25% CPU reduction. Once the memory is stable, the pages_to_scan parameter gets reduced to about 40% of its max value. The new advisor works especially well if the smart scan feature is also enabled. How is defining a target scan time better? =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D For an administrator it is more logical to set a target scan time.. The administrator can determine how many pages are scanned on each scan. Therefore setting a target scan time makes more sense. In addition the administrator might have a good idea about the memory sizing of its respective workloads. The pages_to_scan parameter is per batch. But the administrator has no insight how many batches are needed per scan. Tracing =3D=3D=3D=3D=3D=3D=3D A new tracing event has been added for the scan time advisor. The new trace event is called ksm_advisor. It reports the scan time and the new pages_to_scan setting. Other approaches =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Approach 1: Adapt pages_to_scan after processing each batch. If KSM merges pages, increase the scan rate, if less KSM pages, reduce the the pages_to_scan rate. This doesn't work too well. While it increases the pages_to_scan for a short period, but generally it ends up with a too low pages_to_scan rate. Approach 2: Adapt pages_to_scan after each scan. The problem with that approach is that the calculated scan rate tends to be high. The more aggressive KSM scans, the more pages it can de-duplicate. There have been earlier attempts at an advisor: propose auto-run mode of ksm and its tests (https://marc.info/?l=3Dlinux-mm&m=3D166029880214485&w=3D2) Stefan Roesch (4): mm/ksm: add ksm advisor mm/ksm: add sysfs knobs for advisor mm/ksm: add tracepoint for ksm advisor mm/ksm: document ksm advisor and its sysfs knobs Documentation/admin-guide/mm/ksm.rst | 45 +++++ include/trace/events/ksm.h | 28 ++++ mm/ksm.c | 242 ++++++++++++++++++++++++++- 3 files changed, 314 insertions(+), 1 deletion(-) base-commit: 12d04a7bf0da67321229d2bc8b1a7074d65415a9 --=20 2.39.3