From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BB48E95A91 for ; Mon, 9 Oct 2023 09:48:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A75C68D0040; Mon, 9 Oct 2023 05:48:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A25AE8D0031; Mon, 9 Oct 2023 05:48:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 914418D0040; Mon, 9 Oct 2023 05:48:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 824E28D0031 for ; Mon, 9 Oct 2023 05:48:34 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 5BD741A019E for ; Mon, 9 Oct 2023 09:48:34 +0000 (UTC) X-FDA: 81325448148.27.FF9C7E6 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf21.hostedemail.com (Postfix) with ESMTP id 170631C0003 for ; Mon, 9 Oct 2023 09:48:31 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=AQ5w0qJz; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf21.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696844912; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Jf3TvYWdWhL977BhygUU/Yo5mqDW4sxxg/0A8rYNaRM=; b=V9wyhYXaIHjlt1bLk1hRY9270Va+rZNPNCJsCjl9s4bRAaMSELagusu0AXLhhtPTn7psnx TJ4C//xtk+vKDOb3lGWL1atnTOWbs291gjYzgT4wC5TXD0QKs+3ssopTaThDvzcaRjOV+2 FIkMfDDUbSWQ+7t22tOEkA1HG/qsJzY= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=AQ5w0qJz; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf21.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696844912; a=rsa-sha256; cv=none; b=SHF5uRzJP9BOXjcpa9c+JWC2JcdaUA+S7l8tQyGVzI43ViHyMXcHFCcf5v2GIGyafKl/q5 5ImFprUrM32ZRZ38znUPJTOdrxL4hyiPwq4jSeQDEuIAIhlylZW/NSjYE5QhfOn06KX1V3 kFerCD1jkJtAvjYUaU7O0zLFviZy7mk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696844911; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Jf3TvYWdWhL977BhygUU/Yo5mqDW4sxxg/0A8rYNaRM=; b=AQ5w0qJz9+5yisJmXved0vDzqihTsfNROQPh5pOJ6I+QqFYRZjeX/YBReCNngShXucHncJ g8NIu1fd32LJzMOBpkpAhfJBXiuoS0uJ9a7HbhpTy45cO7cj4LKhQZqvCx8fOZ4K16PN5i 9R6mfqz+7qeegavJl4yVAli/b9GF0Oo= Received: from mail-lf1-f72.google.com (mail-lf1-f72.google.com [209.85.167.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-438-acglDbkxMqGY3oDEG8W5Lw-1; Mon, 09 Oct 2023 05:48:30 -0400 X-MC-Unique: acglDbkxMqGY3oDEG8W5Lw-1 Received: by mail-lf1-f72.google.com with SMTP id 2adb3069b0e04-5043eb2c436so3678701e87.3 for ; Mon, 09 Oct 2023 02:48:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696844908; x=1697449708; h=content-transfer-encoding:in-reply-to:subject:organization:from :references:cc:to:content-language:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Jf3TvYWdWhL977BhygUU/Yo5mqDW4sxxg/0A8rYNaRM=; b=kU0wwKAk2nqpe+iJ0Dp/TO8wjVyULV6Az6OXx5UqZo6tuRKmM59ffjymcCW9wSKtrc kz/bVNzlD/wJG2maoObSxFzyp74OpDO7Cm0BDiqLiP6q9Nkkgc+Lkawqs85cFYxKehNc xMGrTF6nBJZ+xJ5oLWzv0JhShbuG70YjzCiuOxPAQuGCGjnqpZO3I2HaQB+KYzbNc/Ki PgasxSQLmzdObHvBcUH5n5LzhPjSVdBGscvzZiz+Q4JP6g+ZfEfeoBGfVUnqNMYpTSIV BzRg1Mlcyg5dBMcKceZiQ2pM/bWOFB4V0ivnRFJrCpHDwv0ZpZCQskrXdG9zJDwo1mKn iw/g== X-Gm-Message-State: AOJu0YxVtp+wc0QH5k1Rmw9J3Tr6xEiFig5adQboQlOXmbeGa0Pqe0n6 N4Qm+CTPU+BmBeUwTdSK6Gg3XCgCLNw1lLvym/tjO4sQUdzu7BiAPwp4Ran0Bk32T1xnc1i02p6 aUFXgdV9I+kY= X-Received: by 2002:a05:6512:ea5:b0:4f8:77db:1d9e with SMTP id bi37-20020a0565120ea500b004f877db1d9emr18579048lfb.12.1696844908481; Mon, 09 Oct 2023 02:48:28 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEfSbe4VgAbM9GSYCr0y7y4QoQnRmdJHr78+uQDitdeZXQT9PczzBKrIbievTmuvvOGZ3G03Q== X-Received: by 2002:a05:6512:ea5:b0:4f8:77db:1d9e with SMTP id bi37-20020a0565120ea500b004f877db1d9emr18579003lfb.12.1696844907431; Mon, 09 Oct 2023 02:48:27 -0700 (PDT) Received: from ?IPV6:2003:cb:c733:6400:ae10:4bb7:9712:8548? (p200300cbc7336400ae104bb797128548.dip0.t-ipconnect.de. [2003:cb:c733:6400:ae10:4bb7:9712:8548]) by smtp.gmail.com with ESMTPSA id 6-20020a05600c22c600b0040303a9965asm13011530wmg.40.2023.10.09.02.48.26 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 09 Oct 2023 02:48:27 -0700 (PDT) Message-ID: Date: Mon, 9 Oct 2023 11:48:25 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.15.1 To: Stefan Roesch Cc: kernel-team@fb.com, akpm@linux-foundation.org, hannes@cmpxchg.org, riel@surriel.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20231004190249.829015-1-shr@devkernel.io> <4509a3b4-16a6-f63e-1dd5-e20c7eadf87d@redhat.com> <87fs2nhg14.fsf@devkernel.io> From: David Hildenbrand Organization: Red Hat Subject: Re: [PATCH v1 0/4] mm/ksm: Add ksm advisor In-Reply-To: <87fs2nhg14.fsf@devkernel.io> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 170631C0003 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: nbta1caag1kkgpoo1uub8omsa1axo1y7 X-HE-Tag: 1696844911-192673 X-HE-Meta: U2FsdGVkX1/TfhmT7U2mRkLdslDM3z1OqhSxdILlPm0pS+c/KpGQI848m4nyY/KJQrfShgrVXcDnNO8Q2xTAM8OeX5ADuJ8xxBHf1a19TdSjumJ4D0TBOcN4FkVumNLtmOekXWEQJw9HksRn1WsztdxpRDgIJ7Dvc4rU2blr0YXkaD0XrN4rXtQkEu5R6/2PGTlSmOzWPFryHYwZzRzDSX4oqEBogiGQ1Oflr7/2lhnjZKeV402Sc5e2sPmXLw2Dn2NKtibunSLJKnT81Vbvyc3vpmDRro34pOVIOnpYOw5UBb87DScZ+XpZWToxCkUQAGP9woeavkMyGq1ZBkKgqMn5VPq2IFGfjE+E2eK6zhJbxiecMwMB4jhOyPnQW3pTOuXu5zuIH+O2dVe5+PquuKfo/vWPC3k+qqZIX6/ejlcz6Sq+bpjmd+4KPPDExM94eHxkZ7ZiPsKcbuJJpndHGZ1enRInTHcBi3+jeey+d0ekGEMqMPGNOJzEW86dpdehUopO6pDvt4A8dU7iRYyZlJ3N2isCS17l1qFDrbd8bBrzt7cM5yNIIGjVWHIvcnEuktFDG6NxBjHH2g8chOnnZwMaAKsowUpqRsgRRchLKdGrXEGCyJyqm1xlCclrhb3FB8rVgZsIqToy2nY0QVD7llejRbXjd+sDCmmD/q0g+7K+G1XoNlPi6Yp8eJTKMSXle0LdJ5QDYh9tn+Bb6xCuYipGDBnL67uRkB9DHNkU3nvlwjeBybLICTL8pJZW0Zmzj3taBt9QWUN9RrAWw8jHWr785z4HuzEM2BGfPaIo8Dp74YbEvSnCmNl1WKSL4geWmumdA/56O0GOPbTH6sWAxVTzQAXTYpDI95+OrrRIkOLC5imvNKci1sMlrxlOaw3HsX7/laPMsVN2bk2rP43LTlx0arUzx7rz1pncvZg8l5aggGfrYr/dz7mOS6YjLLi6PdUYYcB/UJlSWuNWRR6 oYgqCPDd lLkI17y/AQQCJC6ldeFfNxPH391gsfuYKLUomfmf0PPJnUKfrMQWbOOIcAo42zqzyMfAIf4rTlHuJn4by+id/b54jDd9+23akl1ClOzEdCJDViWw7omrgNDzT4bPHd5bnPQi9oN1U9BNMWQeAz8nz6cRvQMIkIsic8eKQM7STo0xc3RaA8mDNNluV+ybZwr/UsEY/3FDeRWIcn8Xf8NAEFothULy1reW3+y2YBVqGl9fg/lrzlgUjZTibPiDHPTtJXf0AfiKp6NR7w3/dQua/nEEDSANujYwn8wqHUduvULcuXdBZQQtW+RVVifHc3MUD+i3CCCUUkIxRW7Q8YWjDsm0D2/lj2jFcsmwZNyFT3i4XrDbBWOp3TRO77ydRauZYyxDLV47bw2DQXKUPafxdOKEupA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 06.10.23 18:17, Stefan Roesch wrote: > > David Hildenbrand writes: > >> On 04.10.23 21:02, Stefan Roesch wrote: >>> What is the KSM advisor? >>> ========================= >>> The ksm advisor automatically manages the pages_to_scan setting to >>> achieve a target scan time. The target scan time defines how many seconds >>> it should take to scan all the candidate KSM pages. In other words the >>> pages_to_scan rate is changed by the advisor to achieve the target scan >>> time. >>> Why do we need a KSM advisor? >>> ============================== >>> The number of candidate pages for KSM is dynamic. It can often be observed >>> that during the startup of an application more candidate pages need to be >>> processed. Without an advisor the pages_to_scan parameter needs to be >>> sized for the maximum number of candidate pages. With the scan time >>> advisor the pages_to_scan parameter based can be changed based on demand. >>> Algorithm >>> ========== >>> The algorithm calculates the change value based on the target scan time >>> and the previous scan time. To avoid pertubations an exponentially >>> weighted moving average is applied. >>> The algorithm has a max and min >>> value to: >>> - guarantee responsiveness to changes >>> - to avoid to spend too much CPU >>> Parameters to influence the KSM scan advisor >>> ============================================= >>> The respective parameters are: >>> - ksm_advisor_mode >>> 0: None (default), 1: scan time advisor >>> - ksm_advisor_target_scan_time >>> how many seconds a scan should of all candidate pages take >>> - ksm_advisor_min_pages >>> minimum value for pages_to_scan per batch >>> - ksm_advisor_max_pages >>> maximum value for pages_to_scan per batch >>> The parameters are exposed as knobs in /sys/kernel/mm/ksm. >>> By default the scan time advisor is disabled. >> >> What would be the main reason to not have this enabled as default? >> > There might be already exisiting users which directly set pages_to_scan > and tuned the KSM settings accordingly, as the default setting of 100 for > pages_to_scan is too low for typical workloads. Good point. > >> IIUC, it is kind-of an auto-tuning of pages_to_scan. Would "auto-tuning" >> describe it better than "advisor" ? >> >> [...] >> > > I'm fine with auto-tune. I was also thinking about that name, but I > chose advisor, its a bit less strong and it needs input from the user. > I'm not a native speaker, but "adviser" to me implies that no action is taken, only advises are given :) But again, no native speaker. >>> How is defining a target scan time better? >>> =========================================== >>> For an administrator it is more logical to set a target scan time.. The >>> administrator can determine how many pages are scanned on each scan. >>> Therefore setting a target scan time makes more sense. >>> In addition the administrator might have a good idea about the >>> memory sizing of its respective workloads. >> >> Is there any way you could imagine where we could have this just do something >> reasonable without any user input? IOW, true auto-tuning? >> > > True auto-tuning might be difficult as users might want to be able to > choose how aggressive KSM is. Some might want it to be as aggressive as > possible to get the maximum de-duplication rate. Others might want a > more balanced approach that takes CPU-consumption into consideration. > > I guess it depends if you are memory-bound, cpu-bound or both. Agreed, more below. > >> I read above: >>> - guarantee responsiveness to changes >>> - to avoid to spend too much CPU >> >> whereby both things are accountable/measurable to use that as the input for >> auto-tuning? >> > I'm not sure a true auto-tuning can be achieved. I think we need > some input from the user > - How much resources to consume > - How fast memory changes or how stable memory is > (this we might be able to detect) Setting the pages_to_scan is a bit mystical. Setting upper/lower pages_to_scan bounds is similarly mystical, and highly workload dependent. So I agree that a better abstraction to automatically tune the scanning is reasonable. I wonder if we can let the user give better inputs that are less workload dependent. For example, do we need min/max values for pages_to_scan, or can we replace it by something better to the auto-tuning algorithm? IMHO "target scan time" goes into the right direction, but it can still be fairly workload dependent. Maybe a "max CPU consumption" or sth. like that would similarly help to limit CPU waste, and it could be fairly workload dependent. -- Cheers, David / dhildenb