From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DDE74E7D245 for ; Tue, 26 Sep 2023 07:41:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3D2C78D0054; Tue, 26 Sep 2023 03:41:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 381B08D0005; Tue, 26 Sep 2023 03:41:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 248DE8D0054; Tue, 26 Sep 2023 03:41:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 07EA18D0005 for ; Tue, 26 Sep 2023 03:41:41 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id B5AD480857 for ; Tue, 26 Sep 2023 07:41:40 +0000 (UTC) X-FDA: 81277953960.18.1C2CB73 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf09.hostedemail.com (Postfix) with ESMTP id 820C814000F for ; Tue, 26 Sep 2023 07:41:38 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=jOxjFjjZ; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf09.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695714098; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=atusoG9wLQHzNwdybhnCWWKEaRPb6i3DYWEjDPOaKuc=; b=OB4P15JUklBr6eSX9vqehRBGL5hMRUEk73WWJtQTQzfrDIcO0+9NW3/xW/sapEJ/HdU2CO BmXY+FoenpiSn+QluM8FfW+v6M0f94ljCKO9+OfhKr6ClGwLoCTuXcUFdhe2UkEw+9sK42 C1JzY72AaA6R1Ozkv8GRkdeBihEejs8= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=jOxjFjjZ; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf09.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695714098; a=rsa-sha256; cv=none; b=FaVkpL7jC7963rFPiE2gg4x9iP5UnUV7PYTQApEt7ou2VT9Zl6UP3buk8EIN5Bh9HoOZ/M uDbDWzMFY1C00olzxFZo74wGOfJlDIOZrW0xCCIgYW2c3t1x0PCIbYo8Bh9AKxJsWwYmmm 9EUB26VLpKD0sX7Glk/T7utLQm1qNEg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695714097; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=atusoG9wLQHzNwdybhnCWWKEaRPb6i3DYWEjDPOaKuc=; b=jOxjFjjZpe02XAH6eoMjprTJGjkGQiuOJu3jfXfI0AkBxxxXeZhk9shwz6In/UgiI/IMev D0TggMu4NJJKJS/f/FemVwFCgA/IDlgwrrRIRs8yhC8P0zENDfB0DN7dgYG62c52+UE7Xz ChrRXq4ob/r4R2kYP/DKJhGh8TDE024= Received: from mail-ej1-f70.google.com (mail-ej1-f70.google.com [209.85.218.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-73-WtVDokOjNKCF7rm5xvWyAw-1; Tue, 26 Sep 2023 03:41:36 -0400 X-MC-Unique: WtVDokOjNKCF7rm5xvWyAw-1 Received: by mail-ej1-f70.google.com with SMTP id a640c23a62f3a-9adad8f306fso673752466b.0 for ; Tue, 26 Sep 2023 00:41:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695714095; x=1696318895; h=content-transfer-encoding:in-reply-to:organization:from :content-language:references:cc:to:subject:user-agent:mime-version :date:message-id:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=atusoG9wLQHzNwdybhnCWWKEaRPb6i3DYWEjDPOaKuc=; b=OKCOlaYNBai5VbpGXC+hDv0VUQcz6WNMrSYX7AEbkExGZQlzb9xMcMoPF5NPvTwxbv FwjCTIOFYe/6LLpgfkTaCrEW9nkFXA+yKKwHJVLdxmpmpNL7TNFL72Uusknv+AgeNpcS cdgThVCQR0Cvqzx2971bVP5v5RdPHPDgrrXR2Q3tVwGW1GalIScGTkHjSqek8jwEEWZC crnpQ0wvNeQPMgHrY9mxZBow8kLaMOKCpcdQKM3blhoIxtzFaMn5nxCeskyiQrVyQfCh lpTMbDFWUgn2nzqkovExCNtSwRVKNsfcOZJYH6KE58RiO1DJj6lgWXEaVuY8mWp5NFlM XhZg== X-Gm-Message-State: AOJu0Yx3EexsNg5vWQdshzudRdkTV7qZCXYsKS+Z/mmUFULDGgqFZVsG iia/f2lZfMcfXM1HsTZ+WVdD9jXhyYuCo7aYYBSckcGKbN459XpGB1sK0HI5SLdcu62+o/xIeUb To3CDQEByNkY= X-Received: by 2002:a17:907:7e84:b0:9ae:621b:af03 with SMTP id qb4-20020a1709077e8400b009ae621baf03mr10616497ejc.39.1695714095232; Tue, 26 Sep 2023 00:41:35 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGZltpF2NY+qzQbeBmRGD2q15t7/K/DfBoa93SjpUV40Z+HXpuF36wm4hWjCH3lMzLPvhAg4g== X-Received: by 2002:a17:907:7e84:b0:9ae:621b:af03 with SMTP id qb4-20020a1709077e8400b009ae621baf03mr10616468ejc.39.1695714094803; Tue, 26 Sep 2023 00:41:34 -0700 (PDT) Received: from ?IPV6:2003:cb:c73f:600:933b:ca69:5a80:230d? (p200300cbc73f0600933bca695a80230d.dip0.t-ipconnect.de. [2003:cb:c73f:600:933b:ca69:5a80:230d]) by smtp.gmail.com with ESMTPSA id rh27-20020a17090720fb00b009930c80b87csm7493458ejb.142.2023.09.26.00.41.33 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 26 Sep 2023 00:41:34 -0700 (PDT) Message-ID: Date: Tue, 26 Sep 2023 09:41:33 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: [PATCH v3 1/4] mm/ksm: add "smart" page scanning mode To: Stefan Roesch , kernel-team@fb.com Cc: akpm@linux-foundation.org, hannes@cmpxchg.org, riel@surriel.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20230926040939.516161-1-shr@devkernel.io> <20230926040939.516161-2-shr@devkernel.io> From: David Hildenbrand Organization: Red Hat In-Reply-To: <20230926040939.516161-2-shr@devkernel.io> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspam-User: X-Stat-Signature: wcnryzknrgu3gyj9uo35yp6ya7xzmip3 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 820C814000F X-HE-Tag: 1695714098-755831 X-HE-Meta: U2FsdGVkX19EwDmtf/JKN23M8PQ/vQINUBf3V0r1+tMtTk534BsjY9nRQr2pezQ2mXQrPsrio+wCvZ4gpMkCAsiMhvbO7sK9sGZJUc5E7e+Cm76uqw7m59/1U0js9VaRD9A7FUOmoWO16TTzt2buxJYtyyFfVvqi7JaFbDUB3XOvaxitetB/8IouyKkX4oPDnEeh6nn6lVpUbu7SXcjjJ5kXyXHq9wS0Qvk6atneM6s2JqoSh4kpYsK+Xid4uxJCyCleHrnBXLiD4N09iK0irGX79MitREUYkNLw/JFjL/p7AbLKQlcBtAGv8RC/xfrph+zhFO6tbJPkoLXKQRjuLD1b8XTDtuGzmO/G2uJTIlE1oVQH/woV9iEU55Y7qKuHU3R6nZCRJHznnAS6qoRNygUsJcyrnP6d8qZIBUp6lI7AnJL5CwndHbxYh2FWE+vkr7fnxhZd+znWyFj2zhQUFFtro1f1/IpdsZYBee8ywtrw/j9A+SaO3VBdYMEiRql/H61s52zVMO76ipDaYnLxZ8zeqmj2diTDPQtSqhY76eQbCzSd0UYM+ORtWWsB7n1chMN+//JSkm5U0KYIN/Rj17U2GwDkVIYbP3j9ALmLUeqNM3WRyd/oSi3z4zTtE0XP1ms5EE/xG4uPRT18+0K/DRICj2vi101GBilq9PzXpioy0FfKqnyy4ug4wLUm+JVjZaBkg1kD/L8qmY3yKZpdR9KlSuxyQNn63yax5JAEUnvxQZdpvDKgmUfAFvJz6AJjSLsyrHs0p5RFuZFX03d+V6jcfYnr8ZPpzwfOPoEEmeeVzdFZZrLcCtPHOr8p4QDb0wZYNaJ87F8YXqFoYmpZB5SsHa9WGL7gjRzW7mejxJ52ynNom7wUkK3m+H7K4YYbxQC3ls5ZV7+n9+SY5Dxb7lqHRdaj8S077TbeDIMDevwtyfBrtbuOfEL+Hj4YBGtSDMRYTcJyx+c2uJKdZLG 2Z7zTLxs cb4iS/1Y1jrq3M3VOkUsDl350ZUIufyyuGrVH/tVnyIMfGsaagljfFiLC5NRwg993KP1UMSPdqAS6J0zkAz7HxHRZvOI8vlFpRVpTCNqPjeeU38y7uFQt7Om5OnmGhdB0LhXB26MqSmBnFAEDCUsA5x1zDDvJTPBmNuYZ+2L7w9NE/B2Nd8aJNlQiYgodg37gFJ6KgjKcdUSYnohVxCcq0PL3e6ESh3bdS5xQ4XKtLHy/YbAyW2fp8Fg1lLDUCeobaS4zXN+HF795lkUUcpMSnJs+8LUVEBn0u7RkJdAl51FPzNRyNI0sy7PsflBbGB2kwIge05520eSV1JjACoRTx6EZrBRlwi573r/lj1tgFkjkvhTAndi6FA9SxTC1UjNOlybC X-Bogosity: Ham, tests=bogofilter, spamicity=0.020163, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 26.09.23 06:09, Stefan Roesch wrote: > This change adds a "smart" page scanning mode for KSM. So far all the > candidate pages are continuously scanned to find candidates for > de-duplication. There are a considerably number of pages that cannot be > de-duplicated. This is costly in terms of CPU. By using smart scanning > considerable CPU savings can be achieved. > > This change takes the history of scanning pages into account and skips > the page scanning of certain pages for a while if de-deduplication for > this page has not been successful in the past. > > To do this it introduces two new fields in the ksm_rmap_item structure: > age and remaining_skips. age, is the KSM age and remaining_skips > determines how often scanning of this page is skipped. The age field is > incremented each time the page is scanned and the page cannot be de- > duplicated. age updated is capped at U8_MAX. > > How often a page is skipped is dependent how often de-duplication has > been tried so far and the number of skips is currently limited to 8. > This value has shown to be effective with different workloads. > > The feature is currently disable by default and can be enabled with the > new smart_scan knob. > > The feature has shown to be very effective: upt to 25% of the page scans > can be eliminated; the pages_to_scan rate can be reduced by 40 - 50% and > a similar de-duplication rate can be maintained. Thinking about it, what are the cons of just enabling this always and not exposing new toggles? Alternatively, we could make this a compile-time option. In general, LGTM, just curious if we really have to make this configurable. -- Cheers, David / dhildenb