From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20737C7EE2A for ; Tue, 24 Jun 2025 08:35:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AB0DC6B00B2; Tue, 24 Jun 2025 04:35:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A88636B00B3; Tue, 24 Jun 2025 04:35:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C5576B00B5; Tue, 24 Jun 2025 04:35:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 87B086B00B2 for ; Tue, 24 Jun 2025 04:35:43 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 386ECC0126 for ; Tue, 24 Jun 2025 08:35:43 +0000 (UTC) X-FDA: 83589635766.28.3D92940 Received: from mailgw.kylinos.cn (mailgw.kylinos.cn [124.126.103.232]) by imf04.hostedemail.com (Postfix) with ESMTP id A0D0D4000A for ; Tue, 24 Jun 2025 08:35:39 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; spf=pass (imf04.hostedemail.com: domain of xialonglong@kylinos.cn designates 124.126.103.232 as permitted sender) smtp.mailfrom=xialonglong@kylinos.cn ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1750754141; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NRbB6c/OkouVl9vhSSOj93Pk3j2HERgbyA9DrFKLlgQ=; b=1KJTNKMXwTGcVV6hu/RZNsLCnPzyqqesU8bAtgh45lqJMA3IgvpgxddVJBNC6MSbqCzjtj XI6swM9FCC/eGClT70z9wSsm40h61AzL8UNGnkySNpAvDR+ci/EHvbPXNt9YH3RRWD1t3T Qe5iaCv3dZINBvkB8Z9RewlJiOYmdrs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1750754141; a=rsa-sha256; cv=none; b=VL2bCoNbfbjPAgDnPUI194QZHjEe7Z2qLuXM+EhRalHGvjIG13rhlrfhWPmHmFVzkDeg/T tPGq37TM7NBsgCigYxUbJKt5FfBwaPCEC9a/T+1mF1SMstS2Yjn6cD31wBcXYiTN5x7oLU wymjNoKSk2sMrQ/+8OXGRq7nj4w1jEs= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; spf=pass (imf04.hostedemail.com: domain of xialonglong@kylinos.cn designates 124.126.103.232 as permitted sender) smtp.mailfrom=xialonglong@kylinos.cn; dmarc=none X-UUID: 3122758250d611f0b29709d653e92f7d-20250624 X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.45,REQID:2399aed0-3dbb-4733-8ad9-882089a3a0b0,IP:0,U RL:0,TC:0,Content:0,EDM:0,RT:0,SF:0,FILE:0,BULK:0,RULE:Release_Ham,ACTION: release,TS:0 X-CID-META: VersionHash:6493067,CLOUDID:e4d7ca572d61b1a74d34c9c0b6b80160,BulkI D:nil,BulkQuantity:0,Recheck:0,SF:80|81|82|83|102,TC:nil,Content:0|52,EDM: -3,IP:nil,URL:0,File:nil,RT:nil,Bulk:nil,QS:nil,BEC:nil,COL:0,OSI:0,OSA:0, AV:0,LES:1,SPR:NO,DKR:0,DKP:0,BRR:0,BRE:0,ARC:0 X-CID-BVR: 0,NGT X-CID-BAS: 0,NGT,0,_ X-CID-FACTOR: TF_CID_SPAM_SNR X-UUID: 3122758250d611f0b29709d653e92f7d-20250624 Received: from node4.com.cn [(10.44.16.170)] by mailgw.kylinos.cn (envelope-from ) (Generic MTA) with ESMTP id 917961575; Tue, 24 Jun 2025 16:35:32 +0800 Received: from node4.com.cn (localhost [127.0.0.1]) by node4.com.cn (NSMail) with SMTP id 5B686160038C1; Tue, 24 Jun 2025 16:35:32 +0800 (CST) X-ns-mid: postfix-685A6354-1975593342 Received: from [172.25.120.23] (unknown [172.25.120.23]) by node4.com.cn (NSMail) with ESMTPA id 9885616001A03; Tue, 24 Jun 2025 08:35:28 +0000 (UTC) Message-ID: <910a462a-8d4d-4b5d-941c-ba1396e287dc@kylinos.cn> Date: Tue, 24 Jun 2025 16:35:27 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 1/1] mm/ksm: add ksm_pages_sharing for each process to calculate profit more accurately To: xu.xin16@zte.com.cn, david@redhat.com Cc: akpm@linux-foundation.org, chengming.zhou@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, shr@devkernel.io, corbet@lwn.net, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, vbabka@suse.cz References: <202506181714096412Nvp5B3BkFpi3-CKLQ9ep@zte.com.cn> From: Longlong Xia In-Reply-To: <202506181714096412Nvp5B3BkFpi3-CKLQ9ep@zte.com.cn> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: A0D0D4000A X-Stat-Signature: 7yubz73tf1sk6ns3fxsrqyj649fekma9 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1750754139-895858 X-HE-Meta: U2FsdGVkX1/W4FmJmB/MlpEGtCpdHmb2vks0t2v90wYwhjhkMoYVY+rptxmIZDvGSApgdeRK0GI6hYeCyATYrym2ePA2XVcgIENSb3gDJtNQH2mPF03LBxRHSE8UylPB+GpYO+TYDjmM7a+eLmQi5tU4JS8eyeDMlFkqyGDfB+RdrGKf4kWrN5QZu/pLnhj5yetpK0YrqZCm5OumgfzVFJlsVLTfwXcOgq8ukqdIZtYpLEjXC09AtDDlgtIYQuZF+BkKHaK0ufHwIXJyUxScNuJ1Zg7I/PHJllVxiROvioN7zSxslGA7Ada7FqYhDIyxMV37ehNTlLYSa9X/69pQjuk9MUpslxC+0hglvQIwmNdvAinCohx5AMM4rKMdevwE7T4VvmSElrZjoL+PBf0u/Y1RttxCQm5786Gob/uUvZRo/KkzMNfnX2aY5O0ecXu9ZXnGuMtn9OZCaU1mXvojQmoiefDWExz4vsdxAbUDcxzc45jEp59GSCCURo/kPnOi58nGJ8fWd1Vz3ECnmqeAn45c/5leA2NCN0YECAefqFGYtY/d791M6uFclwDl1U1hxDtCKwW+0DppJPC+bbjpkchrfE+n/dATeFwX0VS/I18MJ8QBjFguETNraTSupQ9vc8sCgk1JhG6o/y2SP5q3tdrwLHN3ReQEd6n2HH2+wA8YOKSQ5o45ZDyJR/1ttxo2zuwT8U1p4z2ZTfg85avYLxH96XO3DTfSnMSFnGMB9jBH+Z7qvel0MfUQ/cq3wxjn+Db1TXHpvJla7dmQX5zNj0E+sNF+El8s+9hnXhVKaL9B4D+ET8rQQzkRDt0PEnViUtL1SCcIB/1++GsTvM7MufIj5Cntn4sdxj7ueEt9K7EkhoaK4eO7O5eVQKjc1gn2bEu5PHTp1d35TqjzGVV2sf79CZMT8UlKWP5w8FamfkGLAVISLHZN9PVCQBhRq7Tk13GtiAND+TRO2AAk8iu 4eaZU0Y8 jgZVKe1eWiY8kwMDNW2/7B3UIzFyms1pTt8cGeyK4vSNu8oLqVCqcezdJDGlVFgXRFGb65CHPp2LIoxllJr8At8hLp8Lz3ckhkfriS/oD3OVnGj3ZjqiG+atpsEL/g1/AEfl6Ng1SF4ptyJpjyCE8QCy/wJK3Dqs5bydmAlhfVscsUzUKW4w1NxvWbDb7i/bKHvZPecJe2wDTEWdly8bkQ17/vTe91iuzdvkyFLiYuNsgkkY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: =E5=9C=A8 2025/6/18 17:14, xu.xin16@zte.com.cn =E5=86=99=E9=81=93: >>>> and /proc/self/ksm_stat/ to indicate the saved pages of this process= . >>>> (not including ksm_zero_pages) >>> Curious, why is updating ksm_process_profit() insufficient and we als= o >>> have to expose ksm_pages_sharing? >>> >> Since ksm_process_profit() uses ksm_merging_pages(pages_sharing + >> pages_shared) to calculate the profit for individual processes, >> >> while general_profit uses pages_sharing for profit calculation, this c= an >> lead to the total profit calculated for each process being greater tha= n >> that of general_profit. >> >> Additionally, exposing ksm_pages_sharing under /proc/self/ksm_stat/ ma= y >> be sufficient. >> > Hi, > > Althorugh it's true, however, this patch maybe not okay. It can only en= sure > that the sum of each process's profit roughly equals the system's gener= al_profit > , but gives totally wrong profit result for some one process. For examp= le, when > two pages from two different processes are merged, one process's page_s= hared > increments by +1, while the other's pages_sharing increments by +1, whi= ch > resulting in different calculated profits for the two processes, even t= hough > their actual profits are identical. If in more extreme cases, this coul= d even > render a process's profit entirely unreadable. > > Lastly, do we really need each process=E2=80=99s profit sum to perfectl= y match the general > profit, or we just want a rough estimate of the process=E2=80=99s profi= t from KSM ? > Hi, In extreme cases, stable nodes may be distributed quite unevenly, which=20 is due to stable nodes not being per mm, of course. There are also situations where there are 1000 pairs of pages, with the=20 pages within each pair being identical, while each pair is different=20 from all other pages. This results in the number of page_sharing and page_shared being the=20 same. This way, using ksm_merging_pages(page_sharing + page_shared)=20 averages a 50% error. In practical testing, we may only need to enable KSM for specific=20 applications and calculate the total benefits of these processes. Since page_shared is also included in the statistics, this may lead to=20 the calculated benefits being higher than the actual ones. In practical testing, the error may reach 20%. For example, in one test,=20 the total benefits of all processes were estimated to be around 528MB, while the profit calculated through general_profit was only around 428MB. The theoretical error may be around 50%. If we expose the ksm_pages_sharing for each process, we can not only=20 calculate the actual benefits but also determine how many ksm_pages_shared there are by the difference=20 between ksm_merging_pages and ksm_pages_sharing of each process. >> >>> Hm, I am wondering if that works. Stable nodes are not per MM, so >>> can't we create an accounting imbalance for one MM somehow? >>> >>> (did not look into all the details, just something that came to mind) >>> >> Indeed, using the method in this patch to calculate ksm_pages_sharing >> for each process to determine ksm_pages_shared >> >> can sometimes result in negative values for ksm_pages_shared. >> >> example for calculate mm->ksm_pages_shared=EF=BC=9A >> >> if (rmap_item->hlist.next) { >> ksm_pages_sharing--; >> rmap_item->mm->ksm_pages_sharing--; >> >> } else { >> ksm_pages_shared--; >> rmap_item->mm->ksm_pages_shared--; // can be negative >> } >> >> rmap_item->mm->ksm_merging_pages--; >> >> >> Would it be possible to compare the ratio of each process's rmap_item = to >> the total rmap_item and the ratio of the process's page_shared to the >> total page_shared >> >> to assess this imbalance? For now, I don't have any better ideas. > Although stable_node is not per-mm, if you really add ksm_shared to mm, > it won't cause negative ksm_pages_shared, because the count of ksm_shar= ed > will only be attributed to the process of the first rmap_item. Yes, it was the incorrect method I used during testing that led to the=20 negative values. After the improvement, it has not occurred again. Thank you for your time. Best regards, Longlong Xia