From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4C38C5321D for ; Mon, 26 Aug 2024 19:46:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6A7036B0083; Mon, 26 Aug 2024 15:46:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 657FA6B0085; Mon, 26 Aug 2024 15:46:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4F6A66B0088; Mon, 26 Aug 2024 15:46:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 2D1D26B0083 for ; Mon, 26 Aug 2024 15:46:34 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id E1F1E813A9 for ; Mon, 26 Aug 2024 19:46:33 +0000 (UTC) X-FDA: 82495428666.19.EBB3A85 Received: from mail-qv1-f45.google.com (mail-qv1-f45.google.com [209.85.219.45]) by imf29.hostedemail.com (Postfix) with ESMTP id 1B7E1120015 for ; Mon, 26 Aug 2024 19:46:31 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=XWTC4uTr; spf=pass (imf29.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.219.45 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724701548; a=rsa-sha256; cv=none; b=oXpYgJjEBNAE4B68w/hyR2RvTRXgjFMfmW6Kpwxjmo+/QddEgDC/usMXWR4vRVBdJ1DuYM qfakF8ocijSZg4j1qm4pNHj2dE1L5OTimpFCkXXDZn6FtQpFtnWY55DNjJmfnS054z7EcJ tzjlfcxm9Sci6hZTHPcAUy7sb2Bx9kw= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=XWTC4uTr; spf=pass (imf29.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.219.45 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724701548; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xkcwhp5wUP0PARWUzg25EzXMSiZptOh2qK3AVZk7sM4=; b=w3spJ/6PO6qq/q7alKPF7Bjb7/2sPZsdPu9TfsyuBFZjnMozB3zgbbUTKrbriSQNb2eYRA fNDgBAfp9W4uPfSmjFXnScRVPsghnhnAYwKKFzPfxNekjm/ErScM0Puz7PYBPT/bO3UCwX sn4U33sMBAjsyc5HD1E/eCzQJAMSmgc= Received: by mail-qv1-f45.google.com with SMTP id 6a1803df08f44-6bf8a662467so23146036d6.3 for ; Mon, 26 Aug 2024 12:46:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724701591; x=1725306391; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=xkcwhp5wUP0PARWUzg25EzXMSiZptOh2qK3AVZk7sM4=; b=XWTC4uTrSB3CaYJEezsASQjKqIn89WCd0xar0Dq8R4yq7XAz4/XmNP0GbY0PbMgjPn LpA5w0PmnG1u0NKPtFhY8ouKESEjuIBORjga8Ut3hvz9FzNpu211awpJNMcXZup4kH9l SJNe2MHxRh4W6djsb6wLH7bOI6bMT5IJH0LuUavDv+t2stwf34dInRyyQC6vbP7EPWkv PR1l5UzEtycRsxsfrFHa0ABc9DzQbSgNKp4chTSKty9YHGC5+kwYo3N9IA/KHjqUxeU/ miTD0tnQbjbhzO59J1v+DA1bHdtrQFfBBTO87A2Kueh5Ins0ETgfOI6pWtkCVXFBjmcH uEKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724701591; x=1725306391; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xkcwhp5wUP0PARWUzg25EzXMSiZptOh2qK3AVZk7sM4=; b=ftd0wgs152eLog/a8X3TB9q4aV8oO/mKOUHqE31j8lhiehos3J+j1bARBlZ6QkInCk 3aQRL+oUHgj7M6y+Qjr88H36dhXjnMEmV0I8fPJpVBJbjc5Gz/zLbbow/phSbOU24nQQ N7dphvgtunSEhEZgv2LGbp5jkP6/QoKa/EEqWB80V85c79D5d3j7ctj+q5+DhdMMRKTA ucvl0uOQ58PJ03jZadojO9VsP7wRX2cDRR1OK4lYolcodrMayViIqPpmphDb7W8CFeQg 7EgA3Ou+KHfUluCC3srtD0yKwAKFJXY5oJoCQnJIMOx+1cb5e6fRjF8U1RqwBj8p+mE2 bFUQ== X-Forwarded-Encrypted: i=1; AJvYcCXB39E56Sf028SWCNmq8ayakS0zfHRDPi4wiwJomyAfL1Z1XUJoCRp5+28qbh2RjeDGIReFa94/rA==@kvack.org X-Gm-Message-State: AOJu0YwUycvJyVdKrxwRByNtxLdALaWc1mp9Vl2nwwVzbD6hBiiskIt8 01C6qO/acbzRYnPRgIIuHT4XMuTKjI8JZmA8BInfJgNEM1xOe5Hlxka2htVLp1IEV8qo2HD5Ts8 tlMLBXMby4EV5Tns093rbUslfm2c= X-Google-Smtp-Source: AGHT+IEzelaGzWbS77Np0l8c2438Rxo5NWk0QoxlbS0DBGhDscOGwbnwXNj8sR5VP+DAMuyPNrBeK4US8tc9JoLZrN0= X-Received: by 2002:a05:6214:2b84:b0:6bf:6dc2:e002 with SMTP id 6a1803df08f44-6c32b677f0bmr6678776d6.12.1724701590962; Mon, 26 Aug 2024 12:46:30 -0700 (PDT) MIME-Version: 1.0 References: <20240821074541.516249-1-hanchuanhua@oppo.com> <20240821074541.516249-3-hanchuanhua@oppo.com> In-Reply-To: From: Barry Song <21cnbao@gmail.com> Date: Tue, 27 Aug 2024 07:46:19 +1200 Message-ID: Subject: Re: [PATCH v7 2/2] mm: support large folios swap-in for sync io devices To: Shakeel Butt Cc: hanchuanhua@oppo.com, akpm@linux-foundation.org, linux-mm@kvack.org, baolin.wang@linux.alibaba.com, chrisl@kernel.org, david@redhat.com, hannes@cmpxchg.org, hughd@google.com, kaleshsingh@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, mhocko@suse.com, minchan@kernel.org, nphamcs@gmail.com, ryan.roberts@arm.com, senozhatsky@chromium.org, shy828301@gmail.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, xiang@kernel.org, ying.huang@intel.com, yosryahmed@google.com, hch@infradead.org, ryncsn@gmail.com, Tangquan Zheng Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: pszbgfjsqxx64txbqjngt3bnzwxp188s X-Rspamd-Queue-Id: 1B7E1120015 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1724701591-239147 X-HE-Meta: U2FsdGVkX1/tvQnIW5NDBRzdJeSRCwVs1tsKULEMRuk19c4vewd2W1xjU15tGBm/USzXfewSvqyD876A9l80au89swO5+/AUuIjvRw+4liveFTXgburaXaYicSv0Xo2Pnkpk4YCdjKiXLvimnvBoUsc0K1WL2lhHlY7K3tIGb0SjivPETJdKYkxbJaqTKPyVhDjdWo7Piv5z/CW3a8cGpIArKsH0qedbIDZNYhhtpYXc3pSdN5U+XCpiEGmqBmgwtYtEY1qd7BdNfA4UKgixgzjbHzk+L+qdmabL5borR0n9L6Ycu9hkHGAVhaBfqvOhJsAGDhTIkaxR14AVnrz7T2Pek99tVdVXuaY+r/jevFvEnVBm0G+xxjNqERBE+6hJNuydjVTQR4QCTmwj7q08f1mozULaJZpqbOIwzRBHUaeF3FNAq7Re6MU67+5Alh0SUH63lfQS5+hXFwt5mGuPh54Ty+s7NQGu6uUXKl3fSvDp4kJoj6v7q8FPn/iirZxsTkfmXumUqfQsqhaHxcgXEA+ugHIDpxiU6p2ywSmIQwbpVfTt6PIOWTODwX9VbLPy7ZuNpszbQDnf7ERX2+/sPhGs39Iqdvs5tCPAUjlTbokJp9P8Cpr4z2WgSwjfaYd+PC0MyyK9hS66EN9d1shZa1Zevfbmyy6SF7XLo4HO6ugv0iboqBCA+UPE/CGFVoDIRRkKMb0JbkYFmv7RMcjqsA2ZOGNLjGIXUqT62+jIthezsylu27qo99K/G5Nt0SLabGsIwDCBOGwqnU6TyL7cOGscBYBy4n0iOuGwjbT5+TPzdZAi/F1gD12j9WKtwaSGfz6CJACTlufYwAO9nE2PLRICh3mDWexxRa+Vwglyf7E3/g7LGmPdgnkFN47woGFP0KOFbVj2r8WrN9/RYr6qN8eOkgDZUrhZz3T7XL+fEhZlhS/cSqTX2JeHh3t+Uy6dgbBQo906Gfz9MYs4cnn NmPlComz U18+IKKENv/jzAdJsc+xsIpCjzpLYqVxm2KQI40ci/gOJIbjY7gzz/hbEjGYlsdAoh2dHZaFhqdxRpvL6d0ffifG3M0OM6CXsG8UMjRQJhTlBkbkefp/4zsEboQRDJ1wHpK8Tw8NDHzSp41p8K+9EtGecCMCHy4He7A3Uv2ArX5+x9FbL6hBXkfCNWurTasazrP114vNs8o/EOXlqoKR2qikzv8EopJt/Iutjif7hzZGrm1vC9IhqpwSPH8lDou6vZvcywqLqK7sc5yNT1WVTFkNs03bGBoX5rpIx9WUzTcYSkEWck9AJe3dbPzNXz5Z1XG1UrWbhqPCr5uy1fKoKAPwT1Fn/TrZ024AcEPEaExBATUDs42aKJliwpjouhfma7sBRnJlhsdBUWcu10EnhboFnSLfBi30gcUdw X-Bogosity: Ham, tests=bogofilter, spamicity=0.000971, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sat, Aug 24, 2024 at 5:56=E2=80=AFAM Shakeel Butt wrote: > > Hi Barry, > > On Thu, Aug 22, 2024 at 05:13:06AM GMT, Barry Song wrote: > > On Thu, Aug 22, 2024 at 1:31=E2=80=AFAM Shakeel Butt wrote: > > > > > > On Wed, Aug 21, 2024 at 03:45:40PM GMT, hanchuanhua@oppo.com wrote: > > > > From: Chuanhua Han > > > > > > > > > > > > 3. With both mTHP swap-out and swap-in supported, we offer the opti= on to enable > > > > zsmalloc compression/decompression with larger granularity[2]. T= he upcoming > > > > optimization in zsmalloc will significantly increase swap speed = and improve > > > > compression efficiency. Tested by running 100 iterations of swap= ping 100MiB > > > > of anon memory, the swap speed improved dramatically: > > > > time consumption of swapin(ms) time consumption o= f swapout(ms) > > > > lz4 4k 45274 90540 > > > > lz4 64k 22942 55667 > > > > zstdn 4k 85035 186585 > > > > zstdn 64k 46558 118533 > > > > > > Are the above number with the patch series at [2] or without? Also ca= n > > > you explain your experiment setup or how can someone reproduce these? > > > > Hi Shakeel, > > > > The data was recorded after applying both this patch (swap-in mTHP) and > > patch [2] (compressing/decompressing mTHP instead of page). However, > > without the swap-in series, patch [2] becomes useless because: > > > > If we have a large object, such as 16 pages in zsmalloc: > > do_swap_page will happen 16 times: > > 1. decompress the whole large object and copy one page; > > 2. decompress the whole large object and copy one page; > > 3. decompress the whole large object and copy one page; > > .... > > 16. decompress the whole large object and copy one page; > > > > So, patchset [2] will actually degrade performance rather than > > enhance it if we don't have this swap-in series. This swap-in > > series is a prerequisite for the zsmalloc/zram series. > > Thanks for the explanation. > > > > > We reproduced the data through the following simple steps: > > 1. Collected anonymous pages from a running phone and saved them to a f= ile. > > 2. Used a small program to open and read the file into a mapped anonymo= us > > memory. > > 3. Do the belows in the small program: > > swapout_start_time > > madv_pageout() > > swapout_end_time > > > > swapin_start_time > > read_data() > > swapin_end_time > > > > We calculate the throughput of swapout and swapin using the difference = between > > end_time and start_time. Additionally, we record the memory usage of zr= am after > > the swapout is complete. > > > > Please correct me if I am wrong but you are saying in your experiment, > 100 MiB took 90540 ms to compress/swapout and 45274 ms to > decompress/swapin if backed by 4k pages but took 55667 ms and 22942 ms > if backed by 64k pages. Basically the table shows total time to compress > or decomress 100 MiB of memory, right? Hi Shakeel, Tangquan(CC'd) collected the data and double-checked the case to confirm the answer to your question. We have three cases: 1. no mTHP swap-in, no zsmalloc/zram multi-pages compression/decompression 2. have mTHP swap-in, no zsmalloc/zram multi-pages compression/decompressio= n 3. have mTHP swap-in, have zsmalloc/zram multi-pages compression/decompress= ion The data was 1 vs 3. To provide more precise data that covers each change, Tangquan tested 1 vs. 2 and 2 vs. 3 yesterday using LZ4 (the hardware might differ from the previous test, but the data shows the same trend) per my request. 1. no mTHP swapin, no zsmalloc/zram patch swapin_ms. 30336 swapout_ms. 65651 2. have mTHP swapin, no zsmalloc/zram patch swapin_ms. 27161 swapout_ms. 61135 3. have mTHP swapin, have zsmalloc/zram patch swapin_ms. 13683 swapout_ms. 43305 The test pseudocode is as follows: addr=3Dmmap(100M) read_anon_data_from_file_to addr(); for(i=3D0;i<100;i++) { swapout_start_time; madv_pageout(); swapout_end_time; swapin_start_time; read_addr_to_swapin(); swapin_end_time; } So, while we saw some improvement from 1 to 2, the significant gains come from using large blocks for compression and decompression. This mTHP swap-in series ensures that mTHPs aren't lost after the first swa= p-in, so the following 99 iterations continue to involve THP swap-out and mTHP swap-in. The improvement from 1 to 2 is due to this mTHP swap-in series, while the improvement from 2 to 3 comes from the zsmalloc/zram patchset [2] you mentioned. [2] https://lore.kernel.org/all/20240327214816.31191-1-21cnbao@gmail.com/ > > > > > Thanks Barry