From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.4 required=3.0 tests=DKIM_ADSP_ALL,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,LOTS_OF_MONEY, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,NORMAL_HTTP_TO_IP,NUMERIC_HTTP_ADDR, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,T_MONEY_PERCENT,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8EDFFC433E0 for ; Mon, 6 Jul 2020 11:58:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D1D6120720 for ; Mon, 6 Jul 2020 11:58:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="fMjT3u/R" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D1D6120720 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 77FED6B0007; Mon, 6 Jul 2020 07:58:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 758236B0025; Mon, 6 Jul 2020 07:58:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 50E576B0026; Mon, 6 Jul 2020 07:58:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0112.hostedemail.com [216.40.44.112]) by kanga.kvack.org (Postfix) with ESMTP id E4ED66B0007 for ; Mon, 6 Jul 2020 07:58:00 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 9DACD181AEF07 for ; Mon, 6 Jul 2020 11:58:00 +0000 (UTC) X-FDA: 77007502320.26.pipe23_520668c26eab Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin26.hostedemail.com (Postfix) with ESMTP id 71EC81804B660 for ; Mon, 6 Jul 2020 11:58:00 +0000 (UTC) X-HE-Tag: pipe23_520668c26eab X-Filterd-Recvd-Size: 190886 Received: from smtp-fw-6002.amazon.com (smtp-fw-6002.amazon.com [52.95.49.90]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Mon, 6 Jul 2020 11:57:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1594036680; x=1625572680; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ko0vpgy02JEFekxbxwTtiofkc4jbkaq/7gzJ34iGgAE=; b=fMjT3u/RsMc/ePKTtw9Wut3I8GZ0mC3gx6J6jjaOnW/dFISKUaLM3Q1e drBRbcXLWBniWpiILiqMm32dixEXbp/LabYn3Qdsf89fTumjaV6Yhr3lC eCDEZUx2/hJnS6hXT0DgOZbrv/9r24oT8+Eo6FC7bt6Ui8Ce6RLXfSFTo k=; IronPort-SDR: 9Hk813sbLvigrNVVqsCBajQQydwaAaU1y3AKa8p/0BpMkjwkikXr/6Ququq25kKF08lLUX8rFv gQpIV80b9trw== X-IronPort-AV: E=Sophos;i="5.75,318,1589241600"; d="scan'208";a="40227611" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-2a-d0be17ee.us-west-2.amazon.com) ([10.43.8.6]) by smtp-border-fw-out-6002.iad6.amazon.com with ESMTP; 06 Jul 2020 11:57:53 +0000 Received: from EX13MTAUEA002.ant.amazon.com (pdx4-ws-svc-p6-lb7-vlan3.pdx.amazon.com [10.170.41.166]) by email-inbound-relay-2a-d0be17ee.us-west-2.amazon.com (Postfix) with ESMTPS id AA633A1E01; Mon, 6 Jul 2020 11:57:50 +0000 (UTC) Received: from EX13D31EUA004.ant.amazon.com (10.43.165.161) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 6 Jul 2020 11:57:49 +0000 Received: from u886c93fd17d25d.ant.amazon.com (10.43.160.100) by EX13D31EUA004.ant.amazon.com (10.43.165.161) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 6 Jul 2020 11:57:31 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v17 12/15] Documentation/admin-guide/mm: Add a document for DAMON Date: Mon, 6 Jul 2020 13:53:19 +0200 Message-ID: <20200706115322.29598-13-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200706115322.29598-1-sjpark@amazon.com> References: <20200706115322.29598-1-sjpark@amazon.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Originating-IP: [10.43.160.100] X-ClientProxiedBy: EX13D21UWA003.ant.amazon.com (10.43.160.184) To EX13D31EUA004.ant.amazon.com (10.43.165.161) X-Rspamd-Queue-Id: 71EC81804B660 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit adds a document for DAMON under `Documentation/admin-guide/mm/damon/`. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/api.rst | 20 ++ .../admin-guide/mm/damon/damon_heatmap.png | Bin 0 -> 8366 bytes .../admin-guide/mm/damon/damon_wss_change.png | Bin 0 -> 7211 bytes .../admin-guide/mm/damon/damon_wss_dist.png | Bin 0 -> 6173 bytes Documentation/admin-guide/mm/damon/eval.rst | 222 +++++++++++++ Documentation/admin-guide/mm/damon/faq.rst | 59 ++++ .../admin-guide/mm/damon/freqmine_heatmap.png | Bin 0 -> 9999 bytes .../admin-guide/mm/damon/freqmine_wss_sz.png | Bin 0 -> 5589 bytes .../mm/damon/freqmine_wss_time.png | Bin 0 -> 6550 bytes Documentation/admin-guide/mm/damon/guide.rst | 194 ++++++++++++ Documentation/admin-guide/mm/damon/index.rst | 35 +++ .../admin-guide/mm/damon/mechanisms.rst | 159 ++++++++++ Documentation/admin-guide/mm/damon/plans.rst | 29 ++ Documentation/admin-guide/mm/damon/start.rst | 117 +++++++ .../mm/damon/streamcluster_heatmap.png | Bin 0 -> 42210 bytes .../mm/damon/streamcluster_wss_sz.png | Bin 0 -> 6327 bytes .../mm/damon/streamcluster_wss_time.png | Bin 0 -> 8830 bytes Documentation/admin-guide/mm/damon/usage.rst | 296 ++++++++++++++++++ Documentation/admin-guide/mm/index.rst | 1 + 19 files changed, 1132 insertions(+) create mode 100644 Documentation/admin-guide/mm/damon/api.rst create mode 100644 Documentation/admin-guide/mm/damon/damon_heatmap.png create mode 100644 Documentation/admin-guide/mm/damon/damon_wss_change.p= ng create mode 100644 Documentation/admin-guide/mm/damon/damon_wss_dist.png create mode 100644 Documentation/admin-guide/mm/damon/eval.rst create mode 100644 Documentation/admin-guide/mm/damon/faq.rst create mode 100644 Documentation/admin-guide/mm/damon/freqmine_heatmap.p= ng create mode 100644 Documentation/admin-guide/mm/damon/freqmine_wss_sz.pn= g create mode 100644 Documentation/admin-guide/mm/damon/freqmine_wss_time.= png create mode 100644 Documentation/admin-guide/mm/damon/guide.rst create mode 100644 Documentation/admin-guide/mm/damon/index.rst create mode 100644 Documentation/admin-guide/mm/damon/mechanisms.rst create mode 100644 Documentation/admin-guide/mm/damon/plans.rst create mode 100644 Documentation/admin-guide/mm/damon/start.rst create mode 100644 Documentation/admin-guide/mm/damon/streamcluster_heat= map.png create mode 100644 Documentation/admin-guide/mm/damon/streamcluster_wss_= sz.png create mode 100644 Documentation/admin-guide/mm/damon/streamcluster_wss_= time.png create mode 100644 Documentation/admin-guide/mm/damon/usage.rst diff --git a/Documentation/admin-guide/mm/damon/api.rst b/Documentation/a= dmin-guide/mm/damon/api.rst new file mode 100644 index 000000000000..649409828eab --- /dev/null +++ b/Documentation/admin-guide/mm/damon/api.rst @@ -0,0 +1,20 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +API Reference +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Kernel space programs can use every feature of DAMON using below APIs. = All you +need to do is including ``damon.h``, which is located in ``include/linux= /`` of +the source tree. + +Structures +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +.. kernel-doc:: include/linux/damon.h + + +Functions +=3D=3D=3D=3D=3D=3D=3D=3D=3D + +.. kernel-doc:: mm/damon.c diff --git a/Documentation/admin-guide/mm/damon/damon_heatmap.png b/Docum= entation/admin-guide/mm/damon/damon_heatmap.png new file mode 100644 index 0000000000000000000000000000000000000000..ff485f9f59ca2717f5221b282= 4eaaed616b22c0e GIT binary patch literal 8366 zcmZu$2|QHa+rRfV!ysEip^VCs>|`m+$Wmj^E^B2;mZWTDxsghevSl~P8nW-(s3>cp zvW01}Wt+&p%zH<_|Nr;CZ_Q`SxaXYbe4p)n&+`ywpm&;)o{Js;07m>7bwdDvU;qF~ zm|e&S#4Pd&0CoWeI>s6l3I!P<5D3&SA^^Y?B0vG=3D=3D7>1p?OQ0}wR89{CJ96n;o7+-&DLeE)tAL=3D^z&yxTPm zP*tP=3DUEP$u(Uc$qG7M8RfOXh&D((h({6T%COVu5QrKQgV7)!|n=3D7Q&9fZk!k@FN49 ziHkVA7l8Nfg>lFmhV6*39R;T9kE0Nfw;hGxK_L<;h&>cyXJsmxLMBvnHc^^7iADYt z9D#yuXlOvJ+Y8Vi0QMdR4hjK6qJWeXpePGyDggRQfCUb?pa!^T0@v|CfDUj+4|sSM zcwz*knF3knz>9NWxiwg43%)uJwq69^Ie;IXz(Hql)CK(R22EapX0AeHPiXNvwCoM7 z`tI8B+qHRf*H*x;?LZU-Mp15|C?V+WQ1sRv^u|4!^)Q;1aN4B@vyV^gNCE*;AIJbk^u+toJka1ZT1NW$*RK zVRy>quzt>IoWD=3D2V87}MF1bQ((ITG1#RvA69%L@#MVDiNibFu=3DP%Qw|2m*CN zK)o=3D~a13Y^1&CrmlLYWm5_lyIygm*z%K|O(z?&1mTScH%8E8`h+EsxLHK0=3Di=3DsFF& z!vpVifNnjY#{lRx1U?u8AI*SI7C@gB&~F0_*a3qVfgwlW^Ce)|1sHJ$Mm&JgYXHd` z_~HkA^#{HM0pEjxu~1AvFaRImvp45&Ywvi7vXOo`F^83y<0oM?h>G;A(blNnD<8NV~&G^ zQ&EYIMsod|{bu-v(QI`s73`TDg{Z6B_I?Apfk$8Y#n74CL?qEp)j87~#J?^H)P+V+a(jvkjyXJAUa zeRPZ|Xa#!XadpBFjwYy=3DHM7-Xccsk3=3D-q9j!>cupmYX93F@2>TR zOb4Gt{&pV*?i!N7zI{@?l}2us0%M&W&h52FaxFD11k(Gr%4qp?JGu=3Db#imxZtoD#8 zu9tZ`82Dl=3D&h%MJh;PWX8diE~TzjJbDxMY3QyKVRo+u`btW=3DDKg9So5`?l7@HdN0% zP8nD_+1axy;3b$}b~ntgQjvA|_J3A}!_Fh($D}{mchM$g4kOb$|ZUTr`iMs_`LnEoi$?mt{!oo)(G$g|fK?6OZwOls5wR+&Ib$I~Yb{ooRRj^DBE+CJ<&v-!)-16Y4n*i*grM?2 z5E}dcKuB}@13~{!gwy{I0Wmz}Ps2z4Fg(cV{~G>}4V8Z)aQ}%=3D{s)5oAMWsV{Rd!A zBJ3=3DJDr}Hx(8}NQl(_Hwmj zl%J~b*O@*;=3Da@57*&V`ktIdLGn@cOpS|{kiTrleDRO_DNh3|@OD3SVCorY~%FHQE4 z=3DeoN;GA~hAgsFf3rZab*`K(iy?zr=3D(cTwgK!RY*NV{C6+L2d~X>mv9H&)mq#l_uw=3D<4}XY|Mh$o`=3DI$rk_TmS5?30O z&U@V|`w^Y*)N-ocO@9wx58pZ`j=3DSq?BVT?dHL8fZ36NaG111*6$Gf@^4=3Dp9FmQ@1=3DW(uodD;9XE3*T&xez|O+9G`-c06gFBF!nL=3D$2}` zYJBshGNd)wky>MCVgFWr`~F9?oNBCf9V{N`IB5D)m=3DtLfP!;6i>FMw5;qT#}!Fgga z`dePF;W*S#7gXfq>*?v~>yd7hnrzSOA>;MrFef>7xW0bt*;akM(w6;7(?}{)(zxf% zm#1^83*58ojm*b}A6zJT)sD57nvcGBLHRw`Z5dT|i#qtq$-U%jN#cz|w#4bvr|&AE zKPxH!I7V998bCh~7A2+Xx;ItKQ^+8x07(imG{CQWe%-dCgj?RJ5DSX46S^O+wQUjU?~G2wui zY>78_WQSOam{5wlp3kJC6cbg6L*Mq04`3!~p3l5QD=3DJtLD_i(UN~6$7`7M}*1?plz zL{H&t4IKDLWVC-D4f24I7e}ys<6t>&J_2`62@6>SoS)y|AiHs6Fd!ePZpZQ@M0$O% z2QyWm>N+^zkHaCe6QRNkARj>-k@8rD8XW$kXg)0%vU?9PHO!+&u??FnQ*CYuV%xb3IYN>?)}b)FsTr})jmM`cw|Y_GG^Z5^ ziD?sKB+uVC-?wrgYe(=3D|N95wh3!|MNZ01Hx4@+nAIN1?)=3D9)2QxexuJ+!BUW zimwXWy^M;En{KRi4t$!W=3D=3DDpKe{;($_Q*|x zZ$q?-cVK=3D^VM2_~9tbjp;da(X+8j;~3K{Oe4jcj`+kq-2p7yc3Pj*x_7xWHN-b2l0 zmZFFDBHjTagj+~D+!ZH9y72^Q$7wZvzu*J*n%_(OxZ~lZ(8sP%Z9|J>49nS+c6=3DuL zG%9{dR$FvtQ76phz)6#eT3BxM4jhQu)aHI;rb#Mx-;NpKw)za4uQWP$Iz1Ob?N-Aj z1bA`x(suYA$=3D!bBPg}HfPsDFs_o`Z4bFEcTYoKkL`j&{G76w#fkzl_nM$kEaS@YZF zm>X^5P2?lBEh2*Ro-HO(^D9Rm_vr7ZZm5dudghuWe)%&YOG~l!g=3DG8X^9we~dQx#8 zvl4z=3D(=3DO}XSA_#(GcM@NzI6kuou?O~!=3D{E^-?x{h_jDJqNce&Y4<^!-9v|vXxqK*V zzppUqi1xMP?VIWXH~Oq>TSWdT9&K@~i!uo-20i|jqA9W;#L8ifxcgUABd<*j=3Dg%G! zpp2*8y8}0%HW`v+!7%$SeB$Ty+&8;E9m)P(P9`b80>D>n64M*&hf5>1YP^~ zEvJIHRgjsxLIexbN+XMgwpvC_kHG~cyv(V$>Mceq{4S7T+zD&L8jt&UiSLhc4raHVBC?>2Z}bX|D;^T)U8Ci5 zGPHN{-qI_TX~3VoL9H!O2$$T!+AVwSGjhyt{SngHEZy4mK~s10EirrlkoB>9j^8Frz!b;a7VGKRX6cOGMnSCXW@gnfu~e;57?f&Rj< z8o*s?rupawO5}BLWNYP9>fxAVKq7VfCHFMaJ$L@4}MJ9 z+Rp719ueOMy?6i%zf>eJqO9WJcv@F`FZ5MJgSKT& zidfWcGaNDjflv6c;~+OOTsT-YqGZ%|m4;?VZw{;YV=3Dg*+20myi3lmQy_omLiCAyn4f@T`HtV!BM*aS<8`hC8C(61ig&V~d;@D^>HH%dDM z5ava=3D=3DZNv{zHL<;re7=3DZMB1G@@H1BOkgPi`UsvJJq=3D}0#^?)wY5qWmB@bVSOsJ6z= ? z@KK4%jk~f}0xc#ThQ*?O?f&oOfJNmS)R=3DJAkm&P{k>Vmwf|;6_bGuzw7D$YkDtbtB z{7%9z>|K6RlPIR@5aeBbe(BKfm{d7`TTXg3<5G3q{O(0nVl1l!hoZ?*&Q~Wy*>)US z`{i=3DJmE#^&VmGGh{I~V84&%yO?xpwsi%z5uNkLA*2+FF`qABbWriu&FjKAl;!=3Dc4G z>b5r-AwBJC39!#qf|U_`#qJ@(*4<}Q&kSZuoCDuTu?Eb>-emEhXJ8|vxpmalh>y@m zKDW*hj(C71_lX_ z!1cakd)Ot%MDEfd^o872X_Z$6vEjM17uzhDvMu%_I_F~(9vwxw0+5&n@yuDljt{{q zvJs&<)?VS%q$Z#l)i@L}pPq4@8&b zNmF9lmPayVXjb<&vUlrBBJYF=3D4q?{w7@{~LYiJT&#o%1ILWZjF>7T#LjQ-@qI>;#dX{eaizetFV_cdjxiM>th$5s{8aL2C zARjHhm7&{~oYb*cyW2AJY3&FO#|&++MNV*#Es{7uqpV8`r+JfOCXhFs-m(B`_AFh0cq^gOBj3n5r{o&l>Q+SO zPJvy}8Bq{XzEm)~RIrq?l-=3DmPz1cUJX0flUJ2-Hog<;y#tfI0)?)s?|cbAJd!(UXC zmb%5+est-%Y?~P~(dEEB2&`*_rRW z>rh-oou>ewAN| z!8Cli)r!FnYJ+sd?4+s_Pi6NQXkZB!L2>29OBZ``Ax$Mq;+c3!oZ^JcZl^+ZuPTK=3Dj^Bs6xf?3KGm4|sOFjbd8O26ZNYcipbb!%<;?I|J- z#q;ZUIv7}s8N_H#|SUUSFCPEF&v~N}?)aqdF??NshlPdc)IljqYKCD~z82vtt_L?H7 z8Kl8%`7f5vJ`RP$ zImmgkOjk^Ek^`!>{m&rYSViyPC=3Dg!p)P2W*nzx87-`x8PZ-zKs9dQL7ezUtIjhVw?{Y=3Dy9%R zKWbAci3MF}!bTnrgKKZH6z#{zGw;1?1kRhF{TEx*i6?9S8KY)kz(3j7i*#BCkmp+J zGs}BU#Iw}N`vV0LJet+4AjSJEcf0$3N>sABUXgpHk)tO`<=3Dr+ zZY@U%2C=3Dq?$0ShT#r3!R!%lH{m9V5f+ju(> zKSwP}ujfiw3}t(Q4)y+swfll$^{-y;cRVN3_bnm~GSv@Fn0PPWhPR$hhbZMw87io$ z%SV67G{-$#Q15)KkN%gk>HyVh!SsGMw|%$G5gsD3>@?tPvgK~iJ5H_t2o|jVr}^F| zJ$G;V*(h{h%?FI+BQ)bs#yVNsG<@0B+@l$Dw_}$j@V)EmNRIF1aN|WImzkcb&Dg$@ zqn`R57;|T~()scD=3DK<_ycTIy~K8(EH|4B5Y$Nc|^h7>#K-v@RS=3D{5h7SV{jzT!^8{ zlATOL=3Der?*az#U3yM7Dm_y4ggwa{h!Yew=3D`{uM>??~9RM{Vyt2F``^)k-Efv=3DS1%4 z|L>_e{`Z|K{<;(2Ul%uC-USX(8*lC%;`mSQhy3e_|CRf^e?O1lUw5AW(|T$x|4%gS z{|X!Mci6wX#^$fEjhBo#A2A558qOv>&Vmeiuyz;-Y5rasPH-({6~n*N`cOB+{*x;? zauMi^9qwJdr)z@KaAV+|dwBjDNIj~+P8rm%Tvfdz0>c))>35FV?Mxzsz8$<17cpz1 zq)(Gwj?SoFtf)edDh4UAmku3Vkm-2OfEmA#%+`(N(Ofg(0d`(n_=3DJH*V@WpY$ZMNN z4%miG7JjlzNu|YQ4ggKUMN8gNE-$X2g`X>42bfOpB6s{=3D&Ss z$!OcdSohJ0j6*Nz<^7%}Ht?N@`YCf%FqJQtbaej7het&v@NjPn^3N1yeC(t9;WqrT zep!aU@FyNoGG;=3DHJ^2VO-yw^J8Cq1PFO&(~=3DDtx^(F7NPsY4^`F zQ-Lf#0tBY3XBX@6BAM@eMrJK~Zs!K#mwe%s_K$UVs4La<^_tCe2R`wZZFhpMsX#O`lYsHq^K{VS{(>6wzpeB4aFS z?WlF$b-Z}EkCQ%E!fM_@d@f@vYUYXCprEA&{gM_}toPy#cHfK+d5k1?d4IS2pql#y zsSt^4h7KHiDGAYS{^vxejJ^uB)u$;;lcjVZo_;Q?jfD*LC(tH#!8L&*~Ep zPfoCTU*=3DwzA*rk0u8gPq5$vRw=3DkvYCl||}iMXp}^27k&7w46V4Y3i4WRdSHNy4(F8 zRxulS>43Hyf)Ozt4fm%X$4EfSXOdb5=3DmSbR8}Cm?%M9I`v9|b~qks8)ANtjpxuu|; z^TQeSS1wNveqxFX)#^2VJ>Dj2rE6Wh_QUTgi&-fU$Z9Qe*@lkO3$++somp$w#C8q3 z)1e=3DKM$1XhK5L2{70#ZL+jOC32AznL5YLA0tMiMDYe{K5u{BUsDHdtWh_kr$BA=3DLH zc-`+&mXaFa_|=3D#w-?)z^MN@m?JqM?Kd!J+o)BcHU?>tO^ggKB6agFIDIia){r)hJ9 zf-21uHTC5}2b{~F+MGv=3DJeCF9pHy|$d<}!2_p2E1p6Cl!aCQTyJbBhmeB_Xj60A;Y zVHU;4Cpm)8w1#9Y?xKtbGMD6+!6xVus|2f6X0wd8_tV@{L}TVhlJqnT=3D^D61+8AL=3D z!8BTt8oP&nxFRdEE{06sz(5n&)GhH}ugQl{Xi>Q3^R3}_KGZ?HhMxM1Q+C1s1NaT3 A^#A|> literal 0 HcmV?d00001 diff --git a/Documentation/admin-guide/mm/damon/damon_wss_change.png b/Do= cumentation/admin-guide/mm/damon/damon_wss_change.png new file mode 100644 index 0000000000000000000000000000000000000000..36699547c342cd05c9f416c78= bc3fc86b9dc415e GIT binary patch literal 7211 zcmeHMcQ~8xyH6sBhKf-!T7uG|_G&3&SJkKzYDR}mDX|*$+FFzj+N#;sru192N7VRI zwMSKJS8Za|mh(n`<6PG{=3DfCsUxlXR+eV=3DhZ&;5-1bFVxwHq>Q0#(fL|fiUS^(lUWS zU;qRH9Y@fBEimSDgJ4C*_=3D>4Eg+c*aGBPr#FA@X-P)HC8WOG9GQb;5UxE~6sqdbjFA!n3z zG*B8kNF_uHHiLqxtE=3DPVP-PAv2DHWMikk6s5sXS{m+)z%jIjic@??lV^-OZ?#*JcKJ;CPsYY7|J%jk#O>|jo zTpG*e3@F^7`xDoNOJziq^yiraf!p;-ToV) z`ho*%Au;Z$Z6j{1Qp|y=3Dmn1z`2T-We#)Sczm7FtA0iB@@+0E)69iN7bQzkfoNL#Ihc<=3DxQlieMwTZurVu~j(>4zhFk-0z6QLGhJ`8tTJ6tIE(7sCH$EVXHU zYJSv0k`v}jx($yN%Pt;_3jbhWXAdB;U?V?hXMg&kHtx!M-8hKW`;9B>MEQHW2|^%cg-|TgY)l zzEM_`czHNHIcZchrVob(^o0Nnc^dZ|Zy3GW{dOA9yIIA)*nd-Ge0_(#>rDdhXO`{6 z+!M8k*)XfonX!0vEStFjv)Y@;t1aHudM# zi~|Cl0{~hqRtbZL9yNn^b1q0%sVMt(sq6>LI5)!Ig#&I)12yUh1za)XdDl^%Oz?~H zNZ7j&Ajd(SB#$EZgY(kC(voSRU$LZX;GpEc{C=3D9ALE)$@?5UIgFBeJiv_GsQD|it@ zjzjUZpoGQ5o#-_1Vpz@r)R$@kJ69Bf3zhO0wd?X(*qbWGE>00o> zH9IOjZ!AvXHL{MtnF!do@<(!XRobV#;+-kGld|8awK39|3%?SUu;JB3)pSGev`{5o zQUXx--Tvpd*<9)fJ+yb01x@&vpnFJVivh}#7rPWP>DE)16lNk{xZCoSg&cfQHnxVA zQS{M2)n2&-C_l1=3DnSjCzVFOQt$kg2pE@Q{BN#b}ZL6v2sgvz2MdA_WLtGfPMj)cla z)m74J!FeuSl0s=3Dv-(a)3mtB$*9d(L|sv=3Dw)CX3=3D#LW?5f?6U@nBA&Ncy1qs4uD2VV z?KuWsfQ3Eb9%Z-N;gCyR4gwn$Ei9>;R*UDR0goJab;KF=3DjU^a%QB^`RP(QFp*w?EXBzk-{~|ps@TKtP=3D{b55J$z9RtVYsFDTF5Nin1bxG>7iMpUp#| z?amekLiRzt*A37qhQ)VhL-j)d+I~ij@V}Vbs0>zlK%5O$;ZDmuX?WH9!CR z6C;?a#Od*MFEgkW?{#g0@(Ck{OjR@##%ll)d@lITuoMcU{WgUo{LdG435pdj`yaPU zL7K)r*#ZFizTJy|ID8GwPl9 zwcKW`8rQ(!8BXNfl)tUy0CJVKu^Lwcv~ucWc7nD=3DdO<`qm0?zp_v$=3Dy^5rtV#I zIX0saVUc&>y+(z3QLnET-%IBOVlAHMR@t$1=3D$(^;o~X$muFV_7-h62utEY*L7mDVR z_5v?_WTNT}U-o@gPq%$*w}nWng`A>*?uEM^RbP4FHQt<$0o-}<-UqT??q4U$>YQDz z-9--Nc5csx-r>~|&bT|+9Hp+U80+0Pc;wG8oq4T4`oyDPuFC!vtVPGDrM6n{JrmR_ z3Dh*4gTAQ=3D0qA|sYOBm&R%jdYFI55gEbb7XHo^=3D?Xi(xWR&Ui;$ZLVN?O|?t^Gyp` z!l3sUU8@aS^-1S{Un+2oEc0+%j&y$E<+Mh4fWwk_8@b=3DH-3W7*F+6CFzpvx;l#ahg zvB<${hOK8T(TNtmC|x+A#IH)2^SVB<{bb?~@?V^Hw;`U`GW#>ZyAOs4_@d0B(gFxB ziN=3D3a3-hbu`0G`K;ESgV%`Afm-nIIw@R|zXQ;vvJS;1`$QUUTsmrW#)WvvXt*z%Ml zW5`h%4O5LZp@SveT`EaOa27}Rp-K{x229xprymb6aMrg)01;RctLSoq(*A2(aRw@W z*IMrrT^})lXeyD2FWl%vD2!x^_Xzu=3D5&m+8?Sf*pim~@bGK&nz5-75h#dFR6@dXbk?1lLTbIs>$@Hgl-yzWWY(l`T6Z(W{m&aT+xS@ag&0sC<=3DlJpbsFLVpiJ_(i^^;ZOG8jCId>ATo(LYCCgNkx+ zUD;JrjIl!XqVPYQJmgt|^4Sn#Fqlb5i2w$3;-5uPorWFs&ea*)y%g;-A?e&PEImE7_F{gC5!`u^w*3kiA8tvf%lx z7f2^<>$3+zuE}kK*QaiMuuU62V*SvgS-I?ARI)XhJ7|zpKXGT7rNm}fb6T9AE)u9!I0gI7 z;#&nM1V3CK4D7A;#)2e=3Dmi}Q*yonGZCfgvok4(2TK z^_EppWKWBD-N*o}C@B1>gkxZjJwbmjLD1mtRtON)7X{sHzUWlTw9^Ie;Vv})$gmzzxDt|k5WSK*9`B%=3DWqiKg^Bfsqw`4Br)`SsykeE+U<{d+=3DPbF*FGw>+vQ z7%=3DmDLawhBSGG9{rgRswsn&1Jwl@#-UplF$W+J#fHkt%$fyQk0(Gr{s^c)9X{Giw6 z>7gA{0^J{!q^hB|G~2}Y;hn=3DYk%96$Yw%TY1P2KRJK(VDdQhE z7Yd{i#|PWH8%d@jb9Mg?PrhU0)9`99`eaNL5EKz~b>QB;ln$Cl5B20b>FbU_dyyCR z8?7Uh!E=3DE8Pf(4L8GANM+4^()notm@+-yAcxi25w8Y@k9Pc_mya*-uK7a6pI6N0v=3D z?&ygSQ|Hs??5xms&GJrrH{aH zj1>d|0BB)JG8haCs)P}72@XF;bJSds3{YytPMKe`w}{fEgFXO&+gPjs2G4~`Kp-sW z;HP1*Bq-F`-(w#gWErLFD+mql6;>xPTN{!N%BQJO7XtnkY9E2ca-(pq&G7vdg*?zV z0!@JMIF0zYo_+dCv^CukiM27Qb}MTJM99!4{6~;o{%GZ$G3zozqD{(mix|x?fS{Zw z*+igqyX%B1@i?;?0cOMQszbL%pRc*NNPU_T!N+8Z!S`Fg!jfv=3Dl2o4_Sq4$@ z_g9#zzOAWAIBx8Ps^i`dro9!R4`?~|l8_KJj*^MM2H9h3vD6`IHxm-$NJdYb{iT``8-uJL(){M6LY>e2D)lF!< zNY8q@E~7FkL9%(wNvGN;O>!)|>Vb8EJ(KKa$60V#^PDCKMMnJd=3DvD*LhAI$TC` zp;LPYKSrwVuWtVpOnYv7_1L-nn57vsjH7fZLKhc&5qx|+69_Hx^x&L5ZgAtig0rqH zOD!RHozjKI^(n3_!|zpDD98k0*;=3DQqSl%* zk-(^Oo%n2KPucc zR`P&qS}7{{HOgH%`PjdH&)l>*1Sn_irdxU|{SEYFjlKxKE?6Z3oO!y%|9yO>ckrhc=3DCo1h$nKI=3Dksf%a}7^;5BHUfuh9sI$s#& zz>IJ^ZPWK|l)zBc|%=3D5){xZZ#MBc_!}h+z(_GqM(%U zaN}%tyNHv(sAMO8eR^opEvTyYT8qWwcbXY7mz3k|oO`veJipehRb!W;ZnZL<;`KV# zv{c$Q68<@QwO`;<^i^zC^mM>gH^YQo&wM;vjc^>pZ*AzDK%u8)6Wc{Z{EeEFP>CA> z4zpaFwoOq_v#c`V|5s;{$ zIF~jy=3Dh|iU0QRYA_Hwp_8j-=3D`^-{eXy#wYyZ9IoRWIaOQe!N3wA4)03Zp_`-@XQQ* z%gY}w`X=3DUuL{7yBCVbm1Nb%4v_FRijuSkWD?h{{btLs&7z)nnzg-uGsFHxpp2E8nU za3$f->YVoB3Xgo1f~rzNg(nXt85P)i-rBWsdUa|;^{~VfW!Cu0LKDBB zle9IT022%%H6N5_g|%yHTb%H9se{|}ePyMMKF>q_Rpu7vCL*razDeh8n7*`?MP!(b zqP&1_G`h=3DA-X3e8{&|f#$di__)hB`%Ag>hP7+J z>c(k|DotnzF?~wPVL%^R`|%F!@^s-#2jkw3$98YVy{rxowV2a;{{i;^r5cP>$TZ+5%b?DJOu7 szI*`uV@MpKGS3h7Qegb&4SI4NLMj77KkXD{OHkYNv<!8&_r>`Anc3kx7f(}GdK+my%t3qNA(E1jz%m{(UvvOydz4>lh* z{}9$k$|faVz?(QrgX}PnogKtO7YN!CL0cNQCqJH+4PCahY=3DHo?U*HN zGjdDx_@fe896GEW3BBs~hb*h^Bei+}XyC$1c~F6Ab-BKC`Ap%~`g3mXFMZh;+-x&E z)+lMw^2=3DLt8CS&XZPRmraH9obWUa1Gpb<){%Pr|_kJLa=3D&pj$~;q>d~<;}J*-Gv6Hl!f!xNQ(wy-c~7TgJR?nia~y`hS+Ev=3D*~YlTHkg@$qdON+&?fh&&lf) zHl|#B-`6eotP^GGGJLAWjc@xJj;HhOd~HR0T7~GECLY8t1`yuJzn9eEPa``V+7L5b z>6#}DeKV_!XHfA`&t&qus>!}+p!KXiWH_y`#G@66$I2) z@+@pLaZwFkGchPcPqJ$eCKN~QB}faRHEn8EwGd&`=3DPDbb@aP~)KNkj-D^2>3gV+aV zzUbT3w&x))e_miJL74TeTBls z0rm4}bj^v;(7t`G5$$}o`%3C&YY;`YkV0hBGQ&_x`uwYFR8oTnYJ+1ME%SJM`6ZJk zu@oN(CKv9o-cV}cLv66SBwd*7y_Ipr_l5PsazxYC+=3D;s$5zfT1YJTgIgh+nt@5mKl zq_Yn_eezUgE$iK<%&X39KzyV&+<9)wm-lHwg=3D=3DcCQqsqw^+*9t;^M|Hz*SW&&2=3DW9 zD)sR3n>Xbq#Oi^xn@w$2Xm_uYkn6ingI-f@g!Cy@=3D@>9)_K8Y8Cu>ZsT$6@Tkuu?A zwq%J`Qg3e$0qFD|Jwa{7uS{FU1nVIyJX_LEwelka0pP&9)r+Sk0qU3I6aCHe&~Zx% zvov-Wj4Z0A#k;(idy(Qj<44vI`R-(V6%z`}l5)m9p0A+FfAMzqu4HL>$bYu8G#Cw! zR40I)kt_o*K7p_Kf@&(V)6U)UU+>!+t!`;0bQ0t%z> zJSKp()Omz{UJ0|OtVKdc#y7d-v=3DJm+N4pgCxoKA9d*{XXSB)90H@a3RuiwQ#<9^A5F#p&trB|q}`l2Q-S`0pM z{RwN5vk$e~vJ}2EYr)Mn(mBtNdrt^(8e%%6!L+zau}oVj{2~dIRF1n);H`!8!=3D3!D z?}83tyz+Cvp4yv<%c=3D@~^w9RrAy$Nk(brf`@zQpOc{LGCj>5K`l`(G?G}y)w(0)E) zbPVzHJps)$P&(obJI!bV7bIV(U@wamD=3D6ez@`p_xhGv;FrQG8CT%OAXav=3D^!KL!jr zYzq-jD{v4j4p}iD29`a&g|#d&J^IA8KfF9)G<)HD)6jt7{>iCp00@d z<9GM*`GBygaJg=3Dm7M}PIM^W1oDh7c4U?B>nz%GfD@Z9)lmc3Q=3DDQgd|e8A5R$8*5z zIeg|S{R12sU%}qZ!>g}r=3Dq1$;{a53jwB2(xw=3Dy0`&j|nrG>zcxf*MCYBxnke734VF zH#?_Qq*vr8i7+|#?tlsL&LHdQ1RONPE^{Zz!7Uz!3R;&9{-Ks(LA$8hB;dLMawvXB}x zEG(c7y}M!#nYIdC@X_W3;x#7hQszo?GwdKp{{XmHfV|WNK!`%HXMkeXn-IRI4_}ne zsc>69wxdmY!xXdjALhzfK->UnsZGn>s_Pa|BYHRvBf5O?5(YBG71(Y=3Dg(*Up@Pl@2`)5){L)-@szlBOo5vW%k}(|$yQ6WwLlt8Z=3D$ z=3DBh4^tw7$=3Dx4afomFHAbQ)(eLy*O((@U*CZsr`p_Bh!KMEfVB3*PV9-A#JR>o$X|KG@s#CfU(lK9gv(N%<}(krb`>h( zz=3D`a0#3iRR(0MXh&#F>_VAIvPXJZxv{u)HP9&uZ_wlqbL{Mor0i5ZKo$E3#}ol_d) zg#E6|F;8F8pPYMk)tp+rI%~O3xEsQvned5mw~f(E?2LuVnLlRstQvG%XB?DoN`75^ z(lunQ8=3D<-r+Z=3DMA$$zmBq#gg#_*LY4KVx;9J!$C3o!M~Zo!C(wM39pO-L8FBVj7tD z>!R;K`p=3DrzH5L(e{UvR!7&V6>rVXF0M1@rQ_HLW%73TMmLyV#gcPf=3D?1)_E(1gG^a z;G^@{2hK^>Z#bkzUyVeU=3DP?`>!B&<>T6hR(Au=3DeJ=3Dw6*i6<2U12C0+d-?zxjVga2e z33L&-2IZsB`OH!}Qv868IWC~7`hZ%wg>bPPr+a2?ELhg-f4DoDK?9Ax{De)DVG016 z^|q9b$+n>k9`Rwu=3DPW!;yFH+U{T{3gi=3Dk5n6totEw9!qjEwfT#L7hQvf3Qn4rn z7FMJe<&F9f01AsN@GF0vzs$-#F#8~!S>;eSAKW$FmL!g+D^g~ zFFKw%u;KdqaoULPm7a=3D(9A~1*(wOdywM^0XmbjTutywK%n+vAq1K>e%p)$KwVnt^B z(U&KhSDe-ZOogDe8n`6NPN?(swY7?&p;dMi$eZFqd%0-qWj?4~5B|EWz0j(%MEcyc zIvZTuUpA|5HX$)6YHuBO$^J(8L#Y7xQ&HYL z)l7oWU~qrc&}>20kSUm1{A~%pW{!}vj|ED2ijc3{U=3D|(} z=3D((mgn{-9gjOwNgldlRHEIfbCS@zrn87rKZmj$7`M-?}qG@>efJSzpv>n#EdJ7!;e z#lqhS0mqG*&BW`k5Zn8?gI)f_oN%nLV0z|H1H9WOMc?-|-t+_4M7~oT(s?jZJvH=3D} z>xFH*-a39*8BUi%`hptQKQXA(ZTbiqSHbE2qU~ohN~Q4-Q*4sc8~v}Z?04?BL%|5W zABB!IIS}1;0!rCcrO;w57Otu!MoXu zh99`6V^EpBb&PANe0^iuTZU1UkBmMBu>uFw3+xVgpxx1mteP=3D*!Moktm(w?1$jG7; zPz0pDlqV}n-ru10cEQ6f9>ih9dw7n>)JDNxY8vKH!O{Ib#_DtdIT=3DSj1T~J9z{g;F z22%~TOHe;(pS<8GMc&r8$YzYt^Mq>zHFT7j9yT7bq0>$lxSYfegSTdDi2P;WXR8hX z@oEzuMD8SRhVaz)6QEM3=3D*Cf$0(~r&Q$@r#6BF0?z#&?w`T zEIq~FYs+yZ8lq-%uz(fih^J|8)*Zk{ytrEz*eGCKYHXb2_z(*!8p zKXrx;aX4%ZkiufF4@}kL$T%oCCLi@DojuwJz$3Hk!Tv$?U2jXe=3DDbL-eO_l=3D{yW+o zrHJ})u-^Tt2KEybgl-~l6|y=3D@V1v?r#)@)H$2SGX3OVcskCV!-QNfie`Z{(-Qfk;| zwReHV0LC3v`2p9_)E zE0l0!#t>@7wWrP`g;{Rnxp@LK|1N%WP!DPQpWLQE)n$eH*rxpy$>ks1-B z(B3H9$0yR~g34PGe{ydx3$HET3UwTd!n4EyKZc&2Pxea!n0$d^at)$nS3Q0ydUiVe+#_7J zV?X#$;=3DHm^x>%MV1>;~8s^e>HZRV`C)*gdfnRKyAC?^%|lg)X2zK(^G{v0rb>BQj_OY$BbN9*#K6xJ%U&mS2 zGijqGK@zo!2V5B}@2q;auNYg`yJ0ad+h&_5YqyJZ^J}S(^{lvl_`Kmy38*jROowuR z8j%gC4>;|=3D7Hj{=3DM$3|TF7syftQ&*+;K?9=3DNght_tW!-PJHC$sHw&T_J`qCJdYSIP zpF)*{X(^NOz>1seH*w}3KO>~a7suj5EQo~}R(t{M3nrS1Mg>o!1kAf^a^KeL(ZQwP z&#}#>ks0zs_?9?>%W6R%F zDy5~MUtDD4+x7G-JGb%On!-JOOahz3 z^Xk(+DFo?t%7b^BMp+t(;<>?&)p??(Z~sjA@MCAaBIVe%4431BhDUWdUm8jtl(@=3Dq zp(JXe?$}A}vv2ww%J*iqVp3ptQi?Zfgw3c|y(pA+j)`OjzG5#nJH1p(%W}NlaSRAZZQfcecU)uUGf($$NjY9nPoE9V_i!chW`>2eu;M@tAE z5_mA{PW_|wbM7e4`lDF=3Dv`F9uI1(MukrXLgG4jt1|FiMXeIxmptopCDTBM_fSl^fs zpf0lg3Mv6Kb03EwWCO=3DNG>? zKKT9B`DFkxd9i1BY;x--z#Ib(p9LGoe?G0VubfR4kL(D>_V`x?W_2;Z1$;6YHcG4R z9KTDGWQM)nRGak{jdoYEY$#R!atFKMbC6sr0ql;h*}kfkjC&Pgy;%qjYgE8Ot`{!k z^OTj4UUFA|0k>M%PLgj!Q(Xo?XPf2@=3DjGzzV*&>RkagWOFk#TYN$OqMM-wiCQofOi zF)`bCk2z$i2z)nsIDw8j%op#?__$@PrY_qv(gS}U%OXHzW&OKQ_4menTcnlxbfjHh Rl5p7TIjd!;S)y(m_Fp#ycijL0 literal 0 HcmV?d00001 diff --git a/Documentation/admin-guide/mm/damon/eval.rst b/Documentation/= admin-guide/mm/damon/eval.rst new file mode 100644 index 000000000000..b233890b4e45 --- /dev/null +++ b/Documentation/admin-guide/mm/damon/eval.rst @@ -0,0 +1,222 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Evaluation +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +DAMON is lightweight. It increases system memory usage by only -0.25% a= nd +consumes less than 1% CPU time in most case. It slows target workloads = down by +only 0.94%. + +DAMON is accurate and useful for memory management optimizations. An +experimental DAMON-based operation scheme for THP, 'ethp', removes 31.29= % of +THP memory overheads while preserving 60.64% of THP speedup. Another +experimental DAMON-based 'proactive reclamation' implementation, 'prcl', +reduces 87.95% of residential sets and 29.52% of system memory footprint= while +incurring only 2.15% runtime overhead in the best case (parsec3/freqmine= ). + +Setup +=3D=3D=3D=3D=3D + +On a QEMU/KVM based virtual machine utilizing 20GB of RAM and hosted by = an +Intel i7 machine that running a kernel that v16 DAMON patchset is applie= d, I +measure runtime and consumed system memory while running various realist= ic +workloads with several configurations. I use 13 and 12 workloads in PAR= SEC3 +[3]_ and SPLASH-2X [4]_ benchmark suites, respectively. I use another w= rapper +scripts [5]_ for convenient setup and run of the workloads. + +Measurement +----------- + +For the measurement of the amount of consumed memory in system global sc= ope, I +drop caches before starting each of the workloads and monitor 'MemFree' = in the +'/proc/meminfo' file. To make results more stable, I repeat the runs 5 = times +and average results. + +Configurations +-------------- + +The configurations I use are as below. + +- orig: Linux v5.7 with 'madvise' THP policy +- rec: 'orig' plus DAMON running with virtual memory access recording +- prec: 'orig' plus DAMON running with physical memory access recording +- thp: same with 'orig', but use 'always' THP policy +- ethp: 'orig' plus a DAMON operation scheme, 'efficient THP' +- prcl: 'orig' plus a DAMON operation scheme, 'proactive reclaim [6]_' + +I use 'rec' for measurement of DAMON overheads to target workloads and s= ystem +memory. 'prec' is for physical memory monitroing and recording. It mon= itors +17GB sized 'System RAM' region. The remaining configs including 'thp', = 'ethp', +and 'prcl' are for measurement of DAMON monitoring accuracy. + +'ethp' and 'prcl' are simple DAMON-based operation schemes developed for +proof of concepts of DAMON. 'ethp' reduces memory space waste of THP by= using +DAMON for the decision of promotions and demotion for huge pages, while = 'prcl' +is as similar as the original work. Those are implemented as below:: + + # format: <= action> + # ethp: Use huge pages if a region shows >=3D5% access rate, use reg= ular + # pages if a region >=3D2MB shows <5% access rate for >=3D13 seconds + null null 5 null null null hugepage + 2M null null null 13s null nohugepage + + # prcl: If a region >=3D4KB shows <=3D5% access rate for >=3D7 secon= ds, page out. + 4K null null 5 7s null pageout + +Note that both 'ethp' and 'prcl' are designed with my only straightforwa= rd +intuition because those are for only proof of concepts and monitoring ac= curacy +of DAMON. In other words, those are not for production. For production= use, +those should be more tuned. + +.. [1] "Redis latency problems troubleshooting", https://redis.io/topics= /latency +.. [2] "Disable Transparent Huge Pages (THP)", + https://docs.mongodb.com/manual/tutorial/transparent-huge-pages/ +.. [3] "The PARSEC Becnhmark Suite", https://parsec.cs.princeton.edu/ind= ex.htm +.. [4] "SPLASH-2x", https://parsec.cs.princeton.edu/parsec3-doc.htm#spla= sh2x +.. [5] "parsec3_on_ubuntu", https://github.com/sjp38/parsec3_on_ubuntu +.. [6] "Proactively reclaiming idle memory", https://lwn.net/Articles/78= 7611/ + +Results +=3D=3D=3D=3D=3D=3D=3D + +Below two tables show the measurement results. The runtimes are in seco= nds +while the memory usages are in KiB. Each configuration except 'orig' sh= ows +its overhead relative to 'orig' in percent within parenthesizes.:: + + runtime orig rec (overhead) prec (overh= ead) thp (overhead) ethp (overhead) prcl (overhead) + parsec3/blackscholes 107.228 107.859 (0.59) 108.110 (0.82)= 107.381 (0.14) 106.811 (-0.39) 114.766 (7.03) + parsec3/bodytrack 79.292 79.609 (0.40) 79.777 (0.61)= 79.313 (0.03) 78.892 (-0.50) 80.398 (1.40) + parsec3/canneal 148.887 150.878 (1.34) 153.337 (2.99)= 127.873 (-14.11) 132.272 (-11.16) 167.631 (12.59) + parsec3/dedup 11.970 11.975 (0.04) 12.024 (0.45)= 11.752 (-1.82) 11.921 (-0.41) 13.244 (10.64) + parsec3/facesim 212.800 215.927 (1.47) 215.004 (1.04)= 205.117 (-3.61) 207.401 (-2.54) 220.834 (3.78) + parsec3/ferret 190.646 192.560 (1.00) 192.414 (0.93)= 190.662 (0.01) 192.309 (0.87) 193.497 (1.50) + parsec3/fluidanimate 213.951 216.459 (1.17) 217.578 (1.70)= 209.500 (-2.08) 211.826 (-0.99) 218.299 (2.03) + parsec3/freqmine 291.050 292.117 (0.37) 293.279 (0.77)= 289.553 (-0.51) 291.768 (0.25) 297.309 (2.15) + parsec3/raytrace 118.645 119.734 (0.92) 119.521 (0.74)= 117.715 (-0.78) 118.844 (0.17) 134.045 (12.98) + parsec3/streamcluster 332.843 336.997 (1.25) 337.049 (1.26)= 279.716 (-15.96) 290.985 (-12.58) 346.646 (4.15) + parsec3/swaptions 155.437 157.174 (1.12) 156.159 (0.46)= 155.017 (-0.27) 154.955 (-0.31) 156.555 (0.72) + parsec3/vips 59.215 59.426 (0.36) 59.156 (-0.10= ) 59.243 (0.05) 58.858 (-0.60) 60.184 (1.64) + parsec3/x264 67.445 71.400 (5.86) 71.122 (5.45)= 64.078 (-4.99) 66.027 (-2.10) 71.489 (6.00) + splash2x/barnes 81.826 81.800 (-0.03) 82.648 (1.00)= 74.343 (-9.15) 79.063 (-3.38) 103.785 (26.84) + splash2x/fft 33.850 34.148 (0.88) 33.912 (0.18)= 23.493 (-30.60) 32.684 (-3.44) 48.303 (42.70) + splash2x/lu_cb 86.404 86.333 (-0.08) 86.988 (0.68)= 85.720 (-0.79) 85.944 (-0.53) 89.338 (3.40) + splash2x/lu_ncb 94.908 98.021 (3.28) 96.041 (1.19)= 90.304 (-4.85) 93.279 (-1.72) 97.270 (2.49) + splash2x/ocean_cp 47.122 47.391 (0.57) 47.902 (1.65)= 43.227 (-8.26) 44.609 (-5.33) 51.410 (9.10) + splash2x/ocean_ncp 93.147 92.911 (-0.25) 93.886 (0.79)= 51.451 (-44.76) 71.107 (-23.66) 112.554 (20.83) + splash2x/radiosity 92.150 92.604 (0.49) 93.339 (1.29)= 90.802 (-1.46) 91.824 (-0.35) 104.439 (13.34) + splash2x/radix 31.961 32.113 (0.48) 32.066 (0.33)= 25.184 (-21.20) 30.412 (-4.84) 49.989 (56.41) + splash2x/raytrace 84.781 85.278 (0.59) 84.763 (-0.02= ) 83.192 (-1.87) 83.970 (-0.96) 85.382 (0.71) + splash2x/volrend 87.401 87.978 (0.66) 87.977 (0.66)= 86.636 (-0.88) 87.169 (-0.26) 88.043 (0.73) + splash2x/water_nsquared 239.140 239.570 (0.18) 240.901 (0.74)= 221.323 (-7.45) 224.670 (-6.05) 244.492 (2.24) + splash2x/water_spatial 89.538 89.978 (0.49) 90.171 (0.71)= 89.729 (0.21) 89.238 (-0.34) 99.331 (10.94) + total 3051.620 3080.230 (0.94) 3085.130 (1.10)= 2862.320 (-6.20) 2936.830 (-3.76) 3249.240 (6.48) + + + memused.avg orig rec (overhead) prec = (overhead) thp (overhead) ethp (overhead) prcl = (overhead) + parsec3/blackscholes 1676679.200 1683789.200 (0.42) 1680281= .200 (0.21) 1613817.400 (-3.75) 1835229.200 (9.46) 1407952.= 800 (-16.03) + parsec3/bodytrack 1295736.000 1308412.600 (0.98) 1311988= .000 (1.25) 1243417.400 (-4.04) 1435410.600 (10.78) 1255566.= 400 (-3.10) + parsec3/canneal 1004062.000 1008823.800 (0.47) 1000100= .200 (-0.39) 983976.000 (-2.00) 1051719.600 (4.75) 993055.8= 00 (-1.10) + parsec3/dedup 2389765.800 2393381.000 (0.15) 2366668= .200 (-0.97) 2412948.600 (0.97) 2435885.600 (1.93) 2380172.= 800 (-0.40) + parsec3/facesim 488927.200 498228.000 (1.90) 496683.= 800 (1.59) 476327.800 (-2.58) 552890.000 (13.08) 449143.6= 00 (-8.14) + parsec3/ferret 280324.600 282032.400 (0.61) 282284.= 400 (0.70) 258211.000 (-7.89) 331493.800 (18.25) 265850.4= 00 (-5.16) + parsec3/fluidanimate 560636.200 569038.200 (1.50) 565067.= 400 (0.79) 556923.600 (-0.66) 588021.200 (4.88) 512901.6= 00 (-8.51) + parsec3/freqmine 883286.000 904960.200 (2.45) 886105.= 200 (0.32) 849347.400 (-3.84) 998358.000 (13.03) 622542.8= 00 (-29.52) + parsec3/raytrace 1639370.200 1642318.200 (0.18) 1626673= .200 (-0.77) 1591284.200 (-2.93) 1755088.400 (7.06) 1410261.= 600 (-13.98) + parsec3/streamcluster 116955.600 127251.400 (8.80) 121441.= 000 (3.84) 113853.800 (-2.65) 139659.400 (19.41) 120335.2= 00 (2.89) + parsec3/swaptions 8342.400 18555.600 (122.43) 16581.2= 00 (98.76) 6745.800 (-19.14) 27487.200 (229.49) 14275.60= 0 (71.12) + parsec3/vips 2776417.600 2784989.400 (0.31) 2820564= .600 (1.59) 2694060.800 (-2.97) 2968650.000 (6.92) 2713590.= 000 (-2.26) + parsec3/x264 2912885.000 2936474.600 (0.81) 2936775= .800 (0.82) 2799599.200 (-3.89) 3168695.000 (8.78) 2829085.= 800 (-2.88) + splash2x/barnes 1206459.600 1204145.600 (-0.19) 1177390= .000 (-2.41) 1210556.800 (0.34) 1214978.800 (0.71) 907737.0= 00 (-24.76) + splash2x/fft 9384156.400 9258749.600 (-1.34) 8560377= .800 (-8.78) 9337563.000 (-0.50) 9228873.600 (-1.65) 9823394.= 400 (4.68) + splash2x/lu_cb 510210.800 514052.800 (0.75) 502735.= 200 (-1.47) 514459.800 (0.83) 523884.200 (2.68) 367563.2= 00 (-27.96) + splash2x/lu_ncb 510091.200 516046.800 (1.17) 505327.= 600 (-0.93) 512568.200 (0.49) 524178.400 (2.76) 427981.8= 00 (-16.10) + splash2x/ocean_cp 3342260.200 3294531.200 (-1.43) 3171236= .000 (-5.12) 3379693.600 (1.12) 3314896.600 (-0.82) 3252406.= 000 (-2.69) + splash2x/ocean_ncp 3900447.200 3881682.600 (-0.48) 3816493= .200 (-2.15) 7065506.200 (81.15) 4449224.400 (14.07) 3829931.= 200 (-1.81) + splash2x/radiosity 1466372.000 1463840.200 (-0.17) 1438554= .000 (-1.90) 1475151.600 (0.60) 1474828.800 (0.58) 496636.0= 00 (-66.13) + splash2x/radix 1760056.600 1691719.000 (-3.88) 1613057= .400 (-8.35) 1384416.400 (-21.34) 1632274.400 (-7.26) 2141640.= 200 (21.68) + splash2x/raytrace 38794.000 48187.400 (24.21) 46728.4= 00 (20.45) 41323.400 (6.52) 61499.800 (58.53) 68455.20= 0 (76.46) + splash2x/volrend 138107.400 148197.000 (7.31) 146223.= 400 (5.88) 128076.400 (-7.26) 164593.800 (19.18) 140885.2= 00 (2.01) + splash2x/water_nsquared 39072.000 49889.200 (27.69) 47548.4= 00 (21.69) 37546.400 (-3.90) 57195.400 (46.38) 42994.20= 0 (10.04) + splash2x/water_spatial 662099.800 665964.800 (0.58) 651017.= 000 (-1.67) 659808.400 (-0.35) 674475.600 (1.87) 519677.6= 00 (-21.51) + total 38991500.000 38895300.000 (-0.25) 3778781= 7.000 (-3.09) 41347200.000 (6.04) 40609600.000 (4.15) 36994100= .000 (-5.12) + + +DAMON Overheads +--------------- + +In total, DAMON virtual memory access recording feature ('rec') incurs 0= .94% +runtime overhead and -0.25% memory space overhead. Even though the size= of the +monitoring target region becomes much larger with the physical memory ac= cess +recording ('prec'), it still shows only modest amount of overhead (1.10%= for +runtime and -3.09% for memory footprint). + +For a convenience test run of 'rec' and 'prec', I use a Python wrapper. = The +wrapper constantly consumes about 10-15MB of memory. This becomes a hig= h +memory overhead if the target workload has a small memory footprint. +Nonetheless, the overheads are not from DAMON, but from the wrapper, and= thus +should be ignored. This fake memory overhead continues in 'ethp' and 'p= rcl', +as those configurations are also using the Python wrapper. + + +Efficient THP +------------- + +THP 'always' enabled policy achieves 6.20% speedup but incurs 6.04% memo= ry +overhead. It achieves 44.76% speedup in the best case, but 81.15% memor= y +overhead in the worst case. Interestingly, both the best and worst-case= are +with 'splash2x/ocean_ncp'). + +The 2-lines implementation of data access monitoring based THP version (= 'ethp') +shows 3.76% speedup and 4.15% memory overhead. In other words, 'ethp' r= emoves +31.29% of THP memory waste while preserving 60.64% of THP speedup in tot= al. In +the case of the 'splash2x/ocean_ncp', 'ethp' removes 82.66% of THP memor= y waste +while preserving 52.85% of THP speedup. + + +Proactive Reclamation +--------------------- + +As similar to the original work, I use 4G 'zram' swap device for this +configuration. + +In total, our 1 line implementation of Proactive Reclamation, 'prcl', in= curred +6.48% runtime overhead in total while achieving 5.12% system memory usag= e +reduction. + +Nonetheless, as the memory usage is calculated with 'MemFree' in +'/proc/meminfo', it contains the SwapCached pages. As the swapcached pa= ges can +be easily evicted, I also measured the residential set size of the workl= oads:: + + rss.avg orig rec (overhead) prec = (overhead) thp (overhead) ethp (overhead) prcl = (overhead) + parsec3/blackscholes 590412.200 589991.400 (-0.07) 591716.= 400 (0.22) 591131.000 (0.12) 591055.200 (0.11) 274623.6= 00 (-53.49) + parsec3/bodytrack 32202.200 32297.400 (0.30) 32301.4= 00 (0.31) 32328.000 (0.39) 32169.800 (-0.10) 25311.20= 0 (-21.40) + parsec3/canneal 840063.600 839145.200 (-0.11) 839506.= 200 (-0.07) 835102.600 (-0.59) 839766.000 (-0.04) 833091.8= 00 (-0.83) + parsec3/dedup 1185493.200 1202688.800 (1.45) 1204597= .000 (1.61) 1238071.400 (4.44) 1201689.400 (1.37) 920688.6= 00 (-22.34) + parsec3/facesim 311570.400 311542.000 (-0.01) 311665.= 000 (0.03) 316106.400 (1.46) 312003.400 (0.14) 252646.0= 00 (-18.91) + parsec3/ferret 99783.200 99330.000 (-0.45) 99735.0= 00 (-0.05) 102000.600 (2.22) 99927.400 (0.14) 90967.40= 0 (-8.83) + parsec3/fluidanimate 531780.800 531800.800 (0.00) 531754.= 600 (-0.00) 532009.600 (0.04) 531822.400 (0.01) 479116.0= 00 (-9.90) + parsec3/freqmine 551787.600 551550.600 (-0.04) 551950.= 000 (0.03) 556030.000 (0.77) 553720.400 (0.35) 66480.00= 0 (-87.95) + parsec3/raytrace 895247.000 895240.200 (-0.00) 895770.= 400 (0.06) 895880.200 (0.07) 893516.600 (-0.19) 327339.6= 00 (-63.44) + parsec3/streamcluster 110862.200 110840.400 (-0.02) 110878.= 600 (0.01) 112067.200 (1.09) 112010.800 (1.04) 109763.6= 00 (-0.99) + parsec3/swaptions 5630.000 5580.800 (-0.87) 5599.60= 0 (-0.54) 5624.200 (-0.10) 5697.400 (1.20) 3792.400= (-32.64) + parsec3/vips 31677.200 31881.800 (0.65) 31785.8= 00 (0.34) 32177.000 (1.58) 32456.800 (2.46) 29692.00= 0 (-6.27) + parsec3/x264 81796.400 81918.600 (0.15) 81827.6= 00 (0.04) 82734.800 (1.15) 82854.000 (1.29) 81478.20= 0 (-0.39) + splash2x/barnes 1216014.600 1215462.000 (-0.05) 1218535= .200 (0.21) 1227689.400 (0.96) 1219022.000 (0.25) 650771.0= 00 (-46.48) + splash2x/fft 9622775.200 9511973.400 (-1.15) 9688178= .600 (0.68) 9733868.400 (1.15) 9651488.000 (0.30) 7567077.= 400 (-21.36) + splash2x/lu_cb 511102.400 509911.600 (-0.23) 511123.= 800 (0.00) 514466.800 (0.66) 510462.800 (-0.13) 361014.0= 00 (-29.37) + splash2x/lu_ncb 510569.800 510724.600 (0.03) 510888.= 800 (0.06) 513951.600 (0.66) 509474.400 (-0.21) 424030.4= 00 (-16.95) + splash2x/ocean_cp 3413563.600 3413721.800 (0.00) 3398399= .600 (-0.44) 3446878.000 (0.98) 3404799.200 (-0.26) 3244787.= 400 (-4.94) + splash2x/ocean_ncp 3927797.400 3936294.400 (0.22) 3917698= .800 (-0.26) 7181781.200 (82.85) 4525783.600 (15.22) 3693747.= 800 (-5.96) + splash2x/radiosity 1477264.800 1477569.200 (0.02) 1476954= .200 (-0.02) 1485724.800 (0.57) 1474684.800 (-0.17) 230128.0= 00 (-84.42) + splash2x/radix 1773025.000 1754424.200 (-1.05) 1743194= .400 (-1.68) 1445575.200 (-18.47) 1694855.200 (-4.41) 1769750.= 000 (-0.18) + splash2x/raytrace 23292.000 23284.000 (-0.03) 23292.8= 00 (0.00) 28704.800 (23.24) 26489.600 (13.73) 15753.00= 0 (-32.37) + splash2x/volrend 44095.800 44068.200 (-0.06) 44107.6= 00 (0.03) 44114.600 (0.04) 44054.000 (-0.09) 31616.00= 0 (-28.30) + splash2x/water_nsquared 29416.800 29403.200 (-0.05) 29406.4= 00 (-0.04) 30103.200 (2.33) 29433.600 (0.06) 24927.40= 0 (-15.26) + splash2x/water_spatial 657791.000 657840.400 (0.01) 657826.= 600 (0.01) 657595.800 (-0.03) 656617.800 (-0.18) 481334.8= 00 (-26.83) + total 28475091.000 28368400.000 (-0.37) 2850870= 0.000 (0.12) 31641800.000 (11.12) 29036000.000 (1.97) 21989800= .000 (-22.78) + +In total, 22.78% of residential sets were reduced. + +With parsec3/freqmine, 'prcl' reduced 87.95% of residential sets and 29.= 52% of +system memory usage while incurring only 2.15% runtime overhead. diff --git a/Documentation/admin-guide/mm/damon/faq.rst b/Documentation/a= dmin-guide/mm/damon/faq.rst new file mode 100644 index 000000000000..f55d1d719999 --- /dev/null +++ b/Documentation/admin-guide/mm/damon/faq.rst @@ -0,0 +1,59 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D +Frequently Asked Questions +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D + +Why a new module, instead of extending perf or other user space tools? +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +First, because it needs to be lightweight as much as possible so that it= can be +used online, any unnecessary overhead such as kernel - user space contex= t +switching cost should be avoided. Second, DAMON aims to be used by othe= r +programs including the kernel. Therefore, having a dependency on specif= ic +tools like perf is not desirable. These are the two biggest reasons why= DAMON +is implemented in the kernel space. + + +Can 'idle pages tracking' or 'perf mem' substitute DAMON? +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D + +Idle page tracking is a low level primitive for access check of the phys= ical +address space. 'perf mem' is similar, though it can use sampling to min= imize +the overhead. On the other hand, DAMON is a higher-level framework for = the +monitoring of various address spaces. It is focused on memory managemen= t +optimization and provides sophisticated accuracy/overhead handling mecha= nisms. +Therefore, 'idle pages tracking' and 'perf mem' could provide a subset o= f +DAMON's output, but cannot substitute DAMON. Rather than that, thouse c= ould be +configured as DAMON's low-level primitives for specific address spaces. + + +How can I optimize my system's memory management using DAMON? +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Because there are several ways for the DAMON-based optimizations, we wro= te a +separate document, :doc:`guide`. Please refer to that. + + +Does DAMON support virtual memory only? +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +No. The core of the DAMON is address space independent. The address sp= ace +specific low level primitive parts including monitoring target regions +constructions and actual access checks can be implemented and configured= on the +DAMON core by the users. In this way, DAMON users can monitor any addre= ss +space with any access check technique. + +Nonetheless, DAMON provides a vma tracking and PTE Accessed bit check ba= sed +implementation of the address space dependent functions for the virtual = memory +by default, for a reference and convenient use. In near future, we will= also +provide that for physical memory address space. + + +Can I simply monitor page granularity? +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Yes. You can do so by setting the ``min_nr_regions`` attribute higher t= han the +working set size divided by the page size. Because the monitoring targe= t +regions size is forced to be ``>=3Dpage size``, the region split will ma= ke no +effect. diff --git a/Documentation/admin-guide/mm/damon/freqmine_heatmap.png b/Do= cumentation/admin-guide/mm/damon/freqmine_heatmap.png new file mode 100644 index 0000000000000000000000000000000000000000..4bdde033017856246a47c3380= 84c31761296b27c GIT binary patch literal 9999 zcmZ`<2|QHqyFcfcVT`h5tt?aaU6Ld*wovx$yRw!ng-AH|il~sS7$ii9tdT)QSyQqv zDf^l&TjriAe!qMF_gr$j4v84j*v+6gTbV-y2AN+Qa6BT1MHCZz@w2z#UhRW{2zJnpgUp;0JKK5%>ZN* zNkH2t5>qTGP#c;KQkaOEuEsR8(F0^!;~v>uRf z0Z20fo|ymzW?-o$SZxh9*n%G}f$jESmm}Ec3=3DUrbe_Vwo-Jn?yXz?1f>w4tPgxK0J@BUuckn^1<+#!^x6P@mwY=3D@O5Cs8~El6 zjNSsi2LeBWfw3@P{5CKV2~5NQlMjKZcwjmSn0XA$5`ei(U_Kj|e-11Z0E@-I&oW@C z3RtcIR_cJ2cfjgLV66@K)d{SB1vdJCjbUK(JFqngY|jF}e*(W(ft^jHvH-?qi2)?X z)b8r0*8za`E%_IW7p6T7060KHMbW_L@!XR;F+bG3(~hs}+{b}OccdKc?G3%U<1LMU zdH(X&Sy*YACBNEQ3S1QXN{xbfCWGxX9@!pV>njo)UPSlbr!xt*qj^Z6d1#{%eo#?z z`ONqwmgBK@sN;cL9fsm1O-)Tc*0FUX44DjwCI9|N2zatT7x1#ADJp;afb(7Yj2T?R zyRY~Kp)>jlSjB%YzSEb<-mix6T_@5Y@yo4%H~pfcZ(YDlS78b{+`V8G=3Dfk-sF$cPG zgA|RbwI|g?;u4D@<`K zT>)HX(HWnF4{=3DT7vjPLS1vx{3YUDL4zM0o~fyFxry0mT1{cT!&}fP%%?%l&`|HKL%mO2 zyzf6KZl8dHf2XVaep5{dseNE}%Zl)E=3Dvf8LBVA(P{P~3Y-Ab?|>E@-{d0s0_q8HG7 z?}aD*QT@VkSwdXuIX#k&O&C0u`ZE4z)6+;x_phMHJH`lG9=3DWSdaYqx13t+_|s{Hmq z8Wr2iRGG_}a~E%j)2Yz-^mPcRie;t48>zjt&8zN0Vr>z&`ZJYA$9``=3Di0;}pcpRF& z|DOI2{X=3DdbRRck|mR}bnnb~xlV)q>Tg~DJSLtxN_95|3yVv?P zmWj=3DimdbMD3D;H& zX$A*?;BUvP;0L>)=3DHt{iDN!q2LL*$*d>{2nxmbb@9#863A~GL;>l6Ho8P$)eNYv;s z#t}$JDDX(k?f4XRg5aH!csMc%^&16kwzEK;f3N76I~KP>{rH5Vj^KI@35_h!AT|l z!IN2zASnhw#j%FMc}Bfzpcm49V++jIX7ojE!d*46zT_GD#i9asOiz?dXJ*AUv!eCR zU^w<%th!eB12}$mo=3DC%H^zDgT^aZt zXxM&sMT*xnKzI9cs8KF=3D45qTmS#FHot5d(htfo5w}b+x#cW8^3YZ>3^y=3DIM9rS}o70*y4QP2l z`m#3q_G78-yM)w4o+e0pxrr}Vo9JAs5g|14prXj6GxU}L1^D6cYd)KtdY8`*8Fo(_ zs$-0?iM1m0v?n6b@3_w06+ff**&d{ebSU4y5CePGB&)0)8DZEE_zZfh{=3DPrr_t<%;x^yv>`aL- zrk(Q=3D9TOzt0rYZM?e&!OgKT1jlJ3dsBM{R@Ugm|7;6+5OzGf`u}%!6OpV zesuEPdBlySudMD6XncR+GKC>+@$J_(cSg@224nal&L$rMV~l9l;p*B_m|tmr%>ci4 z)NjKa8v27B%KN5pDo>q=3D7R0GdNE-!aUk4kr;O}G7R>t^}3QPN&PFdfe=3D&lMh*l2+_ znvn%#T#kGfv@XpPC1gY6uSfDJOz!g2HmLvDNQdNkAkJao5#^Mc?~Fnt?<|PUg1GiJ zK8Q2PtS4D>jyA9!Ehl;Ys;JGoqh-^V!R!lGQ|}=3D$(M02@vD=3D|L)NE8^is~0-;di2h zCHyPmLpWlh66WViDfzBXjD5mVWbr-_47@;m0gqdxaD;2jd28n-4#sC+I-$VK|B`1! zLu_Qd6YJAyKs>(`pbWMqJv<9JG=3DIUEcoU@=3D+%-`mc~qcZ1Xkw6RST?7a?Obo5NH3T zfh|;Zn_ocLtH^DDhqYscmJZ2L>^3FD$(hica?3i$9?V|ulcPsJ=3D4^@SnBcO)%KS7T z#-3qXoE(|_mWc8Bngu)DHa{RCZo;xyC3W$p2*&5KP&xhLqRpFU6Ob6kjY5{iKJ1hu_5Il*Ldkm3K^;h3t~#=3D?Ow#^ZxlyITx2Vo9&QuVM?ZW? zeV@e13UNB5J>0s4k`qMBwKQ{bF?T0x$MmRXTM#oZR@4xf7CWP5wPF?+?%y`TIt8W~ zunw4mTeSs$&Es7`LMUldrYF6h&UF^?mhdn zBJT1vbF7Rgy2a2>=3DFCy~s05Uqd0wW6yj&cqJ;kbobcqIRu2C&>6jZ2X>R<`1GMT6_ zv8^(O`kuryxEG6z7><7)`_N!P%r?UJw?v3v6&}(9wKzGSq@~*RF@Wvj+U;B%mRuG^ z4Z$fBYNk7FQE($8_ee-~7))3ALjIB>(Q(3p*l>Yw(jPd{jFNjo z>>eFasW6MkWR`tU@Opo9x} zHBjWr!`P6JI}D4P88X-a55Hg2T{9=3DSh62_`r}7G03asT65^Dk`ex6haT(w)<%*<=3D- z?p_Gk-hNkNOh;S0v(Y-}S2$FCb8Wk;xv<86d9+uJOCeyZtz^P_WAZoMu;2Ba>9L3# zIs-0K|F8MA&CTy_HNFcQvrEh33Vzr9etnzZ8jySQI;nMNV>5f9c5CH{c5HECI?4QBlFbK z;2SP*Nw+8=3D@mr<1;KAikQj$2-)HS-cPi67!r#MfjPDiVTQpD@n zoI3g~8tWAFR}x96NW{kK>_!}f!F&xz*0-YA>3^JVN2BHuw~wGuHAF-o=3D_mjy&LA$* zL%~*vfPd&B{|}vKm+qc`mcswhxyz!2!Q3R9MbZPH;IIFx_=3Dm3e|In@eAG-D~{aXP| zg;X7#J4G`C^$kn>2sJsJD(#AC1nPmc9Hlm&)9Y)W8% z!!$H^R7=3Dem%hoe2^in)ywifdDDXrTzM!FwrLl@oAnR7WZsHd&QdNz_y_)K%msim~O zHWt0ysF`p|$v;&>)Jt8r6Rw-LmFKWb_K&9JUfbAc<|)jW+J}02k#g6Mb#c@@qr{{) zO+{j~n4Yu2%Fjs)uVrJ|dYKVja&1sT)2>^vr)-szJb$bJA#_*z0G^oh`A$`jr)S+Q z*8?7`BHyaggZ4GkX>=3D>FnuyvA%+V%lt0f?35V}}eC6&je5+>Bmdc2mJZ>HShRx1zNkBQbXjS?0ZbT-Phyt3PoLL%dFDyL$ceDGg z5aTIoqra7b-b*(Pc>3LLMyd|z#^dQR_w~2_Gpxp@lVyC z0!~}>$N>P0EaRZgxl1$MPZA)j73xikAZa3e%GX6}DudmuwPRzazOn38Su{%1 z&~D!4d|&OGzP<8h43ekJh`l93-ypH4#nWp4K9vP>dw2@t%HThq)$(^Vipi99-R`e_ z{b83^_JD4lgr3Gz&l%p=3Du1>gyYbr`=3DQ1XQ%c?%1JOnWsJgT8s__SYwI)4OJIP_r2Y zlt=3D8=3Dk|z)0`L^pzyAfXfHLdj+)Fz|EI~I3q`eHX_en{+jm~Emz86&h&ng`-dWPQF6 zcKNm=3DAb$&nLDLU=3Djz540|CS*+x@=3Dx+kFEC%phG=3DETWGSVPhbav1y1iwj+<`nG8H|= 5 z?TU$8oSz1mPS4n~rYP>R^tjNdr!rg(C|xEvm6%=3D)JU+Q=3DJLwG(3obl)c8Q~9`Mr59 zZm%s11V?HZnNRTFHb|V5y1X&uxhtrm0RE-b^_9M4t2HLntGRJtSMlNk^$L!VtdMmh zRCL6O469yf7iZRiK~L+yD-1i|{m2uu=3D-5%bt?fmVCGxHKf#Ml*V_&D~>63^`%1w_A z!tXhF9uh{mT*1ledEBS;pgP`O`8OB!4$!!L;?A)Db+q);o;7p@&dd65W7za3|ETTb z3^}NXw`M4IYp50OXzY^M5R{QS;^l@cN#_Pj&rm;pU-fS((tye#FAQ(XkGvsUYt1xi zMhxw^`E4)yVepgD-5BlS>rODf8|wUUuuPV=3D>=3D%`Rn6uYzFN&d}44Ir5pK847dNJkk zG@S7rj?_aT3N4z#?6vyq$?mVS@_0YT4-HDUn{TYU38x=3Dd2j9I2&HxxZibtLI-m~{qxV2?uwY__~g_^8P8 z-*7WQ7+h_NLVr~_$R`~zkT{HSdGGkH~mX{0pJD=3DkKN_J6?dxJ;l1&!K$NM- zvm0ZBe&(tD4wPO7#R_yt&m@C#n%z}@rk8B=3Drq8(mrbH{w`MdpxksWVRotPi%xarZQ z_v0VziAITZ%&|`GrakZ+F~Wm*?_Z>;zQOK(>0kZEQ53Fs_L^4%EdLJoKiXIQ^x<#c zNp*Y_#r$D1+VJ4sgTMBZH{DR+{ogSI!S>wBm5LB5*_pLS&wqAH&G?$%`L8RP&ED?e zSOs!8+(0+krf^7?XtiR$in1i_Aw36m;`1h^yAu@i`I{ljb%V14#WhNzxP9)sJm zC;s5yE)<&D)T{5T4_|Vy7)<6f3%heJgXO%4M{g5>lk83oevqZ8l{O{XK=3D^uq=3DpIlV zre^!$Ph@G-Y$-4(oEQ1|2aa>JNCIM6aOHt z0#8SSm`rJQ(20LbU{n%yPxKE0AUH2{HPG3Dq2^EU+-O!j_;|R2wB7ma$QXkF2n92* z;sG!Byq9|o%46fr^w98m)Ai^&uDA{E-55A|nqv&6{M zC*A)Yg0w_5>TS|}fAW`NP0|Q?xc2TpIw<7|UZITVL)I76?X@i)=3D_aEibuk~N_WEbOT z5;);K_!SQ$Ld8%m-Zi!Z?+ubXo~n7+3ot;N)MUtdhJfGmQN0B7Ky9|fSivg&^&T&C z3J@s;X(ow#20eWtp|d;2-PAdAl=3D{mt?M8w_FJBKH)ox)ax+Z(@uZ$FGwj3N|W{ooC zb(DTLRaCua+|oo@mYzhQ(oMFhmb%d}!L!Rm zjtF|+?7~GZ&R*)rQs(r3Wge!k`2F*_%AxKUip!aaPdGAkU#^HKj5h^dKJ9CwYKWQS z|6AfrNdcx*A>N9EtFS{_T4h3%=3DP}ibf;Ok@HFl2nLzKw3VXw zUxzQ3#Gfg09D4sFK3alSh6zbVk1jrTOgekIDk3K_>GGIQ>&jU|MB96o;}hvLccw1nd`z@v2JmNo?b4n+cw z%+=3DJ1YgIH>E!2;^fb&1!+OkpanNQ$gr<#^ER5(OS^;c9VOkhPkSg!<7s2AkvX7|KGPrpK* z$A+WcV^yGHvhNJy3shC69-ZbyT_0{O}YQK zCIto<{!`=3Dsq2e%z!Tc%iU5oY_{GT%y75O-;jQFt2xOipHsQ)+y;s5V5)jf2S;lPVi zqUksSdC*?;2Q`SeP}qj%Z(=3Dzp>FHfSBov3SY--#XOc}W+3ek_yQjB4Ed=3DZ*lO?-r; z^!-q3MJ$U-6JMrLS*f~G0z`uFkJ5uJDIq|l5vVZr7pnev& zw3jnjmxjw=3D8chYW1o9whs;|6W{feJ|#u0s?ACdFV;E)|9Wk!iiA)K#pQLW)cYn7K) zW=3D4wmcD4dIz4!IAwaYz~jvL3+b}sbOFfG@%>(O@{4;QvczB=3Dj2_Nn}uUR>cN7EDK^iAoN&OubMqMSc_+mKdu8`YH z%c4-kqdK&mcyWRbx|%@(X`u+VPQ}}|r*-j;oU`{~X*He|Gh#kP#U$-`ZLdTWdHpd9 zknuAZrL<{ek2Av~;!H$GPD9b?mOPGLJ+|XLtl~qGh2;CI#wF6ImAiG+6&y&mC{Zqz zf*eH+kUT>V9oU#6okf8gao5#C58!g?-5GHs&%JvIe-;c;&sM-xv)xNv^Ohxnc zI6@DOzDV4N#zzf6NK)|@*b0&8=3DtKMB{ytrTB>fJp28KroLlReq`dlh96?EmQz zTt_T}D`9Clrb8%bP4zr9oxTd*fiI*h;%uy6@z_Y+_I-#W1m?3WK7e^_RZV@6Cu#*aGp?GSkAQ!s36)rsSaynr}nM(?hk zQwh^OLt;d{42q+@Rfv&{hz9_qghw8IVJ73cAS(=3D>7_T)2U}V^tP%HZ|iPSQdm;d@L z$H-O%M_i$ivAXG~dO;R|Xd`A{b8@wO`DO-bCk&Z>ilc?$2E{Rm6Pe|aOyv??F+Pu} zWm56@iP=3DopID%guTk&%f38&rg>iY6RzyvoJdccQoj)P?}oj8KtHXhz;Q@epYEiehe z6W_q_ezY76KPd`?0=3DBxbApaF=3DA5w_nkr6&56e>23sK-ipIv7?VQk1@cKX0*^fhZ9& zE+&c-a5l!XY4C6mERRIvQqVV=3D_4D^o?Ct#+$h&C=3DyFTJn3e#}mY$C{uHoL6X+a{9W z?8YY;nIRcw`l2QV>KK%_B2m1G(f!iBHDzK})qD+1hqFl&@z=3Dm$z%4s1FdL#TqQvw7 z7!$0|^_zUiJ>&i?Dsbc2ak35dgXCu&e)d`*9ok!!ctv=3D`GZBi;e7p8tVo7FEJ$q*+-geK>^u4>hl8(&ol;*LT?kH63$&%x$8 zB4OL7C7$BE{i4^h;+Pk=3DE~Dfgm-l=3DEovSRFi|9PI=3Dp}UdT2s#mR6W!Gh*@7A&S!9c zejDbmX1JkCY>f~}DanE_1@VI4-G6>Qd|4<9H{!rmA?S}`J3H<;+FhrimBoY_n58(G z#R`6x>#nkoD9d=3DJkA+;MgJ;6P=3DYMJQmQvGykmRIm}0(isUi^r0z+E5$9lEa;|i zs9Buyp>XhZ6HJ*=3DCzf$(?0c>ZoiKX$+4xoZgv?BhoQpMA1z$>h?YrUp+B|`Sw^S=3DO z)J*MDS(ZpZxn8!``ToJGSsvfdp>?HX@XF1vv9~;{ z__SVNut-eV{zVCQu9+DEksGTgs`Hib%aZ%1jhv0J8RZ{WJRcgJzaz&AaS<@^5^*`23^1PI75%n=3Dz-T_p$tZL$x|Gk0~#~fY(80O*tR#R7G{s8-IuH4u9WL12%HYpD9>pIdPjzWE+)jDF#2T4lHzn?IZRr}ek$g? z;lo%fWrO_E-jjEk6kG=3Dj#Kshxj9#4Pk*>Ufv*yEi>ir#$XAX7DHhM%QjX4K#A#W7*0NHq{OFMN1*4A5n^}(xW1ev_ zNU-}*dv*Ha?Ru>lNpup1D&*peph+14zur4j87dorjmhbqhD>=3DbCLg!iWOMQk^m@Dy6G?RRjIBc1p&YiV z61z(0^j&+&Y|SgJy|~kD_xm1oJ9J3Zib|Me=3DU#(BALd_?c(XCEus)7+&gM&X|v!Qa6PeLK)h1m;*|VRAo+mwP(YUp z(E4IYvUA@LsRH*4m=3D9q2-V%@L8%<$F;Y{@35j9J$TQkPo7Rv!m@r_`n}Unm7wix zk9KTTMcAzrHOi1P5m+bd!{xKrKP}DmRfn+VU8}3Fkl5$BlZ|x)qgA2U})Z5$_s}Dc-)r$}Ra- zpFI@5m&@A0j*jt^OvqykKbFp*tCs6axKwp9^bqWq%PJ9Ab}pltcg`Wk1^k@DM)eji nHA(UKE`@8|JIuc!w4vV)+)TUxL=3DJ2>lBpW1+A0M~Ho^Y|9YC~` literal 0 HcmV?d00001 diff --git a/Documentation/admin-guide/mm/damon/freqmine_wss_sz.png b/Doc= umentation/admin-guide/mm/damon/freqmine_wss_sz.png new file mode 100644 index 0000000000000000000000000000000000000000..83cb1bd67734b340be690dd3a= 5a058069d0286cd GIT binary patch literal 5589 zcmb_gc|4SF*S}|m>|0EhY*AT@s4S@&TVyW`Su*(*SqfQ3l(|#LQlteLON%9pB{8zj zRKm|vk|i1j$v&p+S?0Nip7-;-@AJOTU$4)Y`#$GfXSvRGUFUqSiM6pZ;p3Lz1^|HX zl&R4f0Dyx40HbjtAP!tWwFe?JY%J`InM@|c$f@K zMa`xr*`V!x)Ibpc6cGW@&<6yuWDv^)S@ENp+0X~e%=3DTxJ$xLWIOmaIZlfh(USGH4_ zly-9IbtXESDO+D(KQuJt=3DH@0NBZESrva+%o8X5=3D^m?CJ-T>hqK0|9{lJ?kI#NQHkl z0EnDAWu$KxN|>ecguGA|i8?PHGvA^u7_@0S0hh^nya?!9jWUVB53J6d{{6~1>Wozv zWN7Qp21XUX81tD(z8s9v%21+me|tPBW1^p9u7zNdp4v_e@+}3Nn;oiqpA|grKuSaf z9}P#GuAnTzRXF{(xUSa1cj zy@!}E)V64PJGpN(=3D`V@UOJgU=3D-ni-=3D21G>uT~ra@!SLBpm6~fUD`R4xn;a5TmLy>T zJwFcXisgKG-Y*{=3DQvW_3(MNPEIWP~JVo{u(U9#T~eGscu`?^EW)Llx^CsQW;B2MjV zW@MtGoth6Ye#Z-^b16mWL#OO_ZLq}|_mh8x$ED%cE&1$U_%E&2{CVe#y{L`HXlni& z{JBA^N+}VwJABgv6n)!&?5l~LFn8M4sJr#E_R6?U!z!_gA~XJasz=3DQ$C@<5m(Y3R> z6-RcHi`gW&_gQaSyt5zwc25O)5sdzvjQj$xS;=3D z)Yc&yO1B`Y3X#3Mn6xFTd3{fuJXm2aMRp}t6mulb5phdihl2Ru`R-+9I<8z=3DYg#0%80t7=3DWK_TB$9=3D>Jw{N%pcQ zxkqX3RRTp+YWJLiy>J}n%MqZ}J3e*JuMJ<`8HXHn{RPnDkw-24LLlj$T{>2j0l>R! zE)mjruZjm`6zpZtamW$$5>Rch=3DLCy;a|RYPM*Cv7^PYQDU#k=3Dv^)|H_ri8g8nugHc zWswCpEi;&@a*8^O(9Ul`ml;U5pt!I6ke6-b{;B*I5Hl7!j(|o;n#z!Gl)d%fh4=3Dw9 z>oCz)C?Wq#4*qR=3DkqGVAJz@j=3DLvCq1U`5VYApH5m8#aG047p{%A=3D(ijdM&QxJw38C zYHqs+4Cdj%t}Vs;a$>1sx3lvZz|@BfQ2e%REDc!72!A-S3i@4PE4@O21*wSk6;GCk zM+%br**R=3D1<2hZCyl_Vn4Uiy3B!xHY+1^@$j6@>(z7Dnn{DjE+2L3?BVtguHwCwL_ zpyxn{w2i`*CTGZ)h0$}!5r21m ztOi@D_x)RfxRLeiKP;jF%mHPv&OW#MR^9inL2$X-^~f-2M|_gA5GU$g@1uue#H+>}`6i4!g-0 zo)d~jns}dbixaFda^TQ3dxMihPFex7L&sT^`>Esb?0buM*9%3jW0eLNMVJ-UF7=3D2b z5&A5B!kwO*C0SjS1(+%}e^*uLyyJv*_(oA2>m65Jmy0l=3DVq|%?QjbH=3DX_E`P?MD1N zddr=3DLhm}@B@ggcky>6k)1?R+37|xZ99BBq5ftp$w zLbT4H%oH+^dufz*bU!%l9n~b<*|cU@F#?~Ev0-kIe*AsA&^zj*dz4!tawqvo;|@l^ zj|)+D1krh`=3D7q_Ei@&(A?N#y;#!--9B%?QaSR|qA`sQ1V>1D?~$QCw^QUL02N>DE` zX1CG05?k5Y<0%g!VJ3FAA9tahhbEjc;H6Pyyk;5{$Xk;NlPh&R!YjWS8UG0X_CzZ&!wcPVIb}oP>#!S}$m_{PWgh$Eu1j%8>?CPF5F9NO;NkgrrGR=3DZo z0{Mn2`=3DXwn`Fw-Gg{|~72)Ss$iLJ|BzgY6{`^f_#y9>1{Mx=3Dk=3DDCNRJi4}8E^0KK1 zdUbF5`7|CcM5}WCldDR>&TQ6j?E)}a~|%*jfRD4Ee1yJJ3dHK=3DdR<)2I)ZhlunGoQi2Va}%*$xO&`t#+ql zn#Z~+QCsf~5RLO=3Dj2H#JKMd6wAsid;Il?7fctJE&V(t*`K$`h7B`|02#`7u>(xqHa zxCujX&oPomt=3DJhFubg9qIc3eB%q!HgK+Mno)s<-K6H?ja$fZRIEr-0|f;{96`S z5-lo`0@UIj!|(9ZUN;A(sE9#; zAZ7ws+Pt=3DK@cYXJN&)rwVTe_?;)5oA4z$_)-K1|xf|`AQtiGwIuj%H3&nhw6J&J_$ z5RnX8cSb&a3W0?H*kO6(Rjq`I*-=3DrKDMid3V;5lHksE&ee!c$k=3D3etMgSKbZ6pDcS z-PWr1#biYcr&@%xdXRO!X|KqkIHddQFPvQTL7K5nwI=3D`ph?XfJbuBFkLQ%Pnd-nf? z@#6Qte?qZ7Xvs9?jq|| z7p6ZHOs2?!_HhE)71aUu5{@#03hz1m?Q@5BvXz>#3GO4#i-vJa&z-!a!!AD<3-JHF zqPxf8z~_^IPmmn=3Dg^O~{6W(W|z~-63OM7&I#2?V%g4ln#8OK(7QUShFwdqsBy9ZjJ zSYWDN(;+$u=3D$V+2eL+&Si52c(_rF9uV`0Ry`kUbziZq2xdJBG@dT6;?-JpVt_VN8C zTCJDvY;_lETiU{8sH+fyHNd~%wnu(IyXUDUY++c_wM4jb9NHZa74#Q2qx68=3DGH$ru zKgu%4B4#-GW^ij#8-iYsQ&S@6fN`EB8N!~BimlKWp8zr+;|4ViD=3D z4n9;dG6nSpi0*3@$;Ro6;=3DF6BPE?H0o7v}RK&Zaxvd3<;cO&(4R=3Dis*3;NxI#@5K=3D z7gU7EO;v9>lAHpxLawilkI~&7@+d5l6hf%qj~A`+cJVG~Hii(3GIARPzC_~OVnzkj z{6rHM3-W+7D)ijJTs8bMzqbZdCF%z=3DBQjK~L-pX0p!fnCWNCTf{EiPb$}EQ^)*R?9 zxVzHQq*=3DPv4cBi&n>=3Dw_a zuPoHO!a5-(e};Xmr*e*N-!B{_VW&SYoB#0@4cVIe`C!X~8dOex|2e-5ahQ=3DliWe=3Dp zZHpDi(68-EEe%*P_A zi4q7vSif-sgyDaMZyuZ+8Ej-V;h$?6>K*K^MNK{sG#@-k7GH@wSZRCL03xD_fROpL zN07qL`olju_E85qR-Zc#Te3-RK3RADcJP5JbjqW$1n>{qT)#o^%yIbmZ5O6aT1{WV za#au{<2eu0I~6irDteTCVmo7npAgww>ro3GIHe$7)jC$eRWle>*m?;c{*z=3DPW8P1G zi{(r~2%3bjAblmL?BB7ex0Wuj>e2b(NYP+GWGmT?S@AM3Bc;s=3D6X3Mb%vjcTLao;S z!yA2Rv%_`s+SXH>6QJd|~VOO{(Fd}(pZ8MWW-+bmvf`&|NZ zbpm&)qv7rX1hcAZ%nogklBm$O!l~g41;#lVc(&G=3DRMvhj`D;oP>!MqIY-IqST5bZ| z?@TV64N#kqSkRURc?~W|HJLHh8+QfFv>#6Rrfw=3DA z3zxDsYUM-3=3DC_dw0{;Aw!E64tFYYGpHvC!z>xDhz} zqWrD^Hg^`~=3Dh9k!%M&Q~#dFgF2i~md{+&Ls@F2jm%;lK)7j>3nVz5KmjY_FYqt+C| z+-*xZH|qQ*oP8ous~ltKke0Q+YKp&z~@D4 z%Nbq@n^$%H0M}7sO9euUvok{Q>NwzWTopHZ@%rP81jE}!dDtUYFrNK4kIXmi&WbEQ zms;7SuF+~0y?QNXPp{iEXLzb7N1&|6WhPVX_IJrAqe- z1h+&}1$3FUCgai>>V@*OUpr+@eRN65C+9NHE+?k*jV|W~e~PA$ag1hF=3DF!H@LD|2& zzXkSoAxI&cnHgD)gG}-POc@VtejhsN&?)-l^v|2$d)kJarELb8*o2W^t@L->qFWLR z<)$wMk~0xi%$v?we^GKbWNb}7o0<9XE1h#?o#F#D5?yXGKTJvZzlrooVb8DK5a5S5 zcgx7F-kjCoz7jOhp)-&W*^8Q3|J@ehC8A&aG@jFa-3N7*2`kir;M$PiVd)l{{XB3Z3+MY literal 0 HcmV?d00001 diff --git a/Documentation/admin-guide/mm/damon/freqmine_wss_time.png b/D= ocumentation/admin-guide/mm/damon/freqmine_wss_time.png new file mode 100644 index 0000000000000000000000000000000000000000..a61ded5c5cb9d018109fee397= 1ef3620b6126eef GIT binary patch literal 6550 zcmdUTc|25a`~RF{hMBPqV+%2|RkAPHVhEA7KDH1eg``kI6vrS-l(J`=3DWJ`#T7Q3+~ zyA~~$21T+oA1b@~p3(R9d!Fa@d;a|W^PAV1Irn|t*L~mDeO>SCJ=3DZz$HdeNo!YIG8T03iedprKh{3CMkD1n#KVSlAjf7z|iKp-`ANDgps9s0apPX^Bcibax{R z&QCI^1O!B+LJTDmbPItXLI?)Lh+w=3DeI(QJCN~8_J^Mr+kRaI3jEiFAgJtHC_va+&j zYHFxdYF}U9*RNmUE$AvL$xF!XCG?9TA7-sOLFd}T|5X3M>Y(SUi(nIMpiA^sq)ZS$O^=3Dpg* z5g0{?CGt`#;>|c^JlTe5cR>;2LqL3d5D~s0h(v`*42WqzkwJkk5`*H;pi&v|c^K6G z7kP9Bo$|cDmC@Qyt-8)2QW%8h=3DH}0zKfAfPiHnQt>+9#|=3DeM-9QwA0ter%x|w3|A~Q z|5^)GH(v(Hm8ZU{IDwVEGwt2!*_P2?6xmk$r9H^ph%Ihs>fJ}5+bZ5aw-nfpP4ay3 zwsIMOA|?fG%xf=3DQoqQfYciDF@_kb;TrdGc?;=3Du>-@`K0E?CK6TPrT}tnW?z@^Via| zIkGTrE5ls19@}M6@%y#SQb|YyRMT`MLx>HD@z;^KJ|EqS~schZ( zt)3ZTmrwO+(KItFtN1c3b0WVT`lG(YVbP$ls@$@Da;$7#Yrr!sC30|~kuT$6;754o z{=3DC+f(uP0+a8-P&-kgos<{yc?;`pZU_I|ksY!n{$s9#BVTLKHN9RJ7ROHW_oo`I>FH zCi=3DsXj!2(O$Mph{kEOesr6^mI#YvyG-DM>%kw4Z+9kd@a;<4bc1#gtsuJb7=3D*o*p5 zCm!(oAIYZGGUo()dvrEEb^b*P^kuH7(@{O;%2_UUGd_u*4G1e)`s+Hp4GzbL0?7qL zq75G!od5y`CkTZ7SR~Ro5~A$?e>wP_XGo+R2mZD?0)|HM8&W489V7Zx8NznGZNamycRRAopq&@pdL7^tdMCy@EccptzmsZ0RZij5RJ1K3e;@~{+C z>;TLB{e9Unv7{K<-YJ{Y$LkhFnL2DI0ZZynOCnMZ$@Qfb>+rmKpQ8l*=3Ds3IBZ(-&* z_Y^5c`_?sLm7zb&)vC5ubQ~#$YCT@E2FK+;eNc26U=3D#YF^3%f+O%#py+?uvRAIGXrXIo98K@ky}6;^6aRTVcmeJ1Zg$;k<{BQgYObo z3SQST_rv?A-x3B~hLKT5(U|U{ypFfI9SIz8U{82m(G8He6bX$-MS9z$5&?$;u3-!B zqhG-ie29)fijQ`S@?aVXy~cv%GP>h=3D2-mJe;E7}z9$PR7nEAGg5!vI!d%{B^a$IoO zNO;}(BQr0CR5A1}7_UuuOJ@>*j@WO+9S!RK%F?@)3JcaWi0NBP3foU#@c>6W$Y;W^www zs~@P)Q`Wt!Qh;O;IV08F3h+zT3ZSYddQ_g)vi8GuS1#Y>&2o+6NbM58rt`@X%N zBiPF0GhsbnaHjJSDPV&ixsXhE46tUUJOOVvL=3Di!zH}a17{mcR0IiYb4mbA)cSlR@aq)56hR<_;WDxfbUb6it9eI!fuPsXDJKA zG}}hKVg`B`bAoerHeGcO3?DMeMG7zJb}rrP4IDD0UTR?|=3DLTQ0*R?!699-Hnnn(Hs zpiVF~plv}$MYSxj(BHhr!OA94aoB)m@x+>^X9Ik|1$U^{LGJ=3D(^_^}t=3DY>)L>zf@9i1984oczpc>71=3D0VTJNnURmQ5r~EpPFP zAH?Cz53)M)4|%&4D?yt5KO(NHjkF7N!{EDnD~6al&wJXZjYSndU|hc`)LZZ_a@wN- z=3DNKbuW=3DLxuQEjuJS8aWV6oad>GevHRDX#6+IoK~y;lJ9)#djGGPuF<0(yb6bjSACj z)5sad4k~fZx{sb1j!2F(lDbpkXYP^h^<`*kZmbduUY)I8Rrb~lIpWvqkxz$NE zLvJC)q*P;N&}kUk)UI^6C<;n4l%jlRyQ2(#8pdw34DW0aNDpl=3DrQ$Xb&F{ssh7#;U zSmm~aE8tVJ=3DR!}?RX=3Dk)f=3D}jM8SNW!*)oM27>)5IXv^ZNPQPYf^9fn4b00!R`4tNK zi^guL78+9R=3DgKg85s~%~Ow?-NvVrAr`o(Y!^alkgP@zT)b+(Qp8d@~GqBBzF*Xy=3Dy ze4Gqb7zW8Z+C|jGx)VrY$>{xWPp=3DpaqBbE|Z6EEB;8pzju3hJPSE9_fzg2F2V}-$d zdjCt{TA!6Nks5SO=3DrbS7FvR+w!It2$(TsQ$3H2|lRT}giHLK-gYc@PcOAy9gk2IuC z$0u7%mPwG>ZQ%Uu)8)B0|HuNiQO(l7^1dG%sKK5D)z!gJaVDD*+f2>$2(T;Brh5*& zVng~02m=3DQsrL%U_TNI*TNERIc7cY z4_8aTdvY4BBqvz|Eu9G>6Ac7hm3yfVRl@i3K7~Mhqh1{U>IA9|y{5ahw-gWt+GS;( zIxnnLeA*P1TWVrIbNn!8DIgMEDk!S0b@zKTy#miG2y^Ch%ty{g?kV`JZnc?UmvT*3 zAp1)^H2Efp&D!dzT-QU7&bbx+6)sQ%IlK)$NWouJCx{d<2~a~qIF2hf$6I+8^0>|< zBbma(h0C+qOBqvt#^+{kd&AQI`iF!a?njGUDVB|bPi2GYjut)@zI`_SEdt1n`PTX~ zi~BoyY~!SIqs62PDms$Wz?S9mc5H8?m)5mt*k`LQbS3Xi0$~lA1htK6sV7)W!rWSe zJA!ZfQk(d_>$O&|R2r`8RClr0U%@B8CfK2Y(V}~^WCjN5n?5%=3D=3D`(j6jr2wD1Kxjh z!NwmriG+2%kuI6ZHT%jeCW%zf2Rt@k?q>(H{0aP^5YzQ&O)n7$#uq%8?40VaX#XA{%dzf6ub{A^( zO4K%eU+fsjG)e3ZlB4B`bjZoua5bmrD1r3lE)Z5)%S!&`OP-G6Z)Qj1FT|Qs>0X9~ zj=3DxAhBgp)D;>cj>r{GpLZm6b~zm1nIHO=3D>%ubyMHl>H#Ile*fW>H3qJ1-+}bd#IzGRDPp=3Dk^ z6L5|;CCa;X)evDNBTz+SIqtO)3%s8x@g181C%QT63S6pc14)ssVV%IPTs}WjQ0zpZ zP$D!NhW>y(n~TeOCJ4eqWvqz@di9wH4&lng!|VMkVNzL9jkn-3*tyUE;BRIY8OFl=3D zLxOTk%gV)zhmqwu_mMdtuU_7!?hYN~_2+Q_jxuvo04`e<1WH&r68UHOajf=3DmIorI! z??nCP81Mts3#Xy@9ts0?Ijr)eKxXFddxtJ92&k6d=3DKvD`+txANHlag`CCeSreJnm~){X0CNV@1S{&A zV}KxmBxx&6(#2JskeOfgA`l3OKZSg1zJSt%rDHA@Erdc^ppc0mSUeSs3as4AhHH@x znGQy6HGXHWQsi$rMAJX0jLyWOzb!0t{Og{ugKGQ{Q+t6x0_p|mE3pk&p^<0XB@lfT z0lEfXLnAFZe_+?b!`EDHF|2A405^el^_A+5G@+~d;%?K1MaH9kYu(5T4At(&RUujq z(vZMB#Tj^a42xHyJ>>vproB+t-B5rzJp~lo9(oK7HbKyS*f$yq0u{zM zC4JnG6B1Zq`7a&)gOnd0N2$C|_tP#`wXs_x8_UBgV|2(uSfuXx6U3Oqj*}=3DSfKh~l zyLRxSMjJb>3RA|De1{yL<(3}9D%S{eB1IS|a*VZeX~c5bhe+5`A-oPc^R^H2tx3hD zUSWZ9dfU&rb_1$YsR@zOA)%8^u2E_IkVuZPF>=3DdKwqEW#um6cpOsUN?NXftCAW)~p zmI$u|fEosNV2`|syYHWUeR%yRoJUGYF_+XD&_KQq0KECfg1AA7iF$ouMN!*vGR7LT z@>3*`B)&faUV0=3D;!y**=3D1M(fZRy2sY!5>>=3D>Rz6+;%AbFfjYR8VhZFs3_i`=3Diq~T= W zoruIuJNQ+M4SKi9kL=3D?@K6N8f7ht)u0Kf>teiXzFh2OO%3e^I@q99I52krtu%}NSr zm6w65;@HN9SZQJaIacr!Y)BA+vJHYWGdsH5NkORVDf9MJFDD{3kVu_pM`vc0fL3oh z%D5E2;^`Y@R)E~j@tUEloZe0KU0&I`&IZ(dTM1CRG`5?8lZk7Qhv1@YjV>P2ip%01QXB?0@Ijero=3DLG4mc~hcsHz6(f zv1-}(>iWuhujv6wi$^MCUvP#I2RS_P;iY$LHXZskdKih;R3B&GF$SDV8R3T18s$(K z+WW3sCe$zd0?59eRXGXu5$Fw3hQ@cAS{^7zD-yesx-LV@sF$pO!YngC38FJIWYuX; zFdxW~)~KBO&3vE1{@h`j!K zwa$FD-3gs}#~a8!hkpkBK5QjJ8QsKM9}#Skp_H7dRk+{WL$Ce(`^M_rV&}B29=3DE+A z(J-epKm-7_#nqjtwMXxpP$#cAj60YemZvy5--32tg^H&u94ES^Y&UO7j4T~f=3Dl@iC z;hZ1psn5$-HHUcjMU3L0`Ob*0b`ij_v+3S*W}uF zE`+-Me7+nSNx#Qmv%HmAKHj!blRb8)reQWbLH6UvYbFcpXNzWg16RwCoxVV@$A^X>D;gN`;!H%(gYe}`?&GO2kr$8C;hD1GOCLDOt!(u+G@ z4ulD^_W2M-(gXwe%>I4zMS~~Zl=3D|Z0Jf2>RVIBXAs=3D@8{usz^0BpZ)K zZq@#+3bdFcnRn(Y3=3D}l@(J$SKJup$xI6Z9@Y!!+P+&r)p6!F_Hu%TpjzM;9on(B9A z`rlqRB5q8zSn8YgJS3f&Da`8gEw+eAvq%iwQ@)(`-7sZGyJoR6MfGC6$2Rx3%G;*Z zT#s?7S%=3D!trl~t^>F-;H4bxhCTp4+il4cQUz2T#8vVtn*+$FXc8+3W?e!Ckjwpsqw z-a!28*-uH4hghz--K#$qFSTt+RlV5ev3HR1BYB;}AwT0_$|)`!T~(ZTMZWWgND$ z(eqZ_itMIN@`E#Kc~bZ{#)OOHEVFd{IYQ^oW^6X;HU3S&yve<@e=3DiN{l-r7*(L)x# zA_N@+O~G`QStR{iaf?X@c^KZ9=3Dhh`BIH{Ih@suVkmtUg~~U%say_Uhjw^B?WCwuiG9 zJ=3DcHu?&N~slJ4fV=3D(qoh5@Ymu;(9X{sZajdI@KOw2RUnaoaRoCv`8eIYluV@zWw2~ z($`g<7lG^IZc59`?A2w^LtlP5HJ>GG+;8GJ^Xx{G*+ifFb8R^bJ!g&4>C3eDLY<9E zc9}f%U#^1{XAZae%$XNlopG^5QTrcyH7IB6gi3BxN9!h}p3GV}&%WTFd1pU7^4nXo zn>^gjOV3$t7qF54T2>$|5Ttuu?vC0*VRTctt{BTmt(>i11H?O??A9R~?o8c&XGi>X z@sN&+7(>HqA!Mn2BShU9|5bXR0!<$Imw z)5PGI0XA?)?VldSuzHP+3slu>C9F@E)4a(MqLSovsHOzeu3X4XUJ$rxt{!Tcx^l{5iNkpx8 zB*pP9B43h{*V!pY5qpo9{ES!59-W3ZH{{nn)&@^K2!8vKL=3Df)q@Oi>G5N#en5W6z4 nCJqFsf@_N2M!^4u4$A+^o!(TdT%_+C%@mm#TNzavlF0u9cBXpq literal 0 HcmV?d00001 diff --git a/Documentation/admin-guide/mm/damon/guide.rst b/Documentation= /admin-guide/mm/damon/guide.rst new file mode 100644 index 000000000000..15adbe2b4331 --- /dev/null +++ b/Documentation/admin-guide/mm/damon/guide.rst @@ -0,0 +1,194 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Optimization Guide +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +This document helps you estimating the amount of benefit that you could = get +from DAMON-based optimizations, and describes how you could achieve it. = You +are assumed to already read :doc:`start`. + + +Check The Signs +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +No optimization can provide same extent of benefit to every case. There= fore +you should first guess how much improvements you could get using DAMON. = If +some of below conditions match your situation, you could consider using = DAMON. + +- *Low IPC and High Cache Miss Ratios.* Low IPC means most of the CPU t= ime is + spent waiting for the completion of time-consuming operations such as = memory + access, while high cache miss ratios mean the caches don't help it wel= l. + DAMON is not for cache level optimization, but DRAM level. However, + improving DRAM management will also help this case by reducing the mem= ory + operation latency. +- *Memory Over-commitment and Unknown Users.* If you are doing memory + overcommitment and you cannot control every user of your system, a mem= ory + bank run could happen at any time. You can estimate when it will happ= en + based on DAMON's monitoring results and act earlier to avoid or deal b= etter + with the crisis. +- *Frequent Memory Pressure.* Frequent memory pressure means your syste= m has + wrong configurations or memory hogs. DAMON will help you find the rig= ht + configuration and/or the criminals. +- *Heterogeneous Memory System.* If your system is utilizing memory dev= ices + that placed between DRAM and traditional hard disks, such as non-volat= ile + memory or fast SSDs, DAMON could help you utilizing the devices more + efficiently. + + +Profile +=3D=3D=3D=3D=3D=3D=3D + +If you found some positive signals, you could start by profiling your wo= rkloads +using DAMON. Find major workloads on your systems and analyze their dat= a +access pattern to find something wrong or can be improved. The DAMON us= er +space tool (``damo``) will be useful for this. + +We recommend you to start from working set size distribution check using= ``damo +report wss``. If the distribution is ununiform or quite different from = what +you estimated, you could consider `Memory Configuration`_ optimization. + +Then, review the overall access pattern in heatmap form using ``damo rep= ort +heats``. If it shows a simple pattern consists of a small number of mem= ory +regions having high contrast of access temperature, you could consider m= anual +`Program Modification`_. + +If you still want to absorb more benefits, you should develop `Personali= zed +DAMON Application`_ for your special case. + +You don't need to take only one approach among the above plans, but you = could +use multiple of the above approaches to maximize the benefit. + + +Optimize +=3D=3D=3D=3D=3D=3D=3D=3D + +If the profiling result also says it's worth trying some optimization, y= ou +could consider below approaches. Note that some of the below approaches= assume +that your systems are configured with swap devices or other types of aux= iliary +memory so that you don't strictly required to accommodate the whole work= ing set +in the main memory. Most of the detailed optimization should be made on= your +concrete understanding of your memory devices. + + +Memory Configuration +-------------------- + +No more no less, DRAM should be large enough to accommodate only importa= nt +working sets, because DRAM is highly performance critical but expensive = and +heavily consumes the power. However, knowing the size of the real impor= tant +working sets is difficult. As a consequence, people usually equips +unnecessarily large or too small DRAM. Many problems stem from such wro= ng +configurations. + +Using the working set size distribution report provided by ``damo report= wss``, +you can know the appropriate DRAM size for you. For example, roughly sp= eaking, +if you worry about only 95 percentile latency, you don't need to equip D= RAM of +a size larger than 95 percentile working set size. + +Let's see a real example. Below are the heatmap and the working set siz= e +distributions/changes of ``freqmine`` workload in PARSEC3 benchmark suit= e. The +working set size spikes up to 180 MiB, but keeps smaller than 50 MiB for= more +than 95% of the time. Even though you give only 50 MiB of memory space = to the +workload, it will work well for 95% of the time. Meanwhile, you can sav= e the +130 MiB of memory space. + +.. list-table:: + + * - .. kernel-figure:: freqmine_heatmap.png + :alt: the access pattern in heatmap format + :align: center + + The access pattern in heatmap format. + + - .. kernel-figure:: freqmine_wss_sz.png + :alt: the distribution of working set size + :align: center + + The distribution of working set size. + + - .. kernel-figure:: freqmine_wss_time.png + :alt: the chronological changes of working set size + :align: center + + The chronological changes of working set size. + + +Program Modification +-------------------- + +If the data access pattern heatmap plotted by ``damo report heats`` is q= uite +simple so that you can understand how the things are going in the worklo= ad with +your human eye, you could manually optimize the memory management. + +For example, suppose that the workload has two big memory object but onl= y one +object is frequently accessed while the other one is only occasionally +accessed. Then, you could modify the program source code to keep the ho= t +object in the main memory by invoking ``mlock()`` or ``madvise()`` with +``MADV_WILLNEED``. Or, you could proactively evict the cold object usin= g +``madvise()`` with ``MADV_COLD`` or ``MADV_PAGEOUT``. Using both togeth= er +would be also worthy. + +A research work [1]_ using the ``mlock()`` achieved up to 2.55x performa= nce +speedup. + +Let's see another realistic example access pattern for this kind of +optimizations. Below are the visualized access patterns of streamcluste= r +workload in PARSEC3 benchmark suite. We can easily identify the 100 MiB= sized +hot object. + +.. list-table:: + + * - .. kernel-figure:: streamcluster_heatmap.png + :alt: the access pattern in heatmap format + :align: center + + The access pattern in heatmap format. + + - .. kernel-figure:: streamcluster_wss_sz.png + :alt: the distribution of working set size + :align: center + + The distribution of working set size. + + - .. kernel-figure:: streamcluster_wss_time.png + :alt: the chronological changes of working set size + :align: center + + The chronological changes of working set size. + + +Personalized DAMON Application +------------------------------ + +Above approaches will work well for many general cases, but would not en= ough +for some special cases. + +If this is the case, it might be the time to forget the comfortable use = of the +user space tool and dive into the debugfs interface (refer to :doc:`usag= e` for +the detail) of DAMON. Using the interface, you can control the DAMON mo= re +flexibly. Therefore, you can write your personalized DAMON application = that +controls the monitoring via the debugfs interface, analyzes the result, = and +applies complex optimizations itself. Using this, you can make more cre= ative +and wise optimizations. + +If you are a kernel space programmer, writing kernel space DAMON applica= tions +using the API (refer to :doc:`api` for more detail) would be also an opt= ion. + + +Reference Practices +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Referencing previously done successful practices could help you getting = the +sense for this kind of optimizations. There is an academic paper [1]_ +reporting the visualized access pattern and manual `Program +Modification`_ results for a number of realistic workloads. You can als= o get +the visualized access patterns [3]_ [4]_ [5]_ and automated DAMON-based +memory operations results for other realistic workloads that collected w= ith +latest version of DAMON [2]_. + +.. [1] https://dl.acm.org/doi/10.1145/3366626.3368125 +.. [2] https://damonitor.github.io/test/result/perf/latest/html/ +.. [3] https://damonitor.github.io/test/result/visual/latest/rec.heatmap= .1.png.html +.. [4] https://damonitor.github.io/test/result/visual/latest/rec.wss_sz.= png.html +.. [5] https://damonitor.github.io/test/result/visual/latest/rec.wss_tim= e.png.html diff --git a/Documentation/admin-guide/mm/damon/index.rst b/Documentation= /admin-guide/mm/damon/index.rst new file mode 100644 index 000000000000..c6e657f8e90c --- /dev/null +++ b/Documentation/admin-guide/mm/damon/index.rst @@ -0,0 +1,35 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D +DAMON: Data Access MONitor +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D + +DAMON is a data access monitoring framework subsystem for the Linux kern= el. +The core mechanisms of DAMON (refer to :doc:`mechanisms` for the detail)= make +it + + - *accurate* (the monitoring output is useful enough for DRAM level mem= ory + management; It might not appropriate for CPU Cache levels, though), + - *light-weight* (the monitoring overhead is low enough to be applied o= nline), + and + - *scalable* (the upper-bound of the overhead is in constant range rega= rdless + of the size of target workloads). + +Using this framework, therefore, the kernel's memory management mechanis= ms can +make advanced decisions. Experimental memory management optimization wo= rks +that incurring high data accesses monitoring overhead could implemented = again. +In user space, meanwhile, users who have some special workloads can writ= e +personalized applications for better understanding and optimizations of = their +workloads and systems. + +.. toctree:: + :maxdepth: 2 + + start + guide + usage + api + faq + mechanisms + eval + plans diff --git a/Documentation/admin-guide/mm/damon/mechanisms.rst b/Document= ation/admin-guide/mm/damon/mechanisms.rst new file mode 100644 index 000000000000..16066477bb2c --- /dev/null +++ b/Documentation/admin-guide/mm/damon/mechanisms.rst @@ -0,0 +1,159 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Mechanisms +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Address Space Specific Low Level Access Monitoring Primitives +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The target address space independent core logics of DAMON, which mainly +controls the accuracy/overhead of the monitoring, are separated from the +address space dependent low level access monitoring primitives. The pri= mitives +are defined in two parts as below. + +1. Identification of the monitoring target address range for the address= space. +2. Access check of specific address range in the target space. + +For flexible support of various address spaces, DAMON's API further expo= rt an +interface for configuration of the primitives. Therefore, anyone can co= nfigure +DAMON with appropriate implementations for their use cases and even use = their +own implementation if necessary. In this way, DAMON can be expanded for = any +address space while keeping the optimized performance. For example, phy= sical +memory, virtual memory, swap space, those for specific processes, NUMA n= odes, +files, and backing devices would be supportable. Also, if some architec= tures +or kernel module support special access check primitives for specific ad= dress +space, those will be easily configurable. + +DAMON currently provides an implementation of the primitives for the vir= tual +address space. It uses VMA for the target address range identification = and PTE +Accessed bit for the access check. + +Below four sections describe the address independent core mechanisms and= the +five knobs for tuning, that is, ``sampling interval``, ``aggregation +interval``, ``regions update interval``, ``minimum number of regions``, = and +``maximum number of regions``. After that, more details about the DAMON= 's +reference implementation of the primitives for the virtual address space +follows. + + +Basic Access Check +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The output of DAMON says what pages are how frequently accessed for a gi= ven +duration. The resolution of the access frequency is controlled by setti= ng +``sampling interval`` and ``aggregation interval``. In detail, DAMON ch= ecks +access to each page per ``sampling interval`` and aggregates the results= . In +other words, counts the number of the accesses to each page. After each +``aggregation interval`` passes, DAMON calls callback functions that pre= viously +registered by users so that users can read the aggregated results and th= en +clears the results. This can be described in below simple pseudo-code:: + + while monitoring_on: + for page in monitoring_target: + if accessed(page): + nr_accesses[page] +=3D 1 + if time() % aggregation_interval =3D=3D 0: + for callback in user_registered_callbacks: + callback(monitoring_target, nr_accesses) + for page in monitoring_target: + nr_accesses[page] =3D 0 + sleep(sampling interval) + +The monitoring overhead of this mechanism will arbitrarily increase as t= he +size of the target workload grows. + + +Region Based Sampling +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +To avoid the unbounded increase of the overhead, DAMON groups adjacent p= ages +that assumed to have the same access frequencies into a region. As long= as the +assumption (pages in a region have the same access frequencies) is kept,= only +one page in the region is required to be checked. Thus, for each ``samp= ling +interval``, DAMON randomly picks one page in each region, waits for one +``sampling interval``, checks whether the page is accessed meanwhile, an= d +increases the access frequency of the region if so. Therefore, the moni= toring +overhead is controllable by setting the number of regions. DAMON allows= users +to set the minimum and the maximum number of regions for the trade-off. + +This scheme, however, cannot preserve the quality of the output if the +assumption is not guaranteed. + + +Adaptive Regions Adjustment +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D + +At the beginning of the monitoring, DAMON constructs the initial regions= by +evenly splitting the monitoring target memory region into the user-speci= fied +minimum number of regions. In this initial state, the assumption is nor= mally +not kept and therefore the quality would be low. To keep the assumption= as +much as possible, DAMON adaptively merges and splits each region based o= n their +access frequency. + +For each ``aggregation interval``, it compares the access frequencies of +adjacent regions and merges those if the frequency difference is small. = Then, +after it reports and clears the aggregated access frequency of each regi= on, it +splits each region into two or three regions if the total number of regi= ons +will not exceed the user-specified maximum number of regions after the s= plit. + +In this way, DAMON provides its best-effort quality and minimal overhead= while +keeping the bounds users set for their trade-off. + + +Handling Dynamic Target Space Changes +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The monitoring target address range could dynamically changed. For exam= ple, +virtual memory could be dynamically mapped and unmapped. Physical memor= y could +be hot-plugged. + +As the changes could be quite frequent in some cases, DAMON checks the d= ynamic +memory mapping changes and applies it to the abstracted target area only= for +each of a user-specified time interval (``regions update interval``). + + +Virtual Address Space Specific Low Primitives +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +This is for the DAMON's reference implementation of the virtual memory a= ddress +specific low level primitive only. + + +PTE Accessed-bit Based Access Check +----------------------------------- + +The implementation uses PTE Accessed-bit for basic access checks. That = is, it +clears the bit for next sampling target page and checks whether it set a= gain +after one sampling period. To avoid disturbing other Accessed bit users= such +as the reclamation logic, this implementation adjusts the ``PG_Idle`` an= d +``PG_Young`` appropriately, as same to the 'Idle Page Tracking'. + + +VMA-based Target Address Range Construction +------------------------------------------- + +Only small parts in the super-huge virtual address space of the processe= s are +mapped to the physical memory and accessed. Thus, tracking the unmapped +address regions is just wasteful. However, because DAMON can deal with = some +level of noise using the adaptive regions adjustment mechanism, tracking= every +mapping is not strictly required but could even incur a high overhead in= some +cases. That said, too huge unmapped areas inside the monitoring target = should +be removed to not take the time for the adaptive mechanism. + +For the reason, this implementation converts the complex mappings to thr= ee +distinct regions that cover every mapped area of the address space. The= two +gaps between the three regions are the two biggest unmapped areas in the= given +address space. The two biggest unmapped areas would be the gap between = the +heap and the uppermost mmap()-ed region, and the gap between the lowermo= st +mmap()-ed region and the stack in most of the cases. Because these gaps= are +exceptionally huge in usual address spaces, excluding these will be suff= icient +to make a reasonable trade-off. Below shows this in detail:: + + + + + (small mmap()-ed regions and munmap()-ed regions) + + + diff --git a/Documentation/admin-guide/mm/damon/plans.rst b/Documentation= /admin-guide/mm/damon/plans.rst new file mode 100644 index 000000000000..e3aa5ab96c29 --- /dev/null +++ b/Documentation/admin-guide/mm/damon/plans.rst @@ -0,0 +1,29 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Future Plans +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +DAMON is still on its first stage. Below plans are still under developm= ent. + + +Automate Data Access Monitoring-based Memory Operation Schemes Execution +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The ultimate goal of DAMON is to be used as a building block for the dat= a +access pattern aware kernel memory management optimization. It will mak= e +system just works efficiently. However, some users having very special +workloads will want to further do their own optimization. DAMON will au= tomate +most of the tasks for such manual optimizations in near future. Users w= ill be +required to only describe what kind of data access pattern-based operati= on +schemes they want in a simple form. + +By applying a very simple scheme for THP promotion/demotion with a proto= type +implementation, DAMON reduced 60% of THP memory footprint overhead while +preserving 50% of the THP performance benefit. The detailed results can= be +seen on an external web page [1]_. + +Several RFC patchsets for this plan are available [2]_. + +.. [1] https://damonitor.github.io/test/result/perf/latest/html/ +.. [2] https://lore.kernel.org/linux-mm/20200616073828.16509-1-sjpark@am= azon.com/ diff --git a/Documentation/admin-guide/mm/damon/start.rst b/Documentation= /admin-guide/mm/damon/start.rst new file mode 100644 index 000000000000..4b861509565d --- /dev/null +++ b/Documentation/admin-guide/mm/damon/start.rst @@ -0,0 +1,117 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Getting Started +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +This document briefly describes how you can use DAMON by demonstrating i= ts +default user space tool. Please note that this document describes only = a part +of its features for brevity. Please refer to :doc:`usage` for more deta= ils. + + +TL; DR +=3D=3D=3D=3D=3D=3D + +Follow below 5 commands to monitor and visualize the access pattern of y= our +workload. :: + + $ git clone https://github.com/sjp38/linux -b damon/master + /* build the kernel with CONFIG_DAMON=3Dy, install, reboot */ + $ mount -t debugfs none /sys/kernel/debug/ + $ cd linux/tools/damon + $ ./damo record $(pidof ) + $ ./damo report heats --heatmap access_pattern.png + + +Prerequisites +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Kernel +------ + +You should first ensure your system is running on a kernel built with +``CONFIG_DAMON``. If the value is set to ``m``, load the module first:: + + # modprobe damon + + +User Space Tool +--------------- + +For the demonstration, we will use the default user space tool for DAMON= , +called DAMON Operator (DAMO). It is located at ``tools/damon/damo`` of = the +kernel source tree. For brevity, below examples assume you set ``$PATH`= ` to +point it. It's not mandatory, though. + +Because DAMO is using the debugfs interface (refer to :doc:`usage` for t= he +detail) of DAMON, you should ensure debugfs is mounted. Mount it manual= ly as +below:: + + # mount -t debugfs none /sys/kernel/debug/ + +or append below line to your ``/etc/fstab`` file so that your system can +automatically mount debugfs from next booting:: + + debugfs /sys/kernel/debug debugfs defaults 0 0 + + +Recording Data Access Patterns +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D + +Below commands record memory access pattern of a program and save the +monitoring results in a file. :: + + $ git clone https://github.com/sjp38/masim + $ cd masim; make; ./masim ./configs/zigzag.cfg & + $ sudo damo record -o damon.data $(pidof masim) + +The first two lines of the commands get an artificial memory access gene= rator +program and runs it in the background. It will repeatedly access two 10= 0 MiB +sized memory regions one by one. You can substitute this with your real +workload. The last line asks ``damo`` to record the access pattern in +``damon.data`` file. + + +Visualizing Recorded Patterns +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D + +Below three commands visualize the recorded access patterns into three +image files. :: + + $ damo report heats --heatmap access_pattern_heatmap.png + $ damo report wss --range 0 101 1 --plot wss_dist.png + $ damo report wss --range 0 101 1 --sortby time --plot wss_chron_cha= nge.png + +- ``access_pattern_heatmap.png`` will show the data access pattern in a + heatmap, which shows when (x-axis) what memory region (y-axis) is how + frequently accessed (color). +- ``wss_dist.png`` will show the distribution of the working set size. +- ``wss_chron_change.png`` will show how the working set size has + chronologically changed. + +Below are the three images. You can show those made with other realisti= c +workloads at external web pages [1]_ [2]_ [3]_. + +.. list-table:: + + * - .. kernel-figure:: damon_heatmap.png + :alt: the access pattern in heatmap format + :align: center + + The access pattern in heatmap format. + + - .. kernel-figure:: damon_wss_dist.png + :alt: the distribution of working set size + :align: center + + The distribution of working set size. + + - .. kernel-figure:: damon_wss_change.png + :alt: the chronological changes of working set size + :align: center + + The chronological changes of working set size. + +.. [1] https://damonitor.github.io/test/result/visual/latest/rec.heatmap= .1.png.html +.. [2] https://damonitor.github.io/test/result/visual/latest/rec.wss_sz.= png.html +.. [3] https://damonitor.github.io/test/result/visual/latest/rec.wss_tim= e.png.html diff --git a/Documentation/admin-guide/mm/damon/streamcluster_heatmap.png= b/Documentation/admin-guide/mm/damon/streamcluster_heatmap.png new file mode 100644 index 0000000000000000000000000000000000000000..0ad6cdc01e693c7954471ee59= 3c9cfa7c756ce7e GIT binary patch literal 42210 zcmZ^~by!qi)Hb}&3=3DG{3f(U{j(kVHLpdu-VNRD)OOB_WJkre3~Kmh>(0cmhhNokOd zQ9?Q-CFC1_&-;GgKi_j*lV{F4v)0~cuek5EU+8Mzq#|c02LMprzIEd+00YkRfmL6Pr z&>0$g#@*c9Ot6j-kaGe?Uce;)1cZTv1W=3DR#>Q{k|5_o_Hma4!>9eCdc!I~gk8^q{=3D zWPOlr2=3Da|UsWJ4?3~IE1zC46Ft)M;|=3D(`;>;Q-A#LGvz%B{#&H2V%<;vEz-{_eC7~ zBaZ`+Cy$V)!N|WML}xJ3*<+%!Fyg<@h)=3D_bk0Ve=3DFHi?BN%o>he#ek(#FDPWkuJU> z`e`OuE?S;%BIQ7p-s=3DFOUR>(d_(^vpCO=3Dt(Y=3Dt# z?j5sP5sN_ytHyh_Yo+Y6WgNog=3DXgJGvQ=3D`?RGlaO$PKD_Kn*Xby$I^~LH#ArC;*x+ zgJvP{Nf_WnK#LgoEDpX%g0Iq`RR*-lfp&S&p$IyaL6-{nb`5l^f*v){dlU5C2K|~~ zKpPC|f}y)$*Z_RL4@Mq42?F43A!HheY^#XWb@FM{H3<7hZ zVEzeMcm@_Dz^|9!S1ee11(p-RN-9{*0BcyVo(DDxz(x_+ECpK?V7nUpt_M3!V7CSA zwt~Gbu-^*~2EpM7_%i|i%z~piaJ&Rg*1+jDINbw(j|g`bVAv6xBFr(F`>h9F08oGZ z_kzLD5tE$|I1FPoP` zo;EMB$l?5g@RzeIE>fq8uLaB~blOvitOJS%l`R%U#~%rGo}_Yq_tiB_t&d zL`TepB*Jx+uYSEi?Gd4R?kDYYx&NP!yb*KlG_hPt!J~IPBC=3Df+L%dHq=3Dbmo{3Q})Z zMn9*CBaV%k-M)M7XM0pZY-~(SvPrDiCnbN9ms|LOfiIg{C9-!uYBn@X>Ifi}1Jl3x zMr6xnDF<%NFZnn7>>eH_T)OFIfe*W+le4x|jZ1%fedLihWmtn77boYs8`n>(7t^{b zS98l~X77bu4LmtIcm&fdhQ?-f&nl*k#?RGM>pu7A*vLnAKL!RG zIzodc#_XQAHALI z@Hg=3Dg{=3D21WJYoXQJ~epI;LK1b$Fpxjuh^amfP<}>TDC>4?Q1bT$7iqI&f~-0qE5c? zp6a*Gk+d&~6z-=3D#+A^ctTBI6qi=3Dgpzo;fQBastVz0W;trMmaLCZ^Hh#tC1;OW?!kf z$zV9!G?eim|B$?|INB^DoKga(2nIJOzzl!+$Zz9?MtBA+{CbIdziG)r?$GTT3e@@=3DL)(+t> z%s89_Tt740{`)wEIQ-SHO&2qA^YE4Ok{nR{st}FDsWuMP5^sezah(5QAN*8wIV|SC zWB>CrVf4NKKOI7aa3;ybK}SnVKPFXFH&qD3v3dU_8n~G;Z?z1B>BM9QxsSHUgP>18 znQ@yNZAzB}RsN#OT71D_Nb4-sR+dvs)EoD0JBiM7yM37tHF*`qFBS-^g%W8?^elBR8 zB)())RwkGeOIrh)8#xX@P+-avnr+^%vsASES-l*2D&=3Dzjqf~K&hhLQ|LEY_#lHC{n z{x~#&Lbk-q(4P+Zf=3D_jBeyaCz*|6X|ebs!nAnG@J_H!t2y2oN|^Y_d2M`uH`egRwA z{2_azo4;M>nzw7~_9~Hw#iQyQ+08Ut+0-Dkq}EN*)zyqJ1swS4_~V5Eu4$)mQQ@e^!aiumsv#N7(vb!ZQw+k4)xPZLxXp4rMf;GqVy zLvkS}TT25&qoL}X?V_RE$+N{sTsHRxTV5-a~0*8&|~iqwb?sPvh6E9l^cJ1 z|8qR2rMsgfu-nxQwwZ(T-0QnryFWhG7Tj$2+wfd9YI+_U^Uyss`6l+LsbT1tFgZzc zSwEVH-Rn6vk+^_@vx3ov=3D|(dArP1Rh#gw!0^8#C>FLsU}T^anp=3DLVlay&gfqTb)$6 z=3DdsFl@pZFW`#*Z_pV0Z>W+$zpW@3@JnsWzHYb1!RmiLskqGZ{Kh3zk9FKSfoE)4wc z`K?+%`-;Ks$^UzvFg!QyZvnVwi=3Dgat`nUa0$gVy*vbXrT4kH6y~}!>{fQ2r{(rAUxlRIYZD%T-CBNHm8Q7Ca`sby{162qU z;5jA9Tm$y~=3D1#%k{6_pGNhwBn^izO7!|yiiXwZLy&~gfZa^0h&XEx;OkiXJ+vxj8& z|9k1Ejbj>uIl^KcezE7*(&H`}fQ?v+;GOU#S)!)%Bygxw$hMX8n9`R=3D6c|6*84{Q# zd34L4Bu?2WUG|G&=3D0mO_QCH4lzmvnI#k^YC)}X0&(GcA6armR4vzwu(*JiX&lF#g~f)@$md_mex;9wp8P3WUj2J1`@do6nm%NO z{_60j^gr{xlKQPP-vuw47co%(ZD!%yo%S@nzb|xTh&HbGXOfOIl6&TL%dkw24}>cl zUSkQi^_-hi|DHLgS~XI^YGtE;w{vMN@m8c7N1EjwSyxqEUx9d8*EEAaR*dx(t&;Z) z-oDn-l#ai;TK_)!u03Z)jnDaLzsO@&GZR6<&w|O55`0`usXqm(c7m(Vet%;9^>i=3Dt z!R&(qeYbfTxwo0O+oW%A^UV2X=3D6M?Ll+;+B%lw$*9dLP%@nu)(_%hN5_|O^DeA?< z|Ggdh=3DCHX-^Q`nj;nK+xjZ(QptLjnB!44O*=3DbyznBt+dEzqlOvaf)M^6jPBmyDv_{ z`TkYT_KHnd)H7Xst}%v%qEb4wA?#d2zsK?bC_4vFK3s!?t_5Rc#W$ zS#!NgHjC}N+yRTkewTQA-HgoJUn7T83nKSQh4}#1Cf#SYajlID^iYhmx70~KxNT>` zmlt}W_N=3Dl!Fh65KFt&SHlIvA;m#w5@vOkOEItOAY>SLSh)BTo7|9>mxC>2O)|7jNX zG%$XHxqPoKxy}1pLreXd1$(QgO&`5yWD?I@ba$_Uhz*A?@S26N(W=3D9P$1652Q4Urz zxvlbUwQj88SKbQsA}i~C?>p&r)%whkGKJ|)W4!* zP{B)xrGWm)>Rq}Qf_`R?V^IzxGA1)^)x}UbEza)e6zR+RBZA9hn7g8i$)Wp#-9U{k zRn{qtQckd2AZ+_qW#e%EjGPRc19uSbFtgK;_dmlm{N}8P?0N4iEk4tG@XWe)}&(3(k!7vL=3Doioq3XNtHoCpn)iqFgGaU5hChOWDJw4OZTLG`(nzZ zS}k9ns9z0n_`H5Fg_)E)9im=3DQ`?=3DBg!CUIQm?3I0o<)4IwvpSECpXFY9q*+?MYK!cc%%A8F>u-YuE3b9(P3ZB}3%+jpJw%Nu zsCNeCH6n~UUgL}=3DQ5EIFI@gO)3#kjL0~$Gvo@+(MG`=3DS^R*nC;y?n;=3DQ7WlW+qZ`|i22oVnDc*Ydf7+uy)MZolX5AW8i%aSLMJ+ zi0A+;+9r5a=3DkvnPZX)b8=3DsnlHrsiq;&Ouf8(40SP=3D%?h)K@y+kkk@ zg8~D3^XJI|PJH*zv6WjE^0L1c&^u6`LUiQ4qssSfuKSmKoN9s)hoj)(Brbq&ivzjr9K=3D$|5)Roz!VMzV!L`zCOw%`LxE=3DO!a>j*?alTcSQbOafZ9ClXY?K9q(MB4hraXOGycPtsH1%x|xy6 zHPlMk&nVc|tC_!|o}Xwah`{o}^*X=3DV08b66N(EhGlrcd0TmyQie)&-;r%%2M?-eIj zyy5`oJcl|3F_ATC9&ggIZJsgb-hE7elVI3BOVcJ~ja`pRyz|f2$4iDYyB!A&fkPlN2Z&xT#Bev;0^8egRv;0f=3DbzkE>7I&f|tn?Dps1}n1Rv_*MQ|HCqAG7=3DI=3DiPz8URup3=3DxQ@>8ok zYcTdlKz!6jde@p)#{HoOPL_{+%$K8! zf9+!vKl?ta^DRz~nF72v8+1aXj9%E+`-@wLRfzEK=3DlYcY%(r}@q351yZLvl~7%*11 zDr|Nk;%uojokY{g^q)WMxC)kT%-D}_f3<}lYV6evQI=3Dai&Os;Sd{|eLka;#Ej;NJR zte$veePLpWcb*Ez4w^=3D`ySJVWPD7aCcM^@CZ%oqRBzsKjQDYyOzh6o!O=3Dg1T8;25t zsPN18Kk&d1c(&SGJY}m_@s%+u;}$qhiKnZ{0=3DW$JROQ{Em<_Y9TAu+Oc}rZZsSu!v zl1qoe*|4649N#J9IKDL6cC6A;UU3F*b&4L8ZZn0o^7CQC6F{iQUZW%wyMWr2^^*doqkDD6gQ`xK94z3V86=3D zYD;n0=3Dej3GUZ4RO>=3D8>~hqs@#R*5L(W5QBD#S^)oBgG4He4!Kr7>($39-!V`LR9{} z1`OIdBS^_4(dKd3(VnOLSnDr0yhBWZYrUm;4~Sl3h65luI-E50x|1f}u}2eBs@V{_&_(^}saJm3tfp=3DTJZG4{IF+)h zczFP{MIQ)I%K7Onqg-?HS-_(IDJFa|5S1cSb&KFf09Y*qxWg8hLzVQYyy%08XjF>WcpsbYf=3D^7Vv?UL~Z$@wHFHQl{?{^+VYA znbtLc8Z1F#mJ3gK+pd7DJIv2kR*QCmqC+~dDgJY_2v}U0kAQT1x9CR_^;~}NnFg?A6iIb zZVv02{POF3t{3fio@3{ZZ4%epXtn);2?Q2_Rwg2bnb)(O^;Of(ru$;}-d;wm(azPC%R%{0^IVrR#28VDXu6=3DjG=3Duc8nU(&!htK-9^exDaN1p;GFi021*4z!@3=3D27-^Eb)sP3;lqpK8;4vVcz`7%A0H5A z2%%MheimfwhO#*TP8`P&xVOMGImS9=3DPj1`&F}()qs_=3D%UyM3pi!%%6~I_8x!qXixA zf=3D+HIKPgs}9teTm9%*zdh8URXY9)Dc22CLqrNI&@=3D4t>L?4QI!`E=3Dr-_pj7w z(V8u#zolb6Xvzgt0mC=3D*^mQG`z0Q2^IcfiI;NIMCr$IGpe0gxo70S#flVT3Sl6e^1a(B-O(-%EXI1)soLizEt*#gVHK$HN)XQomJ*HHI}acdBoWM;1r@z{*&4}ILqk5 zMMC zk1-jSOB!VCQRD|~Y62ew{iuPu*f2f0%#t{&;H+DqT?}nk_$FR6D~%4Ur><0HJ3n9~6;{6lfHjtLH<0?@6xWI4AC| zOGfwgNpno8b$I580t};IP7U3;ECyT^$yiB6$r`1uHH$55U;UmfmGRZGKI2=3D(4u*l7 z2f`pcJl&nD-$jw#7?cbQ#H4~FPPA}cv{trF{N3;Vt{*&sfGbjy4kUJ`O<>u$2OX6O z$tmcVyjI;hXzrIORW)Jtx`1B~rD=3DcdyMO~yl#q*blLH2)AP=3DgU`!=3DN5Y)D4!gM8i? zJ<4HN+-Bfm=3Dp7(w1>}rB$d|1@m{LcKA+c>l8Eg5ekcKMAd#51mpU0i6VK1G?D2X0Y zAK`dW&9lp62VPU4omTcPS{zkCxOrp5(#acJ`1dn$p^xB(-E}}Rk=3DGw(gC-v?2)ce> z6-4iVl$8-vQzWB4zzx>TcDk@3UoKsU`$oon_d|5wkpPIhDyNEk(yU4O0!Wz9$S~`G z9%0 z_r6&^25UsfYNLqC%|j$w(;`)tUz_)&Y&pm|Q&4fE49FW!OC zZV>P3m>>Asp;Ie$-_nAEP)_rvxsbqI>uRfxi4CYQiwFC6!-ch4n*UChC%LjG2^tkp ze0-Jfhw}SG+5DEyz1HCpDxKO9@3N)udNf)Vs8wpfJ4DOLp;M&3qmB7zwH&X7& z3I3-}uqPC;8&Ap|xsmaWQaAN4ag9hCQ*fJ);~TBjxJ2@MNz&^p7&Wh--yS@-ET5ym zg@Z+6?mM7%zsPm6RAv;|{3;{mHVk9KIQfAFLwa=3D4S~_9c`adJE?i>vy{({9)NNb)4 zw}H>QM7Axll1fP&bb3UEo}uWwE=3DiB;+9Jk3BX8n`MXmr&=3D!W_n5$%JF(M-wrSy1>U zWN_v0I@tS%)kW?<^8o@Bj@abE;ld;Pu+r+b-y;r<3;yki2tpn=3DRiD@kjL1mfAD0)i zd^NQS+o&!IZv1p~d3nx^ULx}jaMnn=3DB$fQIHxsfQ0s))O(gu^({0hjZJ!1Li^*^MC zOm?qNJk<(0oxtLB#j1T5iqHQev!qQnPA$a!glxpWH)E1O zj8g)y)!s#BSg|u)O-;Z2<68v+;!lb#jf)(f&p4~3esX&h!kDVX)+FTk`|5F!!Lvq$ zycnx=3DlO-Zxb%6`Jp{Ei!;|~;zc@$%fMu+O$H3mome_2tVi~jv0_7@#+pjja1*6N|a zt&jr~ga8?K3aS#_7Fx(7Mz^OL4T9lh|vn5N&#S%ipj=3DJ!ln-tZsiz0;dFKkYV57`e)SO%lOY6UcMK!83OUw1+Bwteguq8zhCbSi{?h zORQy#*izm{PEsn%=3DBS4wamE$*6<~h(5oaM`u~;wW0u%5Z!aI3<&7|1X3)CFe((h_FeaAAr6a2=3Du%JqHqk+PftL1K1Pf~SYMgU9L_@uu#*9uF})`}?_WfU z=3DNCjoEVg+4C^7z)L5gjX08665&zMBDf+treff8iOfcB$upF-{$GLLivmy56fu`SF} z11i|C@uuMvpo#()txQ-LG3o}6t5@=3Dh>-HxH!>jXB9jc@+dq3O_a4TdoS9mr+f{P1l z(#Vz*=3D=3Di_A07-diWhjIIq{}8bXf9;CWJT+UV$CenBcem+$n}^_v4i1Ds6C@AMEwB) z;^vDn1!a8~jcva?pnyB7H5kxRBuEgm>tO6#c*~)ErIH^v#R+>eYFr<;OFN7F+Zr(< zYcs!HktT_0)p0S)S$tG#`Kpt2;VYtiJ4V~ApV5m1PY#x)cokqlB=3D#mdl_68x2oY_u zW?^M1Gn~$bkwg+Ac!}v{e-y05)DwiI3E1F>O*i@J8WaTvxb_R%cx^Eb`Te#s7|pq- zuYW?{WP1;6A<|cpXcCrlZ?!Zx78bZLDg@Aj{$jH#Pn6;C+jwDqV-QsvTSX480+}`j z+#1hf8-@I<FmMlhu6?mRDz`!%!8hS_6BFV{vZz(3be+(CW7 zqFUZJ^#LI({xb^sLf}E!9|*rLr*S**{OZpX2OqsF1s=3DQAM$+L1;XFQK`=3D9bHfLM`ZBuy^$twN!;a!xs<+; zkx3Ukp8YQWr%&+!xe%xd^DI5Q!>i|8F1S66l0=3D$9-11J<*0_HlCAw;dKkzl`oLfH? z`fYDJF}Lxy^#`%u4>2H6$_FeN+|}oS)jXUw|IL&N1MHI{SzC4Wy>~Ia*TfMQs8jur ztQuF}t1S#n>|Ww%lu(&MW~BI$b%;l0er+8MIKZ&F`}EL->UiTDF^Z0=3Dm7r0otRfql zD7@b9j1tI*vyY4C85f2f>xtu#n2!~uR7+|K49kVOQH3pcZh1^ATtzA(LDf73Ze%j0 zduba1`f1URHD6xNIb_423+lDbt(P>Q?AJ0N0=3DPnq4xQ-%CcnaX;os~SzQp=3DrNp!gu z0Vn7MLzH6e_!~MTKR#T+9sEaNOF|ttFJ;h!LgR5q{a2#ik>Znn;ylrUl&jR-nHS!F zxXQhXRWXpnMK?oBcf-SAvN#r|Hv>~i^6Ow_W+*=3D)Aq~R&?dNKdy=3D&$)2FVVgubT#<8ath z8@9y}28PKo`TCE$y6wNL1Z;MHj`S!JI<>cYgLJ*<({h;I6+j~PZr*$v!K$QwtCbC3 zsL3I{>e-_zBP8#jQS>Zf<4rYKnzwF}fii_&8I4*?U10v6yeo;8kIZWs1l-1t+3>ys zV5zoW=3DKYo_C|t6)zN!w^#22rXR&KF@PckzsQc7<>kpfl(aCHu2f5NPu%Mqhq;%W79 zC%K8DBZ#Q{MazK_Fb5cZ*onVSD+5w~NhEgPsPS%GwGuSkgQbF;yYHE-L(JjKf+G$u zxk?150G@A9afkx97J}y*Alz|Fl)pP(65&3^PgLza-Ic|5$n+K%s59Hl@j+FR1W=3D@7 z98nIT9iV)%UWUV}2e9q;ORVRe#=3D1)R7@ zVQ@)Yr)APZ5^0dO|9p173Uqt!29-j{#zS9}zb&7wUyK^N8&u+`Zs*fa6vyUK4H~Z- z#{^74l&4$lXlmF78e505SQhezuNxBSZkvEVR5&icffyq+fpn_%)kaC8MgQ|}1RI3i zg5EGBUzBon%Dlf_#jam|0X9?7X>kIKSzzL&^H-RC zRtj`Pnp@yK_C{{_g^08AQ7EW5tf%T%PcRqTOm(p4t$43AE7#?#!otlzFFMhlfGc| zUjqko1IPJGI4?$RbLoNGfYOqLb~Aa>yTW}w*9orfFBPr<<}T+z%x5|R?$V${7&c9e zh_LsQr=3D?bn6t94B*_BB`CrwOVuR*5p^yo;83-eyY&&@Z0n-6ytDdFV+l{>Fd<7w)}` z)>`pmNkEyU%vCxH1Y~()ieA1wMx9<$o2In@^6J@-ZHoC9OD{qh8XU6cu9lQhttvvh z12U56VL0c5lOs0LVTJ$r!l)K|eu?OPNnFSciAopyw`#-%ws0V1Z^1mU4hW#L5h&WL zUvfm*gYr#DsLICXQqKY4g-ov^#yW-CWnS=3D;A(rGd4;YHr5r>~8AFf}f^etchKhCb! zw^8z0OeQf-vOqsN9|qx_d&E??9oJmo`;M~2d6qgNF2;z6jQ|o_hMIdO>W>oeGv;G$ z7a2ZGWQ$S0;e#_{vpABy$qJ(=3DA{Il=3D-Omr+AzN@POX}>2c!tUCz!-j85d_rmO-R=3Dtp#EK~dSzx}OTH5;3$aw`g zE9Z+YhgBUjQCwbGH9yq{rHAqbgyr8d@L({UJ2C}Y+2Ck^aBs{@CZ(a_lK4^JGMGN* zcg042rK#+@W7d;L-JuldDy?fB_YjUZfdWuWY+EG*s~B!Gz%5^5>^QGzD-AqRaQfiQ zG($I#MMM5FznikZM~E7FT}snTPn_VsG#&^=3Dd=3D!7Wb5xQE9kF4IfTIa1Hb%&zDs<6; z)n;<=3DJZR>+`KoxGl9n&q^yoX^O zrk|St4i^RQpTU=3DOt`8nP3k~N<)XC&3`SgEyTjTsvv3(-JC&U;%P7c|vEs^?KpTn-NF2+X4kPxv>>n_6AkoXkf zepwQyfxxqtL0z3=3D+WeBpwc?};oF4avvYvx65im!E6NYZ+iMO(hvh}mdw=3D7WVCTVgs z)hBq{*m$D#RZ% zrHGXJ>^z0oiqcLcD|lh|_P;Et=3DU>ro+ZH)a1a^1n>R&PSB@aFo%#S*1!ivdiRgUxMT;3Kw2Y2l z>an(6zeY7OVnx_|j2JJ5^d{Phr9eL_WP|_B=3D#k*)0NP>Hw&mwYiA(>0Lrff?WR19A2}>$!@jF%4ciIdDe~P*oD04l zX#%u?+%93yFlbJ=3Dnhrek(tcm9*FV1I zrDlgsC=3D{X38=3D`c*TSlU07kvTtwa<5i##}+anK&P6ln#e3U}t9D?06>G^L6FgL8)Kk zHM^m-ddIxL7p|R(8=3DvZJ+lJX9HguvmQae#G`y3Zj<#$8e9XA7}bm=3Dho`4Au)# zq)Ad?g&(!)k_FGt>3driv%s!Vb?O{B5}?`-njq$0|B+`0BlFd{3`OMYZ$yiDpdy~C zT#QB4&qn=3D`!Tm?#twi2_%%m1dAYOtgS=3Dj3s!M#kGnaF(K#CIkH?Zvm)xQkdHv_~tm z(HF&KltF)SnRJAnJ%-32RlDt-wz2o_+gEGy$r~tZ^?(2pSFIAa|1x@?Au|hrm-qML#P1t_ zupyLgPGSom#{PszT_#W#zW;zP{_nTxn)u*#%sPVBn+1L!^9k@Bj%56gc7qQT}D*7;SMz@Y?T4BuBtTG*~2@J7{*1x7IX0sYIuXDC3-5QK^d z*#5lDhA$pepRvR|9fK(MB-L0roIeL@wUaeiia5TQrD4ci8MYCAhjM=3D9`5T2#(bobN zd=3DvYma5HlrkyJKfgYoGDKtAnh!>ZK+{HbGeQ8;_Z9bT^TC{{wx@kEFM2RBJ1%m$c1 z>D9b__Vv>B>G>;=3D6jAvf@?W4Q5sVens6nJ))o#QdLA1r$@-%GhEkC^;O6?~ zRtm*Lw>=3D7Ljr(b>3c>{YF4yEpV$Oq^$UHHm6j5B|$6r6Lk|dV2vEiTT@NG%AX;gH5 z2mWj((>CJBsicvuiJ_~%3tJ!-P*v^61rO#H0d10$-#wA*PWZYlIP%Lw8 zBWkxlxdzkbjA;Q0;GLx{lDenCe^pu|f}f^rOM%i=3Do%!esWxu^}*e1Wc!V1FuOJsy3 z?n4;;1q}rH{%?V-p?4@qjK^yhND=3Dklw(b(}-1y>mZ~aEsKf>G$6o~}c-E&+YnE`L4 z9U;|pXT#T$q=3D;U28FT{rAtThG z6u9rxhd4<2fA*-9ovjtpxC)z5bdG*Yq{3(TxF9S z!`nHlOD;};R^ND(9<5*v)?Up!F$G1}Tl8K+TG;UF(L;Ez)HHfF7Oiol=3DEKXsh|1lT ztY`YMPQ&}_lv&E^5WiMh>L z0uxE$Hop~xZzB_~-)PPHAHk)Bl=3D9Ts58MYv8Ht}puQ<2~xg5V7;IEg9FrrbdN)?|@ zN#uo-_Ip_q922g6*_X#Dui9lvU|CekDgGV;O68DsyPKIgN(-0d746@gQa&+| zFx{#^cG$wXd~jD}u8X}b90|0$s+j?YuQoWP##}olfvZOroxgh(43nQ;6PWN<15Ng| zr3Y+cZ*@OYmREkB%&AJSY5w4{CF8afSBN9!o)aoT&Kj)ya6HrQ%h&W%fOelsE9I@ePPOcmxA^(&49|K?7+sjVZ zMwr$X9sK-`$!u^p%cge{c<1>jF(U$BtMhxqOK0eL`V?SA2K0>Wa|EFujCv9DCvy`j z3SS?~OGNHtP}FUMXwr%wztaO>O9{2jSi7wuXjAIm5kogVCA?Y=3DM37QPYbu%TRorF0 zy6HU8|3VF!%nLc!+D#6vXn|8o!XhFEiFdQ5AV&L#>(HBBO~ zNaEfUbk#Sn0Y|I)3)2WA2=3Di6+jk_X^_cj|sAk{1L-1U`sh}XWY_8Y85ppM_`m;&6c zDwH)H{Yb>{ke|ejk4q`&sB`FfnN~?n8RlSXV0$}=3D?8Tqvn>UZskLAb>-lgJA-cUTO z-6D$HRy?{m^7lFh&p4y`lBuo~qIGleZhCMq&W?y+7b8;x6i^fxdqa%-)H^V!2Awavmln|p&=3D6+uVll>c>lN{ymwXKDS@9uc1XrHij5ck=3DI zP&mznO)xGHS}**?4PPp>75`8mSDU46^Cf_6rr|NL6uwrR@WzlOtaSO7p2L+)4r^3x zoi_zx1e`I>Wfhf~tJE>+iYlC?LB{ zKMwM`L6|yJM^9wTBj+F$)-!9FO3=3DQ5TrosUj&97H{DwXwnP9Y zxU64!c-e>;_RcD?OKAtX4f{-q-hGX>#u|iC8OP~`(&H*W+R1{v1c5!%5fsCQdjJ)I z>9V@@A&+v*sr4lyh7HpW(O%UkUdNJRowcY1w4w5+4y(I-gz+{a#a@E#qnp#dBKj>q z;>|i0VBgQLCo<^K)@!VL)%WVY4Ell&dIAsd*8?gH>8A8o8g}#Ds{WZ0p%mm@N&3j{ zPTz+hrL3|G+f8bUgGL^AI29{fs~M7C0nZp)5wh3jxfX8|6TPEZOXhM@h%v9{xVUS6 z8J7Q=3D{sWH?GQ9U{LWttV2akjYu9WZuw>`Gx`u^ZD12T8$KtOsP$e5vLiQ$JRLR-VMdTA6N{1|BDd}8-yiyJ7AA@0$z2Vw@)_ipIF z&tT{)yqe5AGT)0mLz$0wxY@ykVu!#5=3D^(b+S{foqzOD#nogV(k8%=3DWjv)@^PH{ekm zJGDH2b9-0!|*gRhL=3D8ULG!B=3Dp#t$iG0K>;P`*oCWli7`ep;lHIWl}PjDrCS`UWs(&Np2amB9Z0< zJ3y@Q@5U+ny(qpaz)tO;Tz%h5)ZUgfe~*aPE&d@eI-$ly66BS511Rvz(}zyrd}AJ^ ze`EKlPE3~a#CLzf`u01W-p-SCCeh@1=3D}DkZmZ-faNgpze97{Pg&;@;e-=3D8sfHIa{j z3@9Ls%7&!T^9)`h@yhoKuFpK>#4Qk!2z;*lMyQHE%)OLc)${1pFXs@8x=3D$WU-$NNk zeHcTRs_9kS7Y_wb1x_TOoz^Q0l?sa0C~QP9r#yqHiiZd>6NDTq3Dd$KX`Hx@eTYpp zQF(TaOBT0ZY`2Uil{a2LFV4R%SJHh0nE$N1^Fs(P6Qo@%6(2Ih_}=3D>>Z1?e7%9Ez@ zNs9KIAp)DYbNNvx?}hJaE0kJY_A^(#e1}bD&Xe)^pJNIC(N+R1AB>S zGo)Ku9#s%->pBDOTU)>kPo*`-H_Cpur$)~{QYoihL-BWU;RA_b0-(<=3DZR>$rK z?s1Y~8t8>L`@al2%t!l7tg!qVwwDX%f1}RKLIHKX@1cGK z*0-DQ-BOS_u-E=3D@ivmYW0=3D`C~0ij3jkkN_;?pSU$UMEl=3D+nO=3Dc_ID}ZBsOuR0g+h= b zBrvNa=3D*9E(#vzdKhfly>?al2l1H^pj;x5>KvSW z#EH&oteZkgf!ZGplqupqBOOVXTC#TPQ}KqqVEN+quMUQ1g5eV7bd%gWiCKXkNN(8~ zuWyGI&s`PD?FKs&(pdqw!CPO>{9!sUKS0ZWVLF#7 zT;a*P(%5K84PH*s<7&;oQORZRrrAyLS<8QuoC>D=3Dr%l!+BQg zh<>en&D&PDnAT(Vy8VdysW^=3D{?b3iNQ;mSg%LF3Y_p8*{G^m1#ogH!O(hAjuZGC1V z^f)n`nJtNRJu3an4s0&zTXF)45GvYr=3D1hf)(Qos4?-ON1D6j_NkXQ>Kk#GfiBl*tu z*MMyH9ot&mkmS{z#qay~N{;>_Iz3Ufxzo)Ym%1~+$ z534}y7pzcujERXpRfKW4$*fNc8Mx9Dpy3PI>q`|SuH^ccWlSTjFPZ6ARtvtk4Gbpo zJD;-hRZK;c2xW+VST2nt32OKUM2ftRYxbpwdbQXS5ec#tVcV>v8JBAQZI;FuHX({QBQR!|8(Q6=3D`(gKcjgEWXZ ziXZ~g-3ASc2#BKi+|T#-dj5o&d+*tMul-)@Y!viz5*dSa1Tf}*1XQY;b=3DkJaeq0aw z=3D+NqTzVYT0)6;0te-EI}*8e=3DUyEVffXqA4w$ zm30BS@G;|3iG1+>nTk;GWllU8c)}UpR1J($pHw!8<;wYKNgso2wq2~*3tEdUM8Xc7Dynh{d z^tWFnDpd}0l-SERwb~E)ima3qQtrx#e8^PNXlg?Xf9@^(c`3+p!f-(O3K&Bs0dxaq zSg{OZh0+p1Pa?=3DyI-Nil7};}mHmCe~fTf1iwO^F*Grw*cPFn5G=3D`aU~FE3Liib}xS z^k&9w3d56Kks}Ic68!~a!?JrXnA7{3TkQ3o#>FHz@?LLXUM>iu0L6wSq-EpTF?x_Rf&#Jk_nyrBI zdrHFPurqmAg(s$e(F_j(H1y|ZBC9?~IaUIj{9Xp^nF&uo-RI6M+6ucs{qCQbVa+oe zOnJf@PEm*96o%a$fA4{(O9?GtcjK`Z{i#b~dy~y;+`q{9>>l^%R`}_8{IUKD)v&_~ z-h1y+hCev@els155?ZkY>GXk$3jt+|C4+-tp|8H(9j8Jywwn)}eFg+l*?Kg0pgU`m zWlKZ@v{;2piO5eXKj8Gs zLB3CYWggt8+f_iih=3D(M>%K{W_^y=3DK8kY)-FX?d=3Dx?zkT|!yF%>!!;Al28T_G`dIu+y z$_IN>)S>a|0wA__c&OPfE;s7YgF)5~N78$8;2D3*`YD=3D^JKLFDIEr(b68?8N{8sbX zcPWc8opDlEINUV5Dj_prn9v~N=3DINRL(RPq4{zHP}m_8F%rgxJyPM&=3D{HT!vM)He^$ z?55-eEfp2#c9nlpG9GB{y0Ay$hBt*8rH#^Uoc?^W;lDU3BJg(-@)`EfAFWQ6D~W(%O1} z1B+olcILKhzIFUvt+rS^csJXummM}y$sh=3Da396c%cTTdsycz(D(1Knp@%sTGs@U>b z9hVRyv_+|^XBZag1CJ5PbI<@YzB@q)t@MZr2>jlWC*AFjT<;X=3DBw&&ybAQZoLqsx? zk0M~S5`;5W!6*Ly+2OJ}f3$Qh6tAexIBSjv=3DuUQKPQv=3DVx+@= OEt zx!`YUg%V^Z8}E7Sf%>z#vQtWabole+v(OwR;f?noM%p7OX>`&+@ceKr#g6~SQWox> z&%p*BB8;TTc;y7vHJ-i_ERl<9s{WxulcL=3Dv6kY@cCffwzbY#)Xy!PNO9bCYo5G7A) z-Y{69@nhd<&Z-=3Dgd;gswmT*l0+#7oeHQE~5X@fX)RQZUbXZeO zYpW7GyB-<^4UfF7xh>p?!S_(0XVV5o=3DKHB(&FB7zC$#-w*>a^TbLEo5?lyK=3DFYyAo zO*l{=3D(KQNwUNqq(pz>`Z-q=3D@K_g)d=3DJT^V!q!=3DbQbYvK~nz&}MlN4uDi$HMGVAwT= | zqm_Jm|Aw2Gkaa@=3DwaBc1{s}F7>CPB$hV?b@17oN#p^QhvXx-d$^T1FtZ_F?t3Jbx- z$*iC`mp1G!s@Kb#QSSHM9}x?`nC^YZg~s>0vqbJMfq9n+hTmzF#GTg7IUN2&$Pp7> z5&9#9nifWJl)wI%&(8;bWBG0bZ3&}{X?2ov5cI{>(gO{NccKZ;ljUmJK*d=3Du$6r{2 zoNLm`M;&CSW!sCDeyI*n!mlB4wyiCzy82f?x|gQsN*lS4oPw9p<|s2I;5a& zRo6x(U0>=3Dpz*{@9f~Fy|&rP4S<@uv>O<4SB5uW0iQAVk0_S>M7F&yEURd@HtvWMwn zfjkN&z;9FVM`WiFqQt2@# zq*SFu?cs>8P=3D*Zn(+S*Uc8d8-mrmsI8Q}gAidt1DyD9@;Wpr?P7$~cGSf)vD#PX_M_8hj>wI6zPWQXE^I;c3i-OvkPX%HiM;4#R&L3s@ z+iAvKV2q)1Sn{SzIyvb9;k+D@gR*w+3lTl0KgC2MZ1%|ALDw<;$^w0)3R)7u(N(Yh z9w4`1AZgl+P%Wgr9xc%Af^G#CJefFhQ{uUQ2JVmC0MZI7Gnd?p_87IIFQ%aMVa^Xf zRP0e@pbk$-j$iXdzYMKZS`~K&>q~&UDlMu95Uv(JYcy&+S)_#5k#$=3D^ zZe`9q6Q1>mlXU&RuJ+Wb2W;J_$F9Ad-n8JG{F{F>b>gP~%6DoI`6F0e9`PN%Id_MA zxQ!YPL^)}KSNqXzGY}z|D(b#0rjCRDp#`SS#1w4+c%A^k4Nbr;2p@Oy!Ey%aK-6)P zp|gIA#q(le!h1k?6O}tJHk^aGZ1D7eMcY(bx70R0CuzZz< zt8y`*e%Q1^>zO|ISAN5(fqXgyiCNQre_)sst+o6yzW2|Nr81E{j6+^rd%Qc)yB+QNPIR3&v0n6c5bnSwOULNEtko8SWshQv{{?aTmRln2v;ji#?{Su($zWRQxO3Im76Qidijj!0)M%)lIt0tW!0R0 zKQq#?)58Y5>Z@yA-;taJuj+|i(GT?h%tQ%sk2Vea00*SsXt7l9tCF7Z*GERR#)t#sLb??Uf-K)=3Dp-O9 z(yh4cfRv3PxnF_?kTJW-x2#(HROTD!2i~`7ljj0pa<2mcIHVG2j|q%ezr^^G`||!` z5NVbAjHxrKn6!XmvCDPj)5k?f64RqP3V06&C22{rsF_C3s>>^ZbZH+p;%CvYk@-9GzRKM4$Iw2onMB#SfW?F zmnKk8RY_X|I~eIK4BVcW(vvUgQUdT7GaKpNVtg6Gu79xwMC~}qx(;&7Xk8F$?FCMZ z6kgK>JpEiR-6o$W2n*zOJOPgwl+BXXtvPNsvh672gP>utoJ~%R8(8=3D-G#`xKe`fWI zcW8rdO>>Kxu_yH2tck4G?XRYqy9uK~(VY|{x6uiO?5^gxhbEJO>o4>|{?~z7LyObR zP5U>9ACcMB6iin^1(JFo3rlpoQaGs->4af8h1iv~=3D1Wr?$T-eV9&!RRK6rKjIUy65 zgd)oVvBbw@IoWLB)u5bn5n~mn^R_Q#sF@w$cDq7oP&5EfSgeM|%-}>(_;F;JBp4(_ z5At<~U|^#(Zxn~6SDVfFYa}Sh!^#0{H&A=3DP-iI zX&*eAT5gB8PuqD=3Dp_H;%cCp7;f`y#$&D~!w|94#KgQd0uLHpJ1^BxyWZ21O&?-ag8ZY0!<7wMy`vfD-;S{;&$dIga1} z(m)2)9hXqz^-s&y|tJ<%js9uDM zFQmq9+&y40PNyWEhQD(Y8yDI451c*kPJ*SGc8#gJmk}nPD_dqG>*O46y@D(mNLkj7 zNMmQlE&Xf8bSE5Gsb@tJUH$9+->WclSWINs(Hzy0CK?#=3DN@=3D{%5qmt6lI zCD4AontEG?Vcof6A~{6v12)qwXjuu;dyOXknWBdMQPC&2y5(U;(q1Tvv}U(1 z8#jC1L!)oDy!lgQbt5#Yo3JPzMX678nNk8wt0*%^v1-JwI9NpVQBc zz7rlHx!YsV)mS{d1~gEliVIh7&YkV1B(B^RjO2%#%(OqqEtHLdR#l5IEP)@szm)-U zutcQGm2&|b>ifoaV8F~Vr1hus#9fJq^gkBVF0&Voia=3DLviXhCyecljWX4SKxrijCJ1j?aP%UUit^#NUDH_~D$3nJf81a0JPQ zb6fzXpjI5c*x{Nc8J^^r>#DF}-f}}zO~9mKKL9lya%O7~y}W03{Q1-57StLP{#Vd@ zTfNavyaK>W1|Yyr-Jj!N#zqbCNGswncaRxZBv1bd@!~rHxj1B8UqDQ<48)kLhPd);=3Df@dgC*zRP|30!xhR zPjulX)c&u_5+gl{gS|&gg{!52^<`+>17=3D*XrG+J(#pI!V4B~0O1M%%O)F8&8J`97O zKy0j53zRi0tjhFKjLt3bFP2Y*s#7%Z%axOJeVu3ExcTlQ{h0#`U|rlw3(5?)%Z*y) z@ps4#(S)zqWi0d-`YhUhSQDe);=3De!K0cB-Napi2YZhS`(pOiPKNx zmRnv&BgnDWB%lX%xw@29M)0tI-l)fz!dE8zSlE}aop!ow`L)^i|81=3D72paF&j6Dh2 zDyi|F^)EAWfFkzg;E)N^Rw9%9?5XJmoM8tSiWzIclR@p2*!eKNJfY7C_2Y!F=3Dj zUhbEnEC*1(wD&=3Dz)&uAXWkAbiLU7urqBC&1FHYcQR7OFngV4iCPhE%8=3DywxU3Hl?q zh8SUbyTb5tVz0}hhhD49xTYRDlO)devBxI>|4(;|;$_CW6J`887? z;Eb|N_ke?SUawp4h%%DQ6*5KyhD@j~InBQJc8mg93V`uWx4SsiTTN7%SmTWvrDS4d z!f>F02foioLtqn&_}eL@Dw^$z5(9qzAP))O`q0v{*x3|xTza?M4Nbb)zsZ-RHs|Lu z2yG&d#r|3HrL0i}KU6dU3;zwg*m4@qt2rL7g8GK}79pasirl;Ac_izj^^*tO9?y&b zfx#(ZI5xB1HQiD_iwvVCu2WXl=3Dy6Ql_{%Fxe&o`cF-&r$2p1AOAa+WJ8W!BbB53}8 zx8>bYvvV(AV=3DFK}$oZ^CQ!|{N0tIt>z#U6i`U{1;oJRGsaG51CDN@5Uwda-| zV-uqN>hW{U2c>s+YuL{1r*HN-UrIH-98G0Yq#yexE%RD}t!0 zq%Ze*T;CttRy7@Mntlmh%83D253-8 zA_276+remyFCzE`we#mYfmvZ=3Df@dqxXYmRm1ZAAA;; z0QICd0V4T74NnCwB>F!q&VFKiD6D7{%S|39=3DWWq?LGBlB zMD$_m0@_HgWudCTxxxuUOx=3D2@jq6F!l1AokOOq1Db@P*PO(QZs)R$|3SrHKfkR5Fdz2u)#YDX&m6ShP^39^VGQ-bu?Sgzuod7t18d*E z3K63Rhi^C74>2wbLa~yZH7^P;G?-wjCV>~(`+wdeDuSo2EMtph11RA51fDzR$6r(A z82{mOTwS)C>sVSR>VF(2-cF#9RF72RM*TeZ!#qR+6JeqNy*&|xqsAYOXR}{7ylb$K znfORQuCw^yQrbs~1j+XKU{%)*ixKtW|+!0b%e#|07jDLxoi=3D!EMfn zd#`?@i2~o@#vG0rke&8Oc)3lEM28Tm`{Lx0*!g5K58?$S&7 zC{pa^amB&GuQCQGuG`%rBjzG9ji%8lR~6=3D+UpT&s-Ph8Us~5b%Z`&ZuaARd*pC`b9 zqA^Ei65l!tA2fbH9A%k+x<~$kS>ulMUJ_{4(SLj(S|&N0B52H$%s*m*=3DKEy%7)ktU zR8qr)qIwO9D+Ea}dnID)g_J9X8WaeAz%V=3Due#A15mCAES47PVk%V){wfIRzl`SG8!B?>ryHU|i22n`%jNs$AeN%N^58N$_(XmkU%Dq9uA%ezeja>6Cm`bZsM` zY!Xzf<`maQ6>d$`6Ip*h-?(Pr%$mZ%%>fi;>L?_eJTQ!b2#1MykOb(wW6-}#!3oHf5X>z452Y_^}1N6*WY zpj_k`{#Jo#?xTtPusuc1{`^#g$gdwrBi`_T@*aA5&d2BFMK48#iRBC)k8=3Df94`S~Z zI{p-$5@R{`fd78v&M7zpyXsDYbe$aV>FJCzilagu^kc~5fl9U0gy?&fUv_|%f<$Bi znsx1rACSy~-WN)`Cjj3J3p*KT{IVCnmh_@(@HQLGyd_x#W5rF%AV~1Xe2nrVq zyA4rc0-t|r<4M2?oJ{1e&KvQxL#>{2E`vPJRv^5nn*Y~^PK+TFzSe3v6RNrI{cuT5 zs30PU&QKP3K1>mKvzlvO4&kqY07_QKb6a&!O#`UaI5b_B^!|H-v;>;!|MsQ)oaXAi z7=3D(I?69JW@dU{s{M5-fKGC^HUFMrF$J+OXXq{~tWsv{?5QS#wzQI)%TNLCA@Jxc zH=3DqAqM7lCfzvjPMp7Swj*$>P*$R8O*RgczJje5f-gJKy_F0I8@tLLrGKU z-I5jw-OLt<*d;^<7g}Heh4Teyn~pJ#p(^V)dH1tINz3Vnv3i7|ND5VkcjPC)ZQDI;5SlQINPxnW3Rl`z8Z8Q=3Dlw=3DTFEmr zP(`znmQ;(A0J}Ml(_ZnyE#4vdOA=3D4QGPyFt&Ov&Jc-f65RLsWK)B-&I8H+8p6Y8DE z)Y+ipLuq}#JkZd#MOTfyEZYAjsD4HE@2|mx9!oo|K{mFF(aOF?r$$usHUVbwDL7FhVP4&a3 z(iwpit~s7i>M#`pFjBmUMUbrOOEJn#lT7$d@+=3DjA=3DtuR>jelbz!ZO|rjHpH zmZxWWqFtp?HsQhOFD#NzsXx}xj{}vphKirGU#JAtLF=3D<|NI^Y z?D%Lrd~Ddp@+%f-`j~L%Sn7g5&9p(1d?X}A!1G}%>Kq!7y?Hx|CcMJnZOEgVGtq=3DN zN=3DuP!WytZli1F{c)$e}L-kapLfBz+DCKCr9#~w>=3DPHx`+`?1bXEa|J-fL(II51uJA zxy2_{)5AwSP5bifcQ$q19}2;&rylx57Ltt#!R=3DG0XV8@I?>);>$2bNF7=3D`9LH_Bjk z-_F^_VTt!IkGHEmC(qLQTWdFno#6%Fqg;S(^9>eilfjIrAI(z(!mn*9A57Jsg+r9N znTFWk*gUGU;)&&qql+<@dj!a@lp0pA2K4YGHI{k> zet?ml?F)U^l?zlTIdd*RwoVqMsJ}oZc@p9Ql9M3Y=3DXNF~q7hlXlSed^y2pB!rYCnb zxezlXK9(mmTA>X~FqZ&50`>JrQT#ZdzG78;`fd7##GM8 zM>p<8h(k7(&;O}wVUgQvf6*k+CI2P?If@Fuz+*g7Vfj*x5Z=3D`5{z#a@bnE-_ZeyRPBZ@$aXm~YxM;hCK{ zfHZ1(s>)x$;!#)UCrKv!B`ALElVkJ8QpR^u5(Aj5|NRYje(oJo!IxY4-Z|8d417|Q z6t7fFzooZ^z57R4wQ7?ePA6q&04sxk3Uzo-zQbfdV$hzGM_DBnFXr+%{Npj+IZNi z6+{lxvdjJZnUW1euSU=3DS$+H3ZHNVPfabeuNGgx|9$$v8dZrnd4U%n3ta`ZZWVx(yh zO@z0LT>&h+y7Z3#qGGs!gRnP6;C0HE3S$-u`W#Y#yx*?IC0%LlgWx;E!am0!FSB0{=3DMf zpM{+#8=3DJhTo<45?((PTVAjG+ohmugGBfy67ZKmqolG_r=3Dzk$jUgoQp*H(6?Lc#3U! zBu7y4!?#hr$m@Cqj43C-G3v&U00)K5AU=3D) zQ+c~PT*?~_0^Fh{k`N((>n-giFQmpcM%Pe~9L4puW8d%yL=3D?#K*%_6N6d5F8Q${H0 z;E{JLZ*;hi|9}))z|MLIuhgBcV}=3D;cOEBRDI&y$YVNj~w1H;tLeYtNv=3DQj0uLw60v ze9cCt1qZ?K>67LKKbP)`oDYd4m#O*eSUH1s?R{t?h4p&X4bK7}y4lw(W^*S< zw%{uoX)Y=3DLAVU#7AirpSM3FH~z8nh*WG-F|lB;KoK?1pSEUInLa(1FZ@Qw^K{uQKT z3-mAye>KKvyOKBif|ObF*dz}^%8FRT-Bv6!Gn<(nP~clocdi*9U6nu zvTY#$Lk*p1iyW0?9?$yMe|>NPBAL0_dMGlda;~E+^9zDKbRVY&dZS!7;ajyt&Aj2LBDmo)QGR)jcko&s)zDSf_dU^dEa}j0*)-?6nGX6IE7Mmyh0#Mjt*-bdFMQ=3Dp=3Dmuh7yNwmNg0}IYz!Qk!1G2dIS`? z1=3DHQr)3=3DZqdYK|;s_KgL4QwDP#aOyFo{VH@04ne9ATc*Muss~;16DN;9R!ES`t&!H z!Jc7Yi5nUy#Dmezugv(asKR(TW1FtsRueVmVU3r=3DLMwVL`VhRAvj(}ni2pX>(fv1C zzjx=3DB_dm)Z{|{dMez#BjH9)q8yGqr7GD#X@nV~I|vOcpuE&+|3;kZG8ju8AEMd0SX zOesfSXhqEWMngMpTKYh@S=3DZG+$f?jsWFmx0T@%P9x*z^=3Dnv^>s2M>0_aPKQL*t{-7 zws|x`Sa>A5GX~?^Yj=3DGVGb}{8Tr-jj%%8O!@44PUR&E7Nv{X%y4O;#3wZgSZKU%}j z-<^g+pD2I?YxF?Q%1xU_F4!RwM_VIvg%4&XWlZv6@S3GPuN1i8>nO^K2|4>w=3Df|n1 z3QxmP?Pl0-n)U&S>#xL7C$6Q}U5g)aU0`et;wKx`ria%f|K)}`HX*8rhdRBTa#40g z6RjGf7PHaFUOBP?7zbH5(Qv59#PtV9W8Le%`|RyQ=3DG;^&a=3DiZEuD|;CnYH)d>yvMA!HQwBcXgHO=3DDqDO>NN>P51#)(g7c z>aAaRGX27tc9>?cY%;1||B4(&dR21f6AiJbd$n&-!489@xXPY^1t9e^re~ia2EMgy z1DuNbNr~z;#@JyQWk?+IWfX+Hky;vi;avepju}7E#0Y*}plYlfbzgYUiy7WE0ptF^ zm`E9+?>G9C5T32KwR~oHEjQwm?#k68TDlpQ9{fb-I~Ksw2Cj(s65l~66M>0 zK{WdlZbE4~AMB2@Z0prq4;@mf$9GSPa8T58C0@wXDH5c(G(Bx4w`u{^*#9^2*~y|# zruioSwPgoL?}t^VM)dK3N5i!13#j2ZsnB{<_ZL>);~*bQoCG7uI<34 z2P!iraNQg#7BkISqi48%WUDwDf+{%sI2DR?5b+4we?8_m>MChT86>B)^o8cS@!87Z;^%- z3nMHTT!}?+se+QGF)U%*kXBG)zMv4QeN;SyB>j#QCQ;5et!p<43&IVuu&^vNyw0~- z@!rj9D9#p1FHG%W+3g2$wPq1?!_qZ8!{ z4vE*9QfjvTyh3OU3Ruz}hy!2s+7;s|t~eQ)@ivJ_pPNE#806R;xj}JxG{#8ZBqXv~ z>#0B}KA>h!7XG+~tey7-cF+4PoNPF_BQk0Aun+^O=3D=3D|I0#Znsh4vOOi_h0c1`-%DP zdOVdZwY6^oph{6>9A?6|<7}U~Q^!MuuUW?L-9v(IAiOsp%uT1x$sX)B$MGN-Srwjx z#tXcuz;@I6W-TP0q+$2v*T6wru8o=3Dc{?(uW$hR4a=3D3dkcO8wF@JyO3Uc8zVR22xp* zeQF-1v4w__#^11n%v4V9mYj95zaik5ExM)%UKqa1=3DR8Gmm>omb~f%{oJG%Jcc# zP4U0Y?j-|#_a?8os+@?(JzzV2_|Xs15$i7bA7#+_@h@1X z$1dhoZ^?v)3?oZA2U6Y$2UVr45b7G>c|NT`xy;>KM=3DWCuJ_}1Ij+ik3FRHFdLXPY}020ruwY5f2h(POa z6UCRUN7~ncWvgrCpqH=3DCvSr5T`69wcauaqxUPR2l25Q@Ew}hm8F8{Lj%WQsnR6N*R z9Z$~`8P@fqN{2$T`E}w&tqr{;)ha{uYFqKgp;^UZ!x<)$o%TnCte3t{zJppHLD~V+?bc?yVx(p6HBWlah__hZzDBD*SXrN2(!nUF zfAs+>yoCCqC~AFr2!}v%&Hh*z2VSLFJ(D@T7;sr2*Q#Kj`Ls?j9F4<7IA zG111{?=3DBZ+#{-6o(h1e2WOdy-iojAI^cR*ezRUr?J)zZ|sE63sfY*%X22>{(|9cHf@No@Ry9niemf9F0R70!RQR@^4LAk<-Klq34 zX!LUGghGrFed%UNZBi8-Bu|)WWPu($kM^-u0uSp*ltiXO6ror3yUr6$R{5|iGZb!3 z`_;_NOnG9{>y_Goo?~{U72ybI{hyf-#kM*P_h<;XTzsAAFYQu~NuW=3Dd9QCv=3DU;zW6 zKCzvPVw;O5A77EX3c`=3DTUNEiiRlrVV6k2wK0$TYIdD%Hz%aGx(H`6c6a^y*pipV8> zuyNm;F&geQ5PsPZV@nz1x=3D!8mV$8na*T+2?BJ@!npBW|S4;mBKc2Ojc#OOPRdxwu} z*>AhCY+f6txMo4yAw7J6T28Um6PZ?!iDRyO*#c>*o}ap~fA2e0I4?+PR=3Do=3DHhQaza z#?aZ~2UR+8eVh-o#VAv)x~IUK#B}Ug#nts#?>L=3D)_LSg)b#0WpQRx1X-N_s2(_JEv z+=3D#sX8#?>MKQzQGjzSGaqv*bp6kBEzLcP7o1krcWtOV!WEiJ)#?y3ZaWJdI}1KP5! z+t9LN#v`h6ckeSf3=3D7M{b@u`JEl$40_~Zak<%PH{y$Zzw*L2xanV-`;otbtk=3Dr`GP z!%-mX*eZyQi8g0IZjd4GT60vHh-S=3Dh)2pJ)zr+qlw#beqC#w~`#}aNWb{;U>=3DA2Ac z&q+ntyi5<_6om6wFbuzZOn0!!e3Y7);yX3j4h5Mkv_i1m1V#-YZwEqGw0DkKC#jyX zmsiMYrh1TBu-1)9MhyNQIjp~>A5JvG5D>=3Dw9b+yMa<{dC(vc zHb)lU;-bLO0@RyeO8@d%y}qdi8f?%-KDSfN=3D+RX)`0K=3DKGCcPCi=3Dc`Z$a2!YOLsi!5OmW#8) zk>TJ@|Cja65+<+G;!CW(V|lT!BS-12;__KNg}L5mJwFb_8rK-(+mhIvjLw`9EFxHz zZC||znNA?DSaWmEv#@QOwTyKs#;6$_-_}oefQUcd=3Dv^-Fg*z^PdZAYxG}6d+s78;C zQ*yT1#jKEQ2?TP%IvOYhMEu1`x=3DifKxmulRwmdp_mJH;-?Xm1?$o9J@E|zPBZe&?s zpGCPCv%KcPy`BO$cdysgPHSa?OQE4JQk-y$cebi6B-wBKonG9pJ+ON6?L~jO$H8I=3D z<&mm$>4@|hz?K6fLQ`5o8glcyze)R!qX;^VjDeEk5oUghzLdo-@4VoL0&bHWI zn4;f}YSv}{r2nrY@c8^!|Dfo%)z#JPQM!hhTyPlk#Pn?TnW-z#a)_#sIOZGuh#STx z6vK;BO8cQRxaD)>`X3yh-36O|D|yZ98>v20X*8*o$rL8E6Zh6{jsyZF7nN@wm&QAy z)l9dC%yM-vQ(vb|?l~ih$}--|JzXDNG#t)=3D7wOn|n4y181F@iP$-*~K!d3~|1D@00 z$j69|h(&oJZHq~_&x!bJ~S~8B)R~yr(Ew95U+)}n7o=3D>l)d2a z{EUB$>d&)<=3Dffn`#Pz+JvO$hPf~sWOo=3DiJKHp$QOP*>g;u=3DFnf8-Y$fZMJTGLmkC^ z<2)No9s0NX+#ECBn#NI&281iyWhg=3D0FZgg)W09YX)gMG4`Ew}c(3Eq2k39ZoCb<1)@34VfDdGJc zm#u{UTWvCW!PZen>^2rrIPdo*+-8OvhznN!QbS*Ev*&=3DF_Jt97-;99q)o6AuxLENZ zrDh%@%|yD15=3DIC$k-66+lq@EkV1N>X@HHz7(b~@mntA4DfVl;9KVOUY2~dz@m4?*6 zuj$jn%?!Y@2Fkz~URnZGl2Fl%{Yzne>*6z0;#K%^=3DXd&4yn6X1XtCnvrZ&_tgb)nf zxor0MfuGEzKR-3hcyz2zU1Ju_a!eG27h=3D4eltdGD2OH#|z?UVTea11v+89-9Yb?;< znnj{YI@=3DCb$DemRz~4IlU}N#s|3DRLU*gK-)#cK_M}}2#{Ksp&6dfziN}8cytx9e7 z6C7nR_Cx!=3D`f~H7ze+WvY5lVU!sjr1FhfB&8!Zq}{WSa(TMFP;fBP~+D5%3U!NdNnRk*L;=3D)7a};@;5X5o>W(`%jiS zfU+;47OB3F6tU8XNZi%giwGAsytP~JZNz%KToY=3DwdY1wASvu)H?_-d@FlX~^UPbJC z=3DMUrZxt7k)7YO)3lj>?o;LO`y?xC48`0ni&+v{#QHV=3D4wIfwnxnv;qOwMi`QLQ?d+|Yzdp*tSbf5Wu8 zj>J@%NQ2Se4{u}Pxa?}jI?AVEZSMX9Z1P@2DRv`1uzZ1K{vZ?VeIsv;MRr9)spak1 zWkckW7d+XzSo=3Dx0BM|56dWYA|u>yKj#jB)lPLhWvNek&;)!?zHXCMG%$ld% z7^g_8W7NK_>hd_y;#4h!dzE>V@(-sh`b3HM$*;_sK7U$66_!VR4WZK3EV7+4k*W@) zVnBK$i$X%Ic6_mfyI2BYa{E>#FUZ6U_m5=3DXwKhK0tZkqXt>Ey-E0wPnlaaJWLoq6$vpNV7@uJp*<=3D2hD5+N9_PwNTqR zq>Ee)d;RYo@HXDzvT3nz?WcFBY?;tOh1@8fx-b%US@Xw{5UMo|+w_9BKC;4^BS(sT zL^3);6a4zm-VtwHx8J7&XvQ}Axq*?Ez(_^m??76^xr;+d`}g!3cUM~dyE8jC8G|T^ z#7IKAQ`4=3D1=3Dl6j>upQ2BF4^Bi@wEtHk`2_g{Ky&31)$2LAjibc#lKXyf)i2_;>5l_ z$h>skVY-A|4LC{w-pE`^LOaNkg#J7&xS=3DV|@;f4Z_pY4DD8{fZYZ4eq07CY>wE4u* zZCXRB&K8<<6$)3?zY${%b}zr~1pR09=3D&P*FnLr>6(b@Mf12J+k>5$=3DKG~M zV03s(zOOBqlj+i%Sr-9gv#Y4y8;>7O*J^}f`F6_hKPprDXvzlV|UDCLXYM`y-HH9qwVoKKj!=3Dem_ilrVC=3D8FFnMs|)zdNh&tr25#v*X)dAN=3DW*nk%%dSusgIG= 1 zEJ#F2h~``eL0jT3)q{#HOG9Fy&E_aT!8L8e8eSE9ALcdbVH!aDpSOtv zG6X~jFDQoMXd)+-7$T2H0Ajbf>>-{+$2eR}F7FWMKYAAExz;{73PsP)=3DGGpcWU9Pk z@Ih%WHR+dLU(EhAzvD*J<=3D|Vr$mV{dI?U&Lwmxu@@{|la)58G`6Isp33cH8e{ON!Q zYOkKx@}>|x+sZdRI;5`8P8L(}Nt9lXFU2^JDolux%~m4yrQUBy+k* z34$;-DHX}3-^J7+5QT~&5*vDo?K_uxk(sX9rwwvG3Vh=3Dyd*X3bHDX|?siA-Qv%bhf zpy3%=3DY+M03{uO!IVbM1!APNF8(#y*;ps3Ssw7`2_z%;YkNdoM3LU_gr%W;HFMD8C` z4pYb9W-}O@IQx7N%>TqB4>IRMdYHPY_1njVYBdr7SsxQzi^=3DDXGOkg}3;$_hId{nY zvE^Clz5hWg5l4H$%ne_b!XBu^K6xQ3sm*AJwmt>w;;B|;B<6(-+&UdQqh;OL1IcIc z<5%;~oVy0j3(CtBxHZz~7pka}R)_sft|g8;|o*d;bK~1x5aa{3JS7i1a4CEyz^n>6mT-n=3Dp|H z*9qfmV8dGIOduXO#|w65w6B;@^&G3X4?>3`2933v=3DIX^ff+8~>>ZI?WTSgA&5JoN{$=3DA$>mg~>yPG;KC!xZcN7u{8?kc}Hc5BNr2E72+}S|D0wfx|*K zjNM+EMPi1lx&Q*c(z1<){SBaKo$+28r}paZ87L=3DLm+0k>Q+Pk=3Dw59*iWDjj|PuZp- z@s_diZmP1B?$GJ0WYtyj>E>670EN$cAR<Fdg+|IyAq@=3D zrz1UFuf&>{`>4tLn!M;frG6Q^gh`oV`eBZI!A9qEdDWZhyJ@;zBaw3TbD(I(mR0Tf zDlIS%R*&;IU}tHlw<$Q(vRSO?;K+QxQH<}Zb;)y$NH{JpQS3Hqum1M`SpYv@X1r78 zQFwrco%%b-P$Mr>hi9(Sw722An2qmHK&n9ON^{Kn22IrRRn@C0KQ0v$daTbsxQ%6ue*(lmX(G- zCBIze6scKX64P94dc$d4yV|q&=3Dz?2og*>HqOlNngKz7Q4XK}>9$MF(Lg9Sga3#BWr z2IQ{;=3DZ>>KNbHq2&rq1=3DGQF6^J~>P~C>wUQBQJ3+m-jH1ww05-%e?xUul$T7ZO)t; z80crd0-iLVGORsd#z)Q8IiLr`<%3bKPW^kg|4d>}<*-&urF8RUuUW?!n7v2hFD7r5 zr>6pbBiVGbyU_5v%d2ERn;?8zFj&_t^jiP(+^^%7RyZ@Br^&8PnkvAOc+*0YyM5R*E3S z0;p6e0Rm(d6f2;ps7Mi1RFoo0hXm=3DCfbb(ANr-ebB%w&C$wlw??&jva%uZ%^-}lPC zw%_bmUK^!?yFlbs67ln+9X*}7YYPPdotCkyiyb|7xt0m@3=3DY4>;dD=3D@+8Vb{&0=3DYK zq%bWvae4Cd$WT0YYGlYFVg5%UzXiJ^!>)>z(Uz8Lu{2ullE~{|S>opjYkkCo`O!WX zewoCX48r27j_S;m24$+pju<;XUl71!50s}_aYxJf6h;RMJ16~ecfO0*x%M+9Ew?+H zyD)?9RJCZ?N0^=3DLbID1VpZr|tqDokp>T$`nSYh;Zs#+!Nt<14VSZ0VY=3DRUXPau+yh zmaB|yE~*I&oLmbo+ch0E#8Ssfn=3D6I#m<6jL$=3DB z<5O*XLFWewT&$K?o6=3DMhRwtzsxu4rQxovixmiX1PbxHiH*7Gw3X_jk~Qvqs;v&K2! z3px`ylb>0t1nv*J+;0xtK|2>!tF=3D#sx$&!6{J+JeHkX{$1s%)tF1f?DmaD5XL%ijg zjLsQJOZ@T_zmht4wZmk!H`g+DeyM;jPuwNqNJg5fRb$FexY~S10Kb&>zf*kc$gw1~ zh{3Pt7lN00ICk!IHpUajq!4Sl(raNtl}rkiR$W{Bw*DHXE0jGljTszVG-h5GqVrg^ zmd*+$wUNc7JyOfz`Quk;Yg1;E0fY1Qae_4?3e?~g^77UYdczMNhI1!AAnwEu9~P~e zkfItJ8qxSJ`xBvZ>RXT4G$otXKd&6Ou}-tTzIKsAmRy@B&nMM-*7jsi4&Wy3dU9*( zqo!gzbMRl;UA0+5L7}uQ+-`i2vITykCxdEEAVuPrJ--m&@q&5=3D?5n z)~)H01-R`cl~E$}Y(*GThD6nc;fi6DYEd?G9cj9wXB|nX)(v1j1Ssk@Lt<<)RS}>q z`5>zB^7I7+2~Pk>_zbmpRaS`42IW3PRe~g&$^SxrBUh+SJPZ%=3D7v2vw3DrjselW>j z*7&3g>)$5G)0rP&I3~l3zXyW~K{71h>;zdJ!gazz2x_Eh3DPvR)=3DHdhpkjkjhB#yx ze{da!D-mJ?YjfOFP%UoC57X20Ih^~)I5TrlPsqayDqMV8&`x)O_)Hj+5kycV;T2&x zGC=3DbYXfBipL>7Y-{cN|yktDP@RRqw8;_OBoEFLGow$wU6 znc^Y=3DO1)^95UjEf#tMcp8{llRWFn5!m>t2dl;sVuna7s1Dk@ksEr`RbgfZ)WEqJR@ zAmIYjXIzXP&GA2Q>Jae-*$yy5>^jtNs}Qb1nQmBUIeHYvN;D+laV4<2mCev$c_|XU zPp~S2wE|;O_$w#z5@9Qb@TvqSh|>ws+6p5$XmFAICGydLQ^_l~M#vC?4Q$Z^nkVY! zmOLxZ@U=3DYGGFhox(I+xTiXZ2 zvDQw(SoOke+L!gTdKP?{_8p8=3D%4{^)UMu zDjvz+!m*cGz8wr!IY~5;`HgV)LI9-!3|kQbEc=3D(n1nFdp<{1-cFaXAKlw_6wG=3Daxv zBzy@xhSvyZbIq~uTY74%dD*a`p~$8Hm|6(6&>NZC*Qrd$cven2goY%N`1?yBE0|0g zk$7!(nY;^D$7z%V;&Bu8*(MMY#wM?)CkY|dUIN@tzr5Zz9t5-q)8lM`U;Yj z3s4e~S2k?*7Oa~45CI{zAq=3DwexiLR>xcm}NKd&H9Cc@$g6ErxRif4uT!0K?wmKFmQ zEB+T=3D_zg1>Rf1HGCW^8fYYkx!H?DqJbAvrTB!)Bq>Nq0;{B1ODUm!H6e)}@`BjP-N zM*4UoH5ytd zv8Y39urxHpVlw&rZgfui&CmA zeto}y7mXEajF5Vs!5J9eua19fq+*aj7$;%pWf*ssCPa)=3D7K`QK@SR{?tTg!{x(Y8ca-NH-auqd@IDVtL%*=3DAVyAg`p}$X4$coro~QF5USW5+?7M~oxcBNP)h;s#7P+Y@d?3>n;m9B#`=3DNyxIB_EkaQjF!_3G|G2F zb!x^1DAX1Pa%HU1rLOa-=3DuX{+2}hD%*)I2W=3DqFGt4170fd}`3MOD`fddc(Wf{;&*@ z#XZz!A)qN)Y*G|3L5JOk4#0J3^iOy#zj$X+o;bKCMCug6)67Hix|L1be7?B1=3D(%{@ zwCo_EBR+k{V-LYEPk=3Dfd#=3DYI__$$YZfRWWKm1YI~!1#C(+Ht8;VTwi_@!*6wW+S`% zY4}F?^+#C~6Ojjj0q4u+Q_jN1p>=3D|e1rfrb%RA-4Wk)QFF{Uo8q;f)vZ5N_wJ*;vO zBAx@TO+Lhyt65=3D5n?7zdypccLo8QLJvNQq2zK_{@!@WqWeGGAMB3xQ1T%ir+s~Uj% zIrkaap!kT#h}R^WUwoS@RwQi5Yu5>~zt${5%`_4K=3D9&u=3DH`tw5Q`Etz9uv9uuHe)O zSf-eL{fXl(CF@=3Du_LE(=3D!y!G_JDcR(fH#E!K1puw2TTCj9}k;Nqh+Ox=3DKH@Y9Zja0 z?993+Hfe5{Jf7MH7sryo!$?u&?Nn=3D0-$wm9lTWjYO_s3=3DPSucu_~20Jd0d>!eW-Li zO>djU79hdh;3YlR#}~fk5j3o!)S6tOgyEv1dk7d`t5LHZZ4!WhcAk%jyC#Yrn(8fW zheEk4j-&EaE)|Y?uJ`xCdJ0e48WN8(MZVk!_|R!hnv4_&weygmf?@bx*>oYeqm6;@ zr0vw4XFFnNjoR}$aqyj}B>bc0xr(vPw~Ugt#!+E7Ty`t`;D0-GuX;|PbD`K38!#P$neAT}m4=3D^4)>`4v- z{Py2H^<2&zj zr&&MMHmKGxG4iBcE&Y_zW;*ML!7HF}L+O3N`=3Di^4f;J6>cJ5=3Dwzz9Qr50>Td5FNl0 zAK9sm3L;ysK7*(!nQvr_{6MaKnj^OJQi5Y*jOKenfUOO-zeE4w;$bO6j)gi3a}#;G z>8nvvnQQnCJCyHkAAxA;8_Kmr)kfjcf6D3^T8paiaAD9N4%Z8sh`DsLAv540b{KJU z#Iv->=3D_t78qB6Shg35#WgMIn2z_2iIqI1Q+{y;@$ShVc9yvbVHBt{)tLrz?G4-=3DPK-(D;MWZ}cvPg`kBewq#&c!%LGLVjT z*7AM!VV5hf=3Dv>mF@Jj9iQr-9a_yMV=3DqV?K_g>1QOiUx!CG?6~xF(Ye`NaTuk#c@w20D02+)fT&Uu1Z=3DWD9oXVWhHFN{ zS+5V0(jjOy&r3MXM_BmBY}d4?{lnX7>|3OY?0{__ssB`HymQ&tEl!J7uM-$B+rcJ( z1LJJ4rDXFqFRYS~LRX>M>wUWYC*}_wQ+>T&lRipeY)+P$^%85(f0;&dRlNMzW9Ims z6#spIi{;Bs{>Z|P{etITUvtjCEef1YgTl3ZZ@+=3D-;b%K4-DX=3D$#74mJQip2^+IeGf zcY`b8kyTK#SfCsDwe*2sN`1k@)|<-iA^NEE3Fiz@S}5Q5h?7FnhY~FxE2JM2Uw0dP zT`*|0UkW^^?JLJdOC?9Q77NRuqp&1C{4kjF%{I*B-s}cp;1NGR?%D@fxMiliyeqwb z)s-@h>NZa_$kWZ+w_GM0%NQClv8%ZE4t`Q>GvegcJ(4jfxL&71dIhw4VRyxG^}E5b zu2SU@Eg|hLTaBWs7$L&Am7HF=3D^jXX7*=3D3ja1|N=3D3BnH!-${3BKH|uRKzB)Ebg6DQ3 zua@5>_Ojw zzBdG!AU*UoP%E#tb2Gx{a4n}jdeh5f(r+au@6Ck2(FK;)3yyzN#YhK>1inQ)uB*^b z$#y;=3D_WCP0+jG|P&|{e{U4&)=3D*nJdR*wE!N-!E6nSTX_DEP;%>kfvz=3D!2DB)LCN_c z3Q#|T+_d)+@VceY7AJI`7DDqabX-J|C^i}>vI)Pr#LW=3D#62lUd=3Dzld@pb=3Dzy4zFLRV9<2TU!Eu ztPh+nfn43N?+yI*5%bM=3DjPhNMMoWOR2)540q-$W!LF_w3aG(sLZ2aO9hy!k%7e^YPee?6p;33TozbYlzDZ$wdpGQs zTqr*iFW(ht6yYYHDX%O!E#Fn+iuFY2;dBB0el89nSHH6cI{85|WGEN%w!@x#1-%(~ zsqRZvS=3D)`>VuRi1Xn;65$UcZ&c=3DX~DbV_K|nr2Vp9jtEDUL?5)WNlkLOzAYS!IqGR z(Qr*!QFl%z(t*0%joaXc`J-yks@^)aTPoM!bPCijW`Ng>VVqmx7Ce+k>`@zmRXLy~~Nzh-h6r(G; zT^5(1eKH+#UGkCMf4pPrEtGvxAC(vA)S`iYHG#>5c*Pe2Y~t3b=3DdD`p#^P0h7GWpd zyn1IKm$~HFE)E`)V~>ND&G)7F%=3D$Km&hW}b?NzNwghP!c>hv ze;t1W?RF;a;`3uW+GC{RPD_5>cA)WA&pzNzw3Uu8bx9u{M8%^)FeNL&Vi4XHjhgS? z@Fq0>f>vRV>e_zNk}=3D0~I_cr9keWX1;Q6k75nT;eMDIRq2myGuV+yYKl$GG*nI4U` z$ewz^uae2$50Vy~4)X;b6bWox^Kq^h+;D z+?ms(P=3DfDJvl=3Do(^p429St<$Oix++)aLZ9wZ2x=3D+Zf@=3DHIN2ySI$PlSp}L$mt9Dt= u zxbeIM*}lu#1(zV+%Pp1Ak+n>$&@au%Z3YlfwE>(uRbYv*Lt9*3~~#ucFQm@_p)>dPrt3-Jh(9G z8}#YRJaP_k5BHh}PL*i>f`FQso*N)_OyjG_V70th) ze2P%=3DhYy>tTiqm%>95Sl=3D%3E!R&LAO=3DhrgXeVcC=3D7JpkUz5qK@QdTG%bkO-TF&cO>oLO7yeU9^1C3*C*mmG*qL~E;H>Vb2m2Jy1Bw4*e&$*C+b}#|Afw`78;^e!Vk79gNe`LodBm(x5@_G z4pm+{JzJ!JXl*cI{r1R*NOjbHk*aUJ4({;B-vJg*D~CLjhDMX>_L8Ws+=3DPQT>-{9# zuIAV@Fr*_oVD=3D2}WXQ$icc9_%hB!L)n3P28 zqBS3ox;a4~$Df$B=3Dhvkj=3DKj<8HGK|eT1~VH&JS#}K#3~igoSVGzmjY|eEV(|o`Xbq zfxqRVyXvu133?#|nyy!W#eg)M`zMI1MMW<=3Dp8s$E{Nt{t{_@y&p8el`edOg|ve3=3DH zFhKl;!JmBB@E5$;zu=3DAiCE)H9{6(<4SHE^sg0J{Fvs%RGt^WD_F1g(*T4m-a?0V&=3D zZZ~g#Ci(cCz=3DP{`=3D$A^dtl?|^l|0H(&jPPR0?+18ruT}JNmhZ`<^Bs|g>Ur5xqs7&{CZqO* z2#!$x&6QTl%@uQb+%}}VwG{mKF?uMv*ROl+F5{!_dS+Z|Ir$AVh^r=3DLc&Q0--!m$o zo>p=3D*z0E6O#C`-rhQ>gbbHSJ0{A*+5wp=3Dl9P#4`k2R3ZXO~|K{T(A``Ln^+yz6)j( zCD$D!*_lD>BP7no1O|I8?+DS$o2#oK>c1>NQs z=3DY7G%=3D;%@G?G?$^rwxyvQtRZ>sT8Jigv6P4R)uNvo$+Dd7dTFuKg1!F;E`|jDa<2 zS%$uyTs8Mjfm+HOFDgm5Moz3>f?3u>|EjvOo{G-(R@d;^m}usB!Ic-KaI~F25XrDTHz;y9BdTHJ$KF*rs7iy_eu_`~9 zk#wLT^5;HEP{TKkGZ|_^rXO3qb}4@mtD=3DNwJN*bR607pa+_UyrXd%#%qp5b@FL#qs z%hsl6cAfQxWi}QGfxAvw?`>NfFg1E_E@b6eP%ew9*%yHx-#|IhOaHE_cIABTCaLmZ z`>2?ucYB`hTI-ewe%A1XnraxFy*avfAu=3D$tmTCE6(x4waewMH*Br;cBrEDK?q@G+VUdK8sKl=3D9mT zWSi&3n(>vu&(1d%GE?^-#<|K0MpwJ3Ew>x2^U5_6q?&YtMxG|#c7S(a|Uc8)nsoenmD|#s~KKi~T zM!6X&w;{$|8*^#1rCMNocin@tVNJ{4IZi%pHxTntb@DitAkJf3RmF$TCC*g(ck~sM z@P+^9xc-ED@%nUL8fI(15ex=3D6I_$PSJi%BdW}dZT{-}|9+`;Zlxu5Og@!ob*uBA=3D- zqadLYqQOhpK;alVPOn|`=3D2_C(7t{6ycI2~$D)A7RUBy2a?UenVsz?{@qi~ zRM%c-;i56=3Dbs;*;WRzQk2ev|S^(6Cb_4_8HmnXpot>+V-j95n`ueY`wdkqc$II`ex zhW42@`vM6S?3BFB&oZ7+;c)t?#%t35rkLM}Cf)D@PvAOfYs3+-M;* z3_kDye6ip4yk1&}`sD)0avrY^c3}iB3OxVl}gA9B9 z8m4vW3zI>Bfn)~ARImeY!eC?&#sryEW_Om13?xb(?u6uogoF+pIACOChYwPIfDAdKAqT-#IJB7X9=3DPc%inar7iP-c0<)0r7&4-D!Jb^-dOir*BM zzj^A{^-V!R3Siv`Kh0fKf6(2fbR>?bqn&|$}olfIS$d|z!aTIQq92p+l$e|z`dAs_LO6T1v=3D?SP% zztQ!>VtwAsP5{#L*9YV=3DJ<2(0tp^0}JpaMYds2$g@u)cui+y}mVEDPm(W%{@mKXhf zj^$o`-{ra1X7yu*Jfx!DQV3=3DUuuag0U-Cq6ZFnW z^^BrVV(T(1D4;shoLqudQHCXoHl2|CX2tl?r4a;d03Eh=3Dvh*|g!+Ahukf#dQf}uCB zEsV-oz$H!7bmT3cD+o-l6Y};~s#W8of_GO(5<$}N1ZQKwfc*L3y*w`~P1~jI%Okm3 zqJCQT^L1$iLn_;d6wbk}5cHh!{@HOd(k~~;*P9NnF>6&lzI|Erj`}w94)r+?TYOB$_(bSmEYx4MM;VIqmA^iGcz1X8#6{Ijr>QMdRIhp&(%UC4 z5YS8Gs8qdBZ(GDKKBO)HUo!r=3DgHnHf&HVMmej;GvV_0$2aVQ+z)R7-H&)Wu+Jv}>S zRPbJ``0ak*!*N3=3DuO;ywBQpjc?(8(OMgp4&6Z(AvV)I$4I-O=3D5DvuBw`*V@qu|dV? zgulZ!w@lcnu&D6;;Ba>{WnE_pE(!pFk(Ols6z>x7AHaRd6BV-y#SlXmuOPe~npS6u z&IJhQrQI1p_co?9bQwJ8e zlvRQ9ab7Z&mBNZr5+BcaL^|gSi`Z>D!ePY+0n_vl$`f0wbe;qq*O+2|cp?v4J5*L*#K0McX7Bb$~1*Bha6`QJxbGW*7BKPApS0XBR#DMRpeb9Q>WnRI! z?URL-E?t0TO?wKJ^H<#GY(Iuebx0W&mjtXGt(G+8L2THB`b=3DuaIb$`W_Y~lj@8Z|E zEkeMdx7O>H*v=3DM}ohz3FviD-vvS41Q}FXHuKd(`|^%^Roh@rbBvY$vvw#Y94arih8iY?-k8{7z|4Pu{V4dIh?xm6mJNx4RTB-IXrS5^GH{W~g9Di2e|Q6mVH-_3gEg_J5B6t%#&4)3hvwj9qOx{2v~}0cPuI&e_GT(Wh97{QA+ucUoxIX^VEW3w@85 zZW);9k`8P;KxV`U$DMb!kTY3^kKT;qWu6u*gW{XSQux?0XF&f;<06Hxcl3GvT>uzT zDRki_30u7#lRj3w6ecYu*vZK=3DOR>m%bjc~B~;2N$CIKwEa!8XHYB zS#E9DGq1z47T$A??P~?k*UBZA;YjJhEA4Zpq)K;y6wQV2t`NCYaq^CX#8mDE(fD&@ zcDAqVFm?7!kZ9Xv)IDsZ-oud=3D6ip^wqn2YO$9lIe<@psRWteys(uV{+QvCSY?GOd!`hln<0&eBBUp z`9xyqk(s%PEeRy1HNZ4~8yaFmCdC-1)$u-O$-Nv`~Jq=3DvNV0*M8 zA7%aHm6vw1x^=3D!qDz9bFY|-WYuGgX4k^MDgeToA@KYgZf@nADPlsTUSe(rf23>w*|KgG?nk8A1<$hX+t_rHF(P0|Fz0>xVD&b;Hiyzdk>9Mc})V8mjle;ySXU{P+;8G+^EG zAm-Xm5yX_>kIm2#wt|DVUigKyt$oDcJvO}$qIxxKi)P4KyQJX_HNFa&npr*^3c!8{_#; zHow^A*AoH`nE6Rw*qZNFT*VXxuC&tB;cy?55YX(^{f`k^{9H08O0|DKyIH+jte{EK^ICycuN6Vd-M@!ox%>6o>N9_05r?M1o3v1 zzl-Hw#MFZqk=3DvslsX&T(Bkaj$p$H= d zl%5cUZm-P7A%N!*9ba|l_p_H#-9a$zrK^W{Y@1&sIKcXMSHqbN(i-3)PF?7oDf4v7 zyAh}opR=3DE&GqCEJ@S(s8h$d{x5<3RL4wzBiO$w*_B|71f9b;+Mt- zn{Z-w%NU|@DOzB+ckw<$xKQnudX29ZVVIX(qjLA%L$uGJhH5R1wg(t7sA??>4D7~3 z{{8flGv>TYtIgAS6tUvEk=3DM5L;S3G|--0#5*Nmf5)Wv)RNr!jN?S_z3ZiHJyI%(Za z>Up6<$@(3*+0(Ma+jyte60l}Mb7l5_d6p z{xVyOm``{{%*#*Y#S;+!?ymzI77A5xEvloqpHVTjVb!OTJ4|qp%hEvt_DzSC4hTR{ z<$>Dk;%;lRchiJe+!`Ama+y2ytv^%zxQZY&xM%}1f3TE#+4mpKfAFwK@^5&FnYs?r zw}p4GHHoR`;XljsKx)3EV(dqyHbuA>UW>;=3DaX))SfcBsc>P*DqO1lc6YrF)&VQ;dTH;a0v}*_;=3Dt$Du7aQpQM*R0?Q0nj(MS zeqo=3DqDw!Su#7ouSgz=3D4s#=3D}7UIL+u$_gBfzPFcK|Id-XiZ#Q=3D6bgZ%i>p7*0%AYm= I z@DQtCKlWcA#X|_A5#8HEa_q?(w3)DD{Tx8NbdY$86zu?Hyqz)XqW-E+p$Qy#&hDj6 zHdBy9D21=3D5Vpe)o`&nt12JIfr&`5J97g$8rA9w(Az@BJ@a{c;bA_hV}$2&tvdF7mt zRC-kO`|0j24akASic5rs3r;#vO|pmn`r{x(vzZmLIcNI?E&i$Oy(7WBoyl3*X=3D_S- z1B%T-;4W4aY&>EOgCE?fAG)(EN89L)Tf_G$zjE86Hug#uyt90d{?J#ZJUd_gwweI> zOBj^Yl^^S=3D`|Q)zyqn%NS^qNdH=3D4fM1@YDw-P48?&rRf7?NF#YyZahZ4pM}Ii?7EgXl>+OawmkDbGD0@`1(lB{wqTN@8 zTb!e*JN9ZU?LeP!Y+SPJ*ZL&~0l9qT?OH!;8(QHFexbx?XJ;*Wt$E5=3D@ zDRzU<17^XJx?Kp*AXXe2mqXp5utqN5mWQzXPnF5v>X`ptwiwyV6E-GA{!`)eKM4F) z75)27JTSszTiN^C;P2C$9Gy=3Dd zJzimWN=3D+sKaf|3k6PpbsGw`4_S>p{Z-|pW)fH?plKt!AwV`|m+I=3DJ4T)w4wYx9Wfa`dX3#1V@T5Dv#{V(pWz zyK6(!ogsO`Kx-H0^OBJ9NF5yFc>w&nkww=3DpKPU%1%ZH)VLu=3D@%ET_tZRML(J)W zd^Yv-AK1$^2d(OsCpq1^XEdA>QY-~)2jsk3E1pC^Ns{US)s~Cxvd+yVo1t!XSCt{q zzGB%k&f1iuPbJMflSe_5V*0V-KWLYR>XR(~xgv^2MX$kE*ivlm=3D8yvdf5(O>rM|K- z&#qL@Fo!iAq+{FL+iH)HORoL=3Dh@2SiXvTMLz|DHJD_6%h-rfM&gEk+;km#$(ZLuyN zPh=3Dh(JkGxn4I9vzu<3SPa(97c+nT6}!OZpKukSNsQ@G))$i+r>YkDO=3Du?w{9sLx+< zI*8#WJ9-z483}~XON8!Kp3+`05iG9dXaDORB5XzV8T*oJx__E9*=3DM^1*XU{*zxBIw zj878<5^bwJs)CbM6Q0-Ds@HelPllW01g~Q6jBjiS8tl<~%&D?}b5_uOR zC3?9LHH&?E0GTWQXi)&MTdm$&bTF%m9Hjk*RH*B@b(-D~!cdIh@4F_CF=3Ds2>ue5v7nSTbA}=3DkekQaHHM5(EU~P1)Qmxc|-L0MhQ4s zmFIQs22+!GEk?x*92egzJwbl7z&UO1*BvzuZzmnyfyhyz?x3h1@_6!8OV~VBn(TGA zGc}vPlYREVm5eHIHY~T3W7QEjesk168}{ai11_W#y>q^lHV&U}Xd|r5S~g^IzgIE? zzlqDY`BKcOVW$z9RpcF^;~PU=3D91o{bhzF1;qP8m6NPGYBbw=3D2#Xp>1=3D z=3DY7a&1Sp<)FnG`H{ZnyYbM9BI4A$_Z9B{`s4lP;C1)yL@bX_-UCjDO&8qI!E!l^SE zM9aGzm7;L+_O9Kebv5H-32LeR%C1yb+%v4wD4M)yLBmL{gWH^$j!#IIf zM87a3;Ljm2$eHfAXLzmQIXGqc1ZMj(o4c}f_}ep>1%WwCCYNq8j%YXgOdxsnTfTjI zf~(nNg#Ao@S=3D3sN6XpfzQPV1P+wLQevknyBns&td0WaYSeI5*N$VmDa`p+HqKZj$B ZK{5}#DsLBdC5qK{{D`sc6K%WD{{rQqV}Aeu literal 0 HcmV?d00001 diff --git a/Documentation/admin-guide/mm/damon/streamcluster_wss_time.pn= g b/Documentation/admin-guide/mm/damon/streamcluster_wss_time.png new file mode 100644 index 0000000000000000000000000000000000000000..84a9fd3de09fb0529aeb1051f= 1ce33a0a209e86a GIT binary patch literal 8830 zcmcI~XH*o?vM$}szy!ib&Pg&1K?Nia2m+EpKv9B#iJS!_G(nW0ASxhnL_`n~6a*v=3D zB8UPqNL13GWDtfVnYp9)ob&Fx>)iF;kN0}@bg!=3D7ySnzSZ&&TAngk0o14a}NiiU=3DU z(a2EGiiQS+XlMWuf|g1FnPms5563J_&gxSr6e z#h#Ks2Us3!zD6Ch^_^hbUi1y4PlDR^HBfk2^- zheGJC$s<$9`0DN!N=3DrAPBAkN5Q-qqDnwp!Not&H|CMLACwe$1y1qB7Uz4r&GV@CQK zo)4g*Vfygr2RxEt;-#SxqA}9bIUADmD;JqwDumI$%)k6G2;Xs7L;TTE{PhfSax5V7 zYcGq8zqW|qH6ALH|8DbdNr9A~I#!sI;}qnJR4(RdaV{PgA4?c{Y{kBo!@ zL5)rjlVCuwRSNDC%()xZ;otFEAcP@6{T&?**~b&e$d%2P^t^?TRnksUN#^E3@8NkYQiE z{KE4|lw|*<_I_YAMIoGC*fC8!t2R)SVmTkp;k!Vy9MI6JpoJP|Fgz^%YF9raoDs=3Da z6pJ3%@)$@;J$*+BW11_uK(743n%=3DbZqfHvWTV~2w-t|<${bl*kss7bZtJZve1)q!! z)|?(4inNKM3M7jhlZ@E(EKM3Y9!TCP$PB}Z_wLT+$zJ?JxE6uKk)<%0aDy81%5+l6 zkPAy0a}~6?R=3Dhj`f`e$7=3D+jI+qiD#D?SUk}`5vWwdev>xW#KKAxHH~F2M#j)*A`Nk z)MOVHF*ulJX;M&#!yQXBddP+_Jgu0J1_GZX#VxIbgt*&8&!GN)c1nwddS02x{TcPY z{rDNW8}ZSQ67ByekeCD0?G|R(Qr*32-=3DuDJ9FF-JdYJ8OIaP2ZtdKe%QqfQ#bw0!} zsrOu1o{2I8f@1$UyZ@lTz@kfYVyvS%>6Z(fXn{Ur(v$gTs@P7wW;(D)=3Dbp7p2c4^Z zdO{>4EO1h@ubEFOfD10r#4JlUH@3d(Tz%X z$>bZ=3D{ZROVoe9vy;RNfTvam_d4Ri=3DBc*aiYbqVv^;}~mM@l~B?rC)rLQ3z znN$a9Q=3DGUfH%;AyA^Q0bS-D~;&GA! zz3+dbI^vU#Gas1)L%fZ}XKocAZ~fYteNVBL+Y>cM3x+|Emh9cI#^W_h?grv7odY~R*w)rcfCR9sX;fBZ*aBK z!!~Guh?i#FwAi1^XSKcQh07LbP}gA>XnRj2q%*@d*iIg`1*X?tJ4JeA-@dfHG=3D(qX zZ|U<4-e)sNo0F#j=3DBeO1BP7KCKj;Vsex)MvAr7ZTMI;Nt&_FT43jD8~jCR1_wtkQQ zPX2fL{y*~izwJZ?ssk-hI(>e6HkB48%kyu<`rla%AUiGyKfAU=3DvD@|$!r%T3;}dgr ziBVK_G-zgkZCGSLy48ygA3|l?LYMZas0~66(`j4K0Gj<`q?$i?-9(<$_GYISerJ&j zxt0q=3D25Ky(VGwgq)Bt$?;bymCBmx8?0!_WZW1Gp?!vsbQ@tGDdt?(;)VBMC|+)e=3Dj zhXvLO&Zj)U)G++2P6WslQ)2J}bx2V<qf;?3ry3%Z^XjN=3D) zBoU^I%h-iYnN*#@tvSUA<5PBFY`>0i+_{t~TWpR~mK7JR{<~uAgTC8v7`+9Hyn&mF zCmP=3D}{bgH_Z#qXcJ*!L$af{K94b=3DY1MK69~5EZ;AnYm?x=3DA*74?Ks_;#%1Li8!Ud) z15K5|9JgtWP}ftvmGpW5#+IgX@w3Gpw$>>U$16GY*S6HLNV8QW>~iaI`U}txo z&d5EhQq8n-r{tOEYjjhg7VvB{G8)@kh4!&=3D9paOICEO|eF^r^`zdCU4)!*^B4er73 zK~LW#nsYFXTn_T+SbOczX%o?Q1WOVdKy^^QflS7N6_+o1Iy9IP98&~g?ZApfHsm=3Dw zqmYcnpOY6q^3g`j5onZ2F|6WnXRYu$uO6_~gh;nI7%by`T%p_LP?2(e|BjmnJ`i}$ zb64}Inm4=3D%`ZY80FV`WIZ#)kwm-Nt|fO*#Fq+j```w+q#r-SWcYhxF;=3DC@-f7#;7~eUQeFGC6H~{!kF?SQ73gI_n3WXruq?_`+aH_)|9;j?-X;Fa|TQn@+sDb_cl&r8rAs0?*ffSv7^5b9lzX)ION z(!uF_gTcLvhtDCw8mPBsa^D67)4ZuI6Y@BhXtC63#5(t{$4tP%IyW)W__8>Ed_ssL zxsxjDOPHsmJ+4jV#?@FzfT|Tw^N08B7uf$!y?4D6L+cd5D1!15I{Kg?8Va>|3Uvz& z1k?yQGZnHZxLi@%d=3DT@q6{B zX{Hhk8BuCL@F_V>*gJqB3=3D^bC&HA662y|EyazPP}--RRi<$F zIXbE2cby-DyHB^>y#~m)R@vP&a{*Eue@lRQte6UZ9=3DX{(htAsJx{z$Bv1v+(hpGa~~AGr}6ImyQFj$_5*s5SG2=3DAaJU5bMrR>)?ap%m3(SL zuiWE~1z}5GSx$%WD)baVZdc1LR6|Ni2AGg%1?fIb85ya-7=3DrGij*yF_p1h@k|FMP? z5?m-PGa{JeJ%j<91GbVv_PCnSj3ncuq;6h0Jw$P-!E5xFpGHO)3nwpA@XA_YVHB#^ z>-CLj=3D(GqD-bLm47`grXmN*8+(5PQh52^+S&;Y z^3goh+?WQ2=3D;Yl5u0`%hiF%i)K+x7t-97RR*wPZRX@2Vthp(H}Mq=3D?Z$H{o8J*syF zrXj@!s_7qkaLu4y;8iN8x^Ew z*X+b7UT+&`CytH|9{ZB-gS_U(dQ8L`C`$-MpA72$a8U1mu3Z%r=3Dxg7(TNYu`%WS4V zRa2E8zX&I>IiY9gHgsd8nY$TU&|zkDs(-LpHJ$3JNzTl_Rk^96@&$gvY^T@+XZKQ& z-9oNn?`rEde9HQiis2=3Db3tH74%5MY*l~?hj97GMlg|J6H$hm8R^3Kk{9v~-q~y0H50b<75PJ2wg(dt0`& zE%oI>R^Hp+gToU;63(QRY%uQxkG2Tj%e)$l>}ZBp$sF(lTcQgb7U(!=3DM8YGI9aKYc zt14#+t0Iekh`@7puUq||(2~!@;WXQ$N@FE~dJ8#D8Exc1-x})Lc-LVw=3Dy*a~W%RNo$LJ7PHJEo1~+ z(pc?*F<_m`zK#iGfCABGd+!c&v!U;EH#kCO&*WM&TsZyQ)~>IV@za9<4h6E5-*gmmPHhxAFw_2$~#e zoPVE-tS$e}LJUAXtGDn5{l#Zz@QmJ^X)c*DDON=3D=3D@ z@I*5k`9&BX=3Db=3D1a`M?h5*%W1Vr~EVHOBx=3D4H_9ecEL;-*C^k312#nOr1MAL*3p-zp zP(Iw10z%Jjl!i;;B^FF@=3DBF`4g9gZ&n#{2@ZiqjMUxf6F>pEG;?LD_H0dGe&o4;La zwoex!Dg0hLiWGKMcqIa`w@PX?>WD=3D{Gma|;uno^wBJim)=3DcUf*Ybd}0t$VAGgKQ?_ z#d*^XS7yn8oRiWBaF~pJ;X_(u~12-<;7niRL1=3D4}>Z$ga; z?E;hQVigAx0n;b-g1=3DaKd_{3xeD7HIvhtD^D{=3DO0NYE_k*xz+OPo3bO z6+Q8&kArNu_!?57Oq?{%5W}~Wqla?_+!TPD4GwK*K;vbY zb39;J#tC@f&Hiab$eSr^;8cc^4lW|B7wdd7?+hWSESaMM8PzKT^kijN0|%!VFvP6z ztagFbMKJGde=3D}S|h65BZ&YK1RK_pj`o$h7olMaruPeBZaO0O#c@=3D+q&?0~s_;* zf{Vddrtd&-8LO<1=3D2D-(vNJ^1825 zGgUptx1hrvDHy{`49K%se40QS>e9#695eSgjs}iW`aYm*%kjF@b!EeN&b2j{-<3U3 z1TX*mOIB$5`aAorf73S)8e?b&P!cSNfSyuonS zpCRir+FxO`Pc#^Y};;TB_b#qeJ){ zVXy{E)~4GnaWIV$O$XCB=3D=3DmIYZ++tBCN)>^9wi%Ka66q5*16eq^4Es>7tQ)~zklTtG0C)f=3D(%r8 z+TX#i=3DxZNRktdJKHa5(2#b&qI=3DFo${UG8PNhF6HK$B5IR+F1O7R0CTX)cz`HPeZ`G zZgn{(qU-6&@3ZUblJdgT#XVTeKWVfa{(}}Mi;cKZ@_~x5m6vJ}NQzF!d)e>*ifj_S zfC)~rOK&X)Sr>PprK^I{)Vll?%xaZclnbFIO%%s3aS2Mnmo$!Bp+kC;b4?4`D&DO( zMsP>Ula3MJFe{XPl)?U&q@;ilX&|i!RwbSxLmIWSj%d&)Jh|Fw?zvD?9m*Bo^J|n4 zrn!9woxb=3Du`1SP;xW-H!sQlZT8nTS7tejWNg$93jthvDiC(vB7J#ybJrkN2s9b!~u zMe#chLPD?x`!#K_E`{t5-|$LYWDq84!}`%LPvN%sK{~3nI1HTygO}aR3(L>-Jayx} z8mr#d_$k*+RtM+GpDWJ-3gnL;sd$Zga~LjyZ${BtZovYrZb7p+LK2*JFDdu7swIZZ z=3DTk77O$X+ExR1P+yVG}G5ux?RuIR2nZo5(4m6$}IIX#X=3D+Mut1J@ zNEt)c_>s9@2Q_*5r91bXv*QZR>jnUMWhK~&E1E3C8dV|Z28h!<)6edRV+uQ~(xxtf zSG5U?pN$^hq?)zMG1zoVOC4bBRz7l0xp)>GFa7{A9C!q*Ya0+DwK=3DX%$>~y9$u5_Kixi9vB?V0FyN(^odi#n*c^;O$!3@3%Fn>|Ge%i7c!dNuw;`WX(mQyz~Kj3IC zzil5Z@tctj;c^KGZ}5v2g_#%%_4b*rS3^fas<+WtXU=3D=3DWt=3D`Z0HQKixuN6SA2AZ*? zPaCPwA*YFA_a&-xnn@iTe+sjqe+lTLlK8J7Ui?I`> z4>DoYNN!Hms4x5DX#I%yz13K7*H~Lzhj2SvtqQfJl9USV=3D4}-+bIGs|aCxaXP|Fjnxsp2S<) zJn`xY+~ia=3D=3Dx7!LVTcJaP!kQ}oLPZT;@=3D994%^?1dW>Y^pRy9%lFgc0LixA#C`*jK z;-(%{KhO8A zQL|_UUpX*A+cvBDOEDMPZ}BA?y&2MCj0xlWOPtsMp2sqHn;nB~SZ-XwtqDHze!Zu3 z?2_rR;1(^eZPPTKtFON)2hr4=3DVwliD&|Dw(%vOU8ZV-<%(BfEZN9+dxHmiI$Mm<`x zR;QfPpJHG|xwd#okLE~J;-6CXKac)9k?#+)+p5z_0-C>;J()kB|1v}_(~~t_i(O4z z@~{^3Q}X1%OjMFy^8>^uUb)Nfm6H9BDCn;q<$X@kbuIX+n%!d@)g1CmH~A^4D+*z} zX}o%o6z&~dpv2&x12?hUugE{1C%QA;B^s3eb+4@)6oAi0O5?r;T&CM>WG#Eb*)8jw z_#-7?C`~j9u^qWu3K_J#%V!7*ttPSd=3DgIc+ocx&tNhNaXZaRO|duD!js{uVDl{h5N zl!Z34Z;|&6ZQ7uXf5)78h4DqdJ^|Kd{(JS(+3p)zQ}5h7=3DfNh&eQ}bb?`KqG?8BXe zodrnLBlG3}5jlmfoui$QyGb%iT)6=3D}@XYH@U~A*Nfk|7EY#jS+qQaEw$rZm+PZIr9jumQE zNER7TSY3Ve;BH(I?;YFLIoFu`KW_QW(^g9*%6ki`4IuaYzv=3D&;AFM!T7H&uK#~WGK z9a@i;wh3~&@WFp3GW=3D6F|NRbK7_D1qkF@xuZM$9zBYRXWg=3Dk(K77*E_44m+9xI_Ea zkCNDGz(}ggQXc8-#GSEMMGVT*Ar+&RGfI`ig4p9O?q@!o8S%&3>6qki7#x3<@;Wy* z{bsq>dd80o^~Zi5?QimelosIucxC%<>t)CbupF(P8oS;bE$$1VM^upSMu*VRO|+@r!o(^8Z!djJL@s6;&@G0M`qZ|EZh3l z1ZvJ@%)#RTOR{Vj0Noeahv9Z#MBdA`5SCHb-o4?Eu?_hL4=3D8;5PzJr#J_I>SU2UjSOhgdHt8 zM_|@Z^V{;4{`vQtegv;W`+cqbtKFHLId-wp*T_-ngNu)Kx22f&^Plx>>)~>eY4T!n eCI6{X3*jh?2aa;Fb!_er$w=3DQ!@0G4|%zpsDRL;Nv literal 0 HcmV?d00001 diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation= /admin-guide/mm/damon/usage.rst new file mode 100644 index 000000000000..9d71f04d12ee --- /dev/null +++ b/Documentation/admin-guide/mm/damon/usage.rst @@ -0,0 +1,296 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Detailed Usages +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +DAMON provides below three interfaces for different users. + +- *DAMON user space tool.* + This is for privileged people such as system administrators who want a + just-working human-friendly interface. Using this, users can use the = DAMON=E2=80=99s + major features in a human-friendly way. It may not be highly tuned fo= r + special cases, though. It supports virtual address space monitoring o= nly. +- *debugfs interface.* + This is for privileged user space programmers who want more optimized = use of + DAMON. Using this, users can use DAMON=E2=80=99s major features by re= ading + from and writing to special debugfs files. Therefore, you can write a= nd use + your personalized DAMON debugfs wrapper programs that reads/writes the + debugfs files instead of you. The DAMON user space tool is also a ref= erence + implementation of such programs. It supports virtual address space + monitoring only. +- *Kernel Space Programming Interface.* + This is for kernel space programmers. Using this, users can utilize e= very + feature of DAMON most flexibly and efficiently by writing kernel space + DAMON application programs for you. You can even extend DAMON for var= ious + address spaces. + +This document does not describe the kernel space programming interface i= n +detail. For that, please refer to :doc:`api`. + + +DAMON User Sapce Tool +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +A reference implementation of the DAMON user space tools which provides = a +convenient user interface is in the kernel source tree. It is located a= t +``tools/damon/damo`` of the tree. + +The tool provides a subcommands based interface. Every subcommand provi= des +``-h`` option, which provides the minimal usage of it. Currently, the t= ool +supports two subcommands, ``record`` and ``report``. + +Below example commands assume you set ``$PATH`` to point ``tools/damon/`= ` for +brevity. It is not mandatory for use of ``damo``, though. + + +Recording Data Access Pattern +----------------------------- + +The ``record`` subcommand records the data access pattern of target work= loads +in a file (``./damon.data`` by default). You can specify the target as = either +process id of running target or a command for execution of it. Below ex= ample +shows a command target usage:: + + # cd /tools/damon/ + # damo record "sleep 5" + +The tool will execute ``sleep 5`` by itself and record the data access p= atterns +of the process. Below example shows a pid target usage:: + + # sleep 5 & + # damo record `pidof sleep` + +The location of the recorded file can be explicitly set using ``-o`` opt= ion. +You can further tune this by setting the monitoring attributes. To know= about +the monitoring attributes in detail, please refer to :doc:`mechanisms`. + + +Analyzing Data Access Pattern +----------------------------- + +The ``report`` subcommand reads a data access pattern record file (if no= t +explicitly specified using ``-i`` option, reads ``./damon.data`` file by +default) and generates human-readable reports. You can specify what typ= e of +report you want using a sub-subcommand to ``report`` subcommand. ``raw`= `, +``heats``, and ``wss`` report types are supported for now. + + +raw +~~~ + +``raw`` sub-subcommand simply transforms the binary record into a +human-readable text. For example:: + + $ damo report raw + start_time: 193485829398 + rel time: 0 + nr_tasks: 1 + pid: 1348 + nr_regions: 4 + 560189609000-56018abce000( 22827008): 0 + 7fbdff59a000-7fbdffaf1a00( 5601792): 0 + 7fbdffaf1a00-7fbdffbb5000( 800256): 1 + 7ffea0dc0000-7ffea0dfd000( 249856): 0 + + rel time: 100000731 + nr_tasks: 1 + pid: 1348 + nr_regions: 6 + 560189609000-56018abce000( 22827008): 0 + 7fbdff59a000-7fbdff8ce933( 3361075): 0 + 7fbdff8ce933-7fbdffaf1a00( 2240717): 1 + 7fbdffaf1a00-7fbdffb66d99( 480153): 0 + 7fbdffb66d99-7fbdffbb5000( 320103): 1 + 7ffea0dc0000-7ffea0dfd000( 249856): 0 + +The first line shows the recording started timestamp (nanosecond). Reco= rds of +data access patterns follows. Each record is separated by a blank line.= Each +record first specifies the recorded time (``rel time``) in relative to t= he +start time, the number of monitored tasks in this record (``nr_tasks``). +Recorded data access patterns of each task follow. Each data access pat= tern +for each task shows the target's pid (``pid``) and a number of monitored +address regions in this access pattern (``nr_regions``) first. After th= at, +each line shows the start/end address, size, and the number of observed +accesses of each region. + + +heats +~~~~~ + +The ``raw`` output is very detailed but hard to manually read. ``heats`= ` +sub-subcommand plots the data in 3-dimensional form, which represents th= e time +in x-axis, address of regions in y-axis, and the access frequency in z-a= xis. +Users can set the resolution of the map (``--tres`` and ``--ares``) and +start/end point of each axis (``--tmin``, ``--tmax``, ``--amin``, and +``--amax``) via optional arguments. For example:: + + $ damo report heats --tres 3 --ares 3 + 0 0 0.0 + 0 7609002 0.0 + 0 15218004 0.0 + 66112620851 0 0.0 + 66112620851 7609002 0.0 + 66112620851 15218004 0.0 + 132225241702 0 0.0 + 132225241702 7609002 0.0 + 132225241702 15218004 0.0 + +This command shows a recorded access pattern in heatmap of 3x3 resolutio= n. +Therefore it shows 9 data points in total. Each line shows each of the = data +points. The three numbers in each line represent time in nanosecond, ad= dress, +and the observed access frequency. + +Users will be able to convert this text output into a heatmap image (rep= resents +z-axis values with colors) or other 3D representations using various too= ls such +as 'gnuplot'. For more convenience, ``heats`` sub-subcommand provides t= he +'gnuplot' based heatmap image creation. For this, you can use ``--heatm= ap`` +option. Also, note that because it uses 'gnuplot' internally, it will f= ail if +'gnuplot' is not installed on your system. For example:: + + $ ./damo report heats --heatmap heatmap.png + +Creates the heatmap image in ``heatmap.png`` file. It supports ``pdf``, +``png``, ``jpeg``, and ``svg``. + +If the target address space is virtual memory address space and you plot= the +entire address space, the huge unmapped regions will make the picture lo= oks +only black. Therefore you should do proper zoom in / zoom out using the +resolution and axis boundary-setting arguments. To make this effort min= imal, +you can use ``--guide`` option as below:: + + $ ./damo report heats --guide + pid:1348 + time: 193485829398-198337863555 (4852034157) + region 0: 00000094564599762944-00000094564622589952 (22827008) + region 1: 00000140454009610240-00000140454016012288 (6402048) + region 2: 00000140731597193216-00000140731597443072 (249856) + +The output shows unions of monitored regions (start and end addresses in= byte) +and the union of monitored time duration (start and end time in nanoseco= nds) of +each target task. Therefore, it would be wise to plot the data points i= n each +union. If no axis boundary option is given, it will automatically find = the +biggest union in ``--guide`` output and set the boundary in it. + + +wss +~~~ + +The ``wss`` type extracts the distribution and chronological working set= size +changes from the records. For example:: + + $ ./damo report wss + # + # pid 1348 + # avr: 66228 + 0 0 + 25 0 + 50 0 + 75 0 + 100 1920615 + +Without any option, it shows the distribution of the working set sizes a= s +above. It shows 0th, 25th, 50th, 75th, and 100th percentile and the ave= rage of +the measured working set sizes in the access pattern records. In this c= ase, +the working set size was zero for 75th percentile but 1,920,615 bytes in= max +and 66,228 bytes on average. + +By setting the sort key of the percentile using '--sortby', you can show= how +the working set size has chronologically changed. For example:: + + $ ./damo report wss --sortby time + # + # pid 1348 + # avr: 66228 + 0 0 + 25 0 + 50 0 + 75 0 + 100 0 + +The average is still 66,228. And, because the access was spiked in very= short +duration and this command plots only 4 data points, we cannot show when = the +access spikes made. Users can specify the resolution of the distributio= n +(``--range``). By giving more fine resolution, the short duration spike= s could +be found. + +Similar to that of ``heats --heatmap``, it also supports 'gnuplot' based= simple +visualization of the distribution via ``--plot`` option. + + +debugfs Interface +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +DAMON exports four files, ``attrs``, ``pids``, ``record``, and ``monitor= _on`` +under its debugfs directory, ``/damon/``. + + +Attributes +---------- + +Users can get and set the ``sampling interval``, ``aggregation interval`= `, +``regions update interval``, and min/max number of monitoring target reg= ions by +reading from and writing to the ``attrs`` file. To know about the monit= oring +attributes in detail, please refer to :doc:`mechanisms`. For example, b= elow +commands set those values to 5 ms, 100 ms, 1,000 ms, 10 and 1000, and th= en +check it again:: + + # cd /damon + # echo 5000 100000 1000000 10 1000 > attrs + # cat attrs + 5000 100000 1000000 10 1000 + + +Target PIDs +----------- + +Users can get and set the pids of monitoring target processes by reading= from +and writing to the ``pids`` file. For example, below commands set proce= sses +having pids 42 and 4242 as the processes to be monitored and check it ag= ain:: + + # cd /damon + # echo 42 4242 > pids + # cat pids + 42 4242 + +Note that setting the pids doesn't start the monitoring. + + +Record +------ + +This debugfs file allows you to record monitored access patterns in a re= gular +binary file. The recorded results are first written in an in-memory buf= fer and +flushed to a file in batch. Users can get and set the size of the buffe= r and +the path to the result file by reading from and writing to the ``record`= ` file. +For example, below commands set the buffer to be 4 KiB and the result to= be +saved in ``/damon.data``. :: + + # cd /damon + # echo "4096 /damon.data" > record + # cat record + 4096 /damon.data + +The recording can be disabled by setting the buffer size zero. + + +Turning On/Off +-------------- + +Setting the files as described above doesn't incur effect unless you +explicitly start the monitoring. You can start, stop, and check the cur= rent +status of the monitoring by writing to and reading from the ``monitor_on= `` +file. Writing ``on`` to the file starts the monitoring and recording of= the +targets with the attributes. Writing ``off`` to the file stops those. = DAMON +also stops if every target process is terminated. Below example command= s turn +on, off, and check the status of DAMON:: + + # cd /damon + # echo on > monitor_on + # echo off > monitor_on + # cat monitor_on + off + +Please note that you cannot write to the above-mentioned debugfs files w= hile +the monitoring is turned on. If you write to the files while DAMON is r= unning, +an error code such as ``-EBUSY`` will be returned. diff --git a/Documentation/admin-guide/mm/index.rst b/Documentation/admin= -guide/mm/index.rst index 11db46448354..d3d0ba373eb6 100644 --- a/Documentation/admin-guide/mm/index.rst +++ b/Documentation/admin-guide/mm/index.rst @@ -27,6 +27,7 @@ the Linux memory management. =20 concepts cma_debugfs + data_access_monitor hugetlbpage idle_page_tracking ksm --=20 2.17.1