From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7C92C6FD18 for ; Wed, 19 Apr 2023 00:34:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4C4738E0003; Tue, 18 Apr 2023 20:34:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 474408E0001; Tue, 18 Apr 2023 20:34:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 33BC88E0003; Tue, 18 Apr 2023 20:34:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 20DA58E0001 for ; Tue, 18 Apr 2023 20:34:18 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id E11AC1C657B for ; Wed, 19 Apr 2023 00:34:17 +0000 (UTC) X-FDA: 80696268954.30.273A6D3 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by imf14.hostedemail.com (Postfix) with ESMTP id 688BD100003 for ; Wed, 19 Apr 2023 00:34:15 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=IB944MbN; spf=pass (imf14.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681864455; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Dx037JVt8A+9b87SgprQvpTJjpN1XyKZNq+des4Z1Ig=; b=xL6tvs8sPBlZ+AxZZv4ma8p5JsaRxknDP8vyicEyrl2/ojWdi6dtDGM68P5eUgvaM105cM 1MAK9O2bqaH2n8v/Bs4CKJ57dbLhjzm+70bpoEk6teuBc6KCL7uDrA7uZoQqDmzVOfD0aw cKJ/4omqYo6AYVIH3CXD45i9TAL83EA= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=IB944MbN; spf=pass (imf14.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681864455; a=rsa-sha256; cv=none; b=P/pKMSvGRLMPVR5PVtcLE9H71KuWhp5xUW7WfjwACd1opW0lsaW6Lz+yFFH5QzVH15mYFP ny++Tuuf2R17h4aiGXOPZ4BXcgpVhk+1OAFrUsgaqTedNuyQvlUw2nT6lYqA0oLNzia0Zh Hyv6w19O1rOfBDVPVx486sNg9HiFyDA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681864455; x=1713400455; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version:content-transfer-encoding; bh=2CA/TpUSdrKGSjoZ9YX0iDPOx1tUppAJIF4fLz1tJE8=; b=IB944MbNyqSlDAPCwPL53wwGkjhN3/tY0MC0VpYhm5Jq7hmEbQxuPIRD la2/wXoipfeiM+wjVe6+ma2+QlRSnv7r8yjGYdZJkO4fbvPsxGwpX9L2r OfnzazjyDchlhY72+PPbM0i1bZJMw3FKgXBTGddCNzq8mEZy7h52nFIt3 qs2ux0g5WwPQfwT9b51lurRTpJIqaPQnt9moa+EQEIrbZuggMMfxyUq1k i9xrOJj/Pl4Kr/3HW/Q24+JIrN6aAwIeTTCmBCHcbhZTw0hzTV1aM6lT+ WP1JIvF0nlczLrXEI1MizaGFcJvrUZyi2q68wW19QT3714eLp8dSSKvGy w==; X-IronPort-AV: E=McAfee;i="6600,9927,10684"; a="345314925" X-IronPort-AV: E=Sophos;i="5.99,208,1677571200"; d="scan'208";a="345314925" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Apr 2023 17:34:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10684"; a="723869369" X-IronPort-AV: E=Sophos;i="5.99,208,1677571200"; d="scan'208";a="723869369" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga001-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Apr 2023 17:34:11 -0700 From: "Huang, Ying" To: Doug Anderson Cc: Andrew Morton , Yu Zhao , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Vlastimil Babka , Mel Gorman Subject: Re: [RFC PATCH] migrate_pages: Never block waiting for the page lock References: <20230413182313.RFC.1.Ia86ccac02a303154a0b8bc60567e7a95d34c96d3@changeid> <87v8hz17o9.fsf@yhuang6-desk2.ccr.corp.intel.com> <87ildvwbr5.fsf@yhuang6-desk2.ccr.corp.intel.com> <87edohvpzk.fsf@yhuang6-desk2.ccr.corp.intel.com> Date: Wed, 19 Apr 2023 08:33:01 +0800 In-Reply-To: (Doug Anderson's message of "Tue, 18 Apr 2023 09:19:21 -0700") Message-ID: <87wn28u2wy.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: ne9sk1omy6y59d59g64m7ajox6ouxzsz X-Rspamd-Queue-Id: 688BD100003 X-HE-Tag: 1681864455-281315 X-HE-Meta: U2FsdGVkX19X9RR8uvr7z56+eC9FllQcTzL5htx/okKfxj7I2RwsWz43G91g5NukPr4GRolUNQVYakzfu//OFvINEEVHZPVySmiZ9IowzDjkpDcFRxgUKfGFj/9EV1Aqib1Y9yDz16B2pe0Oypl/8bdYUZed0lnEiPvGwJ9qYLnF/tiy+dv/vscVp9m15Vx4h7plmi03ly3tO/y+GwnTZqCpjNR/b7qaNCeZqs49GYqNJdDiCWPvizlHVlR/++pADVUVQ4mHsnAdhdljrG8pmpLes5a8gvlD8fjCi2j9gaDxwUeKhCLa0n+wb7OqjL2f5ouAwPp1FnH1mpZfSgn/nXI8OxYD1kQrQ4JyIEGzOfqJCPqWDy9MEeAQRPwqHUOLT5YTHYM3/uWj3yX7GFJJ7iZm7bKBvBs6r6DXrvmM89lGdpAkPYuew4SprXKkKOwmFk3Cf2Tobfwg1YAI6j/ZaQzVwZn7HTmKF2K2X/jpeVqecdFAFFXjCfqvWuGPRTMrs4LsMEyrqNfWwwLxbGWEEWEqvEdSCPSTAaLbs5nguwno/u5QILnd9wpHaMNxWsUPCe8oH8a7MNleIC5/tWAMAvB2u29F9guTQ5gTYRA2rU4qhlzrpKVi5Hji55DdIafc+/iSiIj03VUpSIMStmBsDZj//S7blHMLk0XcpENsECQzKL+8TdzsuGkhOlUPMk0boC1db3+C2rBcr+TOUlCKnmvKJHIlfwcVwAlSHaFt3bI3eJ1SVd8RlYgjvd3M6nwH0NJNl3zSTxW51cLo+P7EZZeysKNXM4U2gXIWfO7iL94PJcCutgrqDcWeFpkzs8CwxUXsn0oARWBqg+XlsWhpzGfBXGEy7lXcebpf2v/VX4q4fZ68Mx9uUdAAWO4/EG82Dc+PA6caLuCQyEfOGP/1ixxJDlJLVTu2uBLQY5LX4EGTHMac8080bY9LIVK1r1uI1ZUsFa1rsFtAkCpUzKS h3JPAeip 1ThweA6J3FW/NnA3UZrN0DA5+MU7Ku7IoTTnrfFEI+tTJgZncNpoP6IhwGBgQ4IeOPnqU+KWQ94yWOLr0NNgFx1y869yROsxnESeAF7OMbMTdCMuKEG5ONqfXItVP1Uz2t8X0XAOcUP+5Y6B93X/YCooFOHBeNM7IeXjywawSsvLdjp3XQPNO6UWG/FZdWoDnnFqGpT7memtjh7Hyw6Y0nWUnrFT10LXr8QGJgjzicPzGYKXPidC9k5wFMIsNYJJghN6aUQlfQQnD78XJPIfvaeGbl0LIcwXoBLKQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Doug Anderson writes: > Hi, > > On Mon, Apr 17, 2023 at 8:18=E2=80=AFPM Huang, Ying wrote: >> >> Doug Anderson writes: >> >> > Hi, >> > >> > On Sun, Apr 16, 2023 at 6:15=E2=80=AFPM Huang, Ying wrote: >> >> >> >> Doug Anderson writes: >> >> >> >> > Hi, >> >> > >> >> > On Thu, Apr 13, 2023 at 8:10=E2=80=AFPM Huang, Ying wrote: >> >> >> >> >> >> Douglas Anderson writes: >> >> >> >> >> >> > Currently when we try to do page migration and we're in "synchro= nous" >> >> >> > mode (and not doing direct compaction) then we'll wait an infini= te >> >> >> > amount of time for a page lock. This does not appear to be a gre= at >> >> >> > idea. >> >> >> > >> >> >> > One issue can be seen when I put a device under extreme memory >> >> >> > pressure. I took a sc7180-trogdor Chromebook (4GB RAM, 8GB zram >> >> >> > swap). I ran the browser along with Android (which runs from a >> >> >> > loopback mounted 128K block-size squashfs "disk"). I then manual= ly ran >> >> >> > the mmm_donut memory pressure tool [1]. The system is completely >> >> >> > unusable both with and without this patch since there are 8 proc= esses >> >> >> > completely thrashing memory, but it was still interesting to loo= k at >> >> >> > how migration was behaving. I put some timing code in and I coul= d see >> >> >> > that we sometimes waited over 25 seconds (in the context of >> >> >> > kcompactd0) for a page lock to become available. Although the 25 >> >> >> > seconds was the high mark, it was easy to see tens, hundreds, or >> >> >> > thousands of milliseconds spent waiting on the lock. >> >> >> > >> >> >> > Instead of waiting, if I bailed out right away (as this patch do= es), I >> >> >> > could see kcompactd0 move forward to successfully to migrate oth= er >> >> >> > pages instead. This seems like a better use of kcompactd's time. >> >> >> > >> >> >> > Thus, even though this didn't make the system any more usable in= my >> >> >> > absurd test case, it still seemed to make migration behave bette= r and >> >> >> > that feels like a win. It also makes the code simpler since we h= ave >> >> >> > one fewer special case. >> >> >> >> >> >> TBH, the test case is too extreme for me. >> >> > >> >> > That's fair. That being said, I guess the point I was trying to make >> >> > is that waiting for this lock could take an unbounded amount of tim= e. >> >> > Other parts of the system sometimes hold a page lock and then do a >> >> > blocking operation. At least in the case of kcompactd there are bet= ter >> >> > uses of its time than waiting for any given page. >> >> > >> >> >> And, we have multiple "sync" mode to deal with latency requirement= , for >> >> >> example, we use MIGRATE_SYNC_LIGHT for compaction to avoid too long >> >> >> latency. If you have latency requirement for some users, you may >> >> >> consider to add new "sync" mode. >> >> > >> >> > Sure. kcompactd_do_work() is currently using MIGRATE_SYNC_LIGHT. I >> >> > guess my first thought would be to avoid adding a new mode and make >> >> > MIGRATE_SYNC_LIGHT not block here. Then anyone that truly needs to >> >> > wait for all the pages to be migrated can use the heavier sync mode= s. >> >> > It seems to me like the current users of MIGRATE_SYNC_LIGHT would n= ot >> >> > want to block for an unbounded amount of time here. What do you thi= nk? >> >> >> >> It appears that you can just use MIGRATE_ASYNC if you think the corre= ct >> >> behavior is "NOT block at all". I found that there are more >> >> fine-grained controls on this in compaction code, please take a look = at >> >> "enum compact_priority" and its comments. >> > >> > Actually, the more I think about it the more I think the right answer >> > is to keep kcompactd as using MIGRATE_SYNC_LIGHT and make >> > MIGRATE_SYNC_LIGHT not block on the folio lock. >> >> Then, what is the difference between MIGRATE_SYNC_LIGHT and >> MIGRATE_ASYNC? > > Aren't there still some differences even if we remove blocking this > one lock? ...or maybe your point is that maybe the other differences > have similar properties? Sorry for confusing words. Here, I asked you to list the implementation difference between MIGRATE_ASYNC and MIGRATE_SYNC_LIGHT after your proposed changes. Which are waited in MIGRATE_SYNC_LIGHT but not in MIGRATE_ASYNC? > OK, so let's think about just using MIGRATE_ASYNC and either leaving > MIGRATE_SYNC_LIGHT alone or deleting it (if there are no users left). > The nice thing is that the only users of MIGRATE_SYNC_LIGHT are in > "compaction.c" and there are only 3 places where it's specified. > > 1. kcompactd_do_work() - This is what I was analyzing and where I > argued that indefinite blocking is less useful than simply trying to > compact a different page. So sure, moving this to MIGRATE_ASYNC seems > like it would be OK? > > 2. proactive_compact_node() - Just like kcompactd_do_work(), this is > called from kcompactd and thus probably should have the same mode. > > 3. compact_zone_order() - This explicitly chooses between > MIGRATE_SYNC_LIGHT and MIGRATE_ASYNC, so I guess I'd keep > MIGRATE_SYNC_LIGHT just for this use case. It looks as if > compact_zone_order() is called for direct compaction and thus making > it synchronous can make sense, especially since it seems to go to the > synchronous case after it failed with the async case. Ironically, > though, the exact lock I was proposing to not wait on _isn't_ ever > waited on in direct reclaim (see the comment in migrate_folio_unmap() > about deadlock), but the other differences between SYNC_LIGHT and > ASYNC come into play. > > If the above sounds correct then I'm OK w/ moving #1 and #2 to > MIGRATE_ASYNC and leaving #3 as the sole user or MIGRATE_SYNC_LIGHT. > >> > kcompactd can accept some blocking but we don't want long / unbounded >> > blocking. Reading the comments for MIGRATE_SYNC_LIGHT, this also seems >> > like it fits pretty well. MIGRATE_SYNC_LIGHT says that the stall time >> > of writepage() is too much. It's entirely plausible that someone else >> > holding the lock is doing something as slow as writepage() and thus >> > waiting on the lock can be just as bad for latency. >> >> IIUC, during writepage(), the page/folio will be unlocked. >> >> But, during page reading, the page/folio will be locked. I don't really >> understand why we can wait for page reading but cannot wait for page >> writeback. > > I'm not sure I totally got your point here. It sorta sounds as if > you're making the same point that I was? Yes, kind of. It is a question, not conclusion. > IIUC by waiting on the lock > we may be implicitly waiting for someone to finish reading which seems > as bad as waiting for writing. That was why I was arguing that with > MIGRATE_SYNC_LIGHT (which says that waiting for the write was too > slow) that we shouldn't wait for the lock (which may be blocking on a > read). Best Regards, Huang, Ying