From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B022AC77B71 for ; Tue, 18 Apr 2023 09:17:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E30EC6B0071; Tue, 18 Apr 2023 05:17:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DBB8E8E0001; Tue, 18 Apr 2023 05:17:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C33C56B0074; Tue, 18 Apr 2023 05:17:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B0D5A6B0071 for ; Tue, 18 Apr 2023 05:17:27 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 706D9401D8 for ; Tue, 18 Apr 2023 09:17:27 +0000 (UTC) X-FDA: 80693958534.05.727145D Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf22.hostedemail.com (Postfix) with ESMTP id 5EED1C000F for ; Tue, 18 Apr 2023 09:17:25 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=WwH0Z2Xy; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=WPJvjY6G; spf=pass (imf22.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681809445; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lrrlZEOMXRldfaSPVNeyMPGMmoVT36um+XV3eepdEbU=; b=pXzR7rVMRWtzj/RMKxz3Ud6XkDmaGaI+OFiEwtjeaDDAdaE6p7mJ/fObCCqC+xeFC0rfYj ZIFQmvMOdMy9HLSLBlxcs03yRnpXXxQewOvdXApk5pnROEXmuhheA6asEvWHSd163LlY+8 SFwWEuAZifrTvDejyNC6jyHNaiwAgZc= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=WwH0Z2Xy; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=WPJvjY6G; spf=pass (imf22.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681809445; a=rsa-sha256; cv=none; b=zd4xwjXX1b9XJ0JcgPs+ebTSErD5i8zmvewmJg7XvhoKMl+NK4R4POMWMGlQ1DcEBlykX/ Zqyangfh9L9N6aOvG5HWx21QGAdVbQP/gkI/Np8SsSi6tGub6ztftNKA1Fb8+r6mtRuvRr cyBEbO4avtW+0POxs9dJ7XUNrpPQ5gE= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 9F40A1F8D6; Tue, 18 Apr 2023 09:17:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1681809443; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lrrlZEOMXRldfaSPVNeyMPGMmoVT36um+XV3eepdEbU=; b=WwH0Z2XykF5KxMGR4buRTbjZGhLlP9MICgQbXewkGCuBRvYnV5OMRZF4JWeaNLr3MQOoJq wGgWUCRufMnnnqSQr8QN0j9j4qAOC8hol7w9SZ7RrYHBCf1nGQVVE5gQsw6UrbFf66B2fl SO9bcGCDI67Z+AD2yuzWkwIdS2ioR+M= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1681809443; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lrrlZEOMXRldfaSPVNeyMPGMmoVT36um+XV3eepdEbU=; b=WPJvjY6GNu1rif4tTjyEY7sgW8ZBOMtbfO+UbtH70StIamHihmy9/p7GIhFFGG0NqYSWaC 6Xgav+sfZUE1mPCA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 8066213581; Tue, 18 Apr 2023 09:17:23 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 8/aeHiNgPmQaIwAAMHmgww (envelope-from ); Tue, 18 Apr 2023 09:17:23 +0000 Message-ID: Date: Tue, 18 Apr 2023 11:17:23 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: [RFC PATCH] migrate_pages: Never block waiting for the page lock Content-Language: en-US To: Doug Anderson , "Huang, Ying" , Mel Gorman Cc: Andrew Morton , Yu Zhao , linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20230413182313.RFC.1.Ia86ccac02a303154a0b8bc60567e7a95d34c96d3@changeid> <87v8hz17o9.fsf@yhuang6-desk2.ccr.corp.intel.com> <87ildvwbr5.fsf@yhuang6-desk2.ccr.corp.intel.com> From: Vlastimil Babka In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: t694jjedfoaxifjjfc3p9kehbobwa4xu X-Rspamd-Queue-Id: 5EED1C000F X-HE-Tag: 1681809445-95197 X-HE-Meta: U2FsdGVkX1+O6cnZ1W74h8pNQBb1+bOqee4H9/EWXRwFcNsoyq47X2SRs+RmFESa5B9nAO4/xxQ8TJjbVBIfyCA5j/0jEvyXUZmUwKSiGAjDfjE1yKn/ROyrEej0aqPdqUqBs8FL78C9zZkXo5SM3W7hQSx3nJFkwoIokq0eov7O45SOcATcbUqe/DkXHvCsSvbYa9rF/XKCyqt4q4vVzEvFD1wqTt92iwniOYmmgD06+NoinQpXkFZ26M9/zh7B5dyp9Ys8NsXdhk7fbjaWXzWyCkvyd2wSBIoslP0FIdhFvWxNHb1zGVr58kCtMNIgTV3EjY9FQysS2hyD0A04BZz05+akLRrw7M0ziXCfL3fIKrzX/DzYwjXXkcsk4n3qt+voiU4sWI/DByHFWpqz7yD7iELZG1WjinDifHDrR1lfRvkYsiXLoFdqHmyWq9M0s4vhRqT1ErzzWKNRUP2pYBovhNIZ0h0tb90v3NpMaph0ukJ/cY+9E8ffLxn9ZUWUcj2lCJ0i1jLMJYDu1jtMxfodnB+FppN9Q5Sm+mwWD1kG5yrme0Svpz/IDIfh9X4Jh0bmEOPrWmRSGZXi136I8GkFeESSGce87vrA1XTatil6Opa2n1K0hSelYAGYPSNcj6tTZ3mboEznKu7t9h9koUYr5LyY5yxWCXhnAGm74BFlZ7Zz2rFxlodG8vcQEOfhAVmxLmAJf3hTbdsqLW0MXwrdJXapr93OnOdh08KUEnaaGY1FHZ87GdTaCxm0Wlfy1+tXYgjRIjSVdNReROSG9v/Nd+DD4x31jARGH8kjaw4/LaPSpV62/5ckIcpc+TxaOHkb4yRqITuz5vp4lQpwKtWVbLdxKSzfPnQCP04BUr/dec/yOwl85e8msUp9JSS9WmrXIzbdvG+fjlZxbwXNvpbv6/BMpAdEPMna5R4MvJXSPMkRZ6VO7DRr6Le3k4wPS9AVLq6LKoOEP71QHN1 qE+MpBOl F9nW0Z9rZA5mV9xYFTi4oIo0wPrCgcA6k/mhXcfBLtU3QLXxgUEcS1DgaaH6O2F83lxrjcqoBDZ6ln1Q9z57arctv04PwpnB/swdnUqufCEaZX/bxW7rJABbomlAj3ld7OWAh+vFwIQepU/V4y0s83n59cVDsD4eoUV9XUuyrRno/RyROZkkGYM9hbOOJXZMHk8dCHyJUzEWhP19Q6EmRUfTT2+BCLw8Lbe/889rORoXqr4plaURwdzke9YE26y3b+Wm3T0YnoEpOirHg2bfv3XPIUBIfHfTzVihu8TkAHvq3Kg6adxzTqonM/Dw7CT/GCWKsx4k+l0bjNW9IjdH+JgkAvw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 4/17/23 16:28, Doug Anderson wrote: > Hi, > > On Sun, Apr 16, 2023 at 6:15 PM Huang, Ying wrote: >> >> Doug Anderson writes: >> >> > Hi, >> > >> > On Thu, Apr 13, 2023 at 8:10 PM Huang, Ying wrote: >> >> >> >> Douglas Anderson writes: >> >> >> >> > Currently when we try to do page migration and we're in "synchronous" >> >> > mode (and not doing direct compaction) then we'll wait an infinite >> >> > amount of time for a page lock. This does not appear to be a great >> >> > idea. >> >> > >> >> > One issue can be seen when I put a device under extreme memory >> >> > pressure. I took a sc7180-trogdor Chromebook (4GB RAM, 8GB zram >> >> > swap). I ran the browser along with Android (which runs from a >> >> > loopback mounted 128K block-size squashfs "disk"). I then manually ran >> >> > the mmm_donut memory pressure tool [1]. The system is completely >> >> > unusable both with and without this patch since there are 8 processes >> >> > completely thrashing memory, but it was still interesting to look at >> >> > how migration was behaving. I put some timing code in and I could see >> >> > that we sometimes waited over 25 seconds (in the context of >> >> > kcompactd0) for a page lock to become available. Although the 25 >> >> > seconds was the high mark, it was easy to see tens, hundreds, or >> >> > thousands of milliseconds spent waiting on the lock. >> >> > >> >> > Instead of waiting, if I bailed out right away (as this patch does), I >> >> > could see kcompactd0 move forward to successfully to migrate other >> >> > pages instead. This seems like a better use of kcompactd's time. >> >> > >> >> > Thus, even though this didn't make the system any more usable in my >> >> > absurd test case, it still seemed to make migration behave better and >> >> > that feels like a win. It also makes the code simpler since we have >> >> > one fewer special case. >> >> >> >> TBH, the test case is too extreme for me. >> > >> > That's fair. That being said, I guess the point I was trying to make >> > is that waiting for this lock could take an unbounded amount of time. >> > Other parts of the system sometimes hold a page lock and then do a >> > blocking operation. At least in the case of kcompactd there are better >> > uses of its time than waiting for any given page. >> > >> >> And, we have multiple "sync" mode to deal with latency requirement, for >> >> example, we use MIGRATE_SYNC_LIGHT for compaction to avoid too long >> >> latency. If you have latency requirement for some users, you may >> >> consider to add new "sync" mode. >> > >> > Sure. kcompactd_do_work() is currently using MIGRATE_SYNC_LIGHT. I >> > guess my first thought would be to avoid adding a new mode and make >> > MIGRATE_SYNC_LIGHT not block here. Then anyone that truly needs to >> > wait for all the pages to be migrated can use the heavier sync modes. >> > It seems to me like the current users of MIGRATE_SYNC_LIGHT would not >> > want to block for an unbounded amount of time here. What do you think? >> >> It appears that you can just use MIGRATE_ASYNC if you think the correct >> behavior is "NOT block at all". I found that there are more >> fine-grained controls on this in compaction code, please take a look at >> "enum compact_priority" and its comments. > > Actually, the more I think about it the more I think the right answer > is to keep kcompactd as using MIGRATE_SYNC_LIGHT and make > MIGRATE_SYNC_LIGHT not block on the folio lock. kcompactd can accept > some blocking but we don't want long / unbounded blocking. Reading the > comments for MIGRATE_SYNC_LIGHT, this also seems like it fits pretty > well. MIGRATE_SYNC_LIGHT says that the stall time of writepage() is > too much. It's entirely plausible that someone else holding the lock > is doing something as slow as writepage() and thus waiting on the lock > can be just as bad for latency. +Cc Mel for potential insights. Sounds like a good compromise at first glance, but it's a tricky area. Also there are other callers of migration than compaction, and we should make sure we are not breaking them unexpectedly. > I'll try to send out a v2 with this approach today and we can see what > people think. Please Cc Mel and myself for further versions. > -Doug >