From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B022C7618E for ; Wed, 26 Apr 2023 10:09:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6DB536B00B1; Wed, 26 Apr 2023 06:09:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 665E36B00B2; Wed, 26 Apr 2023 06:09:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 52CA46B00B3; Wed, 26 Apr 2023 06:09:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 3EED36B00B1 for ; Wed, 26 Apr 2023 06:09:26 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id F06DC16023D for ; Wed, 26 Apr 2023 10:09:25 +0000 (UTC) X-FDA: 80723119890.02.1DB3F52 Received: from outbound-smtp02.blacknight.com (outbound-smtp02.blacknight.com [81.17.249.8]) by imf06.hostedemail.com (Postfix) with ESMTP id F0EAE18000C for ; Wed, 26 Apr 2023 10:09:22 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf06.hostedemail.com: domain of mgorman@techsingularity.net designates 81.17.249.8 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1682503763; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UG/s7xEQecmcWewG2XLEuHHBK7IY1W9Q/psyJVyiesM=; b=VSeIBtKHFTRNLRBKfjnSqMlwIBH6HbXf7I2aAwlCvBvtrk7lKFGssmnhBBWOQNLB9dl0tq 9DUUAHfVLLEfyF04+s5CRsDjD1tUKtphJabsPKySxT1vGzQDZ4dpw8NGfFrcad0PrvWCft N6XPSzvv/xPr0LaFmloiXk5Ssl9z+/8= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf06.hostedemail.com: domain of mgorman@techsingularity.net designates 81.17.249.8 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1682503763; a=rsa-sha256; cv=none; b=jh6Wea5MiZfc69MvaqeQQtXFLB5VAFzg8uDi8qHjZnn9vZXX6PsE+Y70mIUzXvj2m9jzm2 egEEXFPJbrvyrkOKO6XgOYEkWpqDsox0rfp8tOfspmmWRZtTTsEVt0eI2UTSKYsDeLt8Qq gzToOwwjOIApoo9WNtw6bopyzTMFBxE= Received: from mail.blacknight.com (pemlinmail04.blacknight.ie [81.17.254.17]) by outbound-smtp02.blacknight.com (Postfix) with ESMTPS id 0E0B9BAADF for ; Wed, 26 Apr 2023 11:09:21 +0100 (IST) Received: (qmail 9114 invoked from network); 26 Apr 2023 10:09:20 -0000 Received: from unknown (HELO techsingularity.net) (mgorman@techsingularity.net@[84.203.21.103]) by 81.17.254.9 with ESMTPSA (AES256-SHA encrypted, authenticated); 26 Apr 2023 10:09:20 -0000 Date: Wed, 26 Apr 2023 11:09:18 +0100 From: Mel Gorman To: Doug Anderson Cc: Hillf Danton , Andrew Morton , Alexander Viro , Christian Brauner , Linus Torvalds , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Yu Zhao , Matthew Wilcox Subject: Re: [PATCH v2 1/4] mm/filemap: Add folio_lock_timeout() Message-ID: <20230426100918.ku32k6mqoogsnijn@techsingularity.net> References: <20230421221249.1616168-1-dianders@chromium.org> <20230421151135.v2.1.I2b71e11264c5c214bc59744b9e13e4c353bc5714@changeid> <20230422051858.1696-1-hdanton@sina.com> <20230425010917.1984-1-hdanton@sina.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: F0EAE18000C X-Stat-Signature: w1484nxzp1hjo3tastpx4jxpponnxxhe X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1682503762-716665 X-HE-Meta: U2FsdGVkX19F+f6K4RfE0D0immG5yE9GTHL9/GTZKfqTa6b0/9fV2G8hBDCUUW6O1I7CwxlLY/yv+ZKXw6/rk7YPsJ4qAa+DW5PElZZhHicxVcJnPPGbgvUaR2zpsJYLxHFimIrJ9vkGHL0dsnUXtQXgPNYjNpY8oNdkBuxmnncM300K61oDG4WcOAhgTBU3JzRAe9P7ZKZ6VzVwLAgHUnkMv2xIpMYO4VWMTVAjMV+ZhlBZBCIYMWPfCUGiaYtKL606GC8E+E4anoARmrcNUAN/OfBOCdvmgw4ZMKPvjEH5vZG8CwjB4H7krKQmLOozkn4+lrbcuFliiK4Hm9aPxOlxreIyo3p4tB6SRiHmM6toRJ2/zKAMy6hGR8GP6ULVpdY8NtqB3xtseZ35maqhcR0e2WsHImOijyEoygn0xMrzUXNFmOMCy6AqnnnLe6U4jY5l8OVhofGd8hLBc0r1EJysDrr4KDruBeeQ2mz8jwT96yfTG6vKi7pyOk+HIGVMzCkQc53RZRdssRynfP3ETocK6nTIgHqXpdwJ4xLmWt7iy9m9YmnTzKhnqi1RopuZViKDiQStpmYM5pLoHydbphh6lcVsFkv8MFFFwrUC494LhJdUcPd9/KhIVoCZwpyzEuYwm6+YPznQgs8WC7U4mA/8ADeFFNFX/LBTygybJ6iC6TZbFUozVbb+Y450GhRvFVDhmuNcn05oB741bgtmjGDZe4i28amBM22V3hNZ/nxtIzWZTsUunObcYlJWoboAsofgiYR61GiC5oz8Y5R5fKE/MiSfdL9F1kU7eOP6tXve6r3YjIdqT0CwGmpXWRuut1QPPZoK4T/tDM8XtuEbc/24C0+XguxXe9gRTCt7yfE6DefePV+mqwRbmDQRI5UjQ7QHGWMehk5Q623ddBTsy5+HqQn0lK8AJHaH63B3B5RvBW0v4YPlLCme+ehh+ckXPxTPUJ0h/MfMbDBXj9D Fs4TbqrE Wsr1Rm56BusaDCmHbhHCjSLT5Dj8F1NFD+EkckRoaSJRADcTD7MbRCx6OgbkQABf7KHSXFOFoulqifevbMfRYiJChc9T72QUadDlQQTaSZBcdqj/KU3UWzVJor7g/Q7SyNwwKD2dfx6QTAGmG0L5++Ks23OlJXhWsP0abF1M05UTQeLqNH+q36XAeHntnvX0JzP+XotXbiBcRBEJbSbRahjq+OUCAn8TtznmeRyJZ0CjogT7Mavf9sbwTVIDjKOv5RYtTI5VaLq8UHH89pfe2CNLP2StL+IzlXzVtTsN8aaqXcS0bGbtbpPm7e4zYvWCLTyRixzXdfTem2jzATlQQh6N7MNssKYLd3uB4JN/Fs2H33t9cpBuRvdwMoKeM9clAFoS5/luZzkpnCk3xwDnuGfeuovQ9z+YlA4F9 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Apr 25, 2023 at 07:19:48AM -0700, Doug Anderson wrote: > Hi, > > On Mon, Apr 24, 2023 at 6:09???PM Hillf Danton wrote: > > > > On 24 Apr 2023 09:56:58 -0700 Douglas Anderson > > > On Fri, Apr 21, 2023 at 10:19=E2=80=AFPM Hillf Danton wrote: > > > > If kcompactd is waken > > > > up by kswapd, waiting for slow IO is the right thing to do. > > > > > > I don't have enough intuition here, so I'm happy to take others' > > > advice here. I guess my thought was that kcompactd is explicitly not > > > using the full "sync" and instead choosing the "sync light". To me > > > that means we shouldn't block for _too_ long. > > > > Take a look at another case of lock wait [1]. > > > > [1] https://lore.kernel.org/lkml/CAHk-=wgyL9OujQ72er7oXt_VsMeno4bMKCTydBT1WSaagZ_5CA@mail.gmail.com/ > > So is this an explicit NAK on this approach, then? It still feels > worthwhile to me given the current kcompactd design where there is a > single thread that's in charge of going through and cleaning up all of > memory. Any single pags isn't _that_ important for kcompactd to deal > with and it's nice not to block the whole task's ability to make > progress. kcompactd is already very much designed in this model (which > is why SYNC_LIGHT exists in the first place) and that's why my patch > series was relatively simple/short. That being said, if people really > don't think I should pursue this then I won't send another version and > we can drop it. I don't consider it to be an explicit NAK but lets cc Linus because it's a valid question. Linus, the patch is https://lore.kernel.org/lkml/20230421151135.v2.1.I2b71e11264c5c214bc59744b9e13e4c353bc5714@changeid/ asnd it's adding folio_lock_timeout which in older terms is a lock_page_timout. The intended use is kcompactd doing out-of-line compaction (like kswapd does out-of-line reclaim) to try lock a page in MIGRATE_SYNC_LIGHT mode but if it cannot be locked quickly then give up and move on to another migration candidate. The MIGRATE_SYNC_LIGHT is expected to incur some delays while trying to make forward progress and the overall problem is that kcompactd can sometimes stall for many seconds and sometimes minutes on one page. The reason I don't consider this patch a NAK candidate is that this is not conditional locking as such because no special action is taken if the lock cannot be acquired. In the referenced mail, I think the context for the IO NOWAIT stuff is "try lock and if that fails, delegate the work to an async context". That is not necessarily a universal win and it's potentially complex. It's not a universal win because it's unknown how long it would take to acquire the lock and it may be a short enough period to be cheaper than the setup_for_async+context_switch+completion handler. If that happens often enough in a short window then delegation may be slower overall than doing the work synchronously. It's potentially complex because the setup for async handling and completion needs code that must be maintained. The kcompactd case using folio_lock_timeout is different. If the lock fails, it's not being explicitly delegated to another context, the page is simply ignored and kcompactd moves on. Fair enough, another context may end up migrating the same page in direct compaction or kcompactd at a later time but there is no complex setup for that and it's not explicit delegation. It's vaguely similar to how shrink_folio_list() calls folio_trylock and if that fails, keep the page on the LRU for a future attempt with the main difference being that some time is spent on trylock. This is *also* not necessarily a universal win because kcompactd could find a suitable migration candidate quicker by a plain trylock but that's what MIGRATE_ASYNC is for, MIGRATE_SYNC_LIGHT is expected to delay for short periods of time when MIGRATE_ASYNC fails and the problem being solved is the folio lock taking minutes to acquire. -- Mel Gorman SUSE Labs