From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6ABAC77B78 for ; Wed, 26 Apr 2023 15:14:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 50A6B6B00E9; Wed, 26 Apr 2023 11:14:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4BAB06B00EA; Wed, 26 Apr 2023 11:14:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3A9986B00EB; Wed, 26 Apr 2023 11:14:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 24F1D6B00E9 for ; Wed, 26 Apr 2023 11:14:28 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id BA4061602FC for ; Wed, 26 Apr 2023 15:14:27 +0000 (UTC) X-FDA: 80723888574.11.4B89A84 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf27.hostedemail.com (Postfix) with ESMTP id B78BB4000F for ; Wed, 26 Apr 2023 15:14:25 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=lNE3M2oQ; spf=none (imf27.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1682522066; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Yammpa9IiLASda95uMVgcL3LerSIgwp4czXb64R8Cec=; b=VRaCUTZKRbHf0km7B5FTvD12uWdNBaRIHy3p6CTydzcBwbBo+kKTG0nInZL34GlDBrImIg IeP284UeaZulMvY3wYQenbUa0chqJTgQNvWyDMI+GitUUF8OZoyyhqsxzQBmOjSefNy0rc duGpURmUazNti9eeJZerLsHEXIqvA/0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1682522066; a=rsa-sha256; cv=none; b=a8XT/j6jA/J3HXA9XnTPzyPSF4lH9waCps+CmlI25yXFx8nHALXPSO5vdZKZFV1K4ibI+L 4VGrAb2L8KmNoqcjrnBIVjY4MX9ipdccafl1DPN3b+rgy8Xo3FtyXQ0yueJmxZkH7vbqhg H/bMnL36JnVc/qBOBbXIlzN2YyWhG+Q= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=lNE3M2oQ; spf=none (imf27.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=Yammpa9IiLASda95uMVgcL3LerSIgwp4czXb64R8Cec=; b=lNE3M2oQhcK/lgbvTsfH5CFeA6 FlCPQsdftdyG8+wIg+keOf8qqmWL3NO+UGv16+1KFu44peZ955HlTUOUzLPRkzfQOOhv8/yd8mMkX dNySdSp0+1fwJNYrRVecPtnKgypUM0BL3BEdgxL/BrqmZ2CSJhX4ARnSN9xypqIYUVGb0X5y9EPrn WS7cLZBq/p/HSS1e77eNFM2CWJX0vBOwuehZhPH3nIUxKAbfbRwD+memVM8zhS0Disad60d3dwv3l iqIcFyHgG3veoPHr2OceeP3Vrm521mLwUUAUKbRvucWs03tBbvpTv/fFYvZktGCcrtC2LGmKFiE6Y XUQrSmcA==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1prgqH-002a2L-Bk; Wed, 26 Apr 2023 15:14:01 +0000 Date: Wed, 26 Apr 2023 16:14:01 +0100 From: Matthew Wilcox To: Mel Gorman Cc: Doug Anderson , Hillf Danton , Andrew Morton , Alexander Viro , Christian Brauner , Linus Torvalds , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Yu Zhao Subject: Re: [PATCH v2 1/4] mm/filemap: Add folio_lock_timeout() Message-ID: References: <20230421221249.1616168-1-dianders@chromium.org> <20230421151135.v2.1.I2b71e11264c5c214bc59744b9e13e4c353bc5714@changeid> <20230422051858.1696-1-hdanton@sina.com> <20230425010917.1984-1-hdanton@sina.com> <20230426100918.ku32k6mqoogsnijn@techsingularity.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230426100918.ku32k6mqoogsnijn@techsingularity.net> X-Rspam-User: X-Rspamd-Queue-Id: B78BB4000F X-Rspamd-Server: rspam09 X-Stat-Signature: qgh9m4gfgzd6wpyzw6yac3ncodckj9s5 X-HE-Tag: 1682522065-358023 X-HE-Meta: U2FsdGVkX1/BCaTLExbYL4LCirlB0q8rt2YCFrmNFI0VgBDDgj2AnxZHAOB8viOPJ6sCtTjdMy2RFUtYEE0bCQmzVs597TuoA096lGEMpj6l2SFOGTr6NCOwKMd8kOvKTywYaz5sYkbZWJ8pWExpgpoey1Zx3nXyMl1eewjyh/MT8KPLne9l6WQfvdbOF+4+KhKnwgFSeWjht/9iN4RgEA2xjeVCgIDYGPWSA5oWNs0Wd086cci3FXxwLe0bf5Prk1x7bqQcXCDKph0tR9P4yuzX0ArUwwf2i9FfAZJj5tCmpF08/1zaRsqprgrMzlaXx14pigAQM70Q0GlQ6+5rJEYS+guhzgnzBHszmD+XMF80UF13dew//5s4Qva6veVvfjZO8Rrv+HBi2a9aZYcpmtOzPBmoWrdNjkxFGZoVepw9t29MpUFXb4RjiaA1vCoGCDL47dQ5dqVUBKOodaBpXj88aKSMsq7MgoAarPEWjlJZkkbTNyMnpLCB4c9SdHwBcbcMBuoxMOscb5kM+AToETUvJXlBWsJB/O4aRsbTvia8axkLOjUkZzcNzwn/QE58XcnQ8bA0O6q+HIQJvOVg3zxfHCDpYRoKAe3cL+P42HcP4DaPpsrsDjRqSv8EtKaOelUWP7q8Oo0Ar4T7MAf/MtCQYaNZY40qhzG553ZNjRhu+UQ9GwO0QA7sF8AkJA50Mp79P4mP8D358wH6oq/biJyykgMaA0moCgaMJafrzNvoPPVdvVgqq1EK7WWmD06wYabSVMbJwmEg5u5rlFNh+RoiQq4F/11eQiFkbcebdTF7A549HlAMsD3glDKUiPw/eOcGdfkEml5qEdp5/yyOpnoKAe+Ak4HfhamY9uBIiQ1+QxkOwyAHxyQtKbWYOgieJdPqd+5wipy4bHJqq51uUw7m8z0wChH8d7xMCAu5zjNVV6vclc603x1pR+mFxiT6BBibAbM6nUypm4a/A8v WrSID/R3 6KGWZQNoJHjiVH4zBmsvjPG9+WZW31yHonPheqHPEu3FamJX7XY+ERJjszvoTWRpbGlEXMeUog7+KlGro05BHlYSO2W15vIzNRCQGG3ipN8o3KV3a6sNdVBu81UMf1t+Uv03ehSmLSo5kSTQ2fT6oXQT7uN3UXcoUWLYXUg5YCtA3GZ6dbPO2I3WlViPmjUnLRdVHTEowOh844oWurwblJUvMbIuF/qRDvbwPV/xc9YEUq3eStqlu0bwnDBcsAT0dUcV7hCLxXK4QFhaqF6X8mvr2OGC7hhCWkC2VAj2CPfi6Z45Teu2kFnGinnSa6aUlFp/tWN4kqyRv+zU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Apr 26, 2023 at 11:09:18AM +0100, Mel Gorman wrote: > On Tue, Apr 25, 2023 at 07:19:48AM -0700, Doug Anderson wrote: > > On Mon, Apr 24, 2023 at 6:09???PM Hillf Danton wrote: > > > Take a look at another case of lock wait [1]. > > > > > > [1] https://lore.kernel.org/lkml/CAHk-=wgyL9OujQ72er7oXt_VsMeno4bMKCTydBT1WSaagZ_5CA@mail.gmail.com/ > > > > So is this an explicit NAK on this approach, then? It still feels > > worthwhile to me given the current kcompactd design where there is a > > single thread that's in charge of going through and cleaning up all of > > memory. Any single pags isn't _that_ important for kcompactd to deal > > with and it's nice not to block the whole task's ability to make > > progress. kcompactd is already very much designed in this model (which > > is why SYNC_LIGHT exists in the first place) and that's why my patch > > series was relatively simple/short. That being said, if people really > > don't think I should pursue this then I won't send another version and > > we can drop it. > > I don't consider it to be an explicit NAK but lets > cc Linus because it's a valid question. Linus, the patch is > https://lore.kernel.org/lkml/20230421151135.v2.1.I2b71e11264c5c214bc59744b9e13e4c353bc5714@changeid/ > asnd it's adding folio_lock_timeout which in older terms is a > lock_page_timout. The intended use is kcompactd doing out-of-line > compaction (like kswapd does out-of-line reclaim) to try lock a page in > MIGRATE_SYNC_LIGHT mode but if it cannot be locked quickly then give up > and move on to another migration candidate. The MIGRATE_SYNC_LIGHT is > expected to incur some delays while trying to make forward progress and > the overall problem is that kcompactd can sometimes stall for many seconds > and sometimes minutes on one page. > > The reason I don't consider this patch a NAK candidate is that this is not > conditional locking as such because no special action is taken if the lock > cannot be acquired. In the referenced mail, I think the context for the IO > NOWAIT stuff is "try lock and if that fails, delegate the work to an async > context". That is not necessarily a universal win and it's potentially > complex. It's not a universal win because it's unknown how long it would > take to acquire the lock and it may be a short enough period to be cheaper > than the setup_for_async+context_switch+completion handler. If that happens > often enough in a short window then delegation may be slower overall than > doing the work synchronously. It's potentially complex because the setup > for async handling and completion needs code that must be maintained. > > The kcompactd case using folio_lock_timeout is different. If the lock > fails, it's not being explicitly delegated to another context, the page > is simply ignored and kcompactd moves on. Fair enough, another context > may end up migrating the same page in direct compaction or kcompactd > at a later time but there is no complex setup for that and it's not > explicit delegation. It's vaguely similar to how shrink_folio_list() > calls folio_trylock and if that fails, keep the page on the LRU for a > future attempt with the main difference being that some time is spent on > trylock. This is *also* not necessarily a universal win because kcompactd > could find a suitable migration candidate quicker by a plain trylock but > that's what MIGRATE_ASYNC is for, MIGRATE_SYNC_LIGHT is expected to delay > for short periods of time when MIGRATE_ASYNC fails and the problem being > solved is the folio lock taking minutes to acquire. I'm not generally a fan of lock-with-timeout approaches. I think the rationale for this one makes sense, but we're going to see some people try to use this for situations where it doesn't make sense. I almost wonder if we shouldn't spin rather than sleep on this lock, since the window of time we're willing to wait is so short. I'm certainly not willing to NAK this patch since it's clearly fixing a real problem. Hm. If the problem is that we want to wait for the lock unless the lock is being held for I/O, we can actually tell that in the caller. if (folio_test_uptodate(folio)) folio_lock(folio); else folio_trylock(folio); (the folio lock isn't held for writeback, just taken and released; if the folio is uptodate, the folio lock should only be taken for a short time; if it's !uptodate then it's probably being read)