From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F167C48260 for ; Fri, 16 Feb 2024 16:57:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 907EF6B00A2; Fri, 16 Feb 2024 11:57:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8B7916B00A5; Fri, 16 Feb 2024 11:57:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A7246B00A6; Fri, 16 Feb 2024 11:57:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 69CA26B00A2 for ; Fri, 16 Feb 2024 11:57:13 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 2FAFE1A0167 for ; Fri, 16 Feb 2024 16:57:13 +0000 (UTC) X-FDA: 81798272346.16.C28499C Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) by imf16.hostedemail.com (Postfix) with ESMTP id ABBB118000D for ; Fri, 16 Feb 2024 16:57:10 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=MjzGwcxi; dmarc=pass (policy=none) header.from=intel.com; spf=none (imf16.hostedemail.com: domain of tim.c.chen@linux.intel.com has no SPF policy when checking 198.175.65.11) smtp.mailfrom=tim.c.chen@linux.intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708102631; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=I+mJQH7DOUiZAPpinpP124YYLAcDdWEyOpNaofEPcu0=; b=bsJw0yJzcfz9K5ePzN5VPx/PX6UTp73EKvuvyYgaSBB412BHS/x5dFhQlBuP5XMSUzyXfW nk001w2DnEdjVjPc4RhktmdBI4DhcbLlN1mNnmLr442knzyyylR/p6lHECitn829+5UBlG f+K06WsgkLsAQXVavXqNqtNcjhQXV3M= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=MjzGwcxi; dmarc=pass (policy=none) header.from=intel.com; spf=none (imf16.hostedemail.com: domain of tim.c.chen@linux.intel.com has no SPF policy when checking 198.175.65.11) smtp.mailfrom=tim.c.chen@linux.intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708102631; a=rsa-sha256; cv=none; b=oW7/uTATWRq7NxfEBCfcFUHm5uOiHxAtfyL9S9EJO7ms6J+3l9f3dK2JFHPtvuw93zXqHA 44tibZ3WTLx3IdRN1aebN90cWWjmjJ5SlvopnMP2uud+FBmO6QXX2hILAharVaGOMGS3Ln xJgHoQU6LLmVz26P7pTL8erQDI2SHaA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708102631; x=1739638631; h=message-id:subject:from:to:cc:date:in-reply-to: references:content-transfer-encoding:mime-version; bh=Njb939Xj2TSbF8CTUoe/en9KsOGgfKVWdwwSZxqrTVQ=; b=MjzGwcxiZw+UAcdNbkTWHe+yR4HEhtoSd4ccTZLIigWiqZJ2ycrm7zyl DBajzwe6ZLjsZr7uP2zMjvB29+XeMhlE9RFXmyCFndLtfyTriiKRcNpca E+LPGQ3+HlfkLoGVTSJq0EYQ+SrX6mBDqAgKs1iK3dMNmZQbL6frNyiI+ J4nsTlPS0zu9uJHF9ZTIgOsgLSUFrircUGNQJTp+hG5w9zy5HOqqlSQYZ gJRvkTz3m5CP8M7hrA+zjveh7TEbVxJEWNKPrHycfIFiXxfcBFSYaVAwy +6kwB/ffgRhpa/3TBD9x5duUyL2Di620cCbSXsVxqenKFlodid9X8IULk A==; X-IronPort-AV: E=McAfee;i="6600,9927,10986"; a="12780600" X-IronPort-AV: E=Sophos;i="6.06,165,1705392000"; d="scan'208";a="12780600" Received: from fmviesa006.fm.intel.com ([10.60.135.146]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Feb 2024 08:57:09 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,165,1705392000"; d="scan'208";a="4167575" Received: from sbiswas-mobl1.amr.corp.intel.com (HELO [10.255.230.53]) ([10.255.230.53]) by fmviesa006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Feb 2024 08:57:08 -0800 Message-ID: <52d414b85f41a76fc0a7b0082cba95297d9e5874.camel@linux.intel.com> Subject: Re: [PATCH v4] mm: swap: async free swap slot cache entries From: Tim Chen To: Andrew Morton Cc: Chris Li , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Wei Xu , Yu Zhao , Greg Thelen , Chun-Tse Shao , Yosry Ahmed , Michal Hocko , Mel Gorman , Huang Ying , Nhat Pham , Kairui Song , Barry Song Date: Fri, 16 Feb 2024 08:57:07 -0800 In-Reply-To: <20240215201627.5abd1841192feaa262d545ba@linux-foundation.org> References: <20240214-async-free-v4-1-6abe0d59f85f@kernel.org> <20240215161114.6bd444ed839f778eefdf6e0a@linux-foundation.org> <1b9a69d1ecaac45a228eb2993d5d9b8234a84155.camel@linux.intel.com> <20240215201627.5abd1841192feaa262d545ba@linux-foundation.org> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.44.4 (3.44.4-2.fc36) MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: ABBB118000D X-Stat-Signature: 4og96gfbazfpxx9jb7qw445wes5ujz51 X-Rspam-User: X-HE-Tag: 1708102630-40807 X-HE-Meta: U2FsdGVkX18n2PnT/baYcDx0AT2MwvBIodiVN8B+3+jjd91akRNe5Lm+OQ++4OY9+mESmj03x1zbKNQgvw3RvpEFhrFNs0+b2xoQJeLY7A1NIVTiU9+JNwigJJ4Spm7qKEEtp17/kNiFBLSs/IBvw6VdreWw9DS1bbbVeDKqibJnVy6sDpnf2+qaNyHANBvgwPYC/TBspl3axY4WWVOIx4m1MA2p3UNGb3/zDNHH3hAIdz6g//MZxIko9qPX2TfDVJEgRJnJAxvKDYvRSmabVnXBtFfEAbRHiGNVJ3NLj5dF4mvbaMHj0MfvzBdQ9u36Ei27WEwRcJXas87vY46HmX9DFI92ybo1o/9n+7NmLBbpzF9DdB7ezQTBymF2M3XVUFtHn0ZARvJ/BIXALzxPCojwn64QXDjhnc/wJvMeF6ia5yX5DpbeCEySHWK6ovwFDw+G41bLVKWpXv95+wmq+ZuYuCZFHmbdz2YP+kzUkxfyQ9AsCMFZaHXJMMAJOzDAFQZRKxNGy811j0XU32bgqu1U6KMrCxYfSsVkrdS5rNm3Swlfdf0sv5C2qmvCmisHSoIlVlPJGnFSa8e1Q2SDxUd0JP0hpzAa+PKxoz3HeVUVz1X8FEX9HiTxPmDAYI0Wli/8dNGpcGK7DO0yJJdcjiBUJMt3VcxTL+xD9oT/MeabDoYrxTdMWCYfHp2sm2iXIUZFmWMw1Xqx9R2mYoqI/wfkgUgY0aDMvKzaTU3tokuZmqIUDj05v81t5ClphWJurwg1qh4DODVIXIf+JesaTi1/AkxaDnU4DY4iTTU1fyFusFBWBKnIPiWb25WJrveu67T3lKLfsr0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, 2024-02-15 at 20:16 -0800, Andrew Morton wrote: > On Thu, 15 Feb 2024 17:38:38 -0800 Tim Chen = wrote: >=20 > > > What this description lacks is any description of why anyone cares.= =20 > > >=20 > > > The patch clearly decreases overall throughput (speed-vs-latency is a > > > common tradeoff). >=20 > This, please. >=20 > > > And the "we don't know how to fix this properly so punt it into a > > > kernel thread" approach remains lame. For example, the risk that the > > > now-liberated allocator can outpace the async freeing, resulting in > > > unlimited object windup. > >=20 > >=20 > > Andrew, > >=20 > > What you are saying about outpacing asyn free is true for v1 and v2 ver= sions of the patch. > >=20 > > But in this latest version, if another reclaim comes in before the asyn= c free has kicked in, > > we would be freeing the whole cache directly, same as original code, wi= thout waiting > > for the async free. It is different from the first version > > where you go into the free one at a time mode while waiting for the asy= nc free.=C2=A0 > > That was also my objection to the first two versions as you could be in= this > > slow free one at a time mode for a long time. > >=20 > > So now we should not have unlimited object windup. And we would be doi= ng free > > in batch of 64, either still in the direct path or in the async path. > >=20 >=20 > OK, thanks, I didn't read closely enough, >=20 > > If the next swap fault comes in very fast, before the async > > free gets a chance to run. It will directly free all the swap > > cache in the swap fault the same way as previously. >=20 > And might it be a win to cancel the async_work in this case? >=20 Canceling async_work will matter for the case where we push swap hard, and have a better chance of finding async have not yet engaged when we need to free additional swap slots. Chris' tests so far has been for his use cases where swap is lightly loaded. The scenarios you listed are when=C2=A0 we push swap hard close to its max throughput.=C2=A0 It would help answer your concerns if Chris could also test high swap scena= rio. Then we can make sure sustainable swap throughput does not regress and latency is improved. And check whether it is beneficial to cancel outstandi= ng async_work on direct free path. I think the pro of canceling the asyn_work is to skip an extra lock acquisition on the cache. Though there is also some overhead in canceling the work itself. Tim >=20 > Again, without a clear description of the userspace-visible effects of > this problem I am groping in the dark. My hands blindly landed upon > the question: the overall effect here is to leave worst-case latency > unaltered, but to decrease average latency. Does this satisfy the > yet-to-be-described requirements? >=20 >=20 > Also, the V4 patch's quoted quantitative testing results are pasted > from the V2 patch's. V2 was a fundamentally different implementation.= =20 > I think it is fair to say that V4 is "untested", with regard to > satisfying its runtime objectives. >=20