From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD701C3DA6E for ; Sat, 23 Dec 2023 06:11:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 384C66B0071; Sat, 23 Dec 2023 01:11:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 335C16B0072; Sat, 23 Dec 2023 01:11:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1FC2A6B0074; Sat, 23 Dec 2023 01:11:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 0F7286B0071 for ; Sat, 23 Dec 2023 01:11:48 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id DE7FEA1F65 for ; Sat, 23 Dec 2023 06:11:47 +0000 (UTC) X-FDA: 81597061854.08.7F3A3E8 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) by imf18.hostedemail.com (Postfix) with ESMTP id 1DE001C0014 for ; Sat, 23 Dec 2023 06:11:45 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=AwWqCry0; spf=pass (imf18.hostedemail.com: domain of rientjes@google.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=rientjes@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1703311906; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1Abo/6b6Ud8/ygj6vePFRQy4/FPhVI580YuqrRjDjYA=; b=BGMQ72rDHA+X7xBOvgdVKviLM5Cl1Rh1ibDzW21hlrlQuluTP96W+5jjVdFkWWFfvAaqFb dFUCCkygBcXJR+V4vrM83M5sHavxE2j7iNB7RCJEUaOGQkxtpCZHE97V04gZH/LXhIhGEG Vghe10nf3WOHHxRo6GppbVPxcNFSpHk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1703311906; a=rsa-sha256; cv=none; b=AZSPsXz24Xzj3OmfJJbSZwwZrMjxRphuQgFkm4aRh+It7CLWurGMheW6MkPlWDuCHJBpI1 gNV1DB43FQvzSeNqnef6ERH3eivSejYoOFX8/iIM1R5Gmv3tZp1rwEK8JN2CtqYOGhhrZd w6CQcntBZOSgKajq/yS7PdGsl/tnAEY= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=AwWqCry0; spf=pass (imf18.hostedemail.com: domain of rientjes@google.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=rientjes@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-1d3e4637853so224235ad.0 for ; Fri, 22 Dec 2023 22:11:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1703311905; x=1703916705; darn=kvack.org; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=1Abo/6b6Ud8/ygj6vePFRQy4/FPhVI580YuqrRjDjYA=; b=AwWqCry0buTsdml6Mz5g4LPIYNuNgyb2Sqzt7lX2BwUfJx13XBeHWgb/BKYqR63Z4G bQbKAhpmlojgWqxwCDa0mjjoeuOAJarVguIZ68x4jxpEB7SLUzz4ISx83SbI0iSujH7m +ucclhoiIPDsxrgrbGxlawzF/CzDD++bBegQXtxIv73fafBR8RVulEhuBP5R1s8RJvQZ tsXbIe/P6Ewy3oFX5o3KxXXC9kaFZNx3OOmfDFx34cKHHbOruc8nnhxJeO5BPJEzUR5I ERVZuk0+NwS6F6yV/Rduxvp0bDsWNjqeCTny2AFcmQMgOSyp3MW4/wqYRZDswTRaZseu IuFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703311905; x=1703916705; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=1Abo/6b6Ud8/ygj6vePFRQy4/FPhVI580YuqrRjDjYA=; b=IY75bxVUa9nlISzPLSO2vo0CCrYyJBdsX67ivg0uB5ESNaToIfxfCzyV697SYg0xBK dOxk3J7lhLOfRlXPN2G9oyw6HV7b9LhBxr97ke6a1psevx33tFG3hkzLkSS55KqZIkjY f5bN+ZivezbblWDjooSfQuVty4SM1cPWrVGFR2jRJHatKDmvBfY0RTseBxqjLRZ+oLSZ PH5/C64YkHCs+rE2JqdHMJd+Bdg7Xhtcr7RJ6acXIkBvdUUVrEm6RyPmDRZvtm/cQQhB 7GHTEWyBSc2+BQLSgYFN/u6HQ9xhLaiyRR7Fu79JP3UOLJJ4jThKZ1eWFOdCkqvOJ/qV mQ/A== X-Gm-Message-State: AOJu0YyaE+AUPMA7CJEvgPkhgRg0UGTY+7IU6p6txGLCRKtC/uRRQsOs ymE3Bncum7sYrxRVL3o++lRwF/mn75FI X-Google-Smtp-Source: AGHT+IE/W4mVw3DWpGmrlczeA0IjpfFgzLnX0g85PAh4xhNoYsEJy7KSW9tLOeVYqkIg5GlB52iHtw== X-Received: by 2002:a17:902:f983:b0:1d3:d270:de8a with SMTP id ky3-20020a170902f98300b001d3d270de8amr181071plb.25.1703311904688; Fri, 22 Dec 2023 22:11:44 -0800 (PST) Received: from [2620:0:1008:15:ed14:1d0f:e856:8a58] ([2620:0:1008:15:ed14:1d0f:e856:8a58]) by smtp.gmail.com with ESMTPSA id h12-20020a170902eecc00b001cf511aa772sm4373015plb.145.2023.12.22.22.11.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Dec 2023 22:11:44 -0800 (PST) Date: Fri, 22 Dec 2023 22:11:43 -0800 (PST) From: David Rientjes To: Chris Li cc: Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Wei Xu , Yu Zhao , Greg Thelen , Chun-Tse Shao , Suren Baghdasaryan , Yosry Ahmed , Brain Geffon , Minchan Kim , Michal Hocko , Mel Gorman , Huang Ying , Nhat Pham , Johannes Weiner , Kairui Song , Zhongkun He , Kemeng Shi , Barry Song , Hugh Dickins Subject: Re: [PATCH] mm: swap: async free swap slot cache entries In-Reply-To: Message-ID: References: <20231221-async-free-v1-1-94b277992cb0@kernel.org> <20231222115208.ab4d2aeacdafa4158b14e532@linux-foundation.org> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 1DE001C0014 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: rhichsaqhir5gxmyomqrik4fzd55cne4 X-HE-Tag: 1703311905-40541 X-HE-Meta: U2FsdGVkX186UGtQ1OWNFJKGZp/CRIEedbjRpvJQEMSdIonfcFUDy4KvUzYVgXc3VBqewlWzCynnUFaNaoJ3i9F2yA3YLGdh+iXmoMF+pH3Mil0HqO+hjmjsq92zjBvq5ufGAi+Ax40vO7chXC34z662utiP/Ru21qcZcPdaxs55OhTsYvfb3ug6q+N8qgPG8xBDrDU2M39zERRhVAQ2Slu0JgEF2bapBNijBR6IGllA7PCOA7W1CE/D2DugMMVKpSCEesKKNPV9jxBNK1E0Gb1y1LVtVmw0HmlY+uMAXJ9I3xHTshc7giM3ujh2b/Ilc9wmV+JkK2u2t+VBH60+tbPCUfvkRieRzTsIa0T8aKlLsffC1L2ox99pW31ShKD8og4aBn++sm/NHc1QxdZt4iZ1YpYgpWQFVNtgDrzYg1CQEil6WrW+mRn2Nl1cTBHWURAYsoSTzM4aiyvyxbbJy0vMSgfUFUF1aV9r0bj7He6Zn4fqeRv61gUNZH3Irhj1+tAO7ttfAO6NjF7PmTk7+jOJGkVtFbitipqoCD15y+f43tAMlijCPM8RvJkcS3UlBjDL+jwPiALknSbt4+HdtcIn7sYpxLSqqkRwDP6JkdMeNPeutE4UlOaZkBg/3Vwiv+TF/piRN1LdjNfzX98yr5CdMf2BCGR00TZbX9bFkOKYX2jdfq9YRx8NBDD22A10pcy8eklhE5FWAk167lDC8UUV/3v4ajtlxh17jAJk0+wGMV9BXMpLOEf1z9BwMl98hk1ryTrth7czMfZmRkeALqGpeWmo5JliCo/Icq8TrvcytIpQIHeEnMNGDvEbTYWwNc6DBv4bMVIunzjgTP1u8L9ZY85kofqUebr7eDjoXO1YHKXHEipqET27knVRSKsdxe8F5umprPjXkbWqVrId9pIIb3pMTtNvHrnPYvQVSjK1mEIc4nczGQAObevABhawKttM5AXNRTwe9eir6K5 9pHi9dNG kxvZ1xawwowkvJVwZtCc/FyLkJt2nEC48XQf6a7hzNqEN+bzufp9eO3lXZNzVf2CSZ/9qdJOQ/3je+0D8DET5Fs97CDZ5HFAquFNR0HZuHc4X09ongqCNXrc5bUJIob/i8+TNuEbmDTBjbB8EZuxIVHpp85YDj3+yZd3ToGDWJv6phITyBgJF8oQVBi5p7gYCOb0drs0Px1Y6FTVHiAKgTCeQJI+DJiRzN3hkDjcL5ROTYBEUkWdwHgimpejuRiCjx+ve5A7hFz1+yttZJ11tlFUrjNQQY29oFBsaMPtix4n8fuvczyv2j9mBNFD/Fzkm5dLsZyE3BHhJ+TXBk0QGES2D0395yMdmorOf0Dt/+gXDjRnTV7w4YFeK5wlbsqsvIzFHagWFkzQqWHF9zCru2iXfWyfcjFD++0ntUk5A7/35Qh9rlzA/2H6HMoD1zevj6ZW7eZmkTO0HQrOg2A0V5Nc0/Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, 22 Dec 2023, Chris Li wrote: > On Fri, Dec 22, 2023 at 11:52:08AM -0800, Andrew Morton wrote: > > On Thu, 21 Dec 2023 22:25:39 -0800 Chris Li wrote: > > > > > We discovered that 1% swap page fault is 100us+ while 50% of > > > the swap fault is under 20us. > > > > > > Further investigation show that a large portion of the time > > > spent in the free_swap_slots() function for the long tail case. > > > > > > The percpu cache of swap slots is freed in a batch of 64 entries > > > inside free_swap_slots(). These cache entries are accumulated > > > from previous page faults, which may not be related to the current > > > process. > > > > > > Doing the batch free in the page fault handler causes longer > > > tail latencies and penalizes the current process. > > > > > > Move free_swap_slots() outside of the swapin page fault handler into an > > > async work queue to avoid such long tail latencies. > > > > This will require a larger amount of total work than the current > > Yes, there will be a tiny little bit of extra overhead to schedule the job > on to the other work queue. > How do you quantify the impact of the delayed swap_entry_free()? Since the free and memcg uncharge are now delayed, is there not the possibility that we stay under memory pressure for longer? (Assuming at least some users are swapping because of memory pressure.) I would assume that since the free and uncharge itself is delayed that in the pathological case we'd actually be swapping *more* until the async worker can run.