From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFA44C83F03 for ; Fri, 4 Jul 2025 18:18:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 588F36B8064; Fri, 4 Jul 2025 14:18:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 53A286B8059; Fri, 4 Jul 2025 14:18:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4288B6B8064; Fri, 4 Jul 2025 14:18:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 326D56B8059 for ; Fri, 4 Jul 2025 14:18:30 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 8F96B1A021E for ; Fri, 4 Jul 2025 18:18:29 +0000 (UTC) X-FDA: 83627392338.14.25D010B Received: from mail-qk1-f174.google.com (mail-qk1-f174.google.com [209.85.222.174]) by imf18.hostedemail.com (Postfix) with ESMTP id B0ECA1C0007 for ; Fri, 4 Jul 2025 18:18:27 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=j9Hkw9Zh; spf=pass (imf18.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.222.174 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1751653107; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fm6vJJaMXbRf6XqtNvSdcQQA6cTKubPXK6+QlM/U09I=; b=PJ06pxVbX73Ww+/XTnm7Aj1E+Y0mZ4z+vuJ+UnVd5RDMfw8kqoCxcNoA17MuN6mlm2Ord2 Cev8Ih8cvn7YFnUAk0nYA2stxrcR8Lb/uDSf8pRZWrTn8BGzQO/pN4ugF7P41Xt6QbKCsC CKw+0xCsLvsAk4VwkvQtnnUx7LwiuO8= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=j9Hkw9Zh; spf=pass (imf18.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.222.174 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1751653107; a=rsa-sha256; cv=none; b=BT2ynlNChwUeMXsBsQ2OtrnMa5I6nNXkdLikgTURkEFMzqsFq6nbhYyANy6wsoQr0U44Ai 09uDOmUyTm7G+TtRS/nCVnAwN2fpp9h5SqQCHFJvDMn1gMm8tJ0CtFNwVuK44ntB8hJPKQ NfvNqeKNuTUAFbSHUn+AUtuwjqeO1v4= Received: by mail-qk1-f174.google.com with SMTP id af79cd13be357-7d467a1d9e4so230547885a.0 for ; Fri, 04 Jul 2025 11:18:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1751653106; x=1752257906; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=fm6vJJaMXbRf6XqtNvSdcQQA6cTKubPXK6+QlM/U09I=; b=j9Hkw9ZhAqA6OiRXJJ04iHhcmxeV+qbox+zDz2tszUFYmT0o5LFKJ8wW3fDqXVGBP8 g0J8FeLgJLCPIFiYXp8OfDKu+sF+O/5jHKEonYpGYUh7SeTeEoL9vhuIa/mMYAMHu8co CMMe7RpQ2XiNqDJhSH6Y0BwX4Yfahz/9pglADwKl1CHI/xoCOdBZQ97+vkoIkPWsFRmi jv/FG3RJS7+/UgymMUyhRLvJ+x+mW/drdKmk8SI6ZTnZbHo1aj4/XiQnrkFtFCcDIEVO +DfWOAgbVGfaGEFKzi9qHC6ZnWiKQtaKTFk1Bko/6ybWPc0N+5Rsl99g/8tFifmkPOKe gajg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751653106; x=1752257906; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=fm6vJJaMXbRf6XqtNvSdcQQA6cTKubPXK6+QlM/U09I=; b=V69MVftjqY2Xl8SJbTmJeWhFFlTJKAAJ+JLkiWcH6isf0ApwTot5Y55f/GO6jICYnk qTP/YRLbZQJ5Od+BkUmSkn2jlwR6HO3gheTEPSbWSIYvK7ki2gMjbzbpKPB7UrU9rvls tCy2HAKO/LIY/bDjHbFbmwa8uubeJ2gsZHFwoLDF3YTVnuli1QJLt0efpZdR/aniSeoj BAIHmmu3AzBN1efxzB279wv+IbYwS+W1tsLlf1nnX9ka9pMgNDOimKdUhqbkiNsJJ9kG 40Y5xPB/ptPvMlAJ0uprgiwHyWmGrkpRwDevOxEElqsDbxbCVFqU46g43TgHRe/uSv8V 3x4w== X-Gm-Message-State: AOJu0Yy51t82c/DNhAQnjMMf3AnbnlMZ32mReFDRa2Q0zbYdUyMkPuxB Uh9x+n+2jY0MsxVzivtTNrxFPttDOROrUrVCbZSyR0zWHckTgp/tkOCz0JtcnWBn9/o= X-Gm-Gg: ASbGncuuCOIdK7uD2Oz/zzse6lihvL3zFg3h75dd6kppzG2eqHz3pK6H7HrfDcjo4aH +MUA/mSAJwbKobfpFXptRG5jYil2MiolvCLMHJcbkA4kNXv7wmAEzu7MrHTfgQloz2M4f1akIC9 lnyZt93Isuv0wnSgWX9J54WBHiF+arxK/UskYz3sswwU6NCeWq+Jt2lc7MSQ5/1xCBCIPlO6zzq F5l3KGwUQCXF5CY3Y6m2hwMZBYx89jNnJ3cKhgHZpv0R6BeuW9lubI9JfSoxP4mHxhV+dLkclVq Z8ja/4i5u0ez1Lzu8ZN5jsBSNyqEnyxdhLKqx/OY4r7C+0Uwt3XksrWSBFIlOLCqsG0= X-Google-Smtp-Source: AGHT+IEXrMp6Gc6YtTfhlo1IKx9/9SN33lsFuLpyujrzcHT63LT3n6kt97kO/u1UXRNsCA8RvHka+g== X-Received: by 2002:a05:620a:6289:b0:7c5:95e6:62c9 with SMTP id af79cd13be357-7d5dcd47e14mr451163785a.29.1751653105731; Fri, 04 Jul 2025 11:18:25 -0700 (PDT) Received: from KASONG-MC4 ([101.32.222.185]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7d5dbe7c188sm183300585a.59.2025.07.04.11.18.21 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Fri, 04 Jul 2025 11:18:25 -0700 (PDT) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Hugh Dickins , Baolin Wang , Matthew Wilcox , Kemeng Shi , Chris Li , Nhat Pham , Baoquan He , Barry Song , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH v4 6/9] mm/shmem, swap: never use swap cache and readahead for SWP_SYNCHRONOUS_IO Date: Sat, 5 Jul 2025 02:17:45 +0800 Message-ID: <20250704181748.63181-7-ryncsn@gmail.com> X-Mailer: git-send-email 2.50.0 In-Reply-To: <20250704181748.63181-1-ryncsn@gmail.com> References: <20250704181748.63181-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Stat-Signature: gu5k59xesoaanu4z18d7an9z7ceoicsr X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: B0ECA1C0007 X-HE-Tag: 1751653107-492040 X-HE-Meta: U2FsdGVkX1/l08Kl4DcHkDS9X5swHCbiyzQOB279ZxS8NUhvaYWpkVgKGFuPTv+FEFXAex2E6nxy3lWiZ86lQuVZg8fsJzJm/nd72XI/OmGmC2lS1WKYhv64olNbvjb+L+Tsjfr4XW0E0XEAfPq0S2w2Clp4aoueRKOEy4g04mPaupHwnxdHmPfhtJX8jZA8NTLc8qGyXMgj+jx25FCK0C9aBt7/37S0IIJg5uCFHLLeVRIyuuLYsZaVRrJvcvIwkTOLKBUwtf08PWiae5xh0+ZCG94rvV7WXULLQlqEkbPcEqzL3hvX8DP4/W40akz7Z8wz67v+U7S9uoufRDqFUlFSUgYjAsNdTOEH8dthQLxSTev618ElxVOxURPCgAM3Jx7HtWMxSTinu1Xc0tB1MNOwdg6BLOfM03SgTWnOe2vZf/TheGaaiLRvTJY01pcrICzioVOwM7bhOlnerXjninKtGmXB/xRuGt7otq+nOSnyrx3IVUrYPu8EUy90qlHzo9vNA86JbIQiw1ZdVRm5X57pbob7fmdkCsvy9+4l/7FCfeapqys+dvx8HghZhvkLWbUc8l2Qu5aLJxqPXv90SpEBgPfpSnM386DwlHkOUQcl1ElEJno1v6YABEDqRfi8sIIOPS60/lpmf/oeecAYNyqRx05OUICf/qz+HWSeoWThtOZFXHiDdDnWqX+MkB22rMBNbfSUq0CRrJJBwI3DMFH3OtXQCVD7Exdo+dosxwSVcmMLX8DHdPRapnX3MFLjWwkKGoDENsw4zFiziMq+6rU0xkRGzGbSj2GESw1Q3fo7uz3JbzFbYYRy5zr/lROi//DhgqjKFZYh/QXBbl+eS6GpEX+wedsycqtKEWwDcaPlkaj0wzx8tna1KMt24mjffQIdk/IxdrJDXjv0JeNFva0JkbDfp8uEP2lZH2u6QxeucBYGjvO86gL6N7thyvUkLeOg31NNjUZE/0b9R1Y ezoDZ2sm zVfryguQHntY1PnGfBsPeIbqtNVS0C1b46oR9+RpaxNmGD3uXIW6RNU57HPqVNgRucgBYXa2nGF+/X1A12xc6H4QwtA2VrNckZm6bWdAskuk1Ri4ElXUNOFmVAXOTUbI2rbujzfQFMVRvgLvubYrW3og3CkW0H2+UWShefsxXSVVt3JT+W8+TaVUx9XjWVoa/cOpllpaByc7Zx2QulzbvV1MaHPsVkVCD87ieEzrMRf0ngfIKY73ZmQ57uRf7DD1Kto1ItrzlInQff8xzFOJMN/YYfIZNU9Ckh7mo8aVzGluYgyye4Z72fyQ/O9LV4c4++IKeOA9eftsWxLezWtRJKONVt3KQRsBDBZHYRhBlWkGzaLDQ/Lg/QIHyf6Ol3jxKS/WQmCLxV9JZfPAHcohDBvlHWNOa7y6prMlcmM4+7ILzzdQMbeRMwjI8ZWSknZ0TN+NGdi3YR/R3BGX8cXXK0FakpqzzN2SWAEF5voYJsnhPnTUZCkHZI240v/+Vm51Kq0VTCBR4jLCXrVgGz9+Rd2bjvq8p2nGVAj7hUti1wJ2JPnQ/abXnGCRqDZMWvY5DSeBNVibPPMfP90AhU1xByPice67lnMavr5gJ2u2BpIZ9m78OM28P65yR50KnK0lgY7BITZzYzipqGrgWV/SsvEh/Lo23hbtEyWCjhpm/Ofe2yOm0NnsJNwqKUQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Currently if a THP swapin failed due to reasons like partially conflicting swap cache or ZSWAP enabled, it will fallback to cached swapin. Right now the swap cache still has a non-trivial overhead, and readahead is not helpful for SWP_SYNCHRONOUS_IO devices, so we should always skip the readahead and swap cache even if the swapin falls back to order 0. So handle the fallback logic without falling back to the cached read. Signed-off-by: Kairui Song --- mm/shmem.c | 55 +++++++++++++++++++++++++++++++++++------------------- 1 file changed, 36 insertions(+), 19 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 2ab214e2771c..1fe9a3eb92b1 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1975,13 +1975,16 @@ static struct folio *shmem_alloc_and_add_folio(struct vm_fault *vmf, return ERR_PTR(error); } -static struct folio *shmem_swap_alloc_folio(struct inode *inode, +static struct folio *shmem_swapin_direct(struct inode *inode, struct vm_area_struct *vma, pgoff_t index, - swp_entry_t entry, int order, gfp_t gfp) + swp_entry_t swap, swp_entry_t index_entry, + int order, gfp_t gfp) { struct shmem_inode_info *info = SHMEM_I(inode); + swp_entry_t entry = index_entry; int nr_pages = 1 << order; struct folio *new; + gfp_t alloc_gfp; void *shadow; /* @@ -1989,6 +1992,7 @@ static struct folio *shmem_swap_alloc_folio(struct inode *inode, * limit chance of success with further cpuset and node constraints. */ gfp &= ~GFP_CONSTRAINT_MASK; + alloc_gfp = gfp; if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) { if (WARN_ON_ONCE(order)) return ERR_PTR(-EINVAL); @@ -2003,19 +2007,22 @@ static struct folio *shmem_swap_alloc_folio(struct inode *inode, if ((vma && unlikely(userfaultfd_armed(vma))) || !zswap_never_enabled() || non_swapcache_batch(entry, nr_pages) != nr_pages) - return ERR_PTR(-EINVAL); + goto fallback; - gfp = limit_gfp_mask(vma_thp_gfp_mask(vma), gfp); + alloc_gfp = limit_gfp_mask(vma_thp_gfp_mask(vma), gfp); + } +retry: + new = shmem_alloc_folio(alloc_gfp, order, info, index); + if (!new) { + new = ERR_PTR(-ENOMEM); + goto fallback; } - - new = shmem_alloc_folio(gfp, order, info, index); - if (!new) - return ERR_PTR(-ENOMEM); if (mem_cgroup_swapin_charge_folio(new, vma ? vma->vm_mm : NULL, - gfp, entry)) { + alloc_gfp, entry)) { folio_put(new); - return ERR_PTR(-ENOMEM); + new = ERR_PTR(-ENOMEM); + goto fallback; } /* @@ -2030,7 +2037,9 @@ static struct folio *shmem_swap_alloc_folio(struct inode *inode, */ if (swapcache_prepare(entry, nr_pages)) { folio_put(new); - return ERR_PTR(-EEXIST); + new = ERR_PTR(-EEXIST); + /* Try smaller folio to avoid cache conflict */ + goto fallback; } __folio_set_locked(new); @@ -2044,6 +2053,15 @@ static struct folio *shmem_swap_alloc_folio(struct inode *inode, folio_add_lru(new); swap_read_folio(new, NULL); return new; +fallback: + /* Order 0 swapin failed, nothing to fallback to, abort */ + if (!order) + return new; + order = 0; + nr_pages = 1; + alloc_gfp = gfp; + entry = swap; + goto retry; } /* @@ -2309,25 +2327,24 @@ static int shmem_swapin_folio(struct inode *inode, pgoff_t index, count_vm_event(PGMAJFAULT); count_memcg_event_mm(fault_mm, PGMAJFAULT); } - /* Skip swapcache for synchronous device. */ if (data_race(si->flags & SWP_SYNCHRONOUS_IO)) { - folio = shmem_swap_alloc_folio(inode, vma, index, - index_entry, order, gfp); + folio = shmem_swapin_direct(inode, vma, index, swap, + index_entry, order, gfp); if (!IS_ERR(folio)) { - swap = index_entry; + if (folio_test_large(folio)) + swap = index_entry; skip_swapcache = true; goto alloced; } /* - * Fallback to swapin order-0 folio unless the swap entry - * already exists. + * Direct swapin handled order 0 fallback already, + * if it failed, abort. */ error = PTR_ERR(folio); folio = NULL; - if (error == -EEXIST) - goto failed; + goto failed; } /* Cached swapin with readahead, only supports order 0 */ folio = shmem_swapin_cluster(swap, gfp, info, index); -- 2.50.0