From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 310A3E77188 for ; Mon, 30 Dec 2024 17:46:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A83F66B0085; Mon, 30 Dec 2024 12:46:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A33106B0088; Mon, 30 Dec 2024 12:46:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8ACE46B0089; Mon, 30 Dec 2024 12:46:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 6593C6B0085 for ; Mon, 30 Dec 2024 12:46:43 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 237DEA01B4 for ; Mon, 30 Dec 2024 17:46:43 +0000 (UTC) X-FDA: 82952354016.10.9605238 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf22.hostedemail.com (Postfix) with ESMTP id 902A2C0005 for ; Mon, 30 Dec 2024 17:45:52 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BQgilxEB; spf=pass (imf22.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.216.52 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735580780; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ByQEC8FsvBO2uOH/tvc24OCSxXwsTLkJSMXrTAjB9ew=; b=CKKPEspAgmXT62DhFYvzEwVVoT1S0K4F8g1Ll4+zL2hj2SeP9bGZg/wrmVyhWYB62WJ6mG 19bU1Bvo0z3d17+NS7COvmF2nv+LWvpR4erex8R3HI7BdOEaEnfzHgwzr+QitIzIw6JTqT cFd4ACSN5Ve2Cjdvcji1soer2P1x9D4= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BQgilxEB; spf=pass (imf22.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.216.52 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735580780; a=rsa-sha256; cv=none; b=mZlzqxDH9NkA0VzNs4PycJqwBuAp68XaMztw6C60yBuLzaZJcR7wQ3tBeZMzBIZajfHxZ4 zgGbLrk4NR4G2BUn+k59OuQ12slxsVMgQ7qN5OqkcZfe6M95aRNEYgZCq+gvOaxZ9If6KV OKnzNRZswEWaPsDCoQNBNElFPcVWbss= Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-2ee67e9287fso12466495a91.0 for ; Mon, 30 Dec 2024 09:46:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735580799; x=1736185599; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=ByQEC8FsvBO2uOH/tvc24OCSxXwsTLkJSMXrTAjB9ew=; b=BQgilxEBUs51+Kpw53/YBbEIq2aawPLytFErJozsCKpZdx7cejQzFcj//KOK7FiomS glIdoUhhg9XZ71aE+vjuMuNJeBccuQ2csW1Gw3Z/B1VuUcUe/ZEZ7Izjg4lL4imQkom+ PIA19VMRYxCwOp+GGv3Ot/sh2ZstBtmScj/HgbmfEfpcR/fdFlcarXyO/ukb12vaIVac CQwmr3mizr/U/UM/x3BfgDGtGVemDxI6IohKRxjp4Z/2caXvDDcFkXiK9UhFFGEGfzIC 5SS9Zy8muQIj/dDwKQE/TlGucTODD86Be6W1Y+IhkkEtTMEUN3DIhfbMpF8/cJW0I8N1 RwPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735580799; x=1736185599; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ByQEC8FsvBO2uOH/tvc24OCSxXwsTLkJSMXrTAjB9ew=; b=T3mCWzMyVbNpKRC8XwZy1XEEX2eafzag3C6t+z9yH+BBrSbsPWsZID4VWcKEyce25w 2liW+ku3mnFkrLWQOH6bHjZLsjp6x0UakOGFfkLNZA/TjxCDhnHYDTcPIYszsS2RMTLL RS0hV8+8vwFV6/aI3GTpRLTiAXPlzv+jLeON+Ms4dbrXompj5ioUuZiuO2eoSm5zoM+u I6VpL0YO8Qv3M2paup1RA2H5WG/5+rPCeG1DLGPoFK/AX3X7SRhQu5QRCOQNPTng4ti6 qHKmTYrlfM9Da1R0hOmlHIVSPur3KPtPkdps4WbxLDg79w6i0sGfwn0N1SxQOy5d7vRr lJkA== X-Gm-Message-State: AOJu0YzT8RRp+EZ54pYRjIH8G11C0s1C8HKr0iyiivjaOqfX1VAc+ePW KNpSc/jsfKcapbZQ27eQLQDgjRnO/+WcVgz3yrBKtY/40K0s2xBPJNou8/Qvxch3mw== X-Gm-Gg: ASbGncs8XbAP+IKgHwCW/LQ4CENdzCJ+ceBFVjCjwhFH6alnBLBBAovWHJJcNDvFn+4 YdpB49I6sCa+B34bXrbPFXwp0wYTNxnwUkBgC4pSjZWLKwytvQIu0mLU/7JffrdvIaLvQqd7+EH mIvteYMfesWX1xYaE1AOxAhchfrGg4T8BQFtd1Jt0HL3gmd4xw6d9bI8ie11djncVgY6PsZvaXt uqBPFQQZjdsMkfSNa8yHtP8JYvdJF6KCAQa/Mv3W1AaVL8eoj9CNg9YJTH6CZ35t3h5cb83Q0+8 X-Google-Smtp-Source: AGHT+IEpwr2sGmtXS7EYPdFpiPU2k4iF8zS23kwgmM7X2KfZ2TC2ulrhiLKT3yI/wIFjfHbUR/OxOA== X-Received: by 2002:a17:90a:d448:b0:2ee:5edc:489 with SMTP id 98e67ed59e1d1-2f452ee6dddmr47363487a91.26.1735580798893; Mon, 30 Dec 2024 09:46:38 -0800 (PST) Received: from KASONG-MC4.tencent.com ([1.203.117.231]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dc9cdf25sm180687695ad.118.2024.12.30.09.46.35 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 30 Dec 2024 09:46:38 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Chris Li , Barry Song , Ryan Roberts , Hugh Dickins , Yosry Ahmed , "Huang, Ying" , Nhat Pham , Johannes Weiner , Kalesh Singh , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH v3 01/13] mm, swap: minor clean up for swap entry allocation Date: Tue, 31 Dec 2024 01:46:09 +0800 Message-ID: <20241230174621.61185-2-ryncsn@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241230174621.61185-1-ryncsn@gmail.com> References: <20241230174621.61185-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 902A2C0005 X-Stat-Signature: 4xfbw7ae5dnweww4cxm7wkmjz1mbex8q X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1735580752-161925 X-HE-Meta: U2FsdGVkX1+nZM6kw/LN3IHS/K4yy8KHlgXhfwIq5z2Weo8IWfYEAoB/9olLSyVurtgCTcdcBxCtIoandKp5xGUnmxgQs5BQ8efVdDxxNk/XEBYBZnfuPY728xVpPvgJ3rrMPo/tXlA2iyrl99/rdwxiVvEIx83Ital8Ij6vlkjP06gHB6WXSIKYVsdvNXgcC/wqWhOU1ta8QdktOD7qjB7AXWuXxSuU5O4xPkjRA+y8mhjMwt7KrQ3cSR3yT9JkHJghZ9cdaBdcjD6wawvYmPUwjrs8ZupzoGPXPUdts+4MGnS5jVSE61lUNRUZSvrccIQCl3QeG+gm7JtXcw3WZRLL2kN/i7FcRY7QKKDshkCYGjtYqYCKinW3sv1ZldHhorq+PFLkOyCy/9oYD31r1LQWaJOrdYfwON5St5ORMJPR4dtrsI28cTFwlwXRA/j9vfPjU+ZkpVclSYi9HxsEmrhobFKKl+c8mwtLo6y83OsjntZKUqT6TZ58y6gnfdKj2+1ursvxkieqXphuCAt8Sq5xPzicaehJvZdsBWNJ75FoPNELZZ80lQ29yNGFbZ8wfXf5WATVV+peS1p+310f9Yjiq1vTmyhhQUcvvx20R1bCgBHZ9noIJY22fa2vDI22MGFE2zxWM4ejI23mSlE7K09nG4pnTguPKGW71PRMUGN4Cx70CYnZDnjB+DbpNBN1zo1jFjDhN1c+JBNBEOQGmPaKIHSzFu+9RRYRWJKiq6IcGAch5Uju1CuFEBMwOc9TzbF5dz13TKiZoafOSCrawD0Ewwt5Sxx8TGmsRef2XgxBADas+R16s4BphDkOFQH+wiKuXRSGGDxHAte7Zy/AxcVRRVH7LSuMlbAU0t5bEcWk/v370D5SHOW0auz4KItzwVn4ZAHIkBeR/33NOcbWHp1MD3sXU0KLKcrZU8QOO+9/vScmMAb0shB6mWf4064Yy5RnEPafDfBhfR55OsS BReEXtQ2 nsPY+jTANs9EGsIl15cnKCIXHFfFXaX3p8OPOOxyMzlqcxSV3iEhq2ZCpEyfPtuDkDosOcMk1VelU8G60wWY/xg122X1OgRzmunIuMXLE8BMIFxLahQkGkdEVFO2jWbziD78OBkftI+csk/TlcmLIT+93ig/V/ah6be0qjupE75VRIAEu4TlkweFfF+lNK8NV1130Ej3Kn1UOnuWIaK/ym5O+4GpnMtZ7tbPqjpGz8fSgtPvpTBDFlVbJ7PhbWkzDFhxaqFtKmKzLLDhFiU/q4t4995+csG96gcz+9QYCrzXYhKU1zkcAsvuKfu1rLdUh8QSxDXAeMotbOe2RlPecx8Hv+4o/t6H4+kPFLRHjOP9UsRSF3lM6ixjlISb9cPDwu4+NwxRwnxKexDRSRxqQCqpjJCgYSTJLo+rERESvl/maIseQyGhi5FxU7aonKABYrmNyjjL9bqzrKbz2QOShJXfiTYT2bImz7OkHAfpnEu7QERs/ZruAj9bkhwL++NxwgD2u6iW84W0LyybsM6gymOm2XM5Avw2T7vo7G6ZK8OruEWBCkv5nq/3wew== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Direct reclaim can skip the whole folio after reclaimed a set of folio based slots. Also simplify the code for allocation, reduce indention. Signed-off-by: Kairui Song --- mm/swapfile.c | 59 +++++++++++++++++++++++++-------------------------- 1 file changed, 29 insertions(+), 30 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index b0a9071cfe1d..f8002f110104 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -604,23 +604,28 @@ static bool cluster_reclaim_range(struct swap_info_struct *si, unsigned long start, unsigned long end) { unsigned char *map = si->swap_map; - unsigned long offset; + unsigned long offset = start; + int nr_reclaim; spin_unlock(&ci->lock); spin_unlock(&si->lock); - for (offset = start; offset < end; offset++) { + do { switch (READ_ONCE(map[offset])) { case 0: - continue; + offset++; + break; case SWAP_HAS_CACHE: - if (__try_to_reclaim_swap(si, offset, TTRS_ANYWAY | TTRS_DIRECT) > 0) - continue; - goto out; + nr_reclaim = __try_to_reclaim_swap(si, offset, TTRS_ANYWAY | TTRS_DIRECT); + if (nr_reclaim > 0) + offset += nr_reclaim; + else + goto out; + break; default: goto out; } - } + } while (offset < end); out: spin_lock(&si->lock); spin_lock(&ci->lock); @@ -838,35 +843,30 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si, int o &found, order, usage); frags++; if (found) - break; + goto done; } - if (!found) { + /* + * Nonfull clusters are moved to frag tail if we reached + * here, count them too, don't over scan the frag list. + */ + while (frags < si->frag_cluster_nr[order]) { + ci = list_first_entry(&si->frag_clusters[order], + struct swap_cluster_info, list); /* - * Nonfull clusters are moved to frag tail if we reached - * here, count them too, don't over scan the frag list. + * Rotate the frag list to iterate, they were all failing + * high order allocation or moved here due to per-CPU usage, + * this help keeping usable cluster ahead. */ - while (frags < si->frag_cluster_nr[order]) { - ci = list_first_entry(&si->frag_clusters[order], - struct swap_cluster_info, list); - /* - * Rotate the frag list to iterate, they were all failing - * high order allocation or moved here due to per-CPU usage, - * this help keeping usable cluster ahead. - */ - list_move_tail(&ci->list, &si->frag_clusters[order]); - offset = alloc_swap_scan_cluster(si, cluster_offset(si, ci), - &found, order, usage); - frags++; - if (found) - break; - } + list_move_tail(&ci->list, &si->frag_clusters[order]); + offset = alloc_swap_scan_cluster(si, cluster_offset(si, ci), + &found, order, usage); + frags++; + if (found) + goto done; } } - if (found) - goto done; - if (!list_empty(&si->discard_clusters)) { /* * we don't have free cluster but have some clusters in @@ -904,7 +904,6 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si, int o goto done; } } - done: cluster->next[order] = offset; return found; -- 2.47.1