From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C29DC4345F for ; Thu, 2 May 2024 23:05:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1E3D36B0092; Thu, 2 May 2024 19:05:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 16D6E6B0093; Thu, 2 May 2024 19:05:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F289E6B0095; Thu, 2 May 2024 19:05:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id C9DF46B0092 for ; Thu, 2 May 2024 19:05:28 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 749C5C0971 for ; Thu, 2 May 2024 23:05:28 +0000 (UTC) X-FDA: 82074989136.27.EB8669F Received: from mail-yw1-f169.google.com (mail-yw1-f169.google.com [209.85.128.169]) by imf29.hostedemail.com (Postfix) with ESMTP id A4944120024 for ; Thu, 2 May 2024 23:05:26 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=F48YtowQ; spf=pass (imf29.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.128.169 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714691126; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ezx4wmMWJiSJtFNK/efG3+qdlPgKJJwodjHrb/ERaiU=; b=3+Y/U8QEyFtVnvJBhEOrKzq2GDS8QDr/H3Hd2cHdN90UY0ls7bRSvpOn5DZO8PozNpLziS FUsd+XCQilic8sDItYdeAzGku+R/xO8yWb+Kg8zZOSdwTOd+BNT8bE+m7dYMu1dF8fBHH0 T4glSXAHT6umF+iTRtylNucCgwL8IPQ= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=F48YtowQ; spf=pass (imf29.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.128.169 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714691126; a=rsa-sha256; cv=none; b=pef8u6yrjHt86bxKhiN2T8DziKdDPyAHzbVgMKaum3BRVOQ6f9lxwGTBKw0QKQjpe+uGyq cyF+80yYZIVnWGrn80DJF0vFD1b/11hVq0XZhGNp2fBSLGksrDh5rksxPEzVwvaFUw8WKQ Ln+T7rayCKTuMGXQoux0AdgW5Xa27XU= Received: by mail-yw1-f169.google.com with SMTP id 00721157ae682-61d35d266e7so34937407b3.1 for ; Thu, 02 May 2024 16:05:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714691126; x=1715295926; darn=kvack.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=ezx4wmMWJiSJtFNK/efG3+qdlPgKJJwodjHrb/ERaiU=; b=F48YtowQMpcPYoagC8nJBcz0Ge6HaaB7M9yOhrtdfcuy+wC2lasaxE3oOw3bRvNKNA yZbfkUtvcvdJG/uehCBzkdeit2KScgZ99+bXFlLV1OWyC0YARPdp3kIebg9A97tEs3KU 5qAThNl7gpDA22ffBmYzfsv2ziXkV/xnDzwrcxQAkSKD9W9gd2yzgbMnH1Jb/TuVV1fT 14NJBBhlglLsQs4LaIJI2aSs4yWZXi2D4XDR4Pj+kXIuTMLYLvBZKhUvzOb67zRc2Xg4 8gT+ASiyR9VT90PnWxARQyzBMSIFFbPtPFyzyHu2dvkDLwnPnjIzAi78bqjnNy3MFjqL nsBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714691126; x=1715295926; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ezx4wmMWJiSJtFNK/efG3+qdlPgKJJwodjHrb/ERaiU=; b=JzAbd9deFoZAmUVO9JaZzi4pTwamdTKHlaCEXckdvgGMi9oqy6nHlqGVQqQyjvt2xf bqFsGNoc8obgCm5WoT/dCvpSLbvBSZPqVdD9MBEWPbnRJEBesnT0drtCUzxoSJXWi2Z9 R618D3LLVaNOVnC44OE3MBuzKzIK9k8Hbqas35yhE0Z6mMgN0j9FDP2pfkK5WELh0G7b bGEr8gktRM6AHgq4JworN4P7SZbXOo4G24M9Cs4FeX3hn7a3JDXpdqiD1CghyQLz9jA2 3AtRKinxL9qBU2vJIjtrMBPUHJX1A4aQvIthZQow1E/kzzGMJ4IjWmKng88dH8ELgsb5 haJg== X-Forwarded-Encrypted: i=1; AJvYcCUYLHW6VlGH0Ng3cf5KOAfJEW+8Ud7ACS1gMzx2VFKVFXwUsv2AS202IzFyDFvh+lOEWDERX8yWv8EK9KZJcB420PM= X-Gm-Message-State: AOJu0YyJWi2wtBA6BhkMiPw4gTSe8cYCB9DHrBjoZR3IT9sUSMt20YOW wC4nHgoQLV8JZS8aamD8kXcVDG4J0ANUaD0vllf6K4JEsDj/UbKCNfOr4/6GEjOgw6UCQeVFo/3 0oRP1ecFNUy6i2YzfahCp2kQhNTw= X-Google-Smtp-Source: AGHT+IHjZWBVSec7G2zPAVUAhhvRdzBCCiWDLXHQKLDQEXWgwQ/huHlw6Hm5HhVgKDCb7tBv2P6F90dUezNKzNHq2qU= X-Received: by 2002:a81:ff0d:0:b0:60a:17a2:5627 with SMTP id k13-20020a81ff0d000000b0060a17a25627mr847104ywn.50.1714691125648; Thu, 02 May 2024 16:05:25 -0700 (PDT) MIME-Version: 1.0 References: <20240409082631.187483-1-21cnbao@gmail.com> <20240409082631.187483-2-21cnbao@gmail.com> <87y19f2lq3.fsf@yhuang6-desk2.ccr.corp.intel.com> <87jzkz2g3t.fsf@yhuang6-desk2.ccr.corp.intel.com> <87bk6b2elo.fsf@yhuang6-desk2.ccr.corp.intel.com> <877cgy2ifu.fsf@yhuang6-desk2.ccr.corp.intel.com> <87ttk20zns.fsf@yhuang6-desk2.ccr.corp.intel.com> <87jzkw25hl.fsf@yhuang6-desk2.ccr.corp.intel.com> <87le5bxd6n.fsf@yhuang6-desk2.ccr.corp.intel.com> In-Reply-To: From: Barry Song <21cnbao@gmail.com> Date: Fri, 3 May 2024 11:05:13 +1200 Message-ID: Subject: Re: [PATCH v2 1/5] mm: swap: introduce swap_free_nr() for batched swap_free() To: "Huang, Ying" Cc: akpm@linux-foundation.org, linux-mm@kvack.org, baolin.wang@linux.alibaba.com, chrisl@kernel.org, david@redhat.com, hanchuanhua@oppo.com, hannes@cmpxchg.org, hughd@google.com, kasong@tencent.com, ryan.roberts@arm.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, xiang@kernel.org, yosryahmed@google.com, yuzhao@google.com, ziy@nvidia.com, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: nuy9mw7kkdghkz5ghyoe796inq1u7juk X-Rspamd-Queue-Id: A4944120024 X-Rspamd-Server: rspam10 X-Rspam-User: X-HE-Tag: 1714691126-546625 X-HE-Meta: U2FsdGVkX18CP9h2/PmQtdWqON/5w0XWutCVDn1rK92xKjI1+fFN+JW0zAiZgR4evk3wrTtSmhdXKVAMNfuBLMFhvhVmAtGWGGqLP6vOO5G+Rr3TgZa+KKA4DZ2cPGBPknLy756eX14o3S4nUeiPstdSrVu5wToikJT8cekZkXz8I9rn4htmaSQ1hbr6bI2Xci9V5Uk0T4BTnyUl+nYs1OX162nlesQqii896zeDfVf/7z6uppGIQ78mle34nJ2ZalkW0IlAdBxTqbZvgvaw0MsVomZYJ0H0IcMV99sCVotMhJivt2GUW/s9VSP7etWSG+MgzxuWX3JiPOL/k0vM7DjsRID/xHPp+9tz/hXSaXzcfyUJxnnNDlbc200AZJX/NgtDGiUnmV1hRWIBR3qUozj+LaBsG92CPYXdclOvipE18lYu0xFCoQ+pFDyIpq2Nch8X1NPSsV5cJacYcpeV3mfrZb/GBVKcE/5h6dgDbEGmnnhD9AoA4Sm0UKOvbYskSMzCFGKkli8mkiT2ueYzRcEo89GUVyoBGY1HG9yGHbGp7Fp1tkAZ7lOzd/jmb2Qy7rs9haHs0Xbi+TLrqBJCuXYZfy+J0SI2TaFQ0c8MHNYj7W2ocY3NO82E+7v+E1oa8cbAzEpXujN6ka5/X7V8sgsiJK2Vr6DETLzV/STxF9J0N3ZafXcDs3wEtJf9F3jz7pjG5ArNNjtyj5t3U/NBQSuSSufLCEsv35ZcENuuUqLZG68otv5b9v+Cvsq4XWs2A/xEU4oriP6L/Bbg0k3KVAKys2r8OMhdLq8Q4LwhCCoVC5uKPdbHvfure6IxH2iA2DBuWMognrp5mmtnhvXu1HuD1cOFRro6i6aanc9k7Zs7Vx1rMDLfcmVfA1KCoL+Mlz2eVSax2L6aEZG9YZzpMOxpKrwvgMjP0ha5P9Xn9ME6aEyVKkGsA9YByb5zHmpdPMXwWkv9R8ffxDGm2fx MHxk8fx/ vAmA5YgR+3k8AgDJXQJjdLf2wXavlqF/101u0CknKfi8TAz9igXb8BwMMeEFU3Iozb6sOWTcnBDMyay8OERoRdgw6rbEb0cIRnfM/RUye/7j6iKtr71ZaGJZ2fHUl4hsrBH+Jou8cPgzNAlHqH2neCS19hv7ytpJD2QJsXHLf6gdc3yHboOJI7jFsL9BZBOSo/lKtjqc/fWZjCaXNupAE7zIwruMM9hl1xqsvaCYJSq3g3GnlZUAR0QWfiX9EkkUznA4vBeFQNvx+Rh8VepuhI0mNpsX7odENG5/954VlqMPdNBJzSEYSSOB/dCCAbA+H+L2YaAhDuLITFB/BWWKR84sAkveVLALRNKHY5x8/qlwduti0Y2ukPex9tqFBGn9mFOFSpezIK1LZ3+2iUaulOg7K5w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: [snip] > > > > My idea is something as below. It's only build tested. > > > > static void cluster_swap_free_nr(struct swap_info_struct *sis, > > unsigned long offset, int nr_pages) > > { > > struct swap_cluster_info *ci; > > DECLARE_BITMAP(to_free, BITS_PER_LONG) = { 0 }; > > int i, nr; > > > > ci = lock_cluster_or_swap_info(sis, offset); > > while (nr_pages) { > > nr = min(BITS_PER_LONG, nr_pages); > > for (i = 0; i < nr; i++) { > > if (!__swap_entry_free_locked(sis, offset + i, 1)) > > bitmap_set(to_free, i, 1); > > } > > if (!bitmap_empty(to_free, BITS_PER_LONG)) { > > unlock_cluster_or_swap_info(sis, ci); > > for_each_set_bit(i, to_free, BITS_PER_LONG) > > free_swap_slot(swp_entry(sis->type, offset + i)); > > if (nr == nr_pages) > > return; > > bitmap_clear(to_free, 0, BITS_PER_LONG); > > ci = lock_cluster_or_swap_info(sis, offset); > > } > > offset += nr; > > nr_pages -= nr; > > } > > unlock_cluster_or_swap_info(sis, ci); > > } > > > > void swap_free_nr(swp_entry_t entry, int nr_pages) > > { > > int nr; > > struct swap_info_struct *sis; > > unsigned long offset = swp_offset(entry); > > > > sis = _swap_info_get(entry); > > if (!sis) > > return; > > > > while (nr_pages >= 0) { this should be "while (nr_pages) " exactly like "cluster_swap_free_nr", otherwise, we get [ 383.652632] EXT4-fs (vda): error count since last fsck: 85 [ 383.653453] EXT4-fs (vda): initial error at time 1709536044: mb_free_blocks:1937: block 704527 [ 383.654947] EXT4-fs (vda): last error at time 1714689579: ext4_mb_generate_buddy:1213 [ 398.564002] watchdog: BUG: soft lockup - CPU#1 stuck for 26s! [a.out:104] [ 398.564679] Modules linked in: [ 398.565239] irq event stamp: 0 [ 398.565648] hardirqs last enabled at (0): [<0000000000000000>] 0x0 [ 398.566888] hardirqs last disabled at (0): [] copy_process+0x654/0x19a8 [ 398.568255] softirqs last enabled at (0): [] copy_process+0x654/0x19a8 [ 398.568813] softirqs last disabled at (0): [<0000000000000000>] 0x0 [ 398.569481] CPU: 1 PID: 104 Comm: a.out Not tainted 6.9.0-rc4-g3c9251435c61-dirty #216 [ 398.570076] Hardware name: linux,dummy-virt (DT) [ 398.570600] pstate: 01401005 (nzcv daif +PAN -UAO -TCO +DIT +SSBS BTYPE=--) [ 398.571148] pc : lock_acquire+0x3c/0x88 [ 398.571636] lr : lock_acquire+0x3c/0x88 [ 398.572075] sp : ffff800086f13af0 [ 398.572484] x29: ffff800086f13af0 x28: 0000000000000000 x27: ffff0000c1096800 [ 398.573349] x26: 0000000000000001 x25: ffff8000803e4d60 x24: 0000000000000000 [ 398.574139] x23: 0000000000000001 x22: 0000000000000000 x21: 0000000000000000 [ 398.574915] x20: 0000000000000000 x19: ffff0000c3eb0918 x18: 0000000000000000 [ 398.576009] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000 [ 398.576895] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000 [ 398.577732] x11: 0000000000000000 x10: 0000000000000000 x9 : ffff8000814cf7a0 [ 398.578668] x8 : ffff800086f13ab8 x7 : 0000000000000000 x6 : ffff8000803e4d60 [ 398.579705] x5 : 0000000000000000 x4 : 0000000000000001 x3 : ffff800082f39008 [ 398.580528] x2 : 0000000000000003 x1 : 0000000000000002 x0 : 0000000000000001 [ 398.581387] Call trace: [ 398.581740] lock_acquire+0x3c/0x88 [ 398.582159] _raw_spin_lock+0x50/0x70 [ 398.582556] swap_free_nr+0x98/0x2a0 [ 398.582946] do_swap_page+0x568/0xd00 [ 398.583337] __handle_mm_fault+0x76c/0x16d0 [ 398.583853] handle_mm_fault+0x7c/0x3c8 [ 398.584276] do_page_fault+0x188/0x698 [ 398.584731] do_translation_fault+0xb4/0xd8 [ 398.585142] do_mem_abort+0x4c/0xa8 [ 398.585541] el0_da+0x58/0x128 [ 398.585923] el0t_64_sync_handler+0xe4/0x158 [ 398.586355] el0t_64_sync+0x1a4/0x1a8 [ 398.651930] watchdog: BUG: soft lockup - CPU#2 stuck for 26s! [a.out:103] [ 398.652819] Modules linked in: [ 398.653682] irq event stamp: 0 [ 398.654213] hardirqs last enabled at (0): [<0000000000000000>] 0x0 [ 398.654714] hardirqs last disabled at (0): [] copy_process+0x654/0x19a8 [ 398.655242] softirqs last enabled at (0): [] copy_process+0x654/0x19a8 [ 398.655835] softirqs last disabled at (0): [<0000000000000000>] 0x0 [ 398.656657] CPU: 2 PID: 103 Comm: a.out Tainted: G L 6.9.0-rc4-g3c9251435c61-dirty #216 [ 398.657705] Hardware name: linux,dummy-virt (DT) [ 398.658273] pstate: 01401005 (nzcv daif +PAN -UAO -TCO +DIT +SSBS BTYPE=--) [ 398.658743] pc : queued_spin_lock_slowpath+0x5c/0x528 [ 398.659146] lr : do_raw_spin_lock+0xc8/0x120 [ 398.659575] sp : ffff800086d03a60 [ 398.659946] x29: ffff800086d03a60 x28: 0000000000000120 x27: 0800000103050003 [ 398.661238] x26: ffff0000c27d5d40 x25: fffffdffc0000000 x24: ffff800082270d08 [ 398.662431] x23: ffff800083f73a31 x22: ffff800086d03d48 x21: 0000ffff9bf70000 [ 398.663358] x20: ffff800082f39008 x19: ffff0000c27d5d40 x18: 0000000000000000 [ 398.664353] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000 [ 398.665825] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000 [ 398.666715] x11: 0000000000000000 x10: 0000000000000000 x9 : ffff8000801420f8 [ 398.667528] x8 : ffff800086d03a48 x7 : 0000000000000000 x6 : ffff8000803abef0 [ 398.668548] x5 : 0000000000000000 x4 : 0000000000000001 x3 : ffff800082f39008 [ 398.670018] x2 : ffff80012ac81000 x1 : 0000000000000000 x0 : 0000000000000001 [ 398.670831] Call trace: [ 398.671136] queued_spin_lock_slowpath+0x5c/0x528 [ 398.671604] do_raw_spin_lock+0xc8/0x120 [ 398.672144] _raw_spin_lock+0x58/0x70 [ 398.672860] __pte_offset_map_lock+0x98/0x210 [ 398.673695] filemap_map_pages+0x10c/0x7f8 [ 398.674495] __handle_mm_fault+0x11e4/0x16d0 [ 398.674926] handle_mm_fault+0x7c/0x3c8 [ 398.675336] do_page_fault+0x100/0x698 [ 398.675863] do_translation_fault+0xb4/0xd8 [ 398.676425] do_mem_abort+0x4c/0xa8 [ 398.677225] el0_ia+0x80/0x188 [ 398.678004] el0t_64_sync_handler+0x100/0x158 [ 398.678778] el0t_64_sync+0x1a4/0x1a8 [ 422.563920] watchdog: BUG: soft lockup - CPU#1 stuck for 48s! [a.out:104] [ 422.564528] Modules linked in: [ 422.564991] irq event stamp: 0 > > nr = min_t(int, nr_pages, SWAPFILE_CLUSTER - offset % SWAPFILE_CLUSTER); > > cluster_swap_free_nr(sis, offset, nr); > > offset += nr; > > nr_pages -= nr; > > } > > } > > -- > > Best Regards, > > Huang, Ying > Thanks Barry