From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC592C27C75 for ; Wed, 12 Jun 2024 05:46:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 31A146B0148; Wed, 12 Jun 2024 01:46:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2CA4F6B0149; Wed, 12 Jun 2024 01:46:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 191BF6B014A; Wed, 12 Jun 2024 01:46:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id F0CB76B0148 for ; Wed, 12 Jun 2024 01:46:52 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 5D28640213 for ; Wed, 12 Jun 2024 05:46:52 +0000 (UTC) X-FDA: 82221152664.03.393D4FC Received: from mail-yw1-f178.google.com (mail-yw1-f178.google.com [209.85.128.178]) by imf28.hostedemail.com (Postfix) with ESMTP id 96915C001C for ; Wed, 12 Jun 2024 05:46:50 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Czh5J4D+; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf28.hostedemail.com: domain of hughd@google.com designates 209.85.128.178 as permitted sender) smtp.mailfrom=hughd@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718171210; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=moJnHsIsot9+U+qmE6uaTW3P8PCz1p59VpzC+G7DvZY=; b=d5pfhsfmozg0GVA0vG+UOvMR9cY8Q2JirzeDa1+4/frxMf2sW6hbBfpdAOfA8xYL1NMrvJ jI0j2Bqz+7mE8U7e8m4+4p4qDGILc3caHkxNU0sZ48wGpu1tVZFDcrBep3el/tNvUQYoR7 8QtQjYYJaWU4XgbYJKzN7+EijJu/3JM= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Czh5J4D+; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf28.hostedemail.com: domain of hughd@google.com designates 209.85.128.178 as permitted sender) smtp.mailfrom=hughd@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718171210; a=rsa-sha256; cv=none; b=hQ0UOfQr1RzUh+yJslu4jMi9Cl6u/lWo1gVqRQtRNVD09ipKZJxNyp9HMXmfR0wNo/5frw 2cwEL5/CtUoEt2OhgKt3gn8i1GH8lWOKSgR8OyK4MQ26sMEwryoKHz/RVYWpPdCWJ4tZcM 0hE8Rfzi0P8jAA1fcix95kMVYDxmRWU= Received: by mail-yw1-f178.google.com with SMTP id 00721157ae682-62a2424ecb8so64993227b3.1 for ; Tue, 11 Jun 2024 22:46:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1718171209; x=1718776009; darn=kvack.org; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=moJnHsIsot9+U+qmE6uaTW3P8PCz1p59VpzC+G7DvZY=; b=Czh5J4D+Z51UdsLjfnkGeV/ytlj/yNcDcOepDVXE0eiQ+ZaqIwqtZ0wQ1tTUfMTbEP +AU96+Z9EJPrRZwhYlrNn/R/kfMDIKPmMvhLXutgXBAwgYfAmc+jRv1XMHQm75qhLrWM sayNuF6bcL+NCUYPvWv6zj+NIc4I3iww3OlSijuzdTCOgWyl8nmwF2Gy0/A2GgLMtmom rsi4W5VzCqoqVZ0YtO02OwhyC0KeWYI4lFz6kBrr+nuuk6dtcGHfZWzwo+ebTic11XVl KsIxWB8S4OWNJ3/r82J+57TtYFYpcTFMlcR3EKigsiegva9w0x6fos0P9SnJgqYtV84E mYNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718171209; x=1718776009; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=moJnHsIsot9+U+qmE6uaTW3P8PCz1p59VpzC+G7DvZY=; b=lIp2G944MMDK6Dk3oVX1s+h81yH60anS9miW0A3D1xnuZjXMo+FishIaGW5TPYS06Q WxZd9n+lKLT+p4JU4GCq4IyRzQCeBIkzMYIIEV3ItAoOOotKofwwWbpxWCeJaS6qwhQb xJ1lwYPEp+FN0Lhp8/7O/8vp2Z7qHvEwZx+S7SlenHciQjGTyzLO/4BR+tXfOKOl9mMq 3WJgKrh0rbuQGQz8W+8xbjunr41ZcMS74qO22LA86gFkpAkcmCCX4/mbFR8F+YgVjNFZ wf1B1/c0XDO0E51+MbSUxwOMvlMULPlT61LUPMcaS0uiGVKW57C/thWw88VczUli7ph7 THTQ== X-Forwarded-Encrypted: i=1; AJvYcCWOWG2rIHVVrhOmedP7yCn7F1kjiFcgHAEWKHBVtpuLv8BdiRcvXAnpwpuwVUHdgifRPbfacK0S8zx6aU4jglmfSes= X-Gm-Message-State: AOJu0YzCiKYzcLIaDmBoxM+QXp/MGHn90yoKVVBovVQZcguKFFnd22KI p3LZim5orz09JCMVFyV0HUpJ4hheam8rMVBSW5cdHGwFWP7p5VpqTvcHiwXPLQ== X-Google-Smtp-Source: AGHT+IGKs+qtmC+lPU0WP+6G10ji1oSZEVaWsH4gOFl+QYJT8h+ibgE3eVIvlQmm7E1leKVvM6n+ug== X-Received: by 2002:a0d:e6d5:0:b0:60a:66c0:d5fe with SMTP id 00721157ae682-62fba9435edmr8488907b3.13.1718171209034; Tue, 11 Jun 2024 22:46:49 -0700 (PDT) Received: from darker.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id 00721157ae682-62ccad1b679sm22806697b3.66.2024.06.11.22.46.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jun 2024 22:46:48 -0700 (PDT) Date: Tue, 11 Jun 2024 22:46:34 -0700 (PDT) From: Hugh Dickins To: Baolin Wang cc: akpm@linux-foundation.org, hughd@google.com, willy@infradead.org, david@redhat.com, wangkefeng.wang@huawei.com, chrisl@kernel.org, ying.huang@intel.com, 21cnbao@gmail.com, ryan.roberts@arm.com, shy828301@gmail.com, ziy@nvidia.com, ioworker0@gmail.com, da.gomez@samsung.com, p.raghav@samsung.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 0/7] support large folio swap-out and swap-in for shmem In-Reply-To: Message-ID: <7477de0e-e5bb-529d-e2c0-cabd0d39fb5c@google.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 96915C001C X-Stat-Signature: rbxyyzq7zgzhdttoxc5qhtz64j5kndfq X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1718171210-469569 X-HE-Meta: U2FsdGVkX1+svdRD6w8eQZVPsh7qv5Bu9Fs6sQ3erttcFoT8O3WvdDLohuLxy4Qbw2cc4WngNuoeEugLNyQjz1/0JlxHvIo5LpXIb30M4QONY7Kj6DlEmjlkhwVouaI8nyDyoPWMcPk3pLjzBkSn1HkWUeg7yA8qt2ljaNnMrqE4Lf3tdRPMBA3RiWd1UM/WYkhVAaysyGESECHEC5rv+2XevCvoYCv1pFiiJ2VtI7sJ5f8AAnEG4B4upRM8jas0lNhDuhpNYrGqQ656GZwQ8iD9WIZ6RW9UUQ3vZ/PHPAY3OVg4TqCrC2oGdjvprkEDBqqzTalM4W2Ba6oRCFxhdXewSKL5h0d7VaEcTSmTVwqi/k07fSkR/oOltaSXfuu0JHbMV/VH6bGNeIJw1ZCsHr2E3Lce9Qc2sbH4khwzQJDiVspDl7lT0wLtQOaOOEVODWXRvkqd75cgvK0uOAwpTy7ar3eKlJ5kWhHGKQfDho18b4UuHwtocZAk++tFV/rc2g7huEdn9afJ0y5oEyRWhqWzWUz4a3xC3prkn6aJkEb0hAbmDZvs5OdYlNQDF4kSzR8HgftKp1bqtc3orvygerXkPlc7L8Hm8bIfvUCHBEW+lZNF79mEcYu/jfq2qveqxcGjRQW6+J0kR3frXHbTTqKbRrsQXg6+SE1meXZN9zZ+7+NGJz/pqjdg1wTwVOUs5GfBGgCjh3N00/0qWyrmtbhu7Ye5TPjdgJDVCRR+W0HJXH1tn4YpfG4FIMkcpNhzxA/2gEJ0yCwCCuW/iCUVb9GfzehhtCbKP5xWRfgDcIJSxwxFh7UlbclzpsIlHzE0A+mnSNlRX3Ll2wB7+6iZkyiKNPXvsWYMIb8GveUtXpn/HmNMEes/Eev0+AyPhYr+gZ49h803eFJomJjv5rEQzBPet4VHHfWAa8wjtgs4FP3Xq9eWB1DSBsrEvkAis1PfcLRTA+l2aExLzm5PZ2S VsUqDTfD CkWnxGcu6BpOEp7MSl6+234BtaXVLNhK9XINQP6K5oK3y93diN58a+RNzs73bck0lF1vysYoQnWT7is6bjIdFoncIlBhHULbjpTHFONjXUH65cYTrgnomtCgvfFDAzvjaZhn60qDos+QbmcMC47LGdmaih91EDkPewccfup5i7KlLBINkzwrqW5FDtisTGbeQ+9Zs4YIvRmCDUCBCaHsw041v5MXnbyxlcBuIqCZyEeDt0TOs36LkQz4nBW/FpN9MuwGwpf1OuAI45L9lEcWJRlSSJ/yNoKCO2wH3Ue9eKRl0oApolQNKMkt9jmyqyEwkE8BZAdnFgvlsVVk08q24QttiYzIqzKneLsoU0DO4oftmBP97TeOsiYO/bD3m/pcLeyxQU1RI/P06InC4DbuEKD6mTbJuYCrXF1Hyb9FcRszUd3EwmRfsdkH3f3pPxGzmTDd2z6sEoVRKpM7rptFJxHSwh3dH+5W9uBMZ+8VEc48xZgczwJnkPvpJbw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, 6 Jun 2024, Baolin Wang wrote: > Shmem will support large folio allocation [1] [2] to get a better performance, > however, the memory reclaim still splits the precious large folios when trying > to swap-out shmem, which may lead to the memory fragmentation issue and can not > take advantage of the large folio for shmeme. > > Moreover, the swap code already supports for swapping out large folio without > split, and large folio swap-in[3] series is queued into mm-unstable branch. > Hence this patch set also supports the large folio swap-out and swap-in for > shmem. > > [1] https://lore.kernel.org/all/cover.1717495894.git.baolin.wang@linux.alibaba.com/ > [2] https://lore.kernel.org/all/20240515055719.32577-1-da.gomez@samsung.com/ > [3] https://lore.kernel.org/all/20240508224040.190469-6-21cnbao@gmail.com/T/ > > Changes from RFC: > - Rebased to the latest mm-unstable. > - Drop the counter name fixing patch, which was queued into mm-hotfixes-stable > branch. > > Baolin Wang (7): > mm: vmscan: add validation before spliting shmem large folio > mm: swap: extend swap_shmem_alloc() to support batch SWAP_MAP_SHMEM > flag setting > mm: shmem: support large folio allocation for shmem_replace_folio() > mm: shmem: extend shmem_partial_swap_usage() to support large folio > swap > mm: add new 'orders' parameter for find_get_entries() and > find_lock_entries() > mm: shmem: use swap_free_nr() to free shmem swap entries > mm: shmem: support large folio swap out > > drivers/gpu/drm/i915/gem/i915_gem_shmem.c | 1 + > include/linux/swap.h | 4 +- > include/linux/writeback.h | 1 + > mm/filemap.c | 27 ++++++- > mm/internal.h | 4 +- > mm/shmem.c | 58 ++++++++------ > mm/swapfile.c | 98 ++++++++++++----------- > mm/truncate.c | 8 +- > mm/vmscan.c | 22 ++++- > 9 files changed, 140 insertions(+), 83 deletions(-) I wanted to have some tests running, while looking through these and your shmem mTHP patches; but I wasted too much time on that by applying these on top and hitting crash, OOMs and dreadful thrashing - testing did not get very far at all. Perhaps all easily fixed, but I don't have more time to spend on it, and think this series cannot expect to go into 6.11: I'll have another try with it next cycle. I really must turn my attention to your shmem mTHP series: no doubt I'll have minor adjustments to ask there - but several other people are also waiting for me to respond (or given up on me completely). The little crash fix needed in this series appears to be: --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2053,7 +2053,8 @@ static int shmem_swapin_folio(struct ino goto failed; } - error = shmem_add_to_page_cache(folio, mapping, index, + error = shmem_add_to_page_cache(folio, mapping, + round_down(index, nr_pages), swp_to_radix_entry(swap), gfp); if (error) goto failed; Then the OOMs and dreadful thrashing are due to refcount confusion: I did not even glance at these patches to work out what's wanted, but a printk in __remove_mapping() showed that folio->_refcount was 1024 where 513 was expected, so reclaim was freeing none of them. Hugh