From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 825F8C4332F for ; Thu, 10 Nov 2022 03:31:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1F6356B0072; Wed, 9 Nov 2022 22:31:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A6398E0001; Wed, 9 Nov 2022 22:31:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 06ED16B0075; Wed, 9 Nov 2022 22:31:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id EB6146B0072 for ; Wed, 9 Nov 2022 22:31:40 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id CA60A121323 for ; Thu, 10 Nov 2022 03:31:40 +0000 (UTC) X-FDA: 80116107960.14.82336C4 Received: from mail-oa1-f50.google.com (mail-oa1-f50.google.com [209.85.160.50]) by imf12.hostedemail.com (Postfix) with ESMTP id 7E8A940008 for ; Thu, 10 Nov 2022 03:31:40 +0000 (UTC) Received: by mail-oa1-f50.google.com with SMTP id 586e51a60fabf-12c8312131fso963197fac.4 for ; Wed, 09 Nov 2022 19:31:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=BJ/hy/T4wgKRhRUKsrzV8PS7uFOQpd7yYx7DQKU65gA=; b=oCel3QyLVsCQ/D5zATgx2McKI20XasvwBWa6tKvQi/0dJfSOYgbyjmiv99EjUbElwE vj+zBgvX0k6GRumlTEYnlsBt4MjY4d8AVsmZOVpE1EbDkLIAfDxVZsY3PxV/616lEyMO +6oaTDoGrUlzXXwem6FIKo4ccz6NYT6v3acyPfELOxMBAmcdP35S8aGCe9XIb6Z9mwnz C272VQYgutXqU48DPBJb4hkBv3VaHTX/5OSqPusziJ/x3uBWduOcfA+Mt0k7RWVF3Iji 5yn5d0/18w8UNjGyyrQFFzYcfGMx8sgWSMzOQlxS500ZXRSqNH9RZ+HDdW0Qj39mp9T9 eTWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=BJ/hy/T4wgKRhRUKsrzV8PS7uFOQpd7yYx7DQKU65gA=; b=p8jyVLEetOVcBRadmIK2Egx3hVPW8cKhmqdhVnJ98ieSTiluUYmoye8+c9WrDu4Y6E Itl/1/FYTosXEY0USFD9/mkzaw0puhE5StLdkgujq+2XlroqaO+J30N8h2K9t7p/24kM RKrOhY6hkODZCx14rhTu/S9BsqcHwkh1svqN+cAmHIaU3fbAKVNEdu7rxwORkR+3n6iR LaypsSbRA34y4DXa6YJ+VM6tEbSesnSDgAE3hmW12X21ummpx7wlAHOoSt1ifdR7197d BB90tDQfzUn00N7yF/GO0SpXBL9zW3bTbXYvAGLwZM4BsrKHN7nF09bUW13tsRrO0CPW OKTA== X-Gm-Message-State: ACrzQf1NKdYuXxYpEO0dxskLtbVakHzjTCjyf+V4rNDZpM+lUCj0vZO/ 3hUQyc/GXlbxTtfBMsKVBeWTfw== X-Google-Smtp-Source: AMsMyM5k5A7yBnhc7JIYU/ERsGQsnivx+IQ2OUPGw7FNEKphqf0Zn7M+mz1ohVFYJerHPKRFdj1Fvg== X-Received: by 2002:a05:6871:9a:b0:13c:c941:645b with SMTP id u26-20020a056871009a00b0013cc941645bmr1091394oaa.95.1668051099606; Wed, 09 Nov 2022 19:31:39 -0800 (PST) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id cv17-20020a056870c69100b00127d2005ea1sm6976643oab.18.2022.11.09.19.31.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Nov 2022 19:31:39 -0800 (PST) Date: Wed, 9 Nov 2022 19:31:37 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@ripple.attlocal.net To: "Kirill A. Shutemov" cc: Hugh Dickins , Andrew Morton , Linus Torvalds , Johannes Weiner , Matthew Wilcox , David Hildenbrand , Vlastimil Babka , Peter Xu , Yang Shi , John Hubbard , Mike Kravetz , Sidhartha Kumar , Muchun Song , Miaohe Lin , Naoya Horiguchi , Mina Almasry , James Houghton , Zach O'Keefe , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH 3/3] mm,thp,rmap: lock_compound_mapcounts() on THP mapcounts In-Reply-To: <20221105200646.wmfilka6prusrb56@box.shutemov.name> Message-ID: <806c097-4613-de13-a5c-5bd5ab318cc9@google.com> References: <5f52de70-975-e94f-f141-543765736181@google.com> <1b42bd1a-8223-e827-602f-d466c2db7d3c@google.com> <20221105200646.wmfilka6prusrb56@box.shutemov.name> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1668051100; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BJ/hy/T4wgKRhRUKsrzV8PS7uFOQpd7yYx7DQKU65gA=; b=8aXGQQ+i3rqWhIO1OCS25fWjqsx3+ZXc+2b4CoKkKUVAoLFpkl591sp6f23NQOlKOQ/Lvx zeNpj/saxzZtBYiVDY+sif1x6ExNwIih3b1/N17sthqrqzNorJ1AnhDIvKebPTKGRXb01X gbiTZEzX2jKk3Hi3Fd8rMdwfKNHfwFw= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=oCel3QyL; spf=pass (imf12.hostedemail.com: domain of hughd@google.com designates 209.85.160.50 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1668051100; a=rsa-sha256; cv=none; b=hQh8fb0iZfRBWmJEhxug7QVQ5ypC36GawdF3MJvllLvnKzI6QBw0vvR1Kb38UakuJXopxl OqZKP5NptxGMGU4tEVT8HGeqg4kfh+pSWJiRcQSqRQ8q/1bwHh4PUB+Zu1qZ2ApdcJJIt7 Z4VkKQcy1QlCLRiu78+BP8A87VFDwLg= X-Rspam-User: X-Stat-Signature: 4dgqrmh3uomf1nebmpt7ns9z3i7x6faj X-Rspamd-Queue-Id: 7E8A940008 Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=oCel3QyL; spf=pass (imf12.hostedemail.com: domain of hughd@google.com designates 209.85.160.50 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam02 X-HE-Tag: 1668051100-856205 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sat, 5 Nov 2022, Kirill A. Shutemov wrote: > On Wed, Nov 02, 2022 at 06:53:45PM -0700, Hugh Dickins wrote: > > Fix the races in maintaining compound_mapcount, subpages_mapcount and > > subpage _mapcount by using PG_locked in the first tail of any compound > > page for a bit_spin_lock() on such modifications; skipping the usual > > atomic operations on those fields in this case. > > > > Bring page_remove_file_rmap() and page_remove_anon_compound_rmap() > > back into page_remove_rmap() itself. Rearrange page_add_anon_rmap() > > and page_add_file_rmap() and page_remove_rmap() to follow the same > > "if (compound) {lock} else if (PageCompound) {lock} else {atomic}" > > pattern (with a PageTransHuge in the compound test, like before, to > > avoid BUG_ONs and optimize away that block when THP is not configured). > > Move all the stats updates outside, after the bit_spin_locked section, > > so that it is sure to be a leaf lock. > > > > Add page_dup_compound_rmap() to manage compound locking versus atomics > > in sync with the rest. In particular, hugetlb pages are still using > > the atomics: to avoid unnecessary interference there, and because they > > never have subpage mappings; but this exception can easily be changed. > > Conveniently, page_dup_compound_rmap() turns out to suit an anon THP's > > __split_huge_pmd_locked() too. > > > > bit_spin_lock() is not popular with PREEMPT_RT folks: but PREEMPT_RT > > sensibly excludes TRANSPARENT_HUGEPAGE already, so its only exposure > > is to the non-hugetlb non-THP pte-mapped compound pages (with large > > folios being currently dependent on TRANSPARENT_HUGEPAGE). There is > > never any scan of subpages in this case; but we have chosen to use > > PageCompound tests rather than PageTransCompound tests to gate the > > use of lock_compound_mapcounts(), so that page_mapped() is correct on > > all compound pages, whether or not TRANSPARENT_HUGEPAGE is enabled: > > could that be a problem for PREEMPT_RT, when there is contention on > > the lock - under heavy concurrent forking for example? If so, then it > > can be turned into a sleeping lock (like folio_lock()) when PREEMPT_RT. > > > > A simple 100 X munmap(mmap(2GB, MAP_SHARED|MAP_POPULATE, tmpfs), 2GB) > > took 18 seconds on small pages, and used to take 1 second on huge pages, > > but now takes 115 milliseconds on huge pages. Mapping by pmds a second > > time used to take 860ms and now takes 86ms; mapping by pmds after mapping > > by ptes (when the scan is needed) used to take 870ms and now takes 495ms. > > Mapping huge pages by ptes is largely unaffected but variable: between 5% > > faster and 5% slower in what I've recorded. Contention on the lock is > > likely to behave worse than contention on the atomics behaved. > > > > Signed-off-by: Hugh Dickins > > Acked-by: Kirill A. Shutemov Thanks, Kirill; and there's a 4/3 posted to change around that "if (compound) {lock} else if (PageCompound) {lock} else {atomic}" ordering, which Linus hated. But this might be a good place to mention, that Linus (I'd sent private mail to sort out mm-unstable instabilities in a hurry, and discussion ensued from there) does not like this patch very much, and has a good idea for improving it, but has let us move forward with this for now. His idea is for subpages_mapcount not to count all the ptes of subpages, but to count all the subpages which have ptes (or I think that's one way of saying it, but not how he said it): count what the stats need counted. I was sceptical at first, because that was indeed something I had tried at one point, but decided against. I am hoping that it will turn out just to be my prejudice: that I embarked on this job, in large part, to get rid of the scan lurking inside total_mapcount(). And Linus's idea would appear to bring back the unlocked scan in total_mapcount(): but remove all the locked scans in page_add/remove_rmap() - which, setting aside my prejudice, sounds like a big improvement (in the double-mapped case; common cases unchanged). I was not enthusiastic, in that discussion several days ago, but got quite excited once I had a moment to consider (but I've not told him so until now). I'll try to pursue it this weekend: maybe I'll rediscover a good reason why it had to be abandoned, but let's hope it works out. Anyway, what's in mm-unstable is good, and an improvement over the old scans; but I appreciate Linus's frustration that it could be much better. Hugh