From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB9A6C001DE for ; Fri, 11 Aug 2023 15:18:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6F2D66B0071; Fri, 11 Aug 2023 11:18:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6A2FD6B0072; Fri, 11 Aug 2023 11:18:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 545216B0074; Fri, 11 Aug 2023 11:18:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 4222E6B0071 for ; Fri, 11 Aug 2023 11:18:46 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 0CAFB1611D5 for ; Fri, 11 Aug 2023 15:18:46 +0000 (UTC) X-FDA: 81112181052.13.47B034B Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf21.hostedemail.com (Postfix) with ESMTP id DBEA41C002E for ; Fri, 11 Aug 2023 15:18:43 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hHqdrnqz; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf21.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1691767123; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Jo0mEcRfDjKXk0toT3MA/an0CZ6PSxrp9vF9M+Za+G4=; b=qJGecgWAtGUMMgeLtqoNg8mZO919yzjgKmOB5+29gEEDeMh778pFb4A6nzXQbsXXc+xBFZ lorYZ2csyYgpuM0vMXD/LchFmGRHodc0sdhqK51qCqJiJwOCrNI2PCfzVkdbEu6wOlFIZd HHuNu1GgLP3+Eg0+JMNp2iPArob38/M= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hHqdrnqz; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf21.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1691767123; a=rsa-sha256; cv=none; b=eNIpGseu8ffnts0/+9585OB6ZrvoXEvAF54YnESH16pu+zR82iNTc0YvNanMeqy/aNqP0T mAcT0rbX0RF9Gr5xKgiCidqnkRPoOgAMCEFvkcp2990xTLsSVK1E+rWUDjjICoXHXtTBIY ko6QJ6XDh4MCrrV9FW+jI6gSvSCGZJc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691767123; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Jo0mEcRfDjKXk0toT3MA/an0CZ6PSxrp9vF9M+Za+G4=; b=hHqdrnqzdho8A98xW9chKQmYgRgJW2wwIFRL7VFemhR1/cpOX5WgzJLpeXEPdceCWEVr3T a3K+jLHJ32DLtbFs8N7DQAYaWzJ9FMMKGc3WJGb7KwkbYBsjJyYKctidWtzPd+XbcJySvO zOhYr9/gD4p3GCuVXOOAS0scE+Fcl3s= Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-692-ubSjaIz9Mw-a9571LOO6Tw-1; Fri, 11 Aug 2023 11:18:41 -0400 X-MC-Unique: ubSjaIz9Mw-a9571LOO6Tw-1 Received: by mail-qt1-f200.google.com with SMTP id d75a77b69052e-40fd6d83c21so5492391cf.1 for ; Fri, 11 Aug 2023 08:18:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691767121; x=1692371921; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Jo0mEcRfDjKXk0toT3MA/an0CZ6PSxrp9vF9M+Za+G4=; b=Ozc8MGKVAGxztGW5ZSXV1UdoiVspDeh8sjecZp3N9eOe0y3WdF88+6eSpmPiVFbjfk Rx+VFq3YBpx1PNQGyA5fEEdllpa2xqY460FJKgulP3TnZxgJOn3CCPVSS8WeMTfIocHj aTXt5ZlJa64jw/8QCSuMtA9Ny0noEjwYisYWCXrlsMoAkFOEpbyrMzXz0fBCFf79Jl1x W8D3z8HqZIUdA5w5GyvD29jiSuD23neWzOZicMzlkDIQqaDgcjOam3z1/up/F1xQ1uVk Oog2Y2w6dlW8zVrarcXGUUeS27dgHh6Hs1eAnNqljllGqGpWJwzhLc2xQmGGbmUN83Zd LSsA== X-Gm-Message-State: AOJu0YyUh0KZl/5HVIIP2gu0bImEAu90rdj1BKqvtwIxaA19/pzn/0rG t2DtrxX9EP+VCkk05+5+O1NAe5JgGb0dggpJmr6hPOM1M5KAk8k82+yw9plZqAF2v3mNGwbE+sw se0/cgnh5nJA= X-Received: by 2002:a05:622a:1a06:b0:40f:dcda:ea10 with SMTP id f6-20020a05622a1a0600b0040fdcdaea10mr2733835qtb.3.1691767121017; Fri, 11 Aug 2023 08:18:41 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFu3sq9QlwOyun+Cer+VeBozeAoyni7P5APQeEkVKQ5+NJw6tsiiA41v7qX2wELRvM3P8bl9w== X-Received: by 2002:a05:622a:1a06:b0:40f:dcda:ea10 with SMTP id f6-20020a05622a1a0600b0040fdcdaea10mr2733810qtb.3.1691767120739; Fri, 11 Aug 2023 08:18:40 -0700 (PDT) Received: from x1n (cpe5c7695f3aee0-cm5c7695f3aede.cpe.net.cable.rogers.com. [99.254.144.39]) by smtp.gmail.com with ESMTPSA id g3-20020ac870c3000000b0040f200feb4fsm1229705qtp.80.2023.08.11.08.18.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Aug 2023 08:18:40 -0700 (PDT) Date: Fri, 11 Aug 2023 11:18:38 -0400 From: Peter Xu To: David Hildenbrand Cc: Matthew Wilcox , Ryan Roberts , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-doc@vger.kernel.org, Andrew Morton , Jonathan Corbet , Mike Kravetz , Hugh Dickins , Yin Fengwei , Yang Shi , Zi Yan Subject: Re: [PATCH mm-unstable v1] mm: add a total mapcount for large folios Message-ID: References: <20230809083256.699513-1-david@redhat.com> <155bd03e-b75c-4d2d-a89d-a12271ada71b@arm.com> <8222bf8f-6b99-58f4-92cc-44113b151d14@redhat.com> MIME-Version: 1.0 In-Reply-To: <8222bf8f-6b99-58f4-92cc-44113b151d14@redhat.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: DBEA41C002E X-Stat-Signature: tgbwjkhjdr7cx7zzqw9mkoy445ycyiye X-Rspam-User: X-HE-Tag: 1691767123-673781 X-HE-Meta: U2FsdGVkX18jHDLMkPZffDYLisc8sN2fEnYsjioDRSt65o3dKdWbrUNuS4Ft7JVkA5e8RgHoTBOGyGPD5Qnu2xYnQAXc0054e92h57MMHxVqB5HYJUXv5dAUJScohICocy39cyu4Ig75be8RZRG48c3EeT5jFeVTV8vHvh7X5psgNSbzgfrHoqkwc0mS7ix5gWKLmJA8XUSAjWhN1Bz7IZ03A5CmQC57xQ6ql/D2edbwWGmR3iNScPLmi3ZiGlxXTBiAVCGmH8Fwk/I1akkepZkK7/sVQE+x7f9Gdt4BNNrq1epWN83d2cHi4TH7UmHg1OA8QSr6gaeyZdA+DbgCJk5mbrGOqWccRjTcFd60bw/dNlCgRabdPz1jUX2r/IPk7/QDzUG/0UfdgsMg7XuKkI7g/ztuuOv2JaHHWP95Dv24rhBGU+sAgk4Zx1DpBrIaWppc3bIXs4qCR3F5WobBAYtMrBzhWM5OBr2RbrpkI4slYxp0voyLMm/DpVMAAoKqyJrWVDiro+y54temkq8uSVUQQZJIzK7DrjmebU/e2otE40ypHbOX4CGhWzXKX0v5FDmWSc2CItUIKAzYBr7XtqenVamAZqFio9J8G4uk6wtDK+mCjktBzDkavY2kLMEkXZL+sTQJiKFLGZdmkB42TLMN3KisjbbUpYoh26ohrV4uQRMKt9s2iJs7GtmggYkV7I5wUWqpC/AD6Uou22FT1yQEbHw34qlU8l8DWBJcr4Jw5R1cKkWgBdabEw7eQFdxi5gOQaTPvIIsBuHQfQh3f0gK6ODr2XLpIYDC3zWY8J10483loI41mWAFAWxXCWrGdjCdWBR84eBQ7QQAe/ih+jfZGIFp3rQLtz+8IEwksd8WA0Gm3ebIT3NIdlswK9SS2iOYo2icjNBSFM3ap8WFH3yWZbXBliEsGWgUJiuQAeM3a9T+Cwfo8fwVjpgswrE2gaqvNIVv0KOe6XbiPx8 xyHVOEP8 lF8NjZW0Ds/souZJI/ro52dRU0kM41sVojdwSa5KIvCIGBNkuXSsLi++23mn0PbEID+v4yb69GiHCrTzQVzQkv+00USK8GxGT2tWHCFOMrZBOAGWGZwKK3jMmfvSBJj/4Ukx1EZiLRuRDJb+/9+Tk9nS8qQUqQINwGhGUyBaHBVZvydEypm6q3aBvEJMvM17zAZKuJ6LEskVaQj8YvrNK3Ofp7mCmkFz+3kInfWERy5p2Bot5gY+Yv5wmC8sX9Ghx0/jYAoK9LGFPIsSimikMAU2gshg9p+x5N+u2wCro2q65wWLomdkR2jquaY7Q7jSylREOPhwM2+60plN6lTzq4mv7ZgUh/ImZIq40q6eRM/hwi84KygYv6yQNxw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Aug 11, 2023 at 12:27:13AM +0200, David Hildenbrand wrote: > On 10.08.23 23:48, Matthew Wilcox wrote: > > On Thu, Aug 10, 2023 at 04:57:11PM -0400, Peter Xu wrote: > > > AFAICS if that patch was all correct (while I'm not yet sure..), you can > > > actually fit your new total mapcount field into page 1 so even avoid the > > > extra cacheline access. You can have a look: the trick is refcount for > > > tail page 1 is still seems to be free on 32 bits (if that was your worry > > > before). Then it'll be very nice if to keep Hugh's counter all in tail 1. > > > > No, refcount must be 0 on all tail pages. We rely on this in many places > > in the MM. > > Very right. Obviously I could have missed this in the past.. can I ask for an example explaining why refcount will be referenced before knowing it's a head? > > One could theoretically > > 1) Move the compound/entire mapcount to page[2] > 2) Make hugetlb stop using the entire mapcount and only the total > mapcount. > 3) Then leave total_mapcount and nr_pages_mapped in page[1] > 4) Make page_mapcount() use total_mapcount for hugetlb. > > When (un)mapping a PMD-mapped THP, we would go to page[2]. Otherwise, only > page[1]. > > The entire mapcount, similarly to nr_pages_mapped, primarily serves to get > memory stats right; well, and to implement page_mapcount() for THP. > > But I'm not 100% sure yet if the overhead from having nr_pages_mapped in > page[2] is significant enough at this point. Worth trying. Besides the cachelines, cmpxchg should also lock the memory bus and meanwhile affect the whole processor pipelines in some way. Maybe that effect is amplified too when it runs in a loop with one more counter. -- Peter Xu