From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84CF3C61DA3 for ; Thu, 26 Jan 2023 16:59:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 149DD6B0072; Thu, 26 Jan 2023 11:59:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0F9E88E0002; Thu, 26 Jan 2023 11:59:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F043B8E0001; Thu, 26 Jan 2023 11:59:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E1DF56B0072 for ; Thu, 26 Jan 2023 11:59:32 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id AA9691C5E6B for ; Thu, 26 Jan 2023 16:59:32 +0000 (UTC) X-FDA: 80397561384.12.3C2EBCA Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) by imf09.hostedemail.com (Postfix) with ESMTP id C6B4B140026 for ; Thu, 26 Jan 2023 16:59:29 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Y9aKhvCd; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf09.hostedemail.com: domain of jthoughton@google.com designates 209.85.221.47 as permitted sender) smtp.mailfrom=jthoughton@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674752369; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=neDGA3ZoiU4GEtv5HBc+8mGFjglFh5fImPAsKZ4CGNA=; b=f+eIc0PVGQtp3Sz9EU8apGXDz//qe5IG8r0zJ15j+N57knhdYB9LVVgkCCs+fLF2YCCv0i YhhMvRqUiqd7ce6IAYW4HUCfWnpjiy1LigZDLsn7mR+E9FZuk0AqE6DCx4V3RKtHnrmg4z qP+SiFMrIb/IfvDw0XGEHRWh9nqayBs= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Y9aKhvCd; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf09.hostedemail.com: domain of jthoughton@google.com designates 209.85.221.47 as permitted sender) smtp.mailfrom=jthoughton@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674752369; a=rsa-sha256; cv=none; b=KCCEiNCykTDqM2UWCtSX1GDFSnznFt5iUjCsScGv3zEN2FXhS4tgepRmp3KkFuRX5dR1P1 k1Z3FZOtyhWaAf72DNwLKkbJh1Oa3dpLbgHu7BA//DzK9cPTAl1bhyGtbXBtZIUbMMRvkw Zbp4+++/ZkO9P62R5G6D0LlAo3feokU= Received: by mail-wr1-f47.google.com with SMTP id h12so2407525wrv.10 for ; Thu, 26 Jan 2023 08:59:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=neDGA3ZoiU4GEtv5HBc+8mGFjglFh5fImPAsKZ4CGNA=; b=Y9aKhvCdAeSa/x+9sluyhUTp4oQvpT3oy9WWedj87O3Ft+0zx8hCq0sH+J/+hsnW/n PpsJcEESfRYV+/b5qNWeUsplTBvA5dz5gnoadRgGvWYZjy/auuHJvPyIdbWYBZCKB5xp r5rxUcFwA7u0zXU1XJO851zbSyH6RrjaZq1R0pOyF6/POCMmnG8UJ56QcLz4MJY8b+EW PW6KfMGOaG44wSKUDn0Ce7tzkA81kRaoD5E++rtmJlkfTUvlAZMZoFiC5lvb18EM7T9r 3YdTw01gik5GHyWtwHvcFbUU/HquYcvJwIfX3CcPJO0/a2r5V8PmU1t3Ne6agzrUcF0X bCJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=neDGA3ZoiU4GEtv5HBc+8mGFjglFh5fImPAsKZ4CGNA=; b=26Uz21XYr849m/mxc6OM0gUeECRXhquyxuA4U9YRw4+D5t3FC2NhkkpH98GXsNYLSK PuOoWJytwhCdG70yCsLr7rLr3/6djDDK+MJsT8c12tCEjmvwr9gj23bDjv9+TVgSTUyX lqNt79M/j77fY0lS19UtMKa0TP+thN4T572eWVl52Zd5cMzv+0lJT4kQk5vry78iwqei Zg7hoZtBNeW7Za9n+tW8eeMVkw9OOi4H8fuq7vVM4yFdRpVyiafoYQnBCVPbkGmggu7c bz15z1NGsDX0R943v1NqE66GMAp427zFKeGgMZn5S0cQFoU1EhqgyTT0HqJAlDivyjca hJ9Q== X-Gm-Message-State: AO0yUKVrj9QvCGpHNKYTcH96b+B/cNMojV4mJJdFuqq3AqZsqCxrsxp0 wL86ymWTk+RDvLgawpWR9jiWKdVTx/wLf7xqVP6eBg== X-Google-Smtp-Source: AK7set+0Bk/K1r68hC2EVR99RRGeNZ+S3IbX8/b1rRgetk3VUJRNEcZ3nNxrMlDvYshlplWdmB2KgHCX3NBVZw8N5A0= X-Received: by 2002:adf:a54c:0:b0:2bf:b373:149a with SMTP id j12-20020adfa54c000000b002bfb373149amr375162wrb.355.1674752368212; Thu, 26 Jan 2023 08:59:28 -0800 (PST) MIME-Version: 1.0 References: <06423461-c543-56fe-cc63-cabda6871104@redhat.com> <6548b3b3-30c9-8f64-7d28-8a434e0a0b80@redhat.com> In-Reply-To: From: James Houghton Date: Thu, 26 Jan 2023 08:58:51 -0800 Message-ID: Subject: Re: [PATCH 21/46] hugetlb: use struct hugetlb_pte for walk_hugetlb_range To: Mike Kravetz Cc: Peter Xu , David Hildenbrand , Muchun Song , David Rientjes , Axel Rasmussen , Mina Almasry , "Zach O'Keefe" , Manish Mishra , Naoya Horiguchi , "Dr . David Alan Gilbert" , "Matthew Wilcox (Oracle)" , Vlastimil Babka , Baolin Wang , Miaohe Lin , Yang Shi , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: C6B4B140026 X-Stat-Signature: s86aiwm8stqoa7555h5qs6bybw9x1jpw X-HE-Tag: 1674752369-209508 X-HE-Meta: U2FsdGVkX19fEEF4c05vQA4wiq74PfbsFcYe1X8iAbN01kRcXrxDkV0C5Hc+B1PakzFG6a74xSA3HpL90M0IyhE3Q4MqKPUAhAccVF8TcaRDR1b8i2E3cmRpXQqnn8leO/0YEPEB8o3cISzxbzp8Tds3B/TjQQiZdCts6jf8AbD0Ow60Mwrk275AzJmS6LInE1jUI3zca4dypxJcS60CsP/w+XFbyNHB9sMBitW+FSn086I0lEZISlo+3Ry4bGSYn50eymWaOFcK5NGpBD06Spe1mxBudHXTSnJ1mYqhX0deGEhYkkXQ1q0bim0boBYKgZ/lpMjeZy5XspUkywADk1vPbHKsjj2qclZvhz64hSvuUn3IWdXyO1eduLhB9BCG438H06bks6g53pwcFo/gkfZ924vCk7VsJ7/cJJBgpM4zJD1G5kpfKrVJFq4TliQb775dGOV7Z2rc0ac1k7SXMKPXUpxwkAHAKUJntTBzB0icc+Bv/Id/bQQPs/MrUzr1vF3+l6qreC5GjvJl8BTE2B8FaPbMNfoO4KObAVOEazZzlxrrg/wbT2GyS2wQfeyZVCR4QRukBJP5VvWi1Pv3LMzl+ETySgeN14MukU0iN4V3s9wg2dY3oZVgDZyzB/as04kPdCEWLUT+GXHEU9ACJYGr1zmWYyLz8diDRFgqrsHIO0l+4qJSne2M6gq7YdjNgpnfX0Fc/n2GjPWJHzssfhX05usEo1DsbKlk6hXC5Ks8Pg1/RFxPtYbNaADDC2QZ2a7qSU/u1GIdHRdOQqB2uscGvSn/FvW/DFew9ev0FC+LBAT4sStQkHYGHmiJIEmC7Up1C7t9eptO43tdptUGVkMjvs7cXqBAImHdrGpzkEhWqWoAivo8I5j0vMUrhltsrE7n3UirLQdPKjh2bZJYbWvRoKlLv3BzfRzhwISQMiqN3s5Q8QE2kcj1K8OOQW986v/j5eE8Afy9NpaMfXD AVtu/FBk QlyCiSxVivir69KkTlLM+mZn0fQ6n1Da20EfJc7k5B3+o0q04uMbopn8B+kn1PCzjgxyKo2Q/tj6QNPEOIdH4V3Nh5UxOTEkl49rZ/gLgr1TqOByPUP0YsYYTdaZLt6egwj5JGa/iyX0VkKb1/sAQeqbQXPGTIrzaP3/d4q4lJZXjJ6lCwzmzh7WouH4S/16eOUHSIKBLcAGLAzmH60iNTEjr5oXwWnfJ27nN7ne+UGwXSGelreEv9KaAlqJafl/dxm8Q5Xi9/D7R7y2/8sCdVFl4JklKtR2oJcV06z2DGCOSvxC59m5Q5i1yy739MxJlgoxFPlD0BP9qFOgUB0PLEh31fA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Jan 19, 2023 at 11:42 AM James Houghton wrote: > > On Thu, Jan 19, 2023 at 9:32 AM Mike Kravetz wrote: > > > > On 01/19/23 08:57, James Houghton wrote: > > > FWIW, what makes the most sense to me right now is to implement the > > > THP-like scheme and mark HGM as mutually exclusive with the vmemmap > > > optimization. We can later come up with a scheme that lets us retain > > > compatibility. (Is that what you mean by "this can be done somewhat > > > independently", Mike?) > > > > Sort of, I was only saying that getting the ref/map counting right seems > > like a task than can be independently worked. Using the THP-like scheme > > is good. > > Ok! So if you're ok with the intermediate mapping sizes, it sounds > like I should go ahead and implement the THP-like scheme. It turns out that the THP-like scheme significantly slows down MADV_COLLAPSE: decrementing the mapcounts for the 4K subpages becomes the vast majority of the time spent in MADV_COLLAPSE when collapsing 1G mappings. It is doing 262k atomic decrements, so this makes sense. This is only really a problem because this is done between mmu_notifier_invalidate_range_start() and mmu_notifier_invalidate_range_end(), so KVM won't allow vCPUs to access any of the 1G page while we're doing this (and it can take like ~1 second for each 1G, at least on the x86 server I was testing on). - James