From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64744C74A5B for ; Wed, 29 Mar 2023 06:54:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9465E6B0072; Wed, 29 Mar 2023 02:54:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8F5B06B0074; Wed, 29 Mar 2023 02:54:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7E4EF6B0075; Wed, 29 Mar 2023 02:54:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 6F3016B0072 for ; Wed, 29 Mar 2023 02:54:30 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 3A323160BF5 for ; Wed, 29 Mar 2023 06:54:30 +0000 (UTC) X-FDA: 80621022300.09.7B0415F Received: from mail-lj1-f175.google.com (mail-lj1-f175.google.com [209.85.208.175]) by imf08.hostedemail.com (Postfix) with ESMTP id 4BE1B160005 for ; Wed, 29 Mar 2023 06:54:26 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=NdRrpDX6; spf=pass (imf08.hostedemail.com: domain of urezki@gmail.com designates 209.85.208.175 as permitted sender) smtp.mailfrom=urezki@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680072866; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=z64+4NP0Iaq2THmuvx30e3ulOcM6eG7Y0CK87PMVoP4=; b=Q2aJJBM6Rhl1oYUzGRE7qX6hPYa3EBfkXsqMKojBb9UbnnlfCeW/me+/afx4jH/mSWZ6bo sbg6gQbLEkYEfPvV4PIdCPSLGcPYY/tIcEvPhgECr15Nca5v/pb9T8eeXp9ynfsbYep3B/ 2bI6lJrJnsa4rS0J8tQovng4v7LYkdo= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=NdRrpDX6; spf=pass (imf08.hostedemail.com: domain of urezki@gmail.com designates 209.85.208.175 as permitted sender) smtp.mailfrom=urezki@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680072866; a=rsa-sha256; cv=none; b=FOI2akJ5MsqTkhOxh/tLL74+IKLMflq9RLS5Vn76dqMMeK+8ehFodLIubQjbK7+hoLlDl7 aB6qGORzk5IdXs9CRaO0OetN15MIqkWyfTnGnhYtCzSoTRAUCBZ56FAVXbjpX2zgM9hNxc zgSaXkV01MX6ZJCErbSNHv4ukt7k85c= Received: by mail-lj1-f175.google.com with SMTP id x20so15001100ljq.9 for ; Tue, 28 Mar 2023 23:54:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680072864; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:from:to:cc:subject:date:message-id:reply-to; bh=z64+4NP0Iaq2THmuvx30e3ulOcM6eG7Y0CK87PMVoP4=; b=NdRrpDX6sK0yGYNDimTTRFcPrBRKBbkFmoAP4Wr21vi4mlUg+Hjb4JCxFK3ZraDOcD ZLQP1FCcaJe57bk79OMqS0yILOsjUQQJD/8gsx3S/m7e9LQwnbm5FlIGKveYoP7zMPzB m+qpc5c40Zzbj+dYjHd+r4QxnIeSBXSuL5fLbdzWOgM29C7POfr0tmT8yLNKiSyfwhfz TDvmvZbo0uC/Z9oz2XRVd9fv/1/CoMlt5lD0StzahriaLrqZS9Zw56nHfr+65z73iI1r goqu5XdcUaVCK+i9xY/m3WasFx+q034MslmAUwJYDk1LZil6bBbtKE+G/ml1anBwuZ+a UfOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680072864; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=z64+4NP0Iaq2THmuvx30e3ulOcM6eG7Y0CK87PMVoP4=; b=dSDzS5+lZ63JQH9g3jXE9V+On8J9DPgzj0xdB3Im4NADiFqbARkGRSFsbcSJLIRSYT /hgMYwQ2ATpTDwiu527yZj+xn18pesMTBRjcdqsJzaz+H8LqU1R65ULPohSLNA+Xn7nv 7XO0IKjmzDKWYzDOXOxtSPQ5IcK2wLHRr/UmQDo9lFKXB0BCazcTqD6hgD8CCgo1LIxi M5wVHkh0P6Mn/EI1LKpwnwjq7viHSnw9Wvjc8Mg8zlmJyMvv8lb9S3NmJb4tfTULmzly iNOxKBF8d9YaZV7NZlq/ggwtcI5FAVEMNVy9/Cr5PRZbHqbrnCAv/fn26ZdtHMb+fC8x MOSw== X-Gm-Message-State: AAQBX9e9nVs+L2wbV1WmqeyR8Qsb+u/Zb1K0enHCMxzlg0Ujc4ihXLqk 6PbSG8miHGOZGLHJ+Qq46r0= X-Google-Smtp-Source: AKy350ZYvagz4vuBTC3lqDdAYJ+bClEDJbqeIK12P3i/5LCHauUXkBZ85ve9qjn0FCROjTHD6NjMZQ== X-Received: by 2002:a2e:9d87:0:b0:29e:c648:674 with SMTP id c7-20020a2e9d87000000b0029ec6480674mr6145908ljj.46.1680072864233; Tue, 28 Mar 2023 23:54:24 -0700 (PDT) Received: from pc636 (host-90-233-209-50.mobileonline.telia.com. [90.233.209.50]) by smtp.gmail.com with ESMTPSA id o4-20020a2e90c4000000b00299fe6c262csm5375138ljg.77.2023.03.28.23.54.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Mar 2023 23:54:23 -0700 (PDT) From: Uladzislau Rezki X-Google-Original-From: Uladzislau Rezki Date: Wed, 29 Mar 2023 08:54:20 +0200 To: Baoquan He Cc: Uladzislau Rezki , Andrew Morton , linux-mm@kvack.org, LKML , Lorenzo Stoakes , Christoph Hellwig , Matthew Wilcox , Dave Chinner , Oleksiy Avramchenko Subject: Re: [PATCH v3 1/2] mm: vmalloc: Remove a global vmap_blocks xarray Message-ID: References: <20230327170126.406044-1-urezki@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 4BE1B160005 X-Rspam-User: X-Stat-Signature: xuhnz58zmrp58y8u6jemwdypxs6wawhm X-HE-Tag: 1680072866-496325 X-HE-Meta: U2FsdGVkX19NZ2hRhu9o+op7hd2FQfbrx+tS94KJ/JvqOktSe7llS1yXWJ279ynkcKNhijR4ZaaAXqJwx+4x2hGstd0+TA1BwBlB9g3OLo7G2vsgFnnMBX0OG2x01n1TTfTitos3LGnlgD5HXGkMBs0bvRwfdttruFYd6ypwEBAGoxkUE8xlXtKMQc8AYwO7iqzo3vUz47GeK4wbG1nSuV4zRxhBti9qMARDIccIhRUTcY0DFWI5n3KB8RGlRIV7KWB+2A4Y2gAAsHKV9B7YZVVdBGcGKb1/l5mvZx698Vf4y+XJGVeErye4xyg0m0JI1KX8DaFp3S/yGGOe0DFqSOjqvI3FiYB5cmKXb+N7VQ1qS/uvn8n3DDrIxhuI09E2o+IT3A3FeNr/QlSk1P3mti+lFafRkg0PFQ3mVyLbBhzgmLBCcclj5MZJJ0cMg2dKuue8G2P7AoybmOqMfdRVMVAXRRCc3AazK+qff8McMHfhaFIEwVhItYUgoYZolC01wKajLqOMqtsIAT1deN+SPNpNhldTFPNQe9q2UrRoZH+w5RVqShpuT09T77aAbWR5++SBUk9dkJc8jNd5hQ7xdiiEdTYjKqkT7MopC0K8e5LhYqoxVo4LCf1pwKRjpUbi1fgTYJ266rIjAhE3umU7QIiG8uk83bB3qrMBj36W/PEZYAomKwefNMTGjvyO7bjR9RtSXHCaTsuECGBTeQ1ZJs7HoRiPtUN/4EGxUy/LnF+No2eZyq8BD8ovAkxWda9zX1q4137/+DRevqc6rTkz2CeFqP7Cwb6mCmG5FMAjnqMjOhwIL2HH7DFNnI7IZ8wZ1rI3CF8KoAXnwdP38yYFa5cw8zXKjyOyUE3yf1p7pm33Gm7CVA2Q1glTFz3KTZFbku/t7PpJxQ1g4MRi8wt+6TQYLFd7Vwp3Jm2ftc55dwEAlmRP1GxeHHemrbHvj7kgMJ/klHYSKPXJWPRdBZm HRiWTyzv kH84F8oBIAILIpi2pHRRvxBdoa3hONaCFrrJVKxqWNCwkZg46JeWKe0TnOy/r05YRVQ/0U+z+KTThkB+x+txdlB+Cv3pG9/Ho0sRBoB60oXw/mt/CoJJA99C0X6g38r4rolAqOU6xQhEwpNJs2uc4TLQiCr8+5KUCSFkbWmPulwtTygksWX7Mc4JF2dNKzjC4dRZmvNeAvR6FqL8losrmWqXBlzZZDQ7E0+IpYyMGfy/in0DOO6Ioxv6Z5/f092ZuWCbg6ZpnrdawezE4+qpi46JUkhe0FmG1nDE/sQtBtl7pWdu4p8ItAygpyQ6CkeO/hQ65PRfpE0BGLMORBow0BJcMZV2VOK1u1AE0eaCKvN5FSWlnIFKAoQ5NZJYlDRs0rF9q32SKgJ2kkvYHB/PVXuG/jICZ2UDKVMe6NPLji4pUAJGBYc1zRgSkeYbMX0xwkWI/YBcG+NlPk/JxnGwWXsbUlbcUxr8e2R+b5lEi9FTiW+c= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Mar 29, 2023 at 12:33:05PM +0800, Baoquan He wrote: > On 03/28/23 at 02:34pm, Uladzislau Rezki wrote: > ...... > > > > @@ -2003,8 +2037,8 @@ static void *new_vmap_block(unsigned int order, gfp_t gfp_mask) > > > > bitmap_set(vb->used_map, 0, (1UL << order)); > > > > INIT_LIST_HEAD(&vb->free_list); > > > > > > > > - vb_idx = addr_to_vb_idx(va->va_start); > > > > - err = xa_insert(&vmap_blocks, vb_idx, vb, gfp_mask); > > > > + vbq = addr_to_vbq(va->va_start); > > > > + err = xa_insert(&vbq->vmap_blocks, va->va_start, vb, gfp_mask); > > > > > > Using va->va_start as index to access xarray may cost extra memory. > > > Imagine we got a virtual address at VMALLOC_START, its region is > > > [VMALLOC_START, VMALLOC_START+4095]. In the xarray, its sequence order > > > is 0. While with va->va_start, it's 0xffffc90000000000UL on x86_64 with > > > level4 paging mode. That means for the first page size vmalloc area, > > > storing it into xarray need about 10 levels of xa_node, just for the one > > > page size. With the old addr_to_vb_idx(), its index is 0. Only one level > > > height is needed. One xa_node is about 72bytes, it could take more time > > > and memory to access va->va_start. Not sure if my understanding is correct. > > > > > > static unsigned long addr_to_vb_idx(unsigned long addr) > > > { > > > addr -= VMALLOC_START & ~(VMAP_BLOCK_SIZE-1); > > > addr /= VMAP_BLOCK_SIZE; > > > return addr; > > > } > > > > > If the size of array depends on index "length", then, indeed it will require > > more memory. From the other hand we can keep the old addr_to_vb_idx() function > > in order to "cut" a va->va_start index. > > Yeah, the extra 10 levels of xa_node is unnecessary if we keep the old > addr_to_vb_idx(). And the prolonged path will cost more time to reach the > wanted leaf node. E.g on x86_64 with 4 level paging mode, vmalloc area > is 32TB. With the old calculation, its index range is [0, 8M], 4 level > heights of xa_node at most is enough to cover. > Good! I have not analyzed how xarray stores its indexes. I will update the patch to cut indexes so we stay the same as we used to be before. -- Uladzislau Rezki