From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2BA3C433EF for ; Sat, 30 Apr 2022 13:19:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1444C6B0072; Sat, 30 Apr 2022 09:19:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0F37A6B0073; Sat, 30 Apr 2022 09:19:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F23C76B0074; Sat, 30 Apr 2022 09:19:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.28]) by kanga.kvack.org (Postfix) with ESMTP id E1DAC6B0072 for ; Sat, 30 Apr 2022 09:19:42 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A3E412068C for ; Sat, 30 Apr 2022 13:19:42 +0000 (UTC) X-FDA: 79413602604.17.D2D73AC Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) by imf26.hostedemail.com (Postfix) with ESMTP id 6083D14001D for ; Sat, 30 Apr 2022 13:19:39 +0000 (UTC) Received: by mail-pj1-f46.google.com with SMTP id w5-20020a17090aaf8500b001d74c754128so12783576pjq.0 for ; Sat, 30 Apr 2022 06:19:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=0SuQrAfcBYJGPjhTb8t3lZ1fTklP5mGBmOFSXiuMhmc=; b=guX6UFeGOXdCf7G4DGwuOBPyxqnjJ9tCzBu0mRaiqkNC3BSKW2ZIqgtjeZfOvgEbD9 ZeQCYrikPLnMv/JMSNhbPiODTxSl6/modpzf+XACc4XWsMMEktlfhSHq2c6Q6gjaelcq 6xEVzn4UECF6HlJJpYguuboiGQJlNUYNnSFcyr4bdgPnG+0n1F6LqZmikCbgocVqEOVd cWwE4snElvpQCSM8UaAxDBkn1mcfkf7bCtxUoNlZwrOotjC0JiK10oRSI5WTtxSd7YU1 1qWAXjCn5U4/tMdnTZCw3Lp0m0EaJQ2JBbw12sUUjgQxv4ecVZTF472NuGrgPG5MNtrN 8mZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=0SuQrAfcBYJGPjhTb8t3lZ1fTklP5mGBmOFSXiuMhmc=; b=afN7wwJPSsetZjnXn0ERtJB6wli7tiHVzO8Q1ZfuVqZnbu07IhxWQ5Z9Rd6I90WgI6 N43ioaN9mfwudm+51jZF14WN+H/54eQuxTwbeV4hf1x5w26ZE8jxMOEN4KljmVzfGUKW PZtRN+RQAL8Y9jvtQ+Dn8dUjX4rtM0q+3SxxHATtMhb3KuZb2jbAvea7v8ISmbUxWN9d IfBvIXZTcT52ILr2w5y3XQIECr9gD3oMlFhhr1QbXkmS1G35OXmJYhnCCA81RsCuJy6z BV/stjfXvsJJZBBRdEZ9reRrzXDYn81DIU3LykKcFQez5KZlLf2ADnOQequjYFiOCVO1 XWVQ== X-Gm-Message-State: AOAM532Dauw25aCYkg6H33L/qQxaJRAjyLeNVfpqOpmC3EK+mPr1Z473 eTG3PBxUtkIQ4QoTjw4HYC0= X-Google-Smtp-Source: ABdhPJwluFKZsRBqSdXxOv0+QMmM1ka3+CCPGlykURC3WPLkLxCSRzwTEHIoaPzaxD531XvUGDN88g== X-Received: by 2002:a17:902:e74e:b0:15e:92bc:c27c with SMTP id p14-20020a170902e74e00b0015e92bcc27cmr1258558plf.8.1651324780169; Sat, 30 Apr 2022 06:19:40 -0700 (PDT) Received: from localhost (subs32-116-206-28-15.three.co.id. [116.206.28.15]) by smtp.gmail.com with ESMTPSA id j24-20020aa78d18000000b0050dc76281bbsm1552018pfe.149.2022.04.30.06.19.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 30 Apr 2022 06:19:39 -0700 (PDT) Date: Sat, 30 Apr 2022 20:19:35 +0700 From: Bagas Sanjaya To: Qi Zheng Cc: akpm@linux-foundation.org, tglx@linutronix.de, kirill.shutemov@linux.intel.com, mika.penttila@nextfour.com, david@redhat.com, jgg@nvidia.com, tj@kernel.org, dennis@kernel.org, ming.lei@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, songmuchun@bytedance.com, zhouchengming@bytedance.com Subject: Re: [RFC PATCH 18/18] Documentation: add document for pte_ref Message-ID: References: <20220429133552.33768-1-zhengqi.arch@bytedance.com> <20220429133552.33768-19-zhengqi.arch@bytedance.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20220429133552.33768-19-zhengqi.arch@bytedance.com> X-Stat-Signature: 6nbfrab9z5ed58uby9m644ii39izba8g X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 6083D14001D X-Rspam-User: Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=guX6UFeG; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of bagasdotme@gmail.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=bagasdotme@gmail.com X-HE-Tag: 1651324779-885398 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Qi, On Fri, Apr 29, 2022 at 09:35:52PM +0800, Qi Zheng wrote: > +Now in order to pursue high performance, applications mostly use some > +high-performance user-mode memory allocators, such as jemalloc or tcmalloc. > +These memory allocators use madvise(MADV_DONTNEED or MADV_FREE) to release > +physical memory for the following reasons:: > + > + First of all, we should hold as few write locks of mmap_lock as possible, > + since the mmap_lock semaphore has long been a contention point in the > + memory management subsystem. The mmap()/munmap() hold the write lock, and > + the madvise(MADV_DONTNEED or MADV_FREE) hold the read lock, so using > + madvise() instead of munmap() to released physical memory can reduce the > + competition of the mmap_lock. > + > + Secondly, after using madvise() to release physical memory, there is no > + need to build vma and allocate page tables again when accessing the same > + virtual address again, which can also save some time. > + I think we can use enumerated list, like below: -- >8 -- diff --git a/Documentation/vm/pte_ref.rst b/Documentation/vm/pte_ref.rst index 0ac1e5a408d7c6..67b18e74fcb367 100644 --- a/Documentation/vm/pte_ref.rst +++ b/Documentation/vm/pte_ref.rst @@ -10,18 +10,18 @@ Preface Now in order to pursue high performance, applications mostly use some high-performance user-mode memory allocators, such as jemalloc or tcmalloc. These memory allocators use madvise(MADV_DONTNEED or MADV_FREE) to release -physical memory for the following reasons:: - - First of all, we should hold as few write locks of mmap_lock as possible, - since the mmap_lock semaphore has long been a contention point in the - memory management subsystem. The mmap()/munmap() hold the write lock, and - the madvise(MADV_DONTNEED or MADV_FREE) hold the read lock, so using - madvise() instead of munmap() to released physical memory can reduce the - competition of the mmap_lock. - - Secondly, after using madvise() to release physical memory, there is no - need to build vma and allocate page tables again when accessing the same - virtual address again, which can also save some time. +physical memory for the following reasons: + +1. We should hold as few write locks of mmap_lock as possible, + since the mmap_lock semaphore has long been a contention point in the + memory management subsystem. The mmap()/munmap() hold the write lock, and + the madvise(MADV_DONTNEED or MADV_FREE) hold the read lock, so using + madvise() instead of munmap() to released physical memory can reduce the + competition of the mmap_lock. + +2. After using madvise() to release physical memory, there is no + need to build vma and allocate page tables again when accessing the same + virtual address again, which can also save some time. The following is the largest user PTE page table memory that can be allocated by a single user process in a 32-bit and a 64-bit system. > +The following is the largest user PTE page table memory that can be > +allocated by a single user process in a 32-bit and a 64-bit system. > + We can say "assuming 4K page size" here, > ++---------------------------+--------+---------+ > +| | 32-bit | 64-bit | > ++===========================+========+=========+ > +| user PTE page table pages | 3 MiB | 512 GiB | > ++---------------------------+--------+---------+ > +| user PMD page table pages | 3 KiB | 1 GiB | > ++---------------------------+--------+---------+ > + > +(for 32-bit, take 3G user address space, 4K page size as an example; > + for 64-bit, take 48-bit address width, 4K page size as an example.) > + ... instead of here. > +There is also a lock-less scenario(such as fast GUP). Fortunately, we don't need > +to do any additional operations to ensure that the system is in order. Take fast > +GUP as an example:: > + > + thread A thread B > + fast GUP madvise(MADV_DONTNEED) > + ======== ====================== > + > + get_user_pages_fast_only() > + --> local_irq_save(); > + call_rcu(pte_free_rcu) > + gup_pgd_range(); > + local_irq_restore(); > + /* do pte_free_rcu() */ > + I see whitespace warning circa do pte_free_rcu() line above when applying this series. Thanks. -- An old man doll... just what I always wanted! - Clara