From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1529EC433F5 for ; Sat, 30 Apr 2022 13:32:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2BDDA6B0074; Sat, 30 Apr 2022 09:32:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 243BD6B0075; Sat, 30 Apr 2022 09:32:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0BF3D6B0078; Sat, 30 Apr 2022 09:32:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id EAA026B0074 for ; Sat, 30 Apr 2022 09:32:12 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id BDC136091D for ; Sat, 30 Apr 2022 13:32:12 +0000 (UTC) X-FDA: 79413634104.22.F5D6563 Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) by imf10.hostedemail.com (Postfix) with ESMTP id 797AFC006F for ; Sat, 30 Apr 2022 13:31:59 +0000 (UTC) Received: by mail-pj1-f50.google.com with SMTP id l11-20020a17090a49cb00b001d923a9ca99so9469072pjm.1 for ; Sat, 30 Apr 2022 06:32:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=message-id:date:mime-version:user-agent:subject:content-language:to :cc:references:from:in-reply-to:content-transfer-encoding; bh=dbDWgmPkPLStp/l8kRXyOJz5NyE5KuQFyXl2VDskwyU=; b=nxXKBufS8v7zlmdyBhrIqOI6zYhkk5YZIBkJiU12Zysh/RGlGNgqQYhx5g1z5j2AhE ujAPtVqtDXU0WrK5k8bDm10KlHjZMnTHcJERL0TD/hNud6NQ69sCO4ivlgRbMSqsfWUF 1VtYQ2CSj0fWyKVw/UarmeZwlHQrtMTS8aH8PqdF2AMtUZjoafNoGybi0/3GpVD6hCUF qPdixGrPflNOGw8cnppv+b8kl+2BFgtYNRjnYyCGLCo7zDjtDUcQX6RV7zVbXV5MrWf/ 3SJSgdtSOruQiqVhb2QhvcKWrglLGmnLNGAxlKEjjDtAeQk23qgt986PvDycdWdhYnsn sXfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=dbDWgmPkPLStp/l8kRXyOJz5NyE5KuQFyXl2VDskwyU=; b=tD5Sn3Vtc1SBr6GUAKf3Wv+b83q7N7sKxnGIwTGiddHHXyMm/yJcswHrdi+HXM0R1i bvJABFXL++Mg6nzkqQKwHRsxt8MXE2wl+L3rU8W/z0bQY61mt8tVsry83ryD06G56TKs CsIFt1IDh8SxwgGq6uwPhsXYUQ1DXTavYIFUdtKSlONImqIXHaqk4zeQ8hmunDeXYLra qjgQETO40LOyH93VK+Ak1t0056vi8K3pq41dq2h+z2BAMLav2FBjl/zFcVPDtS/Hd1ZD Rm6BufMmsAQ9+KoAbgZCerTNx5eekoqsWT02N1XcPYJqfUgl7C6y7IKbeaCIGkTD0IER doiQ== X-Gm-Message-State: AOAM532EYTllwoYiAtMflRfYzXZeSK3HfCTW9qYqABHmjTArxo6A3Eyy lN+WTb1aWYvf6bEbPWHoAjRsttTCWWA/2g== X-Google-Smtp-Source: ABdhPJxfWSvO+6m1XvVr6FutuZE6Ik/Z5TJmpD/tUHd14ymyAHBgC6TYIizTQ14p2WFMZ1ZUk1AGqg== X-Received: by 2002:a17:90b:1e4c:b0:1db:cbc5:e031 with SMTP id pi12-20020a17090b1e4c00b001dbcbc5e031mr4280582pjb.62.1651325530190; Sat, 30 Apr 2022 06:32:10 -0700 (PDT) Received: from [10.254.246.218] ([139.177.225.240]) by smtp.gmail.com with ESMTPSA id q26-20020a63505a000000b003aa8b87feb5sm8503912pgl.0.2022.04.30.06.32.04 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 30 Apr 2022 06:32:09 -0700 (PDT) Message-ID: <8e4d1e8a-6bf7-df07-2cc6-01d840db2757@bytedance.com> Date: Sat, 30 Apr 2022 21:32:01 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.8.0 Subject: Re: [RFC PATCH 18/18] Documentation: add document for pte_ref Content-Language: en-US To: Bagas Sanjaya Cc: akpm@linux-foundation.org, tglx@linutronix.de, kirill.shutemov@linux.intel.com, mika.penttila@nextfour.com, david@redhat.com, jgg@nvidia.com, tj@kernel.org, dennis@kernel.org, ming.lei@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, songmuchun@bytedance.com, zhouchengming@bytedance.com References: <20220429133552.33768-1-zhengqi.arch@bytedance.com> <20220429133552.33768-19-zhengqi.arch@bytedance.com> From: Qi Zheng In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 797AFC006F X-Stat-Signature: hwi84k9e8gfwtu61k4in4pgkw5c4qbw8 Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=nxXKBufS; dmarc=pass (policy=none) header.from=bytedance.com; spf=pass (imf10.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.216.50 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com X-Rspam-User: X-HE-Tag: 1651325519-845210 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2022/4/30 9:19 PM, Bagas Sanjaya wrote: > Hi Qi, > > On Fri, Apr 29, 2022 at 09:35:52PM +0800, Qi Zheng wrote: >> +Now in order to pursue high performance, applications mostly use some >> +high-performance user-mode memory allocators, such as jemalloc or tcmalloc. >> +These memory allocators use madvise(MADV_DONTNEED or MADV_FREE) to release >> +physical memory for the following reasons:: >> + >> + First of all, we should hold as few write locks of mmap_lock as possible, >> + since the mmap_lock semaphore has long been a contention point in the >> + memory management subsystem. The mmap()/munmap() hold the write lock, and >> + the madvise(MADV_DONTNEED or MADV_FREE) hold the read lock, so using >> + madvise() instead of munmap() to released physical memory can reduce the >> + competition of the mmap_lock. >> + >> + Secondly, after using madvise() to release physical memory, there is no >> + need to build vma and allocate page tables again when accessing the same >> + virtual address again, which can also save some time. >> + > > I think we can use enumerated list, like below: Thanks for your review, LGTM, will do. > > -- >8 -- > > diff --git a/Documentation/vm/pte_ref.rst b/Documentation/vm/pte_ref.rst > index 0ac1e5a408d7c6..67b18e74fcb367 100644 > --- a/Documentation/vm/pte_ref.rst > +++ b/Documentation/vm/pte_ref.rst > @@ -10,18 +10,18 @@ Preface > Now in order to pursue high performance, applications mostly use some > high-performance user-mode memory allocators, such as jemalloc or tcmalloc. > These memory allocators use madvise(MADV_DONTNEED or MADV_FREE) to release > -physical memory for the following reasons:: > - > - First of all, we should hold as few write locks of mmap_lock as possible, > - since the mmap_lock semaphore has long been a contention point in the > - memory management subsystem. The mmap()/munmap() hold the write lock, and > - the madvise(MADV_DONTNEED or MADV_FREE) hold the read lock, so using > - madvise() instead of munmap() to released physical memory can reduce the > - competition of the mmap_lock. > - > - Secondly, after using madvise() to release physical memory, there is no > - need to build vma and allocate page tables again when accessing the same > - virtual address again, which can also save some time. > +physical memory for the following reasons: > + > +1. We should hold as few write locks of mmap_lock as possible, > + since the mmap_lock semaphore has long been a contention point in the > + memory management subsystem. The mmap()/munmap() hold the write lock, and > + the madvise(MADV_DONTNEED or MADV_FREE) hold the read lock, so using > + madvise() instead of munmap() to released physical memory can reduce the > + competition of the mmap_lock. > + > +2. After using madvise() to release physical memory, there is no > + need to build vma and allocate page tables again when accessing the same > + virtual address again, which can also save some time. > > The following is the largest user PTE page table memory that can be > allocated by a single user process in a 32-bit and a 64-bit system. > >> +The following is the largest user PTE page table memory that can be >> +allocated by a single user process in a 32-bit and a 64-bit system. >> + > > We can say "assuming 4K page size" here, > >> ++---------------------------+--------+---------+ >> +| | 32-bit | 64-bit | >> ++===========================+========+=========+ >> +| user PTE page table pages | 3 MiB | 512 GiB | >> ++---------------------------+--------+---------+ >> +| user PMD page table pages | 3 KiB | 1 GiB | >> ++---------------------------+--------+---------+ >> + >> +(for 32-bit, take 3G user address space, 4K page size as an example; >> + for 64-bit, take 48-bit address width, 4K page size as an example.) >> + > > ... instead of here. will do. > >> +There is also a lock-less scenario(such as fast GUP). Fortunately, we don't need >> +to do any additional operations to ensure that the system is in order. Take fast >> +GUP as an example:: >> + >> + thread A thread B >> + fast GUP madvise(MADV_DONTNEED) >> + ======== ====================== >> + >> + get_user_pages_fast_only() >> + --> local_irq_save(); >> + call_rcu(pte_free_rcu) >> + gup_pgd_range(); >> + local_irq_restore(); >> + /* do pte_free_rcu() */ >> + > > I see whitespace warning circa do pte_free_rcu() line above when > applying this series. will fix. Thanks, Qi > > Thanks. > -- Thanks, Qi