From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BBD9C27C75 for ; Fri, 14 Jun 2024 10:57:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4A86C6B00FA; Fri, 14 Jun 2024 06:50:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5A7996B0150; Fri, 14 Jun 2024 06:50:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A89F46B0161; Fri, 14 Jun 2024 06:50:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id CE2146B0155 for ; Fri, 14 Jun 2024 06:49:53 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7C39980668 for ; Fri, 14 Jun 2024 10:49:53 +0000 (UTC) X-FDA: 82229173866.29.1FDE57D Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) by imf22.hostedemail.com (Postfix) with ESMTP id A0A0EC0022 for ; Fri, 14 Jun 2024 10:49:50 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=b6iZnJRL; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf22.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.178 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718362189; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hmqCAHcpbwggVjy6exq+YALJF1pljPcusJTHFPXAlKI=; b=lGA7E44S83oDBLYTAKEZlGk5k4kTXXi8iBwoQ6CthUYjf/DAjz+O+mE6GVzLDnu9a0e5nv wRnvKiuVPF88ojkW+ihtsfL+X8MNBnEYMSIk3kSzFzSZimwPHL8/ljEIVKFJLI+4hPPn7O yLHhtfie+KgLmBOcrGSLSel1PrExuuQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718362189; a=rsa-sha256; cv=none; b=WALgzdOfmWo74KEELLKQrlH21P0F/OK04SAFf+8f02AQWuptJxAG7b7p/6I/vSaZDfDJyr Ei13rhhaxb1rss1E+DA+UWlSVlgSj2FOeNWqP6FAGJG/XhK0O3UujCv4HneAE2vAY/ux7j qrnLMsvAJcK+04rTctfrHZhyFFecRxI= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=b6iZnJRL; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf22.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.178 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-70420956abaso65967b3a.1 for ; Fri, 14 Jun 2024 03:49:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1718362189; x=1718966989; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=hmqCAHcpbwggVjy6exq+YALJF1pljPcusJTHFPXAlKI=; b=b6iZnJRLvSXnuUc2r74U/6UAJlLD2vGNliGTvnr49nrIMoRa/PSzoE/sgssil/6pjx 7o36tWI57hVy478RN722U8ZS3OiShkAjysv++ajyFjwooT7vgWPvWE2mY7vNG/o9V5g7 y2zxnp1/uJHpaK6O6SI2txJ/6csLX1IZdn2Ym/PhRsb0THBTR8jiDWiDsFSq5SqeFYAO /FVXVK2X/o7MJsmzlYjkqIqDHL8lQ7RXXFG/AxvKl/uwtiuNLYQHQ/LsXZaltxRFswjd pnOpkNYs+J9tvln1UeCTM0f+IzG+WDLSqeU2neGwzwCbFUzwaqrV0oBVfzGsFUNjLAUH khzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718362189; x=1718966989; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=hmqCAHcpbwggVjy6exq+YALJF1pljPcusJTHFPXAlKI=; b=d+Ys2bjM8P2gzmlgVhbEHAwoL5hlfpPfheZtQZGKojVZ/XcXX7fi3NTkqWlpWzbkHI VG9EUN1YRhfvU/r8ahHggOWX4ebCwzELdW9b+uS4s9iB2eCO6Cbsv4JsiPmzSQ1XfkDi 5QVTcIdGm3WfhdG8Rp3cOzB2OMHX4P00yLls089zbznezLRCFusRsZKx9LMxmL/5wcGr W971qNYFf4gYbC6pJwmC3vf+JGSBg8GXxNlEsERFJ5JbnQUOMjjRoyzXSKYH6NOuExPl B4M6JGtz8+AVy1Ul0VViUrHIvqjkn4QjZ9uXm68hGX6HVDykaMod/jQlpsHmrs+9Utnp fxrg== X-Forwarded-Encrypted: i=1; AJvYcCWLBbzfF3jwIieU+4uzoz+nNRA5rqAje9rz4FitutTTy6efhET9J42Q+ow4joHxCEun+teEB3jjl1zNijZfWVT6aFw= X-Gm-Message-State: AOJu0YwlaJsWyQyDByVRgKBuM0eDQNJ096InsZmX8Vm1GDb7ub6J3EtX XNlonB1+gireugKCt3ZmKdV5LE0cTR9Jh5s9oGNreZXt2a3uEzvSE4/YJ2JLUQA= X-Google-Smtp-Source: AGHT+IGtCcMdp4KkgsUlEmyCdU6iXnl4cUS8Wg4L+AA4odNhkkdCncUU4TqYUpiClGd3D4OkYRrqlQ== X-Received: by 2002:a05:6a00:8995:b0:705:bf53:aa8d with SMTP id d2e1a72fcca58-705d71c7ef2mr2298021b3a.3.1718362189063; Fri, 14 Jun 2024 03:49:49 -0700 (PDT) Received: from [10.84.144.49] ([203.208.167.148]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-705cc96b65asm2771727b3a.74.2024.06.14.03.49.45 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 14 Jun 2024 03:49:48 -0700 (PDT) Message-ID: Date: Fri, 14 Jun 2024 18:49:40 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH 0/3] asynchronously scan and free empty user PTE pages Content-Language: en-US To: David Hildenbrand Cc: hughd@google.com, willy@infradead.org, mgorman@suse.de, muchun.song@linux.dev, akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <02f8cbd0-8b2b-4c2d-ad96-f854d25bf3c2@redhat.com> <2cda0af6-8fde-4093-b615-7979744d6898@redhat.com> From: Qi Zheng In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: A0A0EC0022 X-Stat-Signature: uh45n3k477wi19aikza9rpndu9kx9za1 X-Rspam-User: X-HE-Tag: 1718362190-91519 X-HE-Meta: U2FsdGVkX1+h7Z8zClMaJrD1x1xz6ZOxjucFsUeaKyhKFiohLEzEe0PV5jH+7/7LVIF9roASg7Xc3upe5H8r4/A7HivNQNQ5NvU+rZLKy1nrbTQWFlCru5fPs17PBZzQ0tSD7UmeMBC9aKw8eR7MmXJXe635Pe+S1DH/JKkdHF1OSQSUF7xd97uPauEiMQRqSrv3BaJMOOMf5VFbXCEFreLgOJ1PxmiLNK/z/aLh2MJ9tasTcU8WCXplqNK8irLmz8fAWi5GrYZq8mOReOiZFk+nZIwXCC49awRJEUZ/hXTQ23oTky+Y56DZ6mY6gCAL+ni0isk6h6kTdrQNuLkzM1XXNMQs0fF/hSqTisStQvqN40fflytJgwyj4tPtfMEmiGMiY2e7NYGZzkeKGInzsFfkH0Latt7BCfktHMxiK/LHET8TifV9rvCrX+HJQvEQ7dAAJruM3T2ga+bC2WqxEukFEW+QXTIQcirY3K7QJd1gH8hHhPl7yABwz/tJVzqwggL8pPBG/FEmo+whgaWbT597Rb7QJBx4xULSYu0EIAP9tgWUxXxVyEn+PHpkkRx9T6Y6q/sN/jmSVRXMRWQAslJfdlGI1A0t+1O/PPhe5kPMsLh5lyN2E6DQNVo2FATgSyXOp9FPfK8QS2bGop9XNeVUB1BWE0er1zy2a4hOAfKk2pKfK3M7G0iOFttePKYBLMT7RwA+D2VlLPIgc9RpF8ZCxH9mwQHrtZ0zttzI143MT1VFVViZSmszCLjKzNK8ptLX09/NrJRjprlmCWCDWMsEcSRCuKo76o+iPOOodW77KCt057ooOYh4BkMM5/G7k8/pgBA3RdKELGNCXkl3+NnyvExO3uCmUvdlCDqm9dJiEfwgqtU0/LjhbE2b3WetB+x5LsEjYYG1HcsLmJoJHd24rD/8Jc+HFFEX2Ok/SN76QcVJw14e8YH6SqzSv3avG/NnujaKz+1W+Of/LqD kQURUfAW gicgDLkncYjkxB+D4huwBNql3DLv89iiAchZVXQEIYVmd7nQ5Yh6sPu+awsvjWRY26MQ/78I0/e1EYMMyliJ3dgiqkuwzqsX+yZg/o8TvUZqbM3pr7cekk08QM17XKnEaRM3xNqhkRtLdCLExiAc4BpvpBHXA92B00Ndmu3os9bS+jDhjvRWgIRvghPzZQEkPLyVLQRP8ynzDDMgNB0CipMMuZuQxqPDYprwG4E7YLHGpda6RfYdIXo7X/0421lN1cGUDer5G13I/TYzT0qdxB7n9t8BoXvwzr8gMFFhH5frrOEgayfO3PcZuVz6smonF5pVLndbgaUMG6joOlumeQ6B1DSTUZEULadHpxVjY+w7EQq0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi David, On 2024/6/14 15:53, David Hildenbrand wrote: >>> My thinking is, we start with a madvise(MADV_PT_RECLAIM) that will >>> synchronously try to reclaim page tables without any asynchronous work. >>> >>> Similar to MADV_COLLAPSE that only does synchronous work. Of course, >> >> This is feasible, but I worry that some user-mode programs may not be >> able to determine when to call it. > > Some yes, but others clearly :) Meaning, it's one step into the right > direction without having to worry about asynchronous work in the kernel > for now. That doesn't mean that asynchronous option is off the table. Got it. I will try to implement a synchronous madvise option in the next version. > >> >> My previous idea was to do something similar to madvise(MADV_HUGEPAGE), >> just mark the vma as being able to reclaim the pgtable, and then hand >> it over to the background thread for asynchronous reclaim. > > That's one option, although there might be workloads where you really > don't have to scan asynchronously and possibly repeatedly. > > For example, after virtio-mem discarded some memory it hotunplugged from > a VM using MADV_DONTNEED (in a sequence of multiple steps), it could > just setup a timer to free up page tables after a while exactly once. No > need to scan repeatedly / multiple times if virtio-mem didn't remove any > memory from a VM. > > For memory allocators it could be similar: trigger it once (from another > thread?) on a range after sufficient freeing happened. If the workload > is mostly idle, there might not be a need to free up memory. Thanks for adding this information! > > (mostly focused on anonymous memory + shmem for now. With file-backed > memory it might be different, but that has so far not been the biggest > consumer we saw regarding page tables.) OK. > > Of course, for real asynchronous/automatic scanning in the kernel, one > could try finding clues when scanning is reasonable: for example, mark > page tables that have been scanned and there was nothing to reclaim, and > mark page tables when modifying them. But such optimizations are rather > future work I guess, because devil is in the detail. Yes, we can optimize it step by step. > >> >>> if we don't need any heavy locking for reclaim, we might also just >>> try reclaiming during MADV_DONTNEED when spanning a complete page >> >> I think the lock held by the current solution is not too heavy and >> should be acceptable. >> >> But for MADV_FREE case, it still needs to be handled by >> madvise(MADV_PT_RECLAIM) or asynchronous work. > > Yes. Interestingly, reclaim code might be able to do that scanning + > reclaim if locking is cheap. Yes, I am also considering implementing another madvise option in the next version: mark the vma, then add its corresponding mm to a global list, and then traverse the list and reclaim it when the memory is tight and enters the system reclaim path. > >> >>> table. That won't sort out all cases where reclaim is possible, but >>> with both approaches we could cover quite a lot that were discovered >>> to really result in a lot of emprt page tables. >> [...] >>> >>> I pushed it to >>>       https://github.com/davidhildenbrand/linux/tree/page_table_reclaim >>> >>> I suspect it's a non-working version (and I assume the locking is >>> broken, there >>> are no VMA checks, etc), it's an old prototype. Just to give you an idea >>> about the >>> lockless scanning and how I started by triggering reclaim only when >>> kicked-off by >>> user space. >> >> Many thanks! But I'm worried that on some platforms disbaling the IRQ >> might be more expensive than holding the lock, such as arm64? Not sure. > > Scanning completely lockless (no mmap lock, not PT locks), means that -- > as long as there is not much to reclaim (for most workloads the common > case!) -- you would not affect the workload at all. > > Take a look at the khugepaged logic that does mmap_read_trylock(mm) and > makes sure to drop the mmap lock frequently due to > khugepaged_pages_to_scan, to not affect the workload too much while > scanning. > OK, I will take a look. >> >>> >>>> >>>>> >>>>> We'll have to double check whether all anon memory cases can >>>>> *properly* >>>>> handle pte_offset_map_lock() failing (not just handling it, but doing >>>>> the right thing; most of that anon-only code didn't ever run into that >>>>> issue so far, so these code paths were likely never triggered). >>>> >>>> Yeah, I'll keep checking this out too. >>>> >>>>> >>>>> >>>>>> For the path that will also free PTE pages in THP, we need to recheck >>>>>> whether the >>>>>> content of pmd entry is valid after holding pmd lock or pte lock. >>>>>> >>>>>> 4. TODO >>>>>> ======= >>>>>> >>>>>> Some applications may be concerned about the overhead of scanning and >>>>>> rebuilding >>>>>> page tables, so the following features are considered for >>>>>> implementation in the >>>>>> future: >>>>>> >>>>>>        - add per-process switch (via prctl) >>>>>>        - add a madvise option (like THP) >>>>>>        - add MM_PGTABLE_SCAN_DELAY/MM_PGTABLE_SCAN_SIZE control (via >>>>>> procfs file) >>>>>> Perhaps we can add the refcount to PTE pages in the future as well, >>>>>> which would >>>>>> help improve the scanning speed. >>>>> >>>>> I didn't like the added complexity last time, and the problem of >>>>> handling situations where we squeeze multiple page tables into a >>>>> single >>>>> "struct page". >>>> >>>> OK, except for refcount, do you think the other three todos above are >>>> still worth doing? >>> >>> I think the question is from where we start: for example, only >>> synchronous >>> reclaim vs. asynchonous reclaim. Synchronous reclaim won't really affect >>> workloads that do not actively trigger it, so it raises a lot less >>> eyebrows. ... >>> and some user space might have a good idea where it makes sense to >>> try to >>> reclaim, and when. >>> >>> So the other things you note here rather affect asynchronous reclaim, >>> and >>> might be reasonable in that context. But not sure if we should start >>> with doing >>> things asynchronously. >> >> I think synchronous and asynchronous have their own advantages and >> disadvantages, and are complementary. Perhaps they can be implemented at >> the same time? > > > No strong opinion, something synchronous sounds to me like the > low-hanging fruit, that could add the infrastructure to be used by > something more advanced/synchronously :) Got it, I will try to do the following in the next version. a. for MADV_DONTNEED case, try synchronous reclaim as you said b. for MADV_FREE case: - add a madvise option for synchronous reclaim - add another madvise option to mark the vma, then add its corresponding mm to a global list, and then traverse the list and reclaim it when the memory is tight and enters the system reclaim path. (maybe there is an option to unmark) c. for s390 case you mentioned, create a CONFIG_FREE_PT first, and then s390 will not select this config until the problem is solved. d. for lockless scan, try using disabling IRQ or (mmap read lock + pte_offset_map_nolock). Thanks, Qi >