From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5709C48BF6 for ; Thu, 29 Feb 2024 18:34:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3FD186B0093; Thu, 29 Feb 2024 13:34:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3AB766B0099; Thu, 29 Feb 2024 13:34:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2736A6B009D; Thu, 29 Feb 2024 13:34:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 1337E6B0093 for ; Thu, 29 Feb 2024 13:34:44 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 74796140458 for ; Thu, 29 Feb 2024 18:34:43 +0000 (UTC) X-FDA: 81845692446.13.D18A7D7 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf05.hostedemail.com (Postfix) with ESMTP id AB2E6100015 for ; Thu, 29 Feb 2024 18:34:41 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=HLtbOETN; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf05.hostedemail.com: domain of 3QM7gZQYKCJUNJO6zD5DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--yuzhao.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3QM7gZQYKCJUNJO6zD5DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--yuzhao.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709231681; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=pUh376UAClaxao6mn8dP1Rs/D93aswrImE1sZtkxksI=; b=2aRTDVkknKz1aZtKwXeAhdKPMrX5Sc1WYvkjhUHwqp/L+p4J7xk+GZrw/cjaBH+z+AhnAM +ROewu0+3hiLsOXdcqt0+a/+TYt+JDQILLeKLdsEwH68bLKW0F29EK88X+J3w0kTveYITq o/X/+FoVNFMDtJZkrJKV3ukzQclz8JQ= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=HLtbOETN; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf05.hostedemail.com: domain of 3QM7gZQYKCJUNJO6zD5DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--yuzhao.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3QM7gZQYKCJUNJO6zD5DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--yuzhao.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709231681; a=rsa-sha256; cv=none; b=pIqJRZjvixi6H9TS6F4LWULHoHG6QYz0bCYftwSEhyWdV1a7J+vFgGbRSWiAEToAHfdYE+ kgfGS0TIwCZup09wZhnXQrZsomA8+csYB6JiHGLkDpZQIfRvNj4zD7YW6jxVElxmVuJbzD M4zNOMqW7nbNj98qhQcC/7QiqMH+u7Y= Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6092bf785d7so22813847b3.0 for ; Thu, 29 Feb 2024 10:34:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709231681; x=1709836481; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=pUh376UAClaxao6mn8dP1Rs/D93aswrImE1sZtkxksI=; b=HLtbOETNvErtD04W7zB3y5V5rKPpXwIYvcx0X0h6f10Jd3/KORm0pmyMzLPAcKzFpL LxV+5ovjWvl23FLKvOG59zF66WsMMpt2bOZW/rKZQ9/CiAa9rMAdVifAwY7QTs9c9D2n DAZSH2Wcdbzuzgo684udU953VExJ2IB+KAazfcRhgmS/972GUpnR9ibaZ0Scf0qpRMFv RbBnSCJYM89CfurMlSCqppJ9UwU4sIZICSPn5KEa++0D53mI2VAhEUXRmxaY4UVjfBcS eCGSSG9GJZbCAJ3tVRrBKlP6F0ce3Q/mHmMY5BOjlxrAyRz5+hGXP9mmXnHKQ5MDFVRO ykgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709231681; x=1709836481; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=pUh376UAClaxao6mn8dP1Rs/D93aswrImE1sZtkxksI=; b=JVlVAhDHIRzosJY+RdohIV+Y5uhKS5GeJ4SkkNiq8MW5LHcsEJEIc7pFp5mjVVvzqx qA7zO0ucUd+rBC+RcBlyTdPgtgGqd4HIYeSwPqhbhvyylRtlBQQ8duYJzwaf6xRzTJ+K rZt9YXJsnKjOq70MaiGRdJA+PYFL/YQN9SKO2fFjPzVGz0AV6d9bqn+S7wcu74yc3ssp JcaZA3WaP1jwKSiq7cDiY0xdxEoPhPzl/f18q1SuB6Dv0mgF95x1U0bUHajZxHZ2Jii+ rwug/mpwxYbQmViAh6ychu57MRCz+y/dGntm9kX3r7kEQeNbTn6G2ImA7Xg0osFI1Rwm U81g== X-Gm-Message-State: AOJu0YzlrmDPORGDAXcL0kmx1C4kB+1D7bUSDASnj/ywKdnc4kzmLGDV oih1TfHmSNAje57yJ/pOMWTFKdw+h6L/fL6hzY04gG6YbsqippX8hDWB2AvpxmNsoTb5XeuggR7 wwQ== X-Google-Smtp-Source: AGHT+IERX7V9sht9jLbMZCuGJF1y9i8x29ZBZQwtrWVZUPVvx/Zo0qpEtbgh45Nep2d+6KTkofE6UvMDufU= X-Received: from yuzhao2.bld.corp.google.com ([100.64.188.49]) (user=yuzhao job=sendgmr) by 2002:a05:690c:3510:b0:608:22f1:c8e4 with SMTP id fq16-20020a05690c351000b0060822f1c8e4mr703221ywb.6.1709231680817; Thu, 29 Feb 2024 10:34:40 -0800 (PST) Date: Thu, 29 Feb 2024 11:34:32 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.44.0.rc1.240.g4c46232300-goog Message-ID: <20240229183436.4110845-1-yuzhao@google.com> Subject: [LSF/MM/BPF TOPIC] TAO: THP Allocator Optimizations From: Yu Zhao To: lsf-pc@lists.linux-foundation.org Cc: linux-mm@kvack.org, Jonathan Corbet , Yu Zhao Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: AB2E6100015 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: oprmid1qupxzjzrrp9ot3op1k4d6qxxc X-HE-Tag: 1709231681-462775 X-HE-Meta: U2FsdGVkX1/ZEfNZoxxtUyGzs1uGJZmvVXM0XVz68EpbvMUTvjxfnT5aN87alzNFdfV4EFvG1MbB4817GsxkB7JHFWochBHvR/noEB/Q7MLNwB7tJpk/Kq5K9IEszA9lQQka2kTew6Lk4mpsePryreC6eWB8Ca+vkrbnDA2ZO4C7e7Y6rfv/TFm8Xn4Dnz8i+4mpF4Vb/jlLzxNXgYYCujKTr5k/7CeSsmSZvnTlJw5ZCll/hvEHoClhso68jaqAGMnK5kE723sIUT7Ro5688qmGThbQHQzMw8v4xN8LuUBqIGTGfqH1d37KG5Cqpy6CxNgUMj/tR4OngY10HylO9LYfn5oZLqp326XO6qKMxqTOEKMc7eoNlE4yQcK/6ZjWtUvbQ1Urvoou73U/sHbLoPUWTB+V5Wi/JaHNG7kLb0YNPyWoQgguNb3tOyWFEKL3bSjvFzrSurCaCG+yeLUzHm7G10RAQ0C2f3g1MfvnXlzN8tXGvGwoj5cOE+iqPm6A/ib+cIfmRhALZz05azHeXjmG66gH3OQTy3M3Fz0v0gtc7tChpzWDMnjehTvGqpdwlnbmLJ5mXVYS5aaBMi2stxJiZP68zElIZSqmXNyyDPLNksT+l/P2rFHHp38Ol347Q0KEaPEy+SbsnwJ134NrQnacZNdfyJRkG9XC8+BVK7KhSHNWje0+DJ7xASIMjqH2DbvkH7n4JLWlFNTebEYorHi3VE0b5MZDeiRq0m090m2ycMfbn7NSuGMq+eEIXp/9nQzoXJoM9GwEqP2LfxckT5AUqvjPB/lprYGLDlzf6xbExLN652MVVJowlIlUcq1wzPvfDxWRh0CO5XBE/yZMPVSu2uCTrS0Jz+7ZyXXjdLF2c2O8LGvt855aKxfEE/0MBsgh5o/y7Cz9FWVbm6uZTLtbQqlED9FGHiIdVHHpUxMPiBFNZCxC1IuW+G7gFobPyp23PZ6eyrGctWfGjv+ IPbhzQA2 /JLwtHXPN8/PvMLjS0TRp6cyyDGjdCH/WEpsCfzcVUatoG2IVC1ZKFrLHuEpzwjGdnR+EbTfzzjklxbiR5pM7ILLc4u/QdTy+JvQPwheZ7F13gCfbzY8ajzfuMY4JmViqT2i1QOGybidHYR8RNpQrwPE82zWSh5mkhO8x91YjrjP/DVtVr34OyDyLu/KGpfMC7el1TvUSafRPoQVMnRpG5LS5YfVald/QvjD79j0ScOvLvfMOUIbRQGCo7wRUGTRr7UE+lLgujb8ZiohbLa3GrcWvndKxXAHbuH/yTejIL3wWGKdNWqGOxQUDSYqNw0bE/ploScSMPH+2bq4Ii7zwEPKJoeKHMzJxYqUSLRY4W2JuckVEAgVd35eFNQOk+0Tt6Bs12iOL6QwXhNyYVJigt0G+CHvY/71zC9oI6u2Guc4nNYri4iRmIb/D1xqPfM0YBpqma8OSEqy0RORrisI2G3Jt+o1qLKhqlzfOXDKVlzsNwmin3l3VNcjrpDNF9FnHAikfgKmXVIeWNWOms2i3ZakaSY4OsP/eqBgTZKKljvMJaXxBy5EhbwAiX75BvecmylT+KDoLLp1Ae6OokLQN4D/A+UyEqPtbU5/xBlZkUVfQTjG1M83i40tP/ZxqA8bdIJq/2ZzEDBfk42tI6PLXFMDe1OSWaB+wZqjpa+e72oqZQSqhV+ha5HGOdYE34v6hgtMu5cgIEC1PXbmzI1Tfd8s9SnqrL3qIvI4UymeTzzSuJzSTOcQbXnGk5Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000004, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: TAO is an umbrella project aiming at a better economy of physical contiguity viewed as a valuable resource. A few examples are: 1. A multi-tenant system can have guaranteed THP coverage while hosting abusers/misusers of the resource. 2. Abusers/misusers, e.g., workloads excessively requesting and then splitting THPs, should be punished if necessary. 3. Good citizens should be awarded with, e.g., lower allocation latency and less cost of metadata (struct page). 4. Better interoperability with userspace memory allocators when transacting the resource. This project puts the same emphasis on the established use case for servers and the emerging use case for clients so that client workloads like Android and ChromeOS can leverage the recent multi-sized THPs [1][2]. Chapter One introduces the cornerstone of TAO: an abstraction called policy (virtual) zones, which are overlayed on the physical zones. This is in line with item 1 above. A new door is open after Chapter One. The following two chapters discuss the reverse of THP collapsing, called THP shattering, and THP HVO, which brings the hugeTLB feature [3] to THP. They are in line with items 2 & 3 above. Advanced use cases are discussed in Epilogue, since they require the cooperation of userspace memory allocators. This is in line with item 4 above. [1] https://lwn.net/Articles/932386/ [2] https://lwn.net/Articles/937239/ [3] https://www.kernel.org/doc/html/next/mm/vmemmap_dedup.html Yu Zhao (4): THP zones: the use cases of policy zones THP shattering: the reverse of collapsing THP HVO: bring the hugeTLB feature to THP Profile-Guided Heap Optimization and THP fungibility .../admin-guide/kernel-parameters.txt | 10 + drivers/virtio/virtio_mem.c | 2 +- include/linux/gfp.h | 24 +- include/linux/huge_mm.h | 6 - include/linux/memcontrol.h | 5 + include/linux/mempolicy.h | 2 +- include/linux/mm.h | 140 ++++++ include/linux/mm_inline.h | 24 + include/linux/mm_types.h | 8 +- include/linux/mmzone.h | 53 +- include/linux/nodemask.h | 2 +- include/linux/rmap.h | 4 + include/linux/vm_event_item.h | 5 +- include/trace/events/mmflags.h | 4 +- init/main.c | 1 + mm/compaction.c | 12 + mm/gup.c | 3 +- mm/huge_memory.c | 304 ++++++++++-- mm/hugetlb_vmemmap.c | 2 +- mm/internal.h | 47 +- mm/madvise.c | 11 +- mm/memcontrol.c | 47 ++ mm/memory-failure.c | 2 +- mm/memory.c | 11 +- mm/mempolicy.c | 14 +- mm/migrate.c | 51 +- mm/mm_init.c | 452 ++++++++++-------- mm/page_alloc.c | 199 +++++++- mm/page_isolation.c | 2 +- mm/rmap.c | 21 +- mm/shmem.c | 4 +- mm/swap_slots.c | 3 +- mm/truncate.c | 6 +- mm/userfaultfd.c | 2 +- mm/vmscan.c | 41 +- mm/vmstat.c | 12 +- 36 files changed, 1194 insertions(+), 342 deletions(-) base-commit: d206a76d7d2726f3b096037f2079ce0bd3ba329b -- 2.44.0.rc1.240.g4c46232300-goog