From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41ED0C5B552 for ; Fri, 30 May 2025 09:29:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DA0936B009B; Fri, 30 May 2025 05:29:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D51976B009C; Fri, 30 May 2025 05:29:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C66BA6B009D; Fri, 30 May 2025 05:29:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A6ECE6B009B for ; Fri, 30 May 2025 05:29:15 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 4A296142B1B for ; Fri, 30 May 2025 09:29:15 +0000 (UTC) X-FDA: 83499050670.18.C8D0038 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by imf18.hostedemail.com (Postfix) with ESMTP id 626D31C0005 for ; Fri, 30 May 2025 09:29:13 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="Tgk7/yzW"; spf=pass (imf18.hostedemail.com: domain of libo.gcs85@bytedance.com designates 209.85.214.175 as permitted sender) smtp.mailfrom=libo.gcs85@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748597353; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qGQ/MSD2aq8Zt0uX+CveEn5QJtfmVtywDoJlykkJ748=; b=6mtDDqXB/c9j7KIOVRV+jj8mjNLrRIzOBDwwhASs8HF/SyZEk0WSgx/OeZoTUihMfKS9D+ UmHt4sgtnmdTur6F/o0W4chbEVSbvHiP7RYA9NUmLSg0M0XLAMUKL2QTSv+DA/vCFyez1p 9dB7pG9pLMFiRnipECmGa50obFf7I18= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="Tgk7/yzW"; spf=pass (imf18.hostedemail.com: domain of libo.gcs85@bytedance.com designates 209.85.214.175 as permitted sender) smtp.mailfrom=libo.gcs85@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748597353; a=rsa-sha256; cv=none; b=mui9PnmHaF1s1lAkEpuADeHKRzY19eo1qrTv1utmOYvXkl8XVLHA3LnQdw2oY+8jj2vDEx 7USsU15xDmkfCFYhbl+UjOx8wxIn5Qee4wsx8SeWHn1GDBBVGKyiSr4ckjuiXMSjfPrIy/ pk+VcwDfuYfxW/lmR7DuHJ0wsAFEkRo= Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-234fcadde3eso21982895ad.0 for ; Fri, 30 May 2025 02:29:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1748597352; x=1749202152; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qGQ/MSD2aq8Zt0uX+CveEn5QJtfmVtywDoJlykkJ748=; b=Tgk7/yzWZD8c7M8ESSM2aqHHtP2zpD4vYUxEQnudNXfntTE8Cq2dWAyH/PFmeHFAVk RtKT1EGwvC6ZBxwl94SJjFkogKXB342pVIFNGL9YTcx5tHpdsWZ6UtNRXdHth0d64bhy kLj37I2tuBxHldpFKyGFvcoatiX9Z4VV3rldYCQz8BYU1xTcYhTz0tDPMWUB0ROboUsn q9ePORn/cstDUztIgNmnELhF8Iv7Jw04AeUB4zPnXlT9G+5KAI5uWEaASPrhya4sj4lT 1SkPJ+jbsP0ORg3TN+HN4RWzOf+q2DaVJwCI+7TLwf4c3SnKaWlVTqjSqj/Pq7jZjfpq 3ZLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1748597352; x=1749202152; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qGQ/MSD2aq8Zt0uX+CveEn5QJtfmVtywDoJlykkJ748=; b=MjyWfITmuvAGtzRTrN43YcQS+fiS17I4+eGyytAVYxUvGP0GI1MY/I/zDTG0JexPDK xdoeIU2p9r2cixsxLhXxlGikPQdbIa8QYYwAohu4mwDshfn0R4D/JZotkiFTA3mZEbjb 9jJMPfXi43bQuc/iYWjgoqeGZWsqfBg1a1PULDx2gFe8WK/ZQgOjVGYFWhSQUote1Qup t49/zQU0OG8isseoMfn7lFppz7/tZRD6dcPJhni+7S7LnMcgDkjvyd0XfAyNJxqHsFG4 QbDCWf/xByzGd4I6hRbKu4HXR3Im3aDSCkDyzcJFzorcRpGxT+EKaFMy2PV33ZAZL9NL udKA== X-Forwarded-Encrypted: i=1; AJvYcCXkEzxJtR5cfO/sLfasnGESkGqZC0sW3M8fZ1wLBtr/d4uEd3AOKlS7eikfYdRs2KzRx18m9g+RxA==@kvack.org X-Gm-Message-State: AOJu0Yw20ET33Yb+ydb7guCFSm0tz5I5WOghxMcNJYNUW0dE6JMM+sjj cxkdvnRLJ19tEluC06c9PCTFSAo9FNjn2N5VRzDXi/EAIS+b8Zd57/JbIjS6+4yI2P8= X-Gm-Gg: ASbGncvl0fRTqC7PU5xjzspf3MfBxqq/+H112/w9WUwBhb4BCsn+vRm+4L3KmsQMbNJ +58lKXS+qOoD+FWfGDFZBG73bdDkAghIJdZXUz6kd3+HbAVSDk69GUF1qqoD53efqlkbaPy9nM1 RmxZ1NgKKeTwslPho4Nom+LS1sbpMzUgnG03eZdjaJbx45l7cxU9D7odn0H7sI1lyfYPZEZvS8i 8n8dpGaxR2efyFzNC8bVb1/n7+VMyVkYjjBFxkGqM2wIuqrgxkUcSlYhfrPjPppuwlIdsqo5/9r BmRbRyqDllpZqZolaviVozheH0OUIIkjaPXWBSsX/s6P6QcCUaqI8H0jnntLuZqlm8FwldPInGz LO5zAd/kENw== X-Google-Smtp-Source: AGHT+IHXNv6SBDXy0RRoZRael/cpWOT8NPAPr0x/A4zO15FUEF6dVXWmKj+6rOHNmQu7F4oEsyx/Vw== X-Received: by 2002:a17:90b:3a82:b0:311:ab20:159d with SMTP id 98e67ed59e1d1-3124163a55amr3859759a91.19.1748597351985; Fri, 30 May 2025 02:29:11 -0700 (PDT) Received: from FQ627FTG20.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3124e29f7b8sm838724a91.2.2025.05.30.02.28.57 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Fri, 30 May 2025 02:29:11 -0700 (PDT) From: Bo Li To: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, luto@kernel.org, kees@kernel.org, akpm@linux-foundation.org, david@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, peterz@infradead.org Cc: dietmar.eggemann@arm.com, hpa@zytor.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, viro@zeniv.linux.org.uk, brauner@kernel.org, jack@suse.cz, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com, jannh@google.com, pfalcato@suse.de, riel@surriel.com, harry.yoo@oracle.com, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, yinhongbo@bytedance.com, dengliang.1214@bytedance.com, xieyongji@bytedance.com, chaiwen.cc@bytedance.com, songmuchun@bytedance.com, yuanzhu@bytedance.com, chengguozhu@bytedance.com, sunjiadong.lff@bytedance.com, Bo Li Subject: [RFC v2 03/35] RPAL: add service registration interface Date: Fri, 30 May 2025 17:27:31 +0800 Message-Id: X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 626D31C0005 X-Stat-Signature: adkuxtos6z9ds7g4xdaqkj3zw3ze5gtm X-Rspam-User: X-HE-Tag: 1748597353-707726 X-HE-Meta: U2FsdGVkX18NWJbD7cAo4lB3dHolsjbCIvSGHvG4mHub+wE3ktmRBHgyI9B4KsLMS8gmuttBl5BFjmZbF3J2dVfwqBBe1ErLIzx1hixSXV4B1Z8A4AcslLtKioeeRxBGFGCNqwYWRanKNroVLylP3vEsHV7AIvogAJ6Ki2147Rv+An0WlCnvMVs6VgwKCVHtkzh2U60oR2PSmeKfnBDdQO5C3NCiqp0IvOxY3pa5eg5QdrSg9Cm7rCtsald8JyNUJSSROOVtVsHCnC2BUqk6V14+4E3DecSXYGQhMDVQMliQdCMPmu1ChSOizVQLcrtcUXVRaK6y0qeTMxPULd9op+7DS4ToAwA2O1o7s6PnPfD6iZhi0DH1p3GvoTe1kJA04z5lh1gd3sxgiq22qYIql24Sn/3uMMV6sDaLnczzqaEU0kJA5TvW8kjHFccPG73WfZp4RrkoB3d6bkedADClT23hK7Yw/F5UgSiUG8nm9l7yuRQXWuzNeIlm1Os6OFBfuwXpsdx8i7waL0imHZH9NX6bupqrkPAdu9oFrO08HUo7HOgjyiPkF7nOELFzF7UvQcY1RkilXMq8unxpxRDB6g7zgVPFWd1TnGmCHDu0phmFCnxMTyZcbk+ovXTLCQp0JqXjrwepUAiaO5jyA12hwtflJYq5I5HDh/6TkW26DuctDU9OEZpjzPxGRHSeOQ+nikUFVf3cNSWdFXQy4jMyjzOi/GOM52W179SvO0NGVNicsiI+QjXTGGXP1tuLftNKKln8a73BeTZ6KkHwO4lj9tO2//iOgmlu8VoUQdVBzdycAtBm1qwYXGaBfJ0QSS5QReAmIz7tMkhRfdlgsVhZiMvHzgl4ieSU2rwVPm+uUF5hWRLvp1Yee6qEoJ3niI4cVR0gSWGz/GavCkWykK4NyzVHpKbZRwyQjahJ+nMo+ton1aJPUt4zXJ7KVIu/caByT3/iAdpkH1fTMmW5JMv Eo2AXvFC 3tlgHBd+Qh5MJpXNQh6H2FbqYhRl2HsqN0SEhH3fvnEmErjrcQg8KlyEFkzcEAUEcWEJ8l5RFvRC0gtt+q1q0fo//e/tWWklSrpyuGQ/eOu2HKkAh0y0QrHBXxXtGa+KD2bR7ADzBsmJAo/Nh7W0bAWglu68rXGLLnY6EHO67uxngtYHs7uy7AaU8cnn1MOoLEmKCUGhLow2p4r8jhPH69tly5E9n8+2lUf7cHmS8TlH+jtuOqjrpPW3xn7+vkbFhwcBYLCP4ikDQ3UIfDRRk5o7NrNJaPHATENejq7dj0+xPj0afvkspuXwtMA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Every rpal service should be registered and managed. Each RPAL service has a 64-bit key as its unique identifier, the key should never repeat before kernel reboot. Each RPAL service has an ID to indicate which 512GB virtual address space it can use. Any alive RPAL service has its unique ID, which will never be reused until the service dead. This patch adds a registration interface for RPAL services. Newly registered rpal_service instances are assigned a key that increments starting from 1. The 64-bit length of the key ensures that keys are almost impossible to exhaust before system reboot. Meanwhile, a bitmap is used to allocate IDs, ensuring no duplicate IDs are assigned. RPAL services are managed via a hash list, which facilitates quick lookup of the corresponding rpal_service by key. Signed-off-by: Bo Li --- arch/x86/rpal/service.c | 130 ++++++++++++++++++++++++++++++++++++++++ include/linux/rpal.h | 31 ++++++++++ 2 files changed, 161 insertions(+) diff --git a/arch/x86/rpal/service.c b/arch/x86/rpal/service.c index c8e609798d4f..609c9550540d 100644 --- a/arch/x86/rpal/service.c +++ b/arch/x86/rpal/service.c @@ -13,13 +13,56 @@ #include "internal.h" +static DECLARE_BITMAP(rpal_id_bitmap, RPAL_NR_ID); +static atomic64_t service_key_counter; +static DEFINE_HASHTABLE(service_hash_table, ilog2(RPAL_NR_ID)); +DEFINE_SPINLOCK(hash_table_lock); static struct kmem_cache *service_cache; +static inline void rpal_free_service_id(int id) +{ + clear_bit(id, rpal_id_bitmap); +} + static void __rpal_put_service(struct rpal_service *rs) { kmem_cache_free(service_cache, rs); } +static int rpal_alloc_service_id(void) +{ + int id; + + do { + id = find_first_zero_bit(rpal_id_bitmap, RPAL_NR_ID); + if (id == RPAL_NR_ID) { + id = RPAL_INVALID_ID; + break; + } + } while (test_and_set_bit(id, rpal_id_bitmap)); + + return id; +} + +static bool is_valid_id(int id) +{ + return id >= 0 && id < RPAL_NR_ID; +} + +static u64 rpal_alloc_service_key(void) +{ + u64 key; + + /* confirm we do not run out keys */ + if (unlikely(atomic64_read(&service_key_counter) == _AC(-1, UL))) { + rpal_err("key is exhausted\n"); + return RPAL_INVALID_KEY; + } + + key = atomic64_fetch_inc(&service_key_counter); + return key; +} + struct rpal_service *rpal_get_service(struct rpal_service *rs) { if (!rs) @@ -37,6 +80,90 @@ void rpal_put_service(struct rpal_service *rs) __rpal_put_service(rs); } +static u32 get_hash_key(u64 key) +{ + return key % RPAL_NR_ID; +} + +struct rpal_service *rpal_get_service_by_key(u64 key) +{ + struct rpal_service *rs, *rsp; + u32 hash_key = get_hash_key(key); + + rs = NULL; + hash_for_each_possible(service_hash_table, rsp, hlist, hash_key) { + if (rsp->key == key) { + rs = rsp; + break; + } + } + return rpal_get_service(rs); +} + +static void insert_service(struct rpal_service *rs) +{ + unsigned long flags; + int hash_key; + + hash_key = get_hash_key(rs->key); + + spin_lock_irqsave(&hash_table_lock, flags); + hash_add(service_hash_table, &rs->hlist, hash_key); + spin_unlock_irqrestore(&hash_table_lock, flags); +} + +static void delete_service(struct rpal_service *rs) +{ + unsigned long flags; + + spin_lock_irqsave(&hash_table_lock, flags); + hash_del(&rs->hlist); + spin_unlock_irqrestore(&hash_table_lock, flags); +} + +struct rpal_service *rpal_register_service(void) +{ + struct rpal_service *rs; + + if (!rpal_inited) + return NULL; + + rs = kmem_cache_zalloc(service_cache, GFP_KERNEL); + if (!rs) + goto alloc_fail; + + rs->id = rpal_alloc_service_id(); + if (!is_valid_id(rs->id)) + goto id_fail; + + rs->key = rpal_alloc_service_key(); + if (unlikely(rs->key == RPAL_INVALID_KEY)) + goto key_fail; + + atomic_set(&rs->refcnt, 1); + + insert_service(rs); + + return rs; + +key_fail: + kmem_cache_free(service_cache, rs); +id_fail: + rpal_free_service_id(rs->id); +alloc_fail: + return NULL; +} + +void rpal_unregister_service(struct rpal_service *rs) +{ + if (!rs) + return; + + delete_service(rs); + + rpal_put_service(rs); +} + int __init rpal_service_init(void) { service_cache = kmem_cache_create("rpal_service_cache", @@ -47,6 +174,9 @@ int __init rpal_service_init(void) return -1; } + bitmap_zero(rpal_id_bitmap, RPAL_NR_ID); + atomic64_set(&service_key_counter, RPAL_FIRST_KEY); + return 0; } diff --git a/include/linux/rpal.h b/include/linux/rpal.h index 73468884cc5d..75c5acf33844 100644 --- a/include/linux/rpal.h +++ b/include/linux/rpal.h @@ -11,13 +11,40 @@ #include #include +#include #include #define RPAL_ERROR_MSG "rpal error: " #define rpal_err(x...) pr_err(RPAL_ERROR_MSG x) #define rpal_err_ratelimited(x...) pr_err_ratelimited(RPAL_ERROR_MSG x) +/* + * The first 512GB is reserved due to mmap_min_addr. + * The last 512GB is dropped since stack will be initially + * allocated at TASK_SIZE_MAX. + */ +#define RPAL_NR_ID 254 +#define RPAL_INVALID_ID -1 +#define RPAL_FIRST_KEY _AC(1, UL) +#define RPAL_INVALID_KEY _AC(0, UL) + +/* + * Each RPAL service has a 64-bit key as its unique identifier, and + * the 64-bit length ensures that the key will never repeat before + * the kernel reboot. + * + * Each RPAL service has an ID to indicate which 512GB virtual address + * space it can use. All alive RPAL processes have unique IDs, ensuring + * their address spaces do not overlap. When a process exits, its ID + * is released, allowing newly started RPAL services to reuse the ID. + */ struct rpal_service { + /* Unique identifier for RPAL service */ + u64 key; + /* virtual address space id */ + int id; + /* Hashtable list for this struct */ + struct hlist_node hlist; /* reference count of this struct */ atomic_t refcnt; }; @@ -40,4 +67,8 @@ struct rpal_service *rpal_get_service(struct rpal_service *rs); * @param rs The struct rpal_service to put. */ void rpal_put_service(struct rpal_service *rs); + +void rpal_unregister_service(struct rpal_service *rs); +struct rpal_service *rpal_register_service(void); +struct rpal_service *rpal_get_service_by_key(u64 key); #endif -- 2.20.1