From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 462DFC3ABC9 for ; Sat, 17 May 2025 00:07:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F29E06B0082; Fri, 16 May 2025 20:07:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EDA7E6B0083; Fri, 16 May 2025 20:07:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DC9616B0085; Fri, 16 May 2025 20:07:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id AFF736B0082 for ; Fri, 16 May 2025 20:07:47 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id EF5A412040C for ; Sat, 17 May 2025 00:07:47 +0000 (UTC) X-FDA: 83450461374.17.D1D23AD Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf01.hostedemail.com (Postfix) with ESMTP id 31B8040008 for ; Sat, 17 May 2025 00:07:45 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Tfu3SD3M; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of 3UNMnaAYKCN0RTQDMAFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--surenb.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3UNMnaAYKCN0RTQDMAFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--surenb.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1747440466; a=rsa-sha256; cv=none; b=CJKBM1AVk1xbOxkILKR6GvHRK5JRH7g8VCaO4oeuOTwOqRZ/8Dk1cI6o7xUcMR3zT94uOi xV2BeMTDlHbut+BxZQu69ESK7EIhkHLQbdI1p5UtD7mqL7Yx0ajspAisBiyahWSi2mY7mz 7ZTjzbpz3kltxhrH5cDgZ+syJ1htXqY= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Tfu3SD3M; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of 3UNMnaAYKCN0RTQDMAFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--surenb.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3UNMnaAYKCN0RTQDMAFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--surenb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1747440466; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=fdTg2aIq+yKN3X+ZqL5WTi+FS98hIL9CGjxbsmV+g0Q=; b=LH6YFtKX4m7fb3clszXzf+pb6BcW3SYxNNQu60JzIscZ6BNwmziGJEtj4rCD4duXd6eE5i 4SsQaHay3/05oY31llJvQdqQ4bgN/ufMe+mRXGGCW0718kK9il1hf5sw5baeqPP6tOpDSw L3n9uN0jypjL5k9TKznQ+5m/5YBvJ8Y= Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2320502023fso810265ad.2 for ; Fri, 16 May 2025 17:07:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747440465; x=1748045265; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=fdTg2aIq+yKN3X+ZqL5WTi+FS98hIL9CGjxbsmV+g0Q=; b=Tfu3SD3MDAj5N5bcEMMNBEIlrJUw+74CVFJfXjQRAYNDbZV7A25yjEDOHujbsXMSpg tVveChlVTTU2M8Xm0g473hY6z1xHc8eTalSlZyzKxNTIqzDkN5Fw5AKINHaDsDoF8rdE +7pjnkdztkQ3q6A7c83sK29ukqaBO0qLE+GFFVHigdAuSMd21ABGQKk0OUPq8H93nhDs 6jI5G/ztkwKgEyVoh0u89nOOX03ysQcTkl8nod1pkLHk7VcXR7kuZq9UcBib6rc6Bdi5 WfW44akvji8NgHcATKdMG6pZK475Yw4qo/UaxXuOqKptErjkiszkZ2QBRhHvQoNE2hvG AkvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747440465; x=1748045265; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=fdTg2aIq+yKN3X+ZqL5WTi+FS98hIL9CGjxbsmV+g0Q=; b=Lke0r1lQNXgUMSOMUDZzN7pZ+78EU2iuH18PVOwEu5XsPOud3pdg/EO7X1ImrbwbKF XyLlXGjM+TxicaXXt9vXlAA0u/jeq3VGpSOrMdEoONeOFhg+il2Xxc2DmjNtfo8NuhtD 9izOufaHMLfSJapEVY/LQKa/IPVRElY/HGAiElHnzFKc1MvbC7mcyZvmm92IBwnN7wth fMQhWFMnD0Tqr0ZkmyxdMpo5/eyZNmACwayxRLJtXbIO2BPkp3HnjxD0ilIyG92LdQDU 92eIum1CTfzTXIUc74sFC7vi0eyWPUR5s4Deb6ssHPlz8+TT5wpBa+Q805zW4XfrENYu Q1AQ== X-Forwarded-Encrypted: i=1; AJvYcCU7+9dbkdK+0FqHLn8rlvJcBe++kc/mjpuzWNPrQ+GnYYs2TngVWMV59PrbzxWENmMtNpRBwkV/uA==@kvack.org X-Gm-Message-State: AOJu0YwuRD0y/fCqCIB2IJuecTjDN1D5c11ZXNSC6L28PpiD/6+cv4fr sUd1m/gC6+5KyqpXA2l7/uOPfYaKxVsjeCI9H8mislR8r62sR38F5OFtX8lT0LKDxOyctdGnT/k 2pFaxaQ== X-Google-Smtp-Source: AGHT+IF4OSk8BpiEzrk7dV2VBPTsTg/7CEypyxMwwI1iF2ySUnXZwnrvr4pqvulJrcNA+KCIz4y5EUcqX9c= X-Received: from ploh6.prod.google.com ([2002:a17:902:f706:b0:22e:3312:15a5]) (user=surenb job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:1a68:b0:227:ac2a:1dd6 with SMTP id d9443c01a7336-231d45273c6mr86050615ad.24.1747440464816; Fri, 16 May 2025 17:07:44 -0700 (PDT) Date: Fri, 16 May 2025 17:07:39 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.49.0.1101.gccaa498523-goog Message-ID: <20250517000739.5930-1-surenb@google.com> Subject: [PATCH 1/1] alloc_tag: allocate percpu counters for module tags dynamically From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: kent.overstreet@linux.dev, 00107082@163.com, dennis@kernel.org, tj@kernel.org, cl@gentwo.org, pasha.tatashin@soleen.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, surenb@google.com Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 31B8040008 X-Rspam-User: X-Stat-Signature: q99z3aycmzutgmuueh33e38yjib76mr5 X-HE-Tag: 1747440465-17128 X-HE-Meta: U2FsdGVkX19mxg73PDS3ZYFkgtBQJ/M87HUMiyCIAc5Y3l/MGLVuRUpnwK9P0mSMT4H8HpMt3jHmIRksbioc1Y/sG65A51brhIzC239NJwVet0dRYH3u9QEMW4SFDQfel5EPBTZvgQtUWwkjiwojywadfs0JNvK/1EMXcDUlSFGTuy2jWzo7XCepOm0Wwv7WMJqqeFdXlt1W/jhPGRBv4O0noLGMeAlh4PIZVg7etzU265c3ncdYPiWcgalyRmKQlLeBB8MEq/Xz1ccXFXJu0n1rDg+w/GQ6Bo2l0dLKPYdDOKEKUW7B1VPXHM+nnJcKt+T+kk6p7DEZGqekIjOSC7IAYZlukZxb+3+tgJPBWJMXlKPkbGNGBNhBzxYhj33KpPC6nYsFOIcd8toiHDcNCn2Pv5O48ZW1L+PvEaWaZ8bbdO1TpUiQm5yHRtT0Z40t5nzUHCjFJbCUCXVWF1nncM8cC0Gg0U73Qmo8x485egEprkT6x1l8XlpEJAW+9yOjiL5F55HbpJnCETTuUsrskBTK0LbA9wuD+/V5K5BKN8J05A+N296ht3U9dcYjYMJYQLCaAnZ5pL4lVClzG+2vEIuzH7RMju2W6V5G3ylwuEaTYFLATSystrDefTYnEQ5NT6EH+8wfuKOIRevrT/SP21Rro1Z5RUQ+TSQfeiOCTgmce2rbGPV8X6A39AqtyX3GvnGQK87KRs30Tlr24WOe0c2AUETyIV1FChNJbMZ+RMmnXvhyUgOsBeTnLct8kxyb898dYNkrDHYnPo2DxAb/GUnxIHElMhg5wF4+Hcc3TPCgKWXrteQGMxXBS+iGZa8z1i03l1KcSA+3Fmk1X5xAZhSylVv3r1xXxzIyXniiL1VZn/HyQDycOh1uTjr+hGRtOnBcGjpm18frpODTVfyIczOqzFYyOX+DmI25T5wQiJvMWfNqngRHCP3oSfgs9BM0RXJm9v7x7an0wj3Ope3 C7e1zOEF zmR93EhbMzfQaxXvsUS2MAmlscJDuP1hzx2HlTGK4r5Y8+RQtdSgc8KGb+pLdcVVfpBH6860HVBKkFv5lLZeTh5gzBKcBWGfEv1mgA+JJ1/Ol5YVnMXSOdO49gAxMcWw0qabVQ8V2m43VopFNqyZKfKrLymYsSYEkUQPqzXHSgwZpthRE1jf1RVoAZ5ycAuBWertjZMmMiHBUkpNy2LArD/6/1hDC8EZEZSqsTmgLCiaeNmqkiE+/9AxzghrtHQS3FRiggi1XF/6YpfYrd4xWqytj0KG77LxkR8gRCLshh1P7UPhnXA6tJEVKvUYmHfUIBp64z5m/PRkUM0LSeGxesuUfjgg1Ns5VR84AJLlzO9wcQIRmTV1qiFkV04CawY6Za4+an/Vt2XUYYJr8BiyCieEiL049ZAJGNgrTwr5sNlheCq1v5Qa6wFPhOeXcqykSkyvt7mGBnRLeE6wthLlgKsou595j67VCYSHGQA7OSA0/GkbiVvdXwuLENwoBqmupjPI6V+Sti5SeIOUjedNmS5Ii4uyaHmbFOJYu7L9hrfl4fntf6bf/Wwj071YZ1uFnRTAGKGpWqJHK4GHX789SCaIDSeTjlEe7KRvHjhzUekKVB5bEPVXss2AjdBoZ1Ms6BzryXHuTJxlkD27E0YEL65/PWi3Trx0pqvZ4FdzKv2sDmSrJqWvQP78YFFyPVgjYxN0bbvRvIdYWvV+yIWDT8F/WRw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When a module gets unloaded it checks whether any of its tags are still in use and if so, we keep the memory containing module's allocation tags alive until all tags are unused. However percpu counters referenced by the tags are freed by free_module(). This will lead to UAF if the memory allocated by a module is accessed after module was unloaded. To fix this we allocate percpu counters for module allocation tags dynamically and we keep it alive for tags which are still in use after module unloading. This also removes the requirement of a larger PERCPU_MODULE_RESERVE when memory allocation profiling is enabled because percpu memory for counters does not need to be reserved anymore. Fixes: 0db6f8d7820a ("alloc_tag: load module tags into separate contiguous memory") Reported-by: David Wang <00107082@163.com> Closes: https://lore.kernel.org/all/20250516131246.6244-1-00107082@163.com/ Signed-off-by: Suren Baghdasaryan --- include/linux/alloc_tag.h | 12 ++++++ include/linux/codetag.h | 8 ++-- include/linux/percpu.h | 4 -- lib/alloc_tag.c | 87 +++++++++++++++++++++++++++++++-------- lib/codetag.c | 5 ++- 5 files changed, 88 insertions(+), 28 deletions(-) diff --git a/include/linux/alloc_tag.h b/include/linux/alloc_tag.h index a946e0203e6d..8f7931eb7d16 100644 --- a/include/linux/alloc_tag.h +++ b/include/linux/alloc_tag.h @@ -104,6 +104,16 @@ DECLARE_PER_CPU(struct alloc_tag_counters, _shared_alloc_tag); #else /* ARCH_NEEDS_WEAK_PER_CPU */ +#ifdef MODULE + +#define DEFINE_ALLOC_TAG(_alloc_tag) \ + static struct alloc_tag _alloc_tag __used __aligned(8) \ + __section(ALLOC_TAG_SECTION_NAME) = { \ + .ct = CODE_TAG_INIT, \ + .counters = NULL }; + +#else /* MODULE */ + #define DEFINE_ALLOC_TAG(_alloc_tag) \ static DEFINE_PER_CPU(struct alloc_tag_counters, _alloc_tag_cntr); \ static struct alloc_tag _alloc_tag __used __aligned(8) \ @@ -111,6 +121,8 @@ DECLARE_PER_CPU(struct alloc_tag_counters, _shared_alloc_tag); .ct = CODE_TAG_INIT, \ .counters = &_alloc_tag_cntr }; +#endif /* MODULE */ + #endif /* ARCH_NEEDS_WEAK_PER_CPU */ DECLARE_STATIC_KEY_MAYBE(CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT, diff --git a/include/linux/codetag.h b/include/linux/codetag.h index d14dbd26b370..0ee4c21c6dbc 100644 --- a/include/linux/codetag.h +++ b/include/linux/codetag.h @@ -36,10 +36,10 @@ union codetag_ref { struct codetag_type_desc { const char *section; size_t tag_size; - void (*module_load)(struct codetag_type *cttype, - struct codetag_module *cmod); - void (*module_unload)(struct codetag_type *cttype, - struct codetag_module *cmod); + void (*module_load)(struct module *mod, + struct codetag *start, struct codetag *end); + void (*module_unload)(struct module *mod, + struct codetag *start, struct codetag *end); #ifdef CONFIG_MODULES void (*module_replaced)(struct module *mod, struct module *new_mod); bool (*needs_section_mem)(struct module *mod, unsigned long size); diff --git a/include/linux/percpu.h b/include/linux/percpu.h index 52b5ea663b9f..85bf8dd9f087 100644 --- a/include/linux/percpu.h +++ b/include/linux/percpu.h @@ -15,11 +15,7 @@ /* enough to cover all DEFINE_PER_CPUs in modules */ #ifdef CONFIG_MODULES -#ifdef CONFIG_MEM_ALLOC_PROFILING -#define PERCPU_MODULE_RESERVE (8 << 13) -#else #define PERCPU_MODULE_RESERVE (8 << 10) -#endif #else #define PERCPU_MODULE_RESERVE 0 #endif diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c index 25ecc1334b67..c7f602fa7b23 100644 --- a/lib/alloc_tag.c +++ b/lib/alloc_tag.c @@ -350,18 +350,28 @@ static bool needs_section_mem(struct module *mod, unsigned long size) return size >= sizeof(struct alloc_tag); } -static struct alloc_tag *find_used_tag(struct alloc_tag *from, struct alloc_tag *to) +static bool clean_unused_counters(struct alloc_tag *start_tag, + struct alloc_tag *end_tag) { - while (from <= to) { + struct alloc_tag *tag; + bool ret = true; + + for (tag = start_tag; tag <= end_tag; tag++) { struct alloc_tag_counters counter; - counter = alloc_tag_read(from); - if (counter.bytes) - return from; - from++; + if (!tag->counters) + continue; + + counter = alloc_tag_read(tag); + if (!counter.bytes) { + free_percpu(tag->counters); + tag->counters = NULL; + } else { + ret = false; + } } - return NULL; + return ret; } /* Called with mod_area_mt locked */ @@ -371,12 +381,16 @@ static void clean_unused_module_areas_locked(void) struct module *val; mas_for_each(&mas, val, module_tags.size) { + struct alloc_tag *start_tag; + struct alloc_tag *end_tag; + if (val != &unloaded_mod) continue; /* Release area if all tags are unused */ - if (!find_used_tag((struct alloc_tag *)(module_tags.start_addr + mas.index), - (struct alloc_tag *)(module_tags.start_addr + mas.last))) + start_tag = (struct alloc_tag *)(module_tags.start_addr + mas.index); + end_tag = (struct alloc_tag *)(module_tags.start_addr + mas.last); + if (clean_unused_counters(start_tag, end_tag)) mas_erase(&mas); } } @@ -561,7 +575,8 @@ static void *reserve_module_tags(struct module *mod, unsigned long size, static void release_module_tags(struct module *mod, bool used) { MA_STATE(mas, &mod_area_mt, module_tags.size, module_tags.size); - struct alloc_tag *tag; + struct alloc_tag *start_tag; + struct alloc_tag *end_tag; struct module *val; mas_lock(&mas); @@ -575,15 +590,22 @@ static void release_module_tags(struct module *mod, bool used) if (!used) goto release_area; - /* Find out if the area is used */ - tag = find_used_tag((struct alloc_tag *)(module_tags.start_addr + mas.index), - (struct alloc_tag *)(module_tags.start_addr + mas.last)); - if (tag) { - struct alloc_tag_counters counter = alloc_tag_read(tag); + start_tag = (struct alloc_tag *)(module_tags.start_addr + mas.index); + end_tag = (struct alloc_tag *)(module_tags.start_addr + mas.last); + if (!clean_unused_counters(start_tag, end_tag)) { + struct alloc_tag *tag; + + for (tag = start_tag; tag <= end_tag; tag++) { + struct alloc_tag_counters counter; + + if (!tag->counters) + continue; - pr_info("%s:%u module %s func:%s has %llu allocated at module unload\n", - tag->ct.filename, tag->ct.lineno, tag->ct.modname, - tag->ct.function, counter.bytes); + counter = alloc_tag_read(tag); + pr_info("%s:%u module %s func:%s has %llu allocated at module unload\n", + tag->ct.filename, tag->ct.lineno, tag->ct.modname, + tag->ct.function, counter.bytes); + } } else { used = false; } @@ -596,6 +618,34 @@ static void release_module_tags(struct module *mod, bool used) mas_unlock(&mas); } +static void load_module(struct module *mod, struct codetag *start, struct codetag *stop) +{ + /* Allocate module alloc_tag percpu counters */ + struct alloc_tag *start_tag; + struct alloc_tag *stop_tag; + struct alloc_tag *tag; + + if (!mod) + return; + + start_tag = ct_to_alloc_tag(start); + stop_tag = ct_to_alloc_tag(stop); + for (tag = start_tag; tag < stop_tag; tag++) { + WARN_ON(tag->counters); + tag->counters = alloc_percpu(struct alloc_tag_counters); + if (!tag->counters) { + while (--tag >= start_tag) { + free_percpu(tag->counters); + tag->counters = NULL; + } + shutdown_mem_profiling(true); + pr_err("Failed to allocate memory for allocation tag percpu counters in the module %s. Memory allocation profiling is disabled!\n", + mod->name); + break; + } + } +} + static void replace_module(struct module *mod, struct module *new_mod) { MA_STATE(mas, &mod_area_mt, 0, module_tags.size); @@ -757,6 +807,7 @@ static int __init alloc_tag_init(void) .needs_section_mem = needs_section_mem, .alloc_section_mem = reserve_module_tags, .free_section_mem = release_module_tags, + .module_load = load_module, .module_replaced = replace_module, #endif }; diff --git a/lib/codetag.c b/lib/codetag.c index 42aadd6c1454..de332e98d6f5 100644 --- a/lib/codetag.c +++ b/lib/codetag.c @@ -194,7 +194,7 @@ static int codetag_module_init(struct codetag_type *cttype, struct module *mod) if (err >= 0) { cttype->count += range_size(cttype, &range); if (cttype->desc.module_load) - cttype->desc.module_load(cttype, cmod); + cttype->desc.module_load(mod, range.start, range.stop); } up_write(&cttype->mod_lock); @@ -333,7 +333,8 @@ void codetag_unload_module(struct module *mod) } if (found) { if (cttype->desc.module_unload) - cttype->desc.module_unload(cttype, cmod); + cttype->desc.module_unload(cmod->mod, + cmod->range.start, cmod->range.stop); cttype->count -= range_size(cttype, &cmod->range); idr_remove(&cttype->mod_idr, mod_id); base-commit: 6e68a205c07b3c311f19a4c9c5de95d191d5a459 -- 2.49.0.1101.gccaa498523-goog