From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 560EEC77B61 for ; Mon, 1 May 2023 16:56:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E7CD8900012; Mon, 1 May 2023 12:56:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E2CF7900002; Mon, 1 May 2023 12:56:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CCD4C900012; Mon, 1 May 2023 12:56:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id BBC2F900002 for ; Mon, 1 May 2023 12:56:30 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 8740980819 for ; Mon, 1 May 2023 16:56:30 +0000 (UTC) X-FDA: 80742289740.01.FC98299 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf04.hostedemail.com (Postfix) with ESMTP id A96FD4000D for ; Mon, 1 May 2023 16:56:28 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=6G5OUnFq; spf=pass (imf04.hostedemail.com: domain of 3O-9PZAYKCIIy0xkthmuumrk.iusrot03-ssq1giq.uxm@flex--surenb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3O-9PZAYKCIIy0xkthmuumrk.iusrot03-ssq1giq.uxm@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1682960188; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lw2pXEBNfkStSvJmyTF2I1dH+bsBIyZ0OpKs2HHzIO8=; b=sTMR74W8gYfMLzQmrbCeGd/Ai20mcfNwP3ulItOCFy4EsRvl2fANnfNPSm/kQNkRTs5YYN oXtYqfUJntZU15sNqoAlfnDPPZjLCrX/ioQ/UgKN5TOtPyzsC1VihXej/AMhVhxknl7bGk htMYuA+TsDQZJjTjruP7op/xhd0Eftc= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=6G5OUnFq; spf=pass (imf04.hostedemail.com: domain of 3O-9PZAYKCIIy0xkthmuumrk.iusrot03-ssq1giq.uxm@flex--surenb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3O-9PZAYKCIIy0xkthmuumrk.iusrot03-ssq1giq.uxm@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1682960188; a=rsa-sha256; cv=none; b=XeZchxnuuS2Cg46pltTXMQYvFihYjTfSrjmS3SLS+hXmpaVhwCXeL2G94D3sMfGmENPkfu kawOrFjjzCuSVXj1qpPLxFgXiheo8zT/fGLbBPVV0wbUM5Aym6+GS/iSVyjkYw2WqVFTrK TO/vtaoZJ7QPNg1d/tnWgc0tr97mOTE= Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-559fb5bed89so20701657b3.0 for ; Mon, 01 May 2023 09:56:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1682960188; x=1685552188; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=lw2pXEBNfkStSvJmyTF2I1dH+bsBIyZ0OpKs2HHzIO8=; b=6G5OUnFqmhYzObxhYE56pVPUPb5/Z5fFLyg/xv1k1ELzqtOEByasEUKmpVLdsnIlKz +HpmKICNn1SXskm4W84Z1YnQTS1wv+HkfBClJgQjOTklB/JflZpG6ZVI9CkbEHZZ4VtX +VU0ocTLcHc6zEEkMfHwEXLiatB8YSa4vtZi59KquEiZLQ8PGIiDrQ9uUD2CTQfE2G7r IqwFOlnQ0w6JmNavUX7vfGowmYc1Kgs8nx0nF3FhUsDCRqWZuUIsUlCL/KEMxoHEQqWr IxEVdl0XX/ZNM2Oo2w3beOsDv/rdq05Vn6EfHRE3NqnRnISnGzk4Nz7tXYADK5fB+Pd5 tQVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682960188; x=1685552188; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=lw2pXEBNfkStSvJmyTF2I1dH+bsBIyZ0OpKs2HHzIO8=; b=jgTWFY0OvD3Gth8vLOss4E9Lz9RSFdlXexy0YM5R3rR84TlkAKbBH3ZOfnsB2eiyRZ /G7dRX6K+4UmC81g2Qwb50ttcBxfSK9b1wvtXAHozwyGEskgZIr+B7qtw8+swo63nKYx 0Nx6Ti927YzV7sv/FgoH8h/IFwSw7NsNh+DfgbsJ/RQVXtuPSZ8jsqe8klNICdW7x85W 6XOxffrgxnDKpQOWcDqAvp/ngPuaaIIhuYNpFGqbFkcCdGyjxHlmWezEz+FQtxHaeKNB qg++bjnG7uG7k1vgwn2Nfah1+kiKxw3Hf5tA1gW7mcfoQr075IG356//GETm+z/mvyTM hRng== X-Gm-Message-State: AC+VfDw/T8oO5WWI91ZBJMoYQCgsMWTBfHop3sisXwvCpoVVPhA8e0uL fYvuXOnctyUHJ+1P8eU8JN8f4Ss4/DM= X-Google-Smtp-Source: ACHHUZ4jn05QWo1AuUjIBZxdqElOhbGVaSLvkqDYZu4aVA/YTY7kkMHxf30EuctFEiweZjTNT8dvorYRrv0= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:6d24:3efd:facc:7ac4]) (user=surenb job=sendgmr) by 2002:a81:4304:0:b0:55a:c44:6151 with SMTP id q4-20020a814304000000b0055a0c446151mr3660331ywa.5.1682960187794; Mon, 01 May 2023 09:56:27 -0700 (PDT) Date: Mon, 1 May 2023 09:54:45 -0700 In-Reply-To: <20230501165450.15352-1-surenb@google.com> Mime-Version: 1.0 References: <20230501165450.15352-1-surenb@google.com> X-Mailer: git-send-email 2.40.1.495.gc816e09b53d-goog Message-ID: <20230501165450.15352-36-surenb@google.com> Subject: [PATCH 35/40] lib: implement context capture support for tagged allocations From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: kent.overstreet@linux.dev, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, roman.gushchin@linux.dev, mgorman@suse.de, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, corbet@lwn.net, void@manifault.com, peterz@infradead.org, juri.lelli@redhat.com, ldufour@linux.ibm.com, catalin.marinas@arm.com, will@kernel.org, arnd@arndb.de, tglx@linutronix.de, mingo@redhat.com, dave.hansen@linux.intel.com, x86@kernel.org, peterx@redhat.com, david@redhat.com, axboe@kernel.dk, mcgrof@kernel.org, masahiroy@kernel.org, nathan@kernel.org, dennis@kernel.org, tj@kernel.org, muchun.song@linux.dev, rppt@kernel.org, paulmck@kernel.org, pasha.tatashin@soleen.com, yosryahmed@google.com, yuzhao@google.com, dhowells@redhat.com, hughd@google.com, andreyknvl@gmail.com, keescook@chromium.org, ndesaulniers@google.com, gregkh@linuxfoundation.org, ebiggers@google.com, ytcoode@gmail.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, bristot@redhat.com, vschneid@redhat.com, cl@linux.com, penberg@kernel.org, iamjoonsoo.kim@lge.com, 42.hyeyoo@gmail.com, glider@google.com, elver@google.com, dvyukov@google.com, shakeelb@google.com, songmuchun@bytedance.com, jbaron@akamai.com, rientjes@google.com, minchan@google.com, kaleshsingh@google.com, surenb@google.com, kernel-team@android.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, kasan-dev@googlegroups.com, cgroups@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: sqe5pxwkc736ysqhpi4xt1q8d7upomgj X-Rspamd-Queue-Id: A96FD4000D X-HE-Tag: 1682960188-677493 X-HE-Meta: U2FsdGVkX18EQ5Pf1BEbeOeiGIyggiFFoTuxzXRfoX1usc5jqWN6qpWvFM8YINzK4ojj4VR9vREXDLN5fO9/Cb4Qg5IVUTVKdsKf42OU21hmyeYeXGLbYKMfiZRFF+hRnUtIBjwv6ja+m3ibu7Sk7sgHVa26xc9gzEvo2V86lkxlN5TEXyg+KkqXzKY7AH2Vgu7pgD9OnnCso9xmAYGDqZ7lYjUz6koVzBN4sPqKu29A2oxXDZJierpc9hOUEiy3HxY3174+S0LMC9ykZgZxvc2B71W4yymIe1DQn4jwYpVKIW6k4U2OcP13IOkEFruIIUBGObDENvVmU6WpCgcnZXpxsA/l2oDAbw/5fFUGeFTc5aVoFpL8ovY5VJLqO/SAOVNDf+MJ2GvhzeDnWkcy0LkXuk60saRwkuKn9bT4/fr9W8sj3Ud+vja4HqiJVSYQ/wOzCrzib6/XQex14pGdz3tiBmlalhAV6YsY3rp2ZkWXPsz40DtmSnREp7zHVLt8LC5mW/cFlbh0ltybHBm8r2VkB79Ny8q0rwdLVwEBUVuRZATNv8iG2C7Eg4QkyNkKw6WQ9A1vMiVRM5vd9Fko5TUgYrR/BrTWCdKgVFzNptS7Oy8r2DNr1bvXTxF+S+L8u/2VdAHUMra6huJUNLKYZhBJ29k3x9U/apwHueUXA+qn7pOm0aPqK9Pxzlas3hmdSbjyW5mifCk9EzNxLwNMazMAC27cCdb0Mo06XFABssJrWM97tDU94q1pFlgMtx6+t0AXGah/bgkk3QLAXGHr/Ww5BYpXyBZBhjL5/48xyJh3/WJI7XNhLondpocX7mDwH/LcUhcCLVR3FIX1X5VfcbzWmLafo+hIMTGI7y0Mh7isw7B0bN5U0f3OKaD9+m1i9rheI6MOG6+k8PQnpARHJrJDp+QLGutM3bLmhNvbPffwLMJvcSXveMIpuK2EZFV7ZL/qZQxEtoHIfBUzfAi 6y8oWfZY 2vji92t4MBYuZIoTTHWHCbRHrsIaa1snDzyVjKWvJl8f2AR/gnFpxI68kFF6b7oXN0nmg84Wt7TJz/1j+mqr6rcDLreeiFRL8qV6OnsYWdL9AOv014vJMhthHGyGzMshnPbsEFM5o13Jmx86Ak4c9uPbitxDTevcGB7YXM7x7yY3lCaikY+SYjK/B2v9bQOC+9kRx8dlFt95lDGeNOp+HTXa1p0wMkzzkjnAX9gz7yy7RSOHRpGXAyyVuSlshQvT9KpE/S3YdKDGsNZE5pS5uGRB99NF1obZ6aShzsqPMGSXvoPex6JThYenBhDTyuBRffWGgbG1r+uO4Bd+ja6PN1KrAZ0XT82bucTKcom/qTk5jlcXH6enPfa/qTXbKyYNZeNM/Dj9J1K2VtnKxeJ752NITmeYxhz5Sw2ivuFfK4m9qONkBqCA+r+n9dnhDKZaDO0UlSuUktR/b+SOWpSBxaQP+2eiCLx8hO4TgufUuhbIq51VG4RmL76iV3UIvF8JKukXtpVI+wqQfTE5/9sFB49MNqtmJ5CFEPSc/zzKMjcwiiveBQtECfpxlsYLzT7tkWS7NKbbHRzh8K1Ibx1CDd4Qfgc43YG5CiJ2TuBH3ujgR4Km//QzVScsgdOeJ4Y6p1wr5GoT/QZKnbCH6at2SSXGd7a8Tnb2rkbJauRqh1kEjK7A= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Implement mechanisms for capturing allocation call context which consists of: - allocation size - pid, tgid and name of the allocating task - allocation timestamp - allocation call stack The patch creates allocations.ctx file which can be written to enable/disable context capture for a specific code tag. Captured context can be obtained by reading allocations.ctx file. Usage example: echo "file include/asm-generic/pgalloc.h line 63 enable" > \ /sys/kernel/debug/allocations.ctx cat allocations.ctx 91.0MiB 212 include/asm-generic/pgalloc.h:63 module:pgtable func:__pte_alloc_one size: 4096 pid: 1551 tgid: 1551 comm: cat ts: 670109646361 call stack: pte_alloc_one+0xfe/0x130 __pte_alloc+0x22/0x90 move_page_tables.part.0+0x994/0xa60 shift_arg_pages+0xa4/0x180 setup_arg_pages+0x286/0x2d0 load_elf_binary+0x4e1/0x18d0 bprm_execve+0x26b/0x660 do_execveat_common.isra.0+0x19d/0x220 __x64_sys_execve+0x2e/0x40 do_syscall_64+0x38/0x90 entry_SYSCALL_64_after_hwframe+0x63/0xcd size: 4096 pid: 1551 tgid: 1551 comm: cat ts: 670109711801 call stack: pte_alloc_one+0xfe/0x130 __do_fault+0x52/0xc0 __handle_mm_fault+0x7d9/0xdd0 handle_mm_fault+0xc0/0x2b0 do_user_addr_fault+0x1c3/0x660 exc_page_fault+0x62/0x150 asm_exc_page_fault+0x22/0x30 ... echo "file include/asm-generic/pgalloc.h line 63 disable" > \ /sys/kernel/debug/alloc_tags.ctx Note that disabling context capture will not clear already captured context but no new context will be captured. Signed-off-by: Suren Baghdasaryan --- include/linux/alloc_tag.h | 25 +++- include/linux/codetag.h | 3 +- include/linux/pgalloc_tag.h | 4 +- lib/Kconfig.debug | 1 + lib/alloc_tag.c | 238 +++++++++++++++++++++++++++++++++++- lib/codetag.c | 20 +-- 6 files changed, 272 insertions(+), 19 deletions(-) diff --git a/include/linux/alloc_tag.h b/include/linux/alloc_tag.h index 07922d81b641..2a3d248aae10 100644 --- a/include/linux/alloc_tag.h +++ b/include/linux/alloc_tag.h @@ -17,20 +17,29 @@ * an array of these. Embedded codetag utilizes codetag framework. */ struct alloc_tag { - struct codetag ct; + struct codetag_with_ctx ctc; struct lazy_percpu_counter bytes_allocated; } __aligned(8); #ifdef CONFIG_MEM_ALLOC_PROFILING +static inline struct alloc_tag *ctc_to_alloc_tag(struct codetag_with_ctx *ctc) +{ + return container_of(ctc, struct alloc_tag, ctc); +} + static inline struct alloc_tag *ct_to_alloc_tag(struct codetag *ct) { - return container_of(ct, struct alloc_tag, ct); + return container_of(ct_to_ctc(ct), struct alloc_tag, ctc); } +struct codetag_ctx *alloc_tag_create_ctx(struct alloc_tag *tag, size_t size); +void alloc_tag_free_ctx(struct codetag_ctx *ctx, struct alloc_tag **ptag); +bool alloc_tag_enable_ctx(struct alloc_tag *tag, bool enable); + #define DEFINE_ALLOC_TAG(_alloc_tag, _old) \ static struct alloc_tag _alloc_tag __used __aligned(8) \ - __section("alloc_tags") = { .ct = CODE_TAG_INIT }; \ + __section("alloc_tags") = { .ctc.ct = CODE_TAG_INIT }; \ struct alloc_tag * __maybe_unused _old = alloc_tag_save(&_alloc_tag) extern struct static_key_true mem_alloc_profiling_key; @@ -54,7 +63,10 @@ static inline void __alloc_tag_sub(union codetag_ref *ref, size_t bytes, if (!ref || !ref->ct) return; - tag = ct_to_alloc_tag(ref->ct); + if (is_codetag_ctx_ref(ref)) + alloc_tag_free_ctx(ref->ctx, &tag); + else + tag = ct_to_alloc_tag(ref->ct); if (may_allocate) lazy_percpu_counter_add(&tag->bytes_allocated, -bytes); @@ -88,7 +100,10 @@ static inline void alloc_tag_add(union codetag_ref *ref, struct alloc_tag *tag, if (!ref || !tag) return; - ref->ct = &tag->ct; + if (codetag_ctx_enabled(&tag->ctc)) + ref->ctx = alloc_tag_create_ctx(tag, bytes); + else + ref->ct = &tag->ctc.ct; lazy_percpu_counter_add(&tag->bytes_allocated, bytes); } diff --git a/include/linux/codetag.h b/include/linux/codetag.h index 9ab2f017e845..b6a2f0287a83 100644 --- a/include/linux/codetag.h +++ b/include/linux/codetag.h @@ -104,7 +104,8 @@ struct codetag_with_ctx *ct_to_ctc(struct codetag *ct) } void codetag_lock_module_list(struct codetag_type *cttype, bool lock); -struct codetag_iterator codetag_get_ct_iter(struct codetag_type *cttype); +void codetag_init_iter(struct codetag_iterator *iter, + struct codetag_type *cttype); struct codetag *codetag_next_ct(struct codetag_iterator *iter); struct codetag_ctx *codetag_next_ctx(struct codetag_iterator *iter); diff --git a/include/linux/pgalloc_tag.h b/include/linux/pgalloc_tag.h index 0cbba13869b5..e4661bbd40c6 100644 --- a/include/linux/pgalloc_tag.h +++ b/include/linux/pgalloc_tag.h @@ -6,6 +6,7 @@ #define _LINUX_PGALLOC_TAG_H #include +#include #ifdef CONFIG_MEM_ALLOC_PROFILING @@ -70,7 +71,8 @@ static inline void pgalloc_tag_split(struct page *page, unsigned int nr) if (!ref->ct) goto out; - tag = ct_to_alloc_tag(ref->ct); + tag = is_codetag_ctx_ref(ref) ? ctc_to_alloc_tag(ref->ctx->ctc) + : ct_to_alloc_tag(ref->ct); page_ext = page_ext_next(page_ext); for (i = 1; i < nr; i++) { /* New reference with 0 bytes accounted */ diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 4157c2251b07..1b83ef17d232 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -969,6 +969,7 @@ config MEM_ALLOC_PROFILING select LAZY_PERCPU_COUNTER select PAGE_EXTENSION select SLAB_OBJ_EXT + select STACKDEPOT help Track allocation source code and record total allocation size initiated at that code location. The mechanism can be used to track diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c index 4a0b95a46b2e..675c7a08e38b 100644 --- a/lib/alloc_tag.c +++ b/lib/alloc_tag.c @@ -1,13 +1,18 @@ // SPDX-License-Identifier: GPL-2.0-only #include +#include #include #include #include #include #include +#include #include +#include #include +#define STACK_BUF_SIZE 1024 + DEFINE_STATIC_KEY_TRUE(mem_alloc_profiling_key); /* @@ -23,6 +28,16 @@ static int __init mem_alloc_profiling_disable(char *s) } __setup("nomem_profiling", mem_alloc_profiling_disable); +struct alloc_call_ctx { + struct codetag_ctx ctx; + size_t size; + pid_t pid; + pid_t tgid; + char comm[TASK_COMM_LEN]; + u64 ts_nsec; + depot_stack_handle_t stack_handle; +} __aligned(8); + struct alloc_tag_file_iterator { struct codetag_iterator ct_iter; struct seq_buf buf; @@ -64,7 +79,7 @@ static int allocations_file_open(struct inode *inode, struct file *file) return -ENOMEM; codetag_lock_module_list(cttype, true); - iter->ct_iter = codetag_get_ct_iter(cttype); + codetag_init_iter(&iter->ct_iter, cttype); codetag_lock_module_list(cttype, false); seq_buf_init(&iter->buf, iter->rawbuf, sizeof(iter->rawbuf)); file->private_data = iter; @@ -125,24 +140,240 @@ static const struct file_operations allocations_file_ops = { .read = allocations_file_read, }; +static void alloc_tag_ops_free_ctx(struct kref *refcount) +{ + kfree(container_of(kref_to_ctx(refcount), struct alloc_call_ctx, ctx)); +} + +struct codetag_ctx *alloc_tag_create_ctx(struct alloc_tag *tag, size_t size) +{ + struct alloc_call_ctx *ac_ctx; + + /* TODO: use a dedicated kmem_cache */ + ac_ctx = kmalloc(sizeof(struct alloc_call_ctx), GFP_KERNEL); + if (WARN_ON(!ac_ctx)) + return NULL; + + ac_ctx->size = size; + ac_ctx->pid = current->pid; + ac_ctx->tgid = current->tgid; + strscpy(ac_ctx->comm, current->comm, sizeof(ac_ctx->comm)); + ac_ctx->ts_nsec = local_clock(); + ac_ctx->stack_handle = + stack_depot_capture_stack(GFP_NOWAIT | __GFP_NOWARN); + add_ctx(&ac_ctx->ctx, &tag->ctc); + + return &ac_ctx->ctx; +} +EXPORT_SYMBOL_GPL(alloc_tag_create_ctx); + +void alloc_tag_free_ctx(struct codetag_ctx *ctx, struct alloc_tag **ptag) +{ + *ptag = ctc_to_alloc_tag(ctx->ctc); + rem_ctx(ctx, alloc_tag_ops_free_ctx); +} +EXPORT_SYMBOL_GPL(alloc_tag_free_ctx); + +bool alloc_tag_enable_ctx(struct alloc_tag *tag, bool enable) +{ + static bool stack_depot_ready; + + if (enable && !stack_depot_ready) { + stack_depot_init(); + stack_depot_capture_init(); + stack_depot_ready = true; + } + + return codetag_enable_ctx(&tag->ctc, enable); +} + +static void alloc_tag_ctx_to_text(struct seq_buf *out, struct codetag_ctx *ctx) +{ + struct alloc_call_ctx *ac_ctx; + char *buf; + + ac_ctx = container_of(ctx, struct alloc_call_ctx, ctx); + seq_buf_printf(out, " size: %zu\n", ac_ctx->size); + seq_buf_printf(out, " pid: %d\n", ac_ctx->pid); + seq_buf_printf(out, " tgid: %d\n", ac_ctx->tgid); + seq_buf_printf(out, " comm: %s\n", ac_ctx->comm); + seq_buf_printf(out, " ts: %llu\n", ac_ctx->ts_nsec); + + buf = kmalloc(STACK_BUF_SIZE, GFP_KERNEL); + if (buf) { + int bytes_read = stack_depot_snprint(ac_ctx->stack_handle, buf, + STACK_BUF_SIZE - 1, 8); + buf[bytes_read] = '\0'; + seq_buf_printf(out, " call stack:\n%s\n", buf); + } + kfree(buf); +} + +static ssize_t allocations_ctx_file_read(struct file *file, char __user *ubuf, + size_t size, loff_t *ppos) +{ + struct alloc_tag_file_iterator *iter = file->private_data; + struct codetag_iterator *ct_iter = &iter->ct_iter; + struct user_buf buf = { .buf = ubuf, .size = size }; + struct codetag_ctx *ctx; + struct codetag *prev_ct; + int err = 0; + + codetag_lock_module_list(ct_iter->cttype, true); + while (1) { + err = flush_ubuf(&buf, &iter->buf); + if (err || !buf.size) + break; + + prev_ct = ct_iter->ct; + ctx = codetag_next_ctx(ct_iter); + if (!ctx) + break; + + if (prev_ct != &ctx->ctc->ct) + alloc_tag_to_text(&iter->buf, &ctx->ctc->ct); + alloc_tag_ctx_to_text(&iter->buf, ctx); + } + codetag_lock_module_list(ct_iter->cttype, false); + + return err ? : buf.ret; +} + +#define CTX_CAPTURE_TOKENS() \ + x(disable, 0) \ + x(enable, 0) + +static const char * const ctx_capture_token_strs[] = { +#define x(name, nr_args) #name, + CTX_CAPTURE_TOKENS() +#undef x + NULL +}; + +enum ctx_capture_token { +#define x(name, nr_args) TOK_##name, + CTX_CAPTURE_TOKENS() +#undef x +}; + +static int enable_ctx_capture(struct codetag_type *cttype, + struct codetag_query *query, bool enable) +{ + struct codetag_iterator ct_iter; + struct codetag_with_ctx *ctc; + struct codetag *ct; + unsigned int nfound = 0; + + codetag_lock_module_list(cttype, true); + + codetag_init_iter(&ct_iter, cttype); + while ((ct = codetag_next_ct(&ct_iter))) { + if (!codetag_matches_query(query, ct, ct_iter.cmod, NULL)) + continue; + + ctc = ct_to_ctc(ct); + if (codetag_ctx_enabled(ctc) == enable) + continue; + + if (!alloc_tag_enable_ctx(ctc_to_alloc_tag(ctc), enable)) { + pr_warn("Failed to toggle context capture\n"); + continue; + } + + nfound++; + } + + codetag_lock_module_list(cttype, false); + + return nfound ? 0 : -ENOENT; +} + +static int parse_command(struct codetag_type *cttype, char *buf) +{ + struct codetag_query query = { NULL }; + char *cmd; + int ret; + int tok; + + buf = codetag_query_parse(&query, buf); + if (IS_ERR(buf)) + return PTR_ERR(buf); + + cmd = strsep_no_empty(&buf, " \t\r\n"); + if (!cmd) + return -EINVAL; /* no command */ + + tok = match_string(ctx_capture_token_strs, + ARRAY_SIZE(ctx_capture_token_strs), cmd); + if (tok < 0) + return -EINVAL; /* unknown command */ + + ret = enable_ctx_capture(cttype, &query, tok == TOK_enable); + if (ret < 0) + return ret; + + return 0; +} + +static ssize_t allocations_ctx_file_write(struct file *file, const char __user *ubuf, + size_t len, loff_t *offp) +{ + struct alloc_tag_file_iterator *iter = file->private_data; + char tmpbuf[256]; + + if (len == 0) + return 0; + /* we don't check *offp -- multiple writes() are allowed */ + if (len > sizeof(tmpbuf) - 1) + return -E2BIG; + + if (copy_from_user(tmpbuf, ubuf, len)) + return -EFAULT; + + tmpbuf[len] = '\0'; + parse_command(iter->ct_iter.cttype, tmpbuf); + + *offp += len; + return len; +} + +static const struct file_operations allocations_ctx_file_ops = { + .owner = THIS_MODULE, + .open = allocations_file_open, + .release = allocations_file_release, + .read = allocations_ctx_file_read, + .write = allocations_ctx_file_write, +}; + static int __init dbgfs_init(struct codetag_type *cttype) { struct dentry *file; + struct dentry *ctx_file; file = debugfs_create_file("allocations", 0444, NULL, cttype, &allocations_file_ops); + if (IS_ERR(file)) + return PTR_ERR(file); + + ctx_file = debugfs_create_file("allocations.ctx", 0666, NULL, cttype, + &allocations_ctx_file_ops); + if (IS_ERR(ctx_file)) { + debugfs_remove(file); + return PTR_ERR(ctx_file); + } - return IS_ERR(file) ? PTR_ERR(file) : 0; + return 0; } static bool alloc_tag_module_unload(struct codetag_type *cttype, struct codetag_module *cmod) { - struct codetag_iterator iter = codetag_get_ct_iter(cttype); + struct codetag_iterator iter; bool module_unused = true; struct alloc_tag *tag; struct codetag *ct; size_t bytes; + codetag_init_iter(&iter, cttype); for (ct = codetag_next_ct(&iter); ct; ct = codetag_next_ct(&iter)) { if (iter.cmod != cmod) continue; @@ -183,6 +414,7 @@ static int __init alloc_tag_init(void) .section = "alloc_tags", .tag_size = sizeof(struct alloc_tag), .module_unload = alloc_tag_module_unload, + .free_ctx = alloc_tag_ops_free_ctx, }; cttype = codetag_register_type(&desc); diff --git a/lib/codetag.c b/lib/codetag.c index d891bbe4481d..cbff146b3fe8 100644 --- a/lib/codetag.c +++ b/lib/codetag.c @@ -27,16 +27,14 @@ void codetag_lock_module_list(struct codetag_type *cttype, bool lock) up_read(&cttype->mod_lock); } -struct codetag_iterator codetag_get_ct_iter(struct codetag_type *cttype) +void codetag_init_iter(struct codetag_iterator *iter, + struct codetag_type *cttype) { - struct codetag_iterator iter = { - .cttype = cttype, - .cmod = NULL, - .mod_id = 0, - .ct = NULL, - }; - - return iter; + iter->cttype = cttype; + iter->cmod = NULL; + iter->mod_id = 0; + iter->ct = NULL; + iter->ctx = NULL; } static inline struct codetag *get_first_module_ct(struct codetag_module *cmod) @@ -128,6 +126,10 @@ struct codetag_ctx *codetag_next_ctx(struct codetag_iterator *iter) lockdep_assert_held(&iter->cttype->mod_lock); + /* Move to the first codetag if search just started */ + if (!iter->ct) + codetag_next_ct(iter); + if (!ctx) return next_ctx_from_ct(iter); -- 2.40.1.495.gc816e09b53d-goog