From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A67A0C388F9 for ; Thu, 22 Oct 2020 13:20:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F228C24182 for ; Thu, 22 Oct 2020 13:20:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="VesxqZfV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F228C24182 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 422016B009B; Thu, 22 Oct 2020 09:20:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D26C6B009D; Thu, 22 Oct 2020 09:20:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2491F6B009E; Thu, 22 Oct 2020 09:20:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0008.hostedemail.com [216.40.44.8]) by kanga.kvack.org (Postfix) with ESMTP id DCD586B009B for ; Thu, 22 Oct 2020 09:19:59 -0400 (EDT) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 75C073625 for ; Thu, 22 Oct 2020 13:19:59 +0000 (UTC) X-FDA: 77399619318.06.idea48_4a1543627251 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin06.hostedemail.com (Postfix) with ESMTP id 4FAD81026C321 for ; Thu, 22 Oct 2020 13:19:59 +0000 (UTC) X-HE-Tag: idea48_4a1543627251 X-Filterd-Recvd-Size: 16172 Received: from mail-qt1-f202.google.com (mail-qt1-f202.google.com [209.85.160.202]) by imf32.hostedemail.com (Postfix) with ESMTP for ; Thu, 22 Oct 2020 13:19:58 +0000 (UTC) Received: by mail-qt1-f202.google.com with SMTP id d22so1039919qtn.0 for ; Thu, 22 Oct 2020 06:19:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=hXyFPEk4foicA9MHUn1LEAgXXkiWJy7tayx5woeXroY=; b=VesxqZfVc4TwIJOMCHCJCamqxdv3ivbVcAf/dPZk4kFR93xcHiGrBpORd1FmKSnz4M LLYv6Ue+5wAxbuGSDmXQxMoN8Y4ep4iGBpMomTNuR0kdJNPaa2GGqQCFz4rEiglKvW8h Mj1BXmRBClmxHgIpHikrc/rCK7c3cBSofpyvsWyoYZ0y1nTw8CDWIJLlKn1zT7zyniuA Xpj4CbezQmIO6q+TMWARr7yMMdERPhlVsoPT9rH67PVkJYge8klVwSHrqAtC6EUYMsnG eyy8fdtPy02ZuttyO1oMl1zmHFMk5+ZPxxEJglbphMNSlZcjZIxAZspqPwEO5HQXRouQ XKkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=hXyFPEk4foicA9MHUn1LEAgXXkiWJy7tayx5woeXroY=; b=FVLynQmtIGJ05zkjtsuwG/EKKNBYViekGEatdSXovwoCuTQ48iEpZW3ZthXr2omI9v ucKjFZWg2yWqSllVrP05pw5GUwASqdbfpqV4mQ9SRm1D+KdyJOAeUpIjNnYGlsgBDG63 43ao6B/2DMVHFeRyPBkYkvsH2EDu6UkmEiAiItdLRGTarFtOK4182FiZg3Uno9j0T15K oWLsKmnhaNKWq4yXlbcirv2FCC3Box6j5FJK14qGrg/74AR0S633LCUELy552PezRu82 ddoqXuMSbuWzi8uK4h7pQGYtEkcCynn30PUrbQnSWRYyRcJtn65P40jIYXDpMyyBkKIx S6cQ== X-Gm-Message-State: AOAM532O9d85+KQWbz7WAbxYTq/6XFYX0tpRaC2eBvl+m9B160Pk4CF3 ZS+uwhKN9MuXN/OykqlThErRjMLr5ixE0AQo X-Google-Smtp-Source: ABdhPJz9rkeUtygs5ZPEaDOWc/yvjgB7QSCk5bCAQdXqW9yO1wZyxT1pgPdZxui+xVFkBbj2LobYJDXK6Aw6OyF4 X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a0c:8d05:: with SMTP id r5mr2364576qvb.31.1603372797908; Thu, 22 Oct 2020 06:19:57 -0700 (PDT) Date: Thu, 22 Oct 2020 15:19:06 +0200 In-Reply-To: Message-Id: <6a4a7626bf280871518656f4fa89cb064740baf7.1603372719.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.0.rc1.297.gfa9743e501-goog Subject: [PATCH RFC v2 14/21] kasan: add and integrate kasan boot parameters From: Andrey Konovalov To: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Alexander Potapenko , Marco Elver Cc: Evgenii Stepanov , Kostya Serebryany , Peter Collingbourne , Serban Constantinescu , Andrey Ryabinin , Elena Petrova , Branislav Rankov , Kevin Brodsky , Andrew Morton , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: TODO: no meaningful description here yet, please see the cover letter for this RFC series. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/If7d37003875b2ed3e0935702c8015c223d6416a4 --- mm/kasan/common.c | 92 +++++++++++++----------- mm/kasan/generic.c | 5 ++ mm/kasan/hw_tags.c | 169 ++++++++++++++++++++++++++++++++++++++++++++- mm/kasan/kasan.h | 9 +++ mm/kasan/report.c | 14 +++- mm/kasan/sw_tags.c | 5 ++ 6 files changed, 250 insertions(+), 44 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 1a5e6c279a72..cc129ef62ab1 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -129,35 +129,37 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, unsigned int redzone_size; int redzone_adjust; - /* Add alloc meta. */ - cache->kasan_info.alloc_meta_offset = *size; - *size += sizeof(struct kasan_alloc_meta); - - /* Add free meta. */ - if (IS_ENABLED(CONFIG_KASAN_GENERIC) && - (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size < sizeof(struct kasan_free_meta))) { - cache->kasan_info.free_meta_offset = *size; - *size += sizeof(struct kasan_free_meta); - } - - redzone_size = optimal_redzone(cache->object_size); - redzone_adjust = redzone_size - (*size - cache->object_size); - if (redzone_adjust > 0) - *size += redzone_adjust; - - *size = min_t(unsigned int, KMALLOC_MAX_SIZE, - max(*size, cache->object_size + redzone_size)); + if (static_branch_unlikely(&kasan_stack)) { + /* Add alloc meta. */ + cache->kasan_info.alloc_meta_offset = *size; + *size += sizeof(struct kasan_alloc_meta); + + /* Add free meta. */ + if (IS_ENABLED(CONFIG_KASAN_GENERIC) && + (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || + cache->object_size < sizeof(struct kasan_free_meta))) { + cache->kasan_info.free_meta_offset = *size; + *size += sizeof(struct kasan_free_meta); + } - /* - * If the metadata doesn't fit, don't enable KASAN at all. - */ - if (*size <= cache->kasan_info.alloc_meta_offset || - *size <= cache->kasan_info.free_meta_offset) { - cache->kasan_info.alloc_meta_offset = 0; - cache->kasan_info.free_meta_offset = 0; - *size = orig_size; - return; + redzone_size = optimal_redzone(cache->object_size); + redzone_adjust = redzone_size - (*size - cache->object_size); + if (redzone_adjust > 0) + *size += redzone_adjust; + + *size = min_t(unsigned int, KMALLOC_MAX_SIZE, + max(*size, cache->object_size + redzone_size)); + + /* + * If the metadata doesn't fit, don't enable KASAN at all. + */ + if (*size <= cache->kasan_info.alloc_meta_offset || + *size <= cache->kasan_info.free_meta_offset) { + cache->kasan_info.alloc_meta_offset = 0; + cache->kasan_info.free_meta_offset = 0; + *size = orig_size; + return; + } } *flags |= SLAB_KASAN; @@ -165,10 +167,12 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, size_t kasan_metadata_size(struct kmem_cache *cache) { - return (cache->kasan_info.alloc_meta_offset ? - sizeof(struct kasan_alloc_meta) : 0) + - (cache->kasan_info.free_meta_offset ? - sizeof(struct kasan_free_meta) : 0); + if (static_branch_unlikely(&kasan_stack)) + return (cache->kasan_info.alloc_meta_offset ? + sizeof(struct kasan_alloc_meta) : 0) + + (cache->kasan_info.free_meta_offset ? + sizeof(struct kasan_free_meta) : 0); + return 0; } struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, @@ -270,8 +274,10 @@ void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, if (!(cache->flags & SLAB_KASAN)) return (void *)object; - alloc_meta = kasan_get_alloc_meta(cache, object); - __memset(alloc_meta, 0, sizeof(*alloc_meta)); + if (static_branch_unlikely(&kasan_stack)) { + alloc_meta = kasan_get_alloc_meta(cache, object); + __memset(alloc_meta, 0, sizeof(*alloc_meta)); + } if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) object = set_tag(object, assign_tag(cache, object, true, false)); @@ -308,15 +314,19 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, rounded_up_size = round_up(cache->object_size, KASAN_GRANULE_SIZE); kasan_poison_memory(object, rounded_up_size, KASAN_KMALLOC_FREE); - if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) || - unlikely(!(cache->flags & SLAB_KASAN))) - return false; + if (static_branch_unlikely(&kasan_stack)) { + if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) || + unlikely(!(cache->flags & SLAB_KASAN))) + return false; + + kasan_set_free_info(cache, object, tag); - kasan_set_free_info(cache, object, tag); + quarantine_put(cache, object); - quarantine_put(cache, object); + return IS_ENABLED(CONFIG_KASAN_GENERIC); + } - return IS_ENABLED(CONFIG_KASAN_GENERIC); + return false; } bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) @@ -355,7 +365,7 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, kasan_poison_memory((void *)redzone_start, redzone_end - redzone_start, KASAN_KMALLOC_REDZONE); - if (cache->flags & SLAB_KASAN) + if (static_branch_unlikely(&kasan_stack) && (cache->flags & SLAB_KASAN)) set_alloc_info(cache, (void *)object, flags); return set_tag(object, tag); diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index d259e4c3aefd..20a1e753e0c5 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -33,6 +33,11 @@ #include "kasan.h" #include "../slab.h" +/* See the comments in hw_tags.c */ +DEFINE_STATIC_KEY_TRUE_RO(kasan_enabled); +EXPORT_SYMBOL(kasan_enabled); +DEFINE_STATIC_KEY_TRUE_RO(kasan_stack); + /* * All functions below always inlined so compiler could * perform better optimizations in each of __asan_loadX/__assn_storeX diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 915142da6b57..bccd781011ad 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -8,6 +8,8 @@ #define pr_fmt(fmt) "kasan: " fmt +#include +#include #include #include #include @@ -17,10 +19,175 @@ #include "kasan.h" +enum kasan_arg_mode { + KASAN_ARG_MODE_OFF, + KASAN_ARG_MODE_PROD, + KASAN_ARG_MODE_FULL, +}; + +enum kasan_arg_stack { + KASAN_ARG_STACK_DEFAULT, + KASAN_ARG_STACK_OFF, + KASAN_ARG_STACK_ON, +}; + +enum kasan_arg_trap { + KASAN_ARG_TRAP_DEFAULT, + KASAN_ARG_TRAP_ASYNC, + KASAN_ARG_TRAP_SYNC, +}; + +enum kasan_arg_fault { + KASAN_ARG_FAULT_DEFAULT, + KASAN_ARG_FAULT_REPORT, + KASAN_ARG_FAULT_PANIC, +}; + +static enum kasan_arg_mode kasan_arg_mode __ro_after_init; +static enum kasan_arg_stack kasan_arg_stack __ro_after_init; +static enum kasan_arg_fault kasan_arg_fault __ro_after_init; +static enum kasan_arg_trap kasan_arg_trap __ro_after_init; + +/* Whether KASAN is enabled at all. */ +DEFINE_STATIC_KEY_FALSE_RO(kasan_enabled); +EXPORT_SYMBOL(kasan_enabled); + +/* Whether to collect alloc/free stack traces. */ +DEFINE_STATIC_KEY_FALSE_RO(kasan_stack); + +/* Whether to use synchronous or asynchronous tag checking. */ +static bool kasan_sync __ro_after_init; + +/* Whether panic or disable tag checking on fault. */ +bool kasan_panic __ro_after_init; + +/* kasan.mode=off/prod/full */ +static int __init early_kasan_mode(char *arg) +{ + if (!arg) + return -EINVAL; + + if (!strcmp(arg, "off")) + kasan_arg_mode = KASAN_ARG_MODE_OFF; + else if (!strcmp(arg, "prod")) + kasan_arg_mode = KASAN_ARG_MODE_PROD; + else if (!strcmp(arg, "full")) + kasan_arg_mode = KASAN_ARG_MODE_FULL; + else + return -EINVAL; + + return 0; +} +early_param("kasan.mode", early_kasan_mode); + +/* kasan.stack=off/on */ +static int __init early_kasan_stack(char *arg) +{ + if (!arg) + return -EINVAL; + + if (!strcmp(arg, "off")) + kasan_arg_stack = KASAN_ARG_STACK_OFF; + else if (!strcmp(arg, "on")) + kasan_arg_stack = KASAN_ARG_STACK_ON; + else + return -EINVAL; + + return 0; +} +early_param("kasan.stack", early_kasan_stack); + +/* kasan.trap=sync/async */ +static int __init early_kasan_trap(char *arg) +{ + if (!arg) + return -EINVAL; + + if (!strcmp(arg, "ASYNC")) + kasan_arg_trap = KASAN_ARG_TRAP_ASYNC; + else if (!strcmp(arg, "sync")) + kasan_arg_trap = KASAN_ARG_TRAP_SYNC; + else + return -EINVAL; + + return 0; +} +early_param("kasan.trap", early_kasan_trap); + +/* kasan.fault=report/panic */ +static int __init early_kasan_fault(char *arg) +{ + if (!arg) + return -EINVAL; + + if (!strcmp(arg, "report")) + kasan_arg_fault = KASAN_ARG_FAULT_REPORT; + else if (!strcmp(arg, "panic")) + kasan_arg_fault = KASAN_ARG_FAULT_PANIC; + else + return -EINVAL; + + return 0; +} +early_param("kasan.fault", early_kasan_fault); + void __init kasan_init_tags(void) { - init_tags(KASAN_TAG_MAX); + if (!cpu_supports_tags()) + return; + + /* First, preset values based on the mode. */ + + switch (kasan_arg_mode) { + case KASAN_ARG_MODE_OFF: + return; + case KASAN_ARG_MODE_PROD: + static_branch_enable(&kasan_enabled); + break; + case KASAN_ARG_MODE_FULL: + static_branch_enable(&kasan_enabled); + static_branch_enable(&kasan_stack); + kasan_sync = true; + break; + } + + /* Now, optionally override the presets. */ + switch (kasan_arg_stack) { + case KASAN_ARG_STACK_OFF: + static_branch_disable(&kasan_stack); + break; + case KASAN_ARG_STACK_ON: + static_branch_enable(&kasan_stack); + break; + default: + break; + } + + switch (kasan_arg_trap) { + case KASAN_ARG_TRAP_ASYNC: + kasan_sync = false; + break; + case KASAN_ARG_TRAP_SYNC: + kasan_sync = true; + break; + default: + break; + } + + switch (kasan_arg_fault) { + case KASAN_ARG_FAULT_REPORT: + kasan_panic = false; + break; + case KASAN_ARG_FAULT_PANIC: + kasan_panic = true; + break; + default: + break; + } + + /* TODO: choose between sync and async based on kasan_sync. */ + init_tags(KASAN_TAG_MAX); pr_info("KernelAddressSanitizer initialized\n"); } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index f7ae0c23f023..00b47bc753aa 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -2,9 +2,18 @@ #ifndef __MM_KASAN_KASAN_H #define __MM_KASAN_KASAN_H +#include #include #include +#ifdef CONFIG_KASAN_HW_TAGS +DECLARE_STATIC_KEY_FALSE(kasan_stack); +#else +DECLARE_STATIC_KEY_TRUE(kasan_stack); +#endif + +extern bool kasan_panic __ro_after_init; + #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) #define KASAN_GRANULE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) #else diff --git a/mm/kasan/report.c b/mm/kasan/report.c index dee5350b459c..426dd1962d3c 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -97,6 +97,10 @@ static void end_report(unsigned long *flags) panic_on_warn = 0; panic("panic_on_warn set ...\n"); } +#ifdef CONFIG_KASAN_HW_TAGS + if (kasan_panic) + panic("kasan.fault=panic set ...\n"); +#endif kasan_enable_current(); } @@ -159,8 +163,8 @@ static void describe_object_addr(struct kmem_cache *cache, void *object, (void *)(object_addr + cache->object_size)); } -static void describe_object(struct kmem_cache *cache, void *object, - const void *addr, u8 tag) +static void describe_object_stacks(struct kmem_cache *cache, void *object, + const void *addr, u8 tag) { struct kasan_alloc_meta *alloc_meta = kasan_get_alloc_meta(cache, object); @@ -188,7 +192,13 @@ static void describe_object(struct kmem_cache *cache, void *object, } #endif } +} +static void describe_object(struct kmem_cache *cache, void *object, + const void *addr, u8 tag) +{ + if (static_branch_unlikely(&kasan_stack)) + describe_object_stacks(cache, object, addr, tag); describe_object_addr(cache, object, addr); } diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c index 4db41f274702..b6d185adf2c5 100644 --- a/mm/kasan/sw_tags.c +++ b/mm/kasan/sw_tags.c @@ -33,6 +33,11 @@ #include "kasan.h" #include "../slab.h" +/* See the comments in hw_tags.c */ +DEFINE_STATIC_KEY_TRUE_RO(kasan_enabled); +EXPORT_SYMBOL(kasan_enabled); +DEFINE_STATIC_KEY_TRUE_RO(kasan_stack); + static DEFINE_PER_CPU(u32, prng_state); void __init kasan_init_tags(void) -- 2.29.0.rc1.297.gfa9743e501-goog