From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0940DC3DA59 for ; Mon, 22 Jul 2024 12:40:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8F4906B0085; Mon, 22 Jul 2024 08:40:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8A2AE6B0088; Mon, 22 Jul 2024 08:40:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 76A676B0089; Mon, 22 Jul 2024 08:40:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 56CA86B0085 for ; Mon, 22 Jul 2024 08:40:25 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id EA10A812EC for ; Mon, 22 Jul 2024 12:40:24 +0000 (UTC) X-FDA: 82367346768.08.FF012B0 Received: from mail-lf1-f52.google.com (mail-lf1-f52.google.com [209.85.167.52]) by imf16.hostedemail.com (Postfix) with ESMTP id D371D180004 for ; Mon, 22 Jul 2024 12:40:22 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=AKF1hL73; spf=pass (imf16.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.52 as permitted sender) smtp.mailfrom=urezki@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721651962; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=P/K2L+QmvMsHM3dk3lxU0tm1Ex9aXPZqsjkrwLpqNMg=; b=6NV8aZPbFrbS7RqVLdt+6Y1h9iY68tuoneyBjdQNqlJzwG2wyuHDWbxt0mK7Z6fTKouknx DSES3meFJjilNjJ6bHXFWW5+YfGmHcx+SkOD6eJ54vNqnDgwx54UbXWabygsEb9ziIBBTJ 8z0ELJ473exqXkRBR1dDwkK1Q4LS9Wc= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=AKF1hL73; spf=pass (imf16.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.52 as permitted sender) smtp.mailfrom=urezki@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721651962; a=rsa-sha256; cv=none; b=8YGRRl9V/e3QFUyN4yfhGevuWvKity4Po6O4Kps/J52LUjIVkcThWWUnZSnhIkD78APPdW EsLDijp4sP2H4hdQkGlYE8I1n8kBrzKg3cgudgB6aEblsRJzmdkC2DjJyd4R2sxOnlk84e TNfIeGM+N3ZqwCMnmNxycl2/XVeZAxk= Received: by mail-lf1-f52.google.com with SMTP id 2adb3069b0e04-52efdf02d13so2095986e87.2 for ; Mon, 22 Jul 2024 05:40:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721652021; x=1722256821; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:from:to:cc:subject:date:message-id:reply-to; bh=P/K2L+QmvMsHM3dk3lxU0tm1Ex9aXPZqsjkrwLpqNMg=; b=AKF1hL73X8vpIDjkBOghrOOgHf8Z23tNfx7KuyKdPfsrcR3gOZnFiTp79dQf5/qCTL k3T73LL9KQ3BTTOixIuxN2AQPvaALU/+wwiySZ/2NqN/m3blu6ThGpmjIaNJ/0Jt1bvD seBQa/s1C4kfc3/A3397Wtm9jaO+SbrYy1Z5xe7SLrW0p6Y46qJeDSGcb+d3Rfm0tcyX YrHOggcgaqTtE/Xe/N9UEgA4E0vaHucOQTVIjqREf2tjYN9n9PljWk1gbkrveLf3JsIR 8TKjZ1UQday1Pr5btX5XMBrI6Pu/+fd8rH4W+NolRnSLT4SA8b1NJsV4pJI4U7L8PGry WHiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721652021; x=1722256821; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=P/K2L+QmvMsHM3dk3lxU0tm1Ex9aXPZqsjkrwLpqNMg=; b=qDI6p7ZdYRZnOUkMVn8xRWso6P8AXi9WmXLVghl8CXOxqN/MFcJT5kDHHAtnvylHFa xWJcOsHWimksYt2S/3Jn3E2BDDbhi3/q62yn+0t5s7UkiPbtNP8w6lhDVlVmb2zNadGW w3p6zXl3Ffa9G3vDs3S/FvUOip26x3NTCAzLrOteuILtN/G8oC1T1qu/zYOULNEiek+K myZeK/oZhb7uVxApaZeWGtB2Rpx7GmW5UBNNSnxLkjHL993rIgVjM8yirYnOKJ36DZuO y1wATV3thWQR5Qq2855MVcJnBSdTxpylxtsMGsDvE+XIF/iv8WC9IHbcpphHiqsVgyVJ iJYw== X-Forwarded-Encrypted: i=1; AJvYcCX01hMo8Mx2u7SepUJyI9gRIk411S5h+wj2jst15K9W3z2HLMvYL1UjHjarXgsUUYTHSD30WFjH2TWDbeeimDDGhwo= X-Gm-Message-State: AOJu0YyluoKognxvKzBpfoqhRaXeQMDoZ7oiPMEZuFBiccN82Zy2l6zL ZJ0RAagpO29J+iBWBdrFBSsasUgCEyt9nQiPBJBo/ypqlGC5V7a1 X-Google-Smtp-Source: AGHT+IEKbnz+YGVqQX//G5lVKXtXLjexoIlhLIMvXmJMiNPw2OIt6hOf41hgO32c6BXv9pWNfjjd/A== X-Received: by 2002:a05:6512:a86:b0:52e:d0f8:2d43 with SMTP id 2adb3069b0e04-52efb53bae5mr4885953e87.17.1721652020702; Mon, 22 Jul 2024 05:40:20 -0700 (PDT) Received: from pc636 (host-90-233-197-231.mobileonline.telia.com. [90.233.197.231]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-52efb97bbd4sm872658e87.60.2024.07.22.05.40.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Jul 2024 05:40:20 -0700 (PDT) From: Uladzislau Rezki X-Google-Original-From: Uladzislau Rezki Date: Mon, 22 Jul 2024 14:40:17 +0200 To: Adrian Huang Cc: urezki@gmail.com, ahuang12@lenovo.com, akpm@linux-foundation.org, hch@infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, peterz@infradead.org, sunjw10@lenovo.com Subject: Re: [PATCH 1/1] mm/vmalloc: Add preempt point in purge_vmap_node() when enabling kasan Message-ID: References: <20240722115054.6295-1-ahuang12@lenovo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240722115054.6295-1-ahuang12@lenovo.com> X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: D371D180004 X-Stat-Signature: 8iok1ne3iyosayipub4ph9dngb84rexa X-Rspam-User: X-HE-Tag: 1721652022-475844 X-HE-Meta: U2FsdGVkX19ILpkZWlC47FkG3gS2I1X8ue1lC8Nzh8kOCrsVDQAfZnjia2uuy66TTeqYdx/EjAQYPLFKZkLPpfqgp+AWrDrIHGa4dR2FqCbDHsZLQ3f7KHyTIebVZND/y9dmt7xPhR2gQEtExCMYxx8V8sk4/VOEmUMokIWv6A8uVNobOKBLhijMuKd2SsnkBAaeLbmc8Jj6Kykgv3bvye9WMgy4Qfi1DRNn0XQIBywCMmW9IH7lmECTbDDNch8ja2lsEhP6EBT3iDk5uXo1e8VceuEPOynrIVNlKPUl+vM7xG54TcMWRl5FqFi01K4eixgfyyrPKtCWaqtZZe8SRFGxdBB/fXCFXjCdtDN2jqPfNrq54oHPV8BbuOvZHCvSAbim+sc586VYLfrSdMEHn5EeA5/Y8BopmpKDuGRev3Oqy3WrS1jM9CKQdN1rhRqYSXNM3pqUGjN747r4t6Rbz/DlN0nAhnf40y/MfpYtgfwqE1BNDSlsho5GD1NiS3ijWcG6gv0dgHuzkYFLtIfhFnZTd080LH1nhYwvhsXFxMTDIVQsdcJJoKU1nnl3VUDZOdumwvyhyeR0dLHtLZK8UhHfgScC3ddmM5Mhb/I5Buv0N7ObT4710O9yAtSJArNFYYKw4/h6komRcWm4Ct9ZtWoszHbuOK5g1mKxPU9knrDbjv7qrITBdqRFEENNldbwm7+8B7UhZD8jbbTdiEupukBC1NrV/ff23AEpGFhVJQzzNCotSsB5KeSb6ipi1olL58vD6qsTLb6IXOWGVb6uTUD4Gi3VGQhc6zEy8X0FBGYiNpT52M/jnKVTbvi5fEChl75CsD+G+KtJw0SDS3aU0snWHSZUjzMQnA7AVIgcwc3YzhiDQEqpJFmjpkpkyBuShqa0fG5rEuYXuVqTXj5ju8+9Y5tVs6pbv/SbaUk9/jbOSmRpWmi/KEVOv+FD9Cld/b8pJUTdf0eEk3HmL2z oSGRlj0C bYS+xN2oUZvVSM03ReGDuDfLDPGU0xSfLmOywi3XXOr4azI50MD3Z1uY3gdvzxA+t9LIOu4ZArMJqz4SicOlKWv7dzsXPbcf4fJF7fXgDXpw2cba9+dCn8cvXRmY6+U6H5WEiiEeDysseajg7b4QV75CED3NPuFvB6bNyoEbm58yxga8AQBukakLV42A6wR1J6H8JHPRAKD2HOH2IbDHHF4f+q8uOX6x1uZsyBmyd9Qxr/uugxXkdFQNfTGvuYAUO+y2cKgpWgfuhUOY0lOcadAqWu7ouGN+YajKnwBaan/jYRhECKSYOf5oTzfYPsPxW4lewhkY1R0ej//te01Cj0eb3ITqUBJOTY4t2M6DbtH9Ro0y790v0oRUPSpd4zvv9xcg5m5YNEvAmPzMRkubdaYWNaM1FApuPQlFE//F68Kd+gPuXqQu1YNGH/bxUSWQ+yz+p2zzqokYdNFnp1wez8hm8qs9fC9ohnb1B X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Jul 22, 2024 at 07:50:54PM +0800, Adrian Huang wrote: > > I tried to simulate the reported splat and i can reproduce it with KASAN > > enabled. I use qemu on my 64-core system, it allows me to specify 255 > > cores while running VM. The kernel is 6.10.0-rc5. > > > > The kernel should be built with CONFIG_KASAN=y and CONFIG_KASAN_VMALLOC=y > > > > The "soft lockup" can be triggered when the kernel is compiled within a > > VM using 256 jobs and preemption is disabled: > > > > echo none > /sys/kernel/debug/sched/preempt > > make -C coding/linux.git/ -j256 bzImage > > > > > > watchdog: BUG: soft lockup - CPU#28 stuck for 22s! [kworker/28:1:1760] > > CPU: 28 PID: 1760 Comm: kworker/28:1 Kdump: loaded Not tainted 6.10.0-rc5 #95 > > Workqueue: events drain_vmap_area_work > > RIP: 0010:smp_call_function_many_cond+0x1d8/0xbb0 > > ... > > > > Great to hear you're able to reproduce the issue. > > I keep debugging, and the original patch (https://lore.kernel.org/all/ZogS_04dP5LlRlXN@pc636/T/) shows purge_vmap_node() iteratively releases kasan vmalloc > allocations and flushes tlb for each vmap_area. There are 2805 > flush_tlb_kernel_range() calls in ftrace log. > * One is called in __purge_vmap_area_lazy(). > * Others are called in kasan_release_vmalloc(): Called by purge_vmap_node(). > - [Rough calculation] Each flush_tlb_kernel_range() runs about 7.5ms. > -- 2804 * 7.5ms = 21.03 seconds (That's why a soft lock is trigger) > > If we combine all tlb flush operations into one operation in the call path > 'purge_vmap_node()->kasan_release_vmalloc()', the running time of > drain_vmap_area_work() can be saved greately. The idea is from the > flush_tlb_kernel_range() call in __purge_vmap_area_lazy(). > And, the soft lockup won't not be triggered. Please refer to the following patch. > Here is the test result based on 6.10: > > [6.10 wo/ the patch below] > 1. ftrace latency profiling (record a trace if the latency > 20s): Commands > echo 20000000 > /sys/kernel/debug/tracing/tracing_thresh > echo drain_vmap_area_work > /sys/kernel/debug/tracing/set_graph_function > echo function_graph > /sys/kernel/debug/tracing/current_tracer > echo 1 > /sys/kernel/debug/tracing/tracing_on > > 2. Run `make -j $(nproc)` to compile the kernel source > > 3. Once the soft lockup is reproduced, check the ftace: > cat /sys/kernel/debug/tracing/trace > # tracer: function_graph > # > # CPU DURATION FUNCTION CALLS > # | | | | | | | > 76) $ 50412985 us | } /* __purge_vmap_area_lazy */ > 76) $ 50412997 us | } /* drain_vmap_area_work */ > 76) $ 29165911 us | } /* __purge_vmap_area_lazy */ > 76) $ 29165926 us | } /* drain_vmap_area_work */ > 91) $ 53629423 us | } /* __purge_vmap_area_lazy */ > 91) $ 53629434 us | } /* drain_vmap_area_work */ > 91) $ 28121014 us | } /* __purge_vmap_area_lazy */ > 91) $ 28121026 us | } /* drain_vmap_area_work */ > > > [6.10 w/ the patch below] > 1. Repeat step 1-2 in "[6.10 wo/ the patch below]" > > 2. The soft lockup is not triggered and the ftrace log is empty. > cat /sys/kernel/debug/tracing/trace > # tracer: function_graph > # > # CPU DURATION FUNCTION CALLS > # | | | | | | | > > > 3. Setting 'tracing_thresh' to 10/5 seconds does not get any ftrace log. > > 4. Setting 'tracing_thresh' to 1 second gets ftrace log. > cat /sys/kernel/tracing/trace > # tracer: function_graph > # > # CPU DURATION FUNCTION CALLS > # | | | | | | | > 51) $ 1019695 us | } /* __purge_vmap_area_lazy */ > 51) $ 1019703 us | } /* drain_vmap_area_work */ > 198) $ 1018707 us | } /* __purge_vmap_area_lazy */ > 198) $ 1018718 us | } /* drain_vmap_area_work */ > > 5. Run the following test_vmalloc command without any issues > modprobe test_vmalloc nr_threads=$(nproc) run_test_mask=0x1 nr_pages=4 > > Could you please test this patch in your VM environment? > > --- > diff --git a/include/linux/kasan.h b/include/linux/kasan.h > index 70d6a8f6e25d..ddbf42a1a7b7 100644 > --- a/include/linux/kasan.h > +++ b/include/linux/kasan.h > @@ -55,6 +55,9 @@ extern p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D]; > int kasan_populate_early_shadow(const void *shadow_start, > const void *shadow_end); > > +#define KASAN_VMALLOC_PAGE_RANGE 0x1 /* Apply existing page range */ > +#define KASAN_VMALLOC_TLB_FLUSH 0x2 /* TLB flush */ > + > #ifndef kasan_mem_to_shadow > static inline void *kasan_mem_to_shadow(const void *addr) > { > @@ -511,7 +514,8 @@ void kasan_populate_early_vm_area_shadow(void *start, unsigned long size); > int kasan_populate_vmalloc(unsigned long addr, unsigned long size); > void kasan_release_vmalloc(unsigned long start, unsigned long end, > unsigned long free_region_start, > - unsigned long free_region_end); > + unsigned long free_region_end, > + unsigned long flags); > > #else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ > > @@ -526,7 +530,8 @@ static inline int kasan_populate_vmalloc(unsigned long start, > static inline void kasan_release_vmalloc(unsigned long start, > unsigned long end, > unsigned long free_region_start, > - unsigned long free_region_end) { } > + unsigned long free_region_end, > + unsigned long flags) { } > > #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ > > @@ -561,7 +566,8 @@ static inline int kasan_populate_vmalloc(unsigned long start, > static inline void kasan_release_vmalloc(unsigned long start, > unsigned long end, > unsigned long free_region_start, > - unsigned long free_region_end) { } > + unsigned long free_region_end, > + unsigned long flags) { } > > static inline void *kasan_unpoison_vmalloc(const void *start, > unsigned long size, > diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c > index d6210ca48dda..88d1c9dcb507 100644 > --- a/mm/kasan/shadow.c > +++ b/mm/kasan/shadow.c > @@ -489,7 +489,8 @@ static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr, > */ > void kasan_release_vmalloc(unsigned long start, unsigned long end, > unsigned long free_region_start, > - unsigned long free_region_end) > + unsigned long free_region_end, > + unsigned long flags) > { > void *shadow_start, *shadow_end; > unsigned long region_start, region_end; > @@ -522,12 +523,17 @@ void kasan_release_vmalloc(unsigned long start, unsigned long end, > __memset(shadow_start, KASAN_SHADOW_INIT, shadow_end - shadow_start); > return; > } > - apply_to_existing_page_range(&init_mm, > + > + > + if (flags & KASAN_VMALLOC_PAGE_RANGE) > + apply_to_existing_page_range(&init_mm, > (unsigned long)shadow_start, > size, kasan_depopulate_vmalloc_pte, > NULL); > - flush_tlb_kernel_range((unsigned long)shadow_start, > - (unsigned long)shadow_end); > + > + if (flags & KASAN_VMALLOC_TLB_FLUSH) > + flush_tlb_kernel_range((unsigned long)shadow_start, > + (unsigned long)shadow_end); > } > } > > diff --git a/mm/vmalloc.c b/mm/vmalloc.c > index e34ea860153f..d66e09135876 100644 > --- a/mm/vmalloc.c > +++ b/mm/vmalloc.c > @@ -2193,8 +2193,15 @@ static void purge_vmap_node(struct work_struct *work) > struct vmap_area *va, *n_va; > LIST_HEAD(local_list); > > + unsigned long start; > + unsigned long end; > + > vn->nr_purged = 0; > > + start = list_first_entry(&vn->purge_list, struct vmap_area, list)->va_start; > + > + end = list_last_entry(&vn->purge_list, struct vmap_area, list)->va_end; > + > list_for_each_entry_safe(va, n_va, &vn->purge_list, list) { > unsigned long nr = (va->va_end - va->va_start) >> PAGE_SHIFT; > unsigned long orig_start = va->va_start; > @@ -2205,7 +2212,8 @@ static void purge_vmap_node(struct work_struct *work) > > if (is_vmalloc_or_module_addr((void *)orig_start)) > kasan_release_vmalloc(orig_start, orig_end, > - va->va_start, va->va_end); > + va->va_start, va->va_end, > + KASAN_VMALLOC_PAGE_RANGE); > > atomic_long_sub(nr, &vmap_lazy_nr); > vn->nr_purged++; > @@ -2218,6 +2226,8 @@ static void purge_vmap_node(struct work_struct *work) > list_add(&va->list, &local_list); > } > > + kasan_release_vmalloc(start, end, start, end, KASAN_VMALLOC_TLB_FLUSH); > + > reclaim_list_global(&local_list); > } > > @@ -4726,7 +4736,8 @@ struct vm_struct **pcpu_get_vm_areas(const unsigned long *offsets, > &free_vmap_area_list); > if (va) > kasan_release_vmalloc(orig_start, orig_end, > - va->va_start, va->va_end); > + va->va_start, va->va_end, > + KASAN_VMALLOC_PAGE_RANGE | KASAN_VMALLOC_TLB_FLUSH); > vas[area] = NULL; > } > > @@ -4776,7 +4787,8 @@ struct vm_struct **pcpu_get_vm_areas(const unsigned long *offsets, > &free_vmap_area_list); > if (va) > kasan_release_vmalloc(orig_start, orig_end, > - va->va_start, va->va_end); > + va->va_start, va->va_end, > + KASAN_VMALLOC_PAGE_RANGE | KASAN_VMALLOC_TLB_FLUSH); > vas[area] = NULL; > kfree(vms[area]); > } > Sure! I will have a look at the patch and check my environment. -- Uladzislau Rezki