From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BDB9C3DA63 for ; Wed, 24 Jul 2024 12:46:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A1A446B0088; Wed, 24 Jul 2024 08:46:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9CA3C6B0089; Wed, 24 Jul 2024 08:46:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 86A986B008A; Wed, 24 Jul 2024 08:46:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 67D466B0088 for ; Wed, 24 Jul 2024 08:46:34 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 160A0160903 for ; Wed, 24 Jul 2024 12:46:34 +0000 (UTC) X-FDA: 82374619908.28.C1E4521 Received: from mail-pg1-f182.google.com (mail-pg1-f182.google.com [209.85.215.182]) by imf06.hostedemail.com (Postfix) with ESMTP id 0F475180013 for ; Wed, 24 Jul 2024 12:46:31 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ALndcQAL; spf=pass (imf06.hostedemail.com: domain of adrianhuang0701@gmail.com designates 209.85.215.182 as permitted sender) smtp.mailfrom=adrianhuang0701@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721825144; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hBJWQ1t6QCM1T5bdDmDCHfCfTpW6DNC9TA/sHMK/e6E=; b=RQDBcOzKaSfYnEgYZ9XaGFlfVI+3SyM22GZLUoSOseAwIYkzmTV3ByinCRbUgw5vHY9PiH sF5dgtRAPJ/4M41Tgcs4YW4AdZ1WsxIBWFvFfQ6Zg+1MapXOTaSdb42AjNTtOKjpZpLm/E ND/MqrJD5aywvpcEtr6fBg0sEHhDeuI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721825144; a=rsa-sha256; cv=none; b=nUL2lWSZxfr7ENU3VYL5nTHR9fHj5nq/mg+Pz7chNw+AxWRbei4lSfIJCRyQL7sFEEU3Cp BAbRP6d7b8PMc9v2myvcvJg9/EEMJK4XvH6/62AsyN/XAmpu+dkJcEgXWXt5fpaJ6Xgwfa 0oaT3i2WIT8zsO8dkP6Pu6zHrI7w1Vk= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ALndcQAL; spf=pass (imf06.hostedemail.com: domain of adrianhuang0701@gmail.com designates 209.85.215.182 as permitted sender) smtp.mailfrom=adrianhuang0701@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pg1-f182.google.com with SMTP id 41be03b00d2f7-7a0e8b76813so1208251a12.3 for ; Wed, 24 Jul 2024 05:46:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721825191; x=1722429991; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hBJWQ1t6QCM1T5bdDmDCHfCfTpW6DNC9TA/sHMK/e6E=; b=ALndcQALlW16nMMzQ0nqWmSlD7necI3hvNaZUd3YWH7LHKBnCN57OfkjR0Xz3TNkof CHw7LM4RrP5LOQhAXhFabaLZUvcWyAP0+w6S56yYLXSQJ9h5dyluBOXqcx07HqZKDNqG 5A6FOhFJ5/dzvtkrj9rWID1kT9O6LLRhbTv4EBAwlTYCJMbs3Y+r3wZOXaK6XC0eyF0Q Y4v14CdBa8b2fUxmy9X0DoTpQB8y/R2nU8ybTdmaST7qPbpPjgIT2LKBymIuy/2ysqmN cAAIuPCr6GUhh+D7nP8ETmO4WBlEtb04wfKKDjwCkIH7ThwflRT5xzia4ldWpRCJELJG CR2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721825191; x=1722429991; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hBJWQ1t6QCM1T5bdDmDCHfCfTpW6DNC9TA/sHMK/e6E=; b=stv+Jm1m6Wi2IHO+zKdMbv65r2VzXLus/Gc0Yaua37Qjfe+6Pq2cpJ6NbTk31Njs6h okU2VsErryEV3q51bKq3uZJrSyicOK/J+ayt0sW9Da+g+17NN3iyTsSv5leelC2hzVDK 8qHuryXOBf2opa7Ssuq5zcHFTaA+BiORSygD149KhHV0FOXQQxULqSJrMZ/e4DNpEkUn yL/qoAJIFnIe+4U5RDfRrU/Vee/UZfUo6OW9Xw9IzMgV9Tqvn6DydeSQX9j5m0X3x6ht yldNmUgpj7tRCyQPURzuubhwD1O4lNVDlz6TAgY8YTr9HjBirVnby8sZ4OQ1YU45nvoE bBiQ== X-Forwarded-Encrypted: i=1; AJvYcCW/JGjcTObuD4TRxZ/RXRzH1opmh71e1/jQrVg7lNnovTuo2Jc8CAwHRn/t8C1idsiVL6g0Fk6qbPsO/cw8G7Kju6g= X-Gm-Message-State: AOJu0YzMcdjg5meHnyOo/wb0c1OLI/HtBNQq14vWW8zx0m9IP+/R+wSi W9FHVINCq0dQwpBzLTrINf0SArPdrel7r9SlajnM0W3ivLbJx/a/ X-Google-Smtp-Source: AGHT+IH6M0HvSoPJFLKPUL5jhfc+xNB6S7usqtaEDwG6DOU1XH4t2Wf/WLzP9XxIEsHXMbYpzciKtA== X-Received: by 2002:a05:6a21:9989:b0:1c3:b1b3:75cf with SMTP id adf61e73a8af0-1c45e5d127fmr3556808637.14.1721825190542; Wed, 24 Jul 2024 05:46:30 -0700 (PDT) Received: from AHUANG12-3ZHH9X.lenovo.com (220-143-182-11.dynamic-ip.hinet.net. [220.143.182.11]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1fd6f28bc6fsm92998615ad.65.2024.07.24.05.46.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jul 2024 05:46:30 -0700 (PDT) From: Adrian Huang X-Google-Original-From: Adrian Huang To: urezki@gmail.com Cc: adrianhuang0701@gmail.com, ahuang12@lenovo.com, akpm@linux-foundation.org, hch@infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, peterz@infradead.org, sunjw10@lenovo.com Subject: Re: [PATCH 1/1] mm/vmalloc: Add preempt point in purge_vmap_node() when enabling kasan Date: Wed, 24 Jul 2024 20:46:24 +0800 Message-Id: <20240724124624.27673-1-ahuang12@lenovo.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Stat-Signature: omkw9cyqrfyea37hdu81cxhjdjmya7h5 X-Rspamd-Queue-Id: 0F475180013 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1721825191-397512 X-HE-Meta: U2FsdGVkX1/422uZomLxeJqj0Eo2Tk7f+KbhWf0orpx2wdbKlag57x0ZrdUJ1HsFMj+wx83eQJrgLu9DGYadxQlpyD26Vb40Uw7HfakkSoCGEa2o/DAXkIjzlAjIqJExHZA2ZAMJFaRz30kF/MdDIqfB0Cy1DHGpq5BypmOkfiIHTAo/D2v5MFNnnqNN8BAN+vUEj7xKfT4GMPjzEI+NUtqyYef3AVzwILeTCM9Z5TaID3du4Jf4rmvHaTlF4FLeMiSD0UXmBdWAOmULQ+SwT5/qpP1TOAKE7MgkBAQNr2kx9hl+qMjLltrUFLTmcsDshn4Po6XMzpTDAOEeIrFJ0PkTp56+lMFHS+l5R963pQJJwWTMp2AiFxiINvbRvwql1HJkOc1qZceNSMaK6+NZp6OH9ob4NXWP11wQdshGoOMScMrCXWHLA3fQ6G3t9QmujA/xkFGYjJMHU+K/4ynTtjxx5RwHrYTf2AxrS34FvVcT/IyDphqj7fa299a/vDLSWzEAfysMBZGJGPaEtvhIv/s7+obNuW7ps02QdZbs2DUo3LXLm+SR/S0nZcDT6NfakUoZqsSAYZHG0gV9X2IiMHSLPpirpB/tnpz/125KMBJzGromgLWDsEXGSdxz4ro5OcTkAl8qjVNjgfMcTwe6HNxsoaLf1N673GmQqrVSBzX3fiL5sZE3ZI+VsrafgdG4x0dqcQn77mKk+03bDeI0+EKdSPsFEbIeX39eIubjNqAtKkvn1Z44rlck5NIxPFOJLIrY9DbOWMEfRv6SB8ynJ9FJMfPEetOyxVDPA8w3bxD2SHGmOnJUxi+yQgv7eBYQ3oYFVy2FcuZyavZDaiVf6Bd3QJEwIU8GrkCtKCEBsWXqXDkV2NEmknvalJhF4W56Tp+mj5VC6+6KkrN9fVY5ReY+XsbWrfRxuxzVPHsmw8E+5Vd3XNxethbCYreObbMYloQZLfxi/5fxhRGRQnq et/Rgmy2 GLqaASLAYug5LxFnH/teYFsl+BGowA/y6tz96Asrs4S2PIumaAW8B5f6cyS+5OJOPtZPlK7iWtjjHCGhmIEHpKM5tbSdKh7tTqWbKs8Vm5tkEyel8+0ctQHytbELNkkTflW7M9f4gmHAI/AYWwO0vs64bBwYc7C7qlqJI/BK9Tt1eF7vhXhM/vXqbeMLVI7aeMBncibqGz9LMO/QZVWkUYp5l05PC4uVIauEwgBAkZ2TiP6mKbcCYC4TSgWUyhaa8aOcCIAVZLX/9gvNykn4OtDdIn9yQ7Whty+u6P2Ziq+ug6vIxzZAs0MsuMnmkjm3KsD8BvJZ9YaTG8wPs4SayJuE8tg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: > It works great and does not generate the soft-lock-up splat :) > See below some comments: Great. Thanks for the confirmation. >> + kasan_release_vmalloc(start, end, start, end, KASAN_VMALLOC_TLB_FLUSH); >> + >> > Do we need it here? We just did the TLB flush for en entire range in the > __purge_vmap_area_lazy(). So, it is two times invoked and looks odd to me. > > Am i missing something? 1. The TLB flush for the entire range in __purge_vmap_area_lazy() is for the vmalloc virtual address (VMALLOC_START->VMALLOC_END). 2. The TLB flush in purge_vmap_node() is for the KASAN shadow virtual address (the shadow offset 'CONFIG_KASAN_SHADOW_OFFSET' is defined in .config). BTW, I found my first patch has the potential risk. We need to flush TLB of the KASAN shadow virtual address firstly. Please see the following patch for detail. (I put the comment in the following patch). The following patch also works well on my 256-core machine. If you're ok with the patch, I'll submit it for upstream review. And, may I have your tag(s): tested-by/reviewed-by? (If possible, could you please have a test for the following patch). Thanks. --- diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 70d6a8f6e25d..ddbf42a1a7b7 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -55,6 +55,9 @@ extern p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D]; int kasan_populate_early_shadow(const void *shadow_start, const void *shadow_end); +#define KASAN_VMALLOC_PAGE_RANGE 0x1 /* Apply exsiting page range */ +#define KASAN_VMALLOC_TLB_FLUSH 0x2 /* TLB flush */ + #ifndef kasan_mem_to_shadow static inline void *kasan_mem_to_shadow(const void *addr) { @@ -511,7 +514,8 @@ void kasan_populate_early_vm_area_shadow(void *start, unsigned long size); int kasan_populate_vmalloc(unsigned long addr, unsigned long size); void kasan_release_vmalloc(unsigned long start, unsigned long end, unsigned long free_region_start, - unsigned long free_region_end); + unsigned long free_region_end, + unsigned long flags); #else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ @@ -526,7 +530,8 @@ static inline int kasan_populate_vmalloc(unsigned long start, static inline void kasan_release_vmalloc(unsigned long start, unsigned long end, unsigned long free_region_start, - unsigned long free_region_end) { } + unsigned long free_region_end, + unsigned long flags) { } #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ @@ -561,7 +566,8 @@ static inline int kasan_populate_vmalloc(unsigned long start, static inline void kasan_release_vmalloc(unsigned long start, unsigned long end, unsigned long free_region_start, - unsigned long free_region_end) { } + unsigned long free_region_end, + unsigned long flags) { } static inline void *kasan_unpoison_vmalloc(const void *start, unsigned long size, diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index d6210ca48dda..88d1c9dcb507 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -489,7 +489,8 @@ static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr, */ void kasan_release_vmalloc(unsigned long start, unsigned long end, unsigned long free_region_start, - unsigned long free_region_end) + unsigned long free_region_end, + unsigned long flags) { void *shadow_start, *shadow_end; unsigned long region_start, region_end; @@ -522,12 +523,17 @@ void kasan_release_vmalloc(unsigned long start, unsigned long end, __memset(shadow_start, KASAN_SHADOW_INIT, shadow_end - shadow_start); return; } - apply_to_existing_page_range(&init_mm, + + + if (flags & KASAN_VMALLOC_PAGE_RANGE) + apply_to_existing_page_range(&init_mm, (unsigned long)shadow_start, size, kasan_depopulate_vmalloc_pte, NULL); - flush_tlb_kernel_range((unsigned long)shadow_start, - (unsigned long)shadow_end); + + if (flags & KASAN_VMALLOC_TLB_FLUSH) + flush_tlb_kernel_range((unsigned long)shadow_start, + (unsigned long)shadow_end); } } diff --git a/mm/vmalloc.c b/mm/vmalloc.c index e34ea860153f..12cdc92cdb83 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2193,8 +2193,22 @@ static void purge_vmap_node(struct work_struct *work) struct vmap_area *va, *n_va; LIST_HEAD(local_list); + unsigned long start; + unsigned long end; + vn->nr_purged = 0; + start = list_first_entry(&vn->purge_list, struct vmap_area, list)->va_start; + + end = list_last_entry(&vn->purge_list, struct vmap_area, list)->va_end; + + /* + * Since node_pool_add_va() returns vmap_area(s) to its pool, the + * returned vmap_area(s) might be grabbed immediately via node_alloc() + * by other core. We need to flush TLB firstly. + */ + kasan_release_vmalloc(start, end, start, end, KASAN_VMALLOC_TLB_FLUSH); + list_for_each_entry_safe(va, n_va, &vn->purge_list, list) { unsigned long nr = (va->va_end - va->va_start) >> PAGE_SHIFT; unsigned long orig_start = va->va_start; @@ -2205,7 +2219,8 @@ static void purge_vmap_node(struct work_struct *work) if (is_vmalloc_or_module_addr((void *)orig_start)) kasan_release_vmalloc(orig_start, orig_end, - va->va_start, va->va_end); + va->va_start, va->va_end, + KASAN_VMALLOC_PAGE_RANGE); atomic_long_sub(nr, &vmap_lazy_nr); vn->nr_purged++; @@ -4726,7 +4741,8 @@ struct vm_struct **pcpu_get_vm_areas(const unsigned long *offsets, &free_vmap_area_list); if (va) kasan_release_vmalloc(orig_start, orig_end, - va->va_start, va->va_end); + va->va_start, va->va_end, + KASAN_VMALLOC_PAGE_RANGE | KASAN_VMALLOC_TLB_FLUSH); vas[area] = NULL; } @@ -4776,7 +4792,8 @@ struct vm_struct **pcpu_get_vm_areas(const unsigned long *offsets, &free_vmap_area_list); if (va) kasan_release_vmalloc(orig_start, orig_end, - va->va_start, va->va_end); + va->va_start, va->va_end, + KASAN_VMALLOC_PAGE_RANGE | KASAN_VMALLOC_TLB_FLUSH); vas[area] = NULL; kfree(vms[area]); }