From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04B7EC433EF for ; Tue, 7 Sep 2021 20:05:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 929BC61100 for ; Tue, 7 Sep 2021 20:05:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 929BC61100 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linuxfoundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id BA3DC900002; Tue, 7 Sep 2021 16:05:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B2C6F6B0072; Tue, 7 Sep 2021 16:05:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F5D8900002; Tue, 7 Sep 2021 16:05:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0156.hostedemail.com [216.40.44.156]) by kanga.kvack.org (Postfix) with ESMTP id 8DB726B0071 for ; Tue, 7 Sep 2021 16:05:41 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 38A7882499A8 for ; Tue, 7 Sep 2021 20:05:41 +0000 (UTC) X-FDA: 78561857682.01.6AEC384 Received: from mail-il1-f169.google.com (mail-il1-f169.google.com [209.85.166.169]) by imf18.hostedemail.com (Postfix) with ESMTP id E59CD400208E for ; Tue, 7 Sep 2021 20:05:40 +0000 (UTC) Received: by mail-il1-f169.google.com with SMTP id u17so269835ilm.13 for ; Tue, 07 Sep 2021 13:05:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=qBeV/3OVpeBGqoC50W6WyV/Bm/mvFYoJ4MmFvZCO8nQ=; b=V3OZERQ7Iaruu0XIKpcZqON5EAPtH4KYFTa7rEhUEp8ixXqsDJEtcDEB3z9/EXE+F9 zBE4ktNYiOqFT7acLIw1mz+WwcaPJ7TMKeAltOam7FdPoWonV0rhKGPypTVMPazrlMq6 X3ngHxfEN4EHE57hKFnPsyzOqJmVS3BeOvmmE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=qBeV/3OVpeBGqoC50W6WyV/Bm/mvFYoJ4MmFvZCO8nQ=; b=Ymhg/BGrmg0K+BYF8m8cOB+jnOVeN8RoPzDoysq01k+A0e9SBBimd2IbUIYFgDaByz FkcNNyl+aXlk3ZNjG+7YXdNDf2OrUy+KP7kyUaqfZRuWfb3Qm6VG33qNBxWwGChwzmNW hnONZKZ+HE35LRvjZ0xZVo1tfvwEulh38zJJMTfHZNTRwcmhclqNpyMxHkPcg5b8uiMi 4vMpnDvWDtTrPlKRDpyWBTwBSV9DbVGdyTKJbZcK7/HyfctcsUKDO8B5oHGz9HtNsKOB Hvf5SA/LCQvTCaGGMlfJsjD/sAWlGqwTy3q/Fbe+Kf9KAA919Fmc2QGdynzcV96RYoxl ujTg== X-Gm-Message-State: AOAM532IT764rbg17HrNtt4B05m/erlAiHowILBijzL/6r2iPNY0IqyC zWlWZV+DsMcPrEg/LYglcIEC3Q== X-Google-Smtp-Source: ABdhPJzgWaSM/EHxvlWBSzY+jZR2X/KwdW76ujEZfr8UVOQJ6v8iRdP84LNEnpfIcspW7drE+x/iCQ== X-Received: by 2002:a05:6e02:f03:: with SMTP id x3mr13623361ilj.315.1631045140169; Tue, 07 Sep 2021 13:05:40 -0700 (PDT) Received: from [192.168.1.112] (c-24-9-64-241.hsd1.co.comcast.net. [24.9.64.241]) by smtp.gmail.com with ESMTPSA id g14sm63699ila.28.2021.09.07.13.05.37 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 07 Sep 2021 13:05:38 -0700 (PDT) Subject: Re: [PATCH 0/6] stackdepot, kasan, workqueue: Avoid expanding stackdepot slabs when holding raw_spin_lock To: Marco Elver , Andrew Morton Cc: Tejun Heo , Lai Jiangshan , Andrey Konovalov , Walter Wu , Andrey Ryabinin , Alexander Potapenko , Dmitry Vyukov , Vijayanand Jitta , Vinayak Menon , "Gustavo A. R. Silva" , kasan-dev@googlegroups.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Aleksandr Nogikh , Taras Madan , Shuah Khan References: <20210907141307.1437816-1-elver@google.com> From: Shuah Khan Message-ID: <69f98dbd-e754-c34a-72cf-a62c858bcd2f@linuxfoundation.org> Date: Tue, 7 Sep 2021 14:05:37 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: <20210907141307.1437816-1-elver@google.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=linuxfoundation.org header.s=google header.b=V3OZERQ7; spf=pass (imf18.hostedemail.com: domain of skhan@linuxfoundation.org designates 209.85.166.169 as permitted sender) smtp.mailfrom=skhan@linuxfoundation.org; dmarc=pass (policy=none) header.from=linuxfoundation.org X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: E59CD400208E X-Stat-Signature: 8ogam8y58cejhbk4uo4bs16biga9ibd1 X-HE-Tag: 1631045140-213010 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 9/7/21 8:13 AM, Marco Elver wrote: > Shuah Khan reported [1]: > > | When CONFIG_PROVE_RAW_LOCK_NESTING=y and CONFIG_KASAN are enabled, > | kasan_record_aux_stack() runs into "BUG: Invalid wait context" when > | it tries to allocate memory attempting to acquire spinlock in page > | allocation code while holding workqueue pool raw_spinlock. > | > | There are several instances of this problem when block layer tries > | to __queue_work(). Call trace from one of these instances is below: > | > | kblockd_mod_delayed_work_on() > | mod_delayed_work_on() > | __queue_delayed_work() > | __queue_work() (rcu_read_lock, raw_spin_lock pool->lock held) > | insert_work() > | kasan_record_aux_stack() > | kasan_save_stack() > | stack_depot_save() > | alloc_pages() > | __alloc_pages() > | get_page_from_freelist() > | rm_queue() > | rm_queue_pcplist() > | local_lock_irqsave(&pagesets.lock, flags); > | [ BUG: Invalid wait context triggered ] > > [1] https://lkml.kernel.org/r/20210902200134.25603-1-skhan@linuxfoundation.org > > PROVE_RAW_LOCK_NESTING is pointing out that (on RT kernels) the locking > rules are being violated. More generally, memory is being allocated from > a non-preemptive context (raw_spin_lock'd c-s) where it is not allowed. > > To properly fix this, we must prevent stackdepot from replenishing its > "stack slab" pool if memory allocations cannot be done in the current > context: it's a bug to use either GFP_ATOMIC nor GFP_NOWAIT in certain > non-preemptive contexts, including raw_spin_locks (see gfp.h and > ab00db216c9c7). > > The only downside is that saving a stack trace may fail if: stackdepot > runs out of space AND the same stack trace has not been recorded before. > I expect this to be unlikely, and a simple experiment (boot the kernel) > didn't result in any failure to record stack trace from insert_work(). > > The series includes a few minor fixes to stackdepot that I noticed in > preparing the series. It then introduces __stack_depot_save(), which > exposes the option to force stackdepot to not allocate any memory. > Finally, KASAN is changed to use the new stackdepot interface and > provide kasan_record_aux_stack_noalloc(), which is then used by > workqueue code. > > Marco Elver (6): > lib/stackdepot: include gfp.h > lib/stackdepot: remove unused function argument > lib/stackdepot: introduce __stack_depot_save() > kasan: common: provide can_alloc in kasan_save_stack() > kasan: generic: introduce kasan_record_aux_stack_noalloc() > workqueue, kasan: avoid alloc_pages() when recording stack > > include/linux/kasan.h | 2 ++ > include/linux/stackdepot.h | 6 +++++ > kernel/workqueue.c | 2 +- > lib/stackdepot.c | 51 ++++++++++++++++++++++++++++++-------- > mm/kasan/common.c | 6 ++--- > mm/kasan/generic.c | 14 +++++++++-- > mm/kasan/kasan.h | 2 +- > 7 files changed, 65 insertions(+), 18 deletions(-) > Thank you. Tested all the 6 patches in this series on Linux 5.14. This problem exists in 5.13 and needs to be marked for both 5.14 and 5.13 stable releases. Here is my Tested-by: Shuah Khan thanks, -- Shuah