From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=3.0 tests=BAYES_00,DKIM_ADSP_ALL, DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95344C63697 for ; Sat, 28 Nov 2020 14:27:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E7EF724686 for ; Sat, 28 Nov 2020 14:27:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=natalenko.name header.i=@natalenko.name header.b="k7ihDQwC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E7EF724686 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=natalenko.name Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0D3EA6B005D; Sat, 28 Nov 2020 09:27:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 084086B0068; Sat, 28 Nov 2020 09:27:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EDBF36B006C; Sat, 28 Nov 2020 09:27:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0060.hostedemail.com [216.40.44.60]) by kanga.kvack.org (Postfix) with ESMTP id D89CD6B005D for ; Sat, 28 Nov 2020 09:27:25 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 9B36B824999B for ; Sat, 28 Nov 2020 14:27:25 +0000 (UTC) X-FDA: 77534054850.02.rod17_36027af27391 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin02.hostedemail.com (Postfix) with ESMTP id 7E02A10097AA0 for ; Sat, 28 Nov 2020 14:27:25 +0000 (UTC) X-HE-Tag: rod17_36027af27391 X-Filterd-Recvd-Size: 7978 Received: from vulcan.natalenko.name (vulcan.natalenko.name [104.207.131.136]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Sat, 28 Nov 2020 14:27:24 +0000 (UTC) Received: from localhost (home.natalenko.name [151.237.229.131]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by vulcan.natalenko.name (Postfix) with ESMTPSA id A189A8B18C9; Sat, 28 Nov 2020 15:27:23 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=natalenko.name; s=dkim-20170712; t=1606573643; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rk41III0tAkUh4u/hIKJEcn4ibWW3TcSORMYy2GTzN8=; b=k7ihDQwCULTU9S/ikzBoxo1P0RtVceKrJkARHROduIcWV2nNClsB0dRBpUryKPk+QZYR7z kMB/UMQqQz3n0IUfkabPbbhnE81kTeAMlEaclWEyzg7dxKAFjnX+Eq5UPEXfdpagu+EcPI 0FsVN+B8TZSEAlacTVifrcfQ64gsZ7o= Date: Sat, 28 Nov 2020 15:27:23 +0100 From: Oleksandr Natalenko To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, Andrew Morton , Sebastian Andrzej Siewior , Steven Rostedt , Mike Galbraith , Thomas Gleixner , linux-rt-users@vger.kernel.org Subject: Re: scheduling while atomic in z3fold Message-ID: <20201128142723.zik6d5skvt3uwu5f@spock.localdomain> References: <20201128140523.ovmqon5fjetvpby4@spock.localdomain> <20201128140924.iyqr2h52z2olt6zb@spock.localdomain> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20201128140924.iyqr2h52z2olt6zb@spock.localdomain> Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sat, Nov 28, 2020 at 03:09:24PM +0100, Oleksandr Natalenko wrote: > > While running v5.10-rc5-rt11 I bumped into the following: > >=20 > > ``` > > BUG: scheduling while atomic: git/18695/0x00000002 > > Preemption disabled at: > > [] z3fold_zpool_malloc+0x463/0x6e0 > > =E2=80=A6 > > Call Trace: > > dump_stack+0x6d/0x88 > > __schedule_bug.cold+0x88/0x96 > > __schedule+0x69e/0x8c0 > > preempt_schedule_lock+0x51/0x150 > > rt_spin_lock_slowlock_locked+0x117/0x2c0 > > rt_spin_lock_slowlock+0x58/0x80 > > rt_spin_lock+0x2a/0x40 > > z3fold_zpool_malloc+0x4c1/0x6e0 > > zswap_frontswap_store+0x39c/0x980 > > __frontswap_store+0x6e/0xf0 > > swap_writepage+0x39/0x70 > > shmem_writepage+0x31b/0x490 > > pageout+0xf4/0x350 > > shrink_page_list+0xa28/0xcc0 > > shrink_inactive_list+0x300/0x690 > > shrink_lruvec+0x59a/0x770 > > shrink_node+0x2d6/0x8d0 > > do_try_to_free_pages+0xda/0x530 > > try_to_free_pages+0xff/0x260 > > __alloc_pages_slowpath.constprop.0+0x3d5/0x1230 > > __alloc_pages_nodemask+0x2f6/0x350 > > allocate_slab+0x3da/0x660 > > ___slab_alloc+0x4ff/0x760 > > __slab_alloc.constprop.0+0x7a/0x100 > > kmem_cache_alloc+0x27b/0x2c0 > > __d_alloc+0x22/0x230 > > d_alloc_parallel+0x67/0x5e0 > > __lookup_slow+0x5c/0x150 > > path_lookupat+0x2ea/0x4d0 > > filename_lookup+0xbf/0x210 > > vfs_statx.constprop.0+0x4d/0x110 > > __do_sys_newlstat+0x3d/0x80 > > do_syscall_64+0x33/0x40 > > entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > ``` > >=20 > > The preemption seems to be disabled here: > >=20 > > ``` > > $ scripts/faddr2line mm/z3fold.o z3fold_zpool_malloc+0x463 > > z3fold_zpool_malloc+0x463/0x6e0: > > add_to_unbuddied at mm/z3fold.c:645 > > (inlined by) z3fold_alloc at mm/z3fold.c:1195 > > (inlined by) z3fold_zpool_malloc at mm/z3fold.c:1737 > > ``` > >=20 > > The call to the rt_spin_lock() seems to be here: > >=20 > > ``` > > $ scripts/faddr2line mm/z3fold.o z3fold_zpool_malloc+0x4c1 > > z3fold_zpool_malloc+0x4c1/0x6e0: > > add_to_unbuddied at mm/z3fold.c:649 > > (inlined by) z3fold_alloc at mm/z3fold.c:1195 > > (inlined by) z3fold_zpool_malloc at mm/z3fold.c:1737 > > ``` > >=20 > > Or, in source code: > >=20 > > ``` > > 639 /* Add to the appropriate unbuddied list */ > > 640 static inline void add_to_unbuddied(struct z3fold_pool *pool, > > 641 struct z3fold_header *zhdr) > > 642 { > > 643 if (zhdr->first_chunks =3D=3D 0 || zhdr->last_chunks =3D=3D = 0 || > > 644 zhdr->middle_chunks =3D=3D 0) { > > 645 struct list_head *unbuddied =3D get_cpu_ptr(pool->unbudd= ied); > > 646 > > 647 int freechunks =3D num_free_chunks(zhdr); > > 648 spin_lock(&pool->lock); > > 649 list_add(&zhdr->buddy, &unbuddied[freechunks]); > > 650 spin_unlock(&pool->lock); > > 651 zhdr->cpu =3D smp_processor_id(); > > 652 put_cpu_ptr(pool->unbuddied); > > 653 } > > 654 } > > ``` > >=20 > > Shouldn't the list manipulation be protected with > > local_lock+this_cpu_ptr instead of get_cpu_ptr+spin_lock? Totally untested: ``` diff --git a/mm/z3fold.c b/mm/z3fold.c index 18feaa0bc537..53fcb80c6167 100644 --- a/mm/z3fold.c +++ b/mm/z3fold.c @@ -41,6 +41,7 @@ #include #include #include +#include #include #include #include @@ -156,6 +157,7 @@ struct z3fold_pool { const char *name; spinlock_t lock; spinlock_t stale_lock; + local_lock_t llock; struct list_head *unbuddied; struct list_head lru; struct list_head stale; @@ -642,14 +644,17 @@ static inline void add_to_unbuddied(struct z3fold_p= ool *pool, { if (zhdr->first_chunks =3D=3D 0 || zhdr->last_chunks =3D=3D 0 || zhdr->middle_chunks =3D=3D 0) { - struct list_head *unbuddied =3D get_cpu_ptr(pool->unbuddied); + struct list_head *unbuddied; + int freechunks; + local_lock(&pool->llock); + unbuddied =3D *this_cpu_ptr(&pool->unbuddied); =20 - int freechunks =3D num_free_chunks(zhdr); + freechunks =3D num_free_chunks(zhdr); spin_lock(&pool->lock); list_add(&zhdr->buddy, &unbuddied[freechunks]); spin_unlock(&pool->lock); zhdr->cpu =3D smp_processor_id(); - put_cpu_ptr(pool->unbuddied); + local_unlock(&pool->llock); } } =20 @@ -887,7 +892,8 @@ static inline struct z3fold_header *__z3fold_alloc(st= ruct z3fold_pool *pool, =20 lookup: /* First, try to find an unbuddied z3fold page. */ - unbuddied =3D get_cpu_ptr(pool->unbuddied); + local_lock(&pool->llock); + unbuddied =3D *this_cpu_ptr(&pool->unbuddied); for_each_unbuddied_list(i, chunks) { struct list_head *l =3D &unbuddied[i]; =20 @@ -905,7 +911,7 @@ static inline struct z3fold_header *__z3fold_alloc(st= ruct z3fold_pool *pool, !z3fold_page_trylock(zhdr)) { spin_unlock(&pool->lock); zhdr =3D NULL; - put_cpu_ptr(pool->unbuddied); + local_unlock(&pool->llock); if (can_sleep) cond_resched(); goto lookup; @@ -919,7 +925,7 @@ static inline struct z3fold_header *__z3fold_alloc(st= ruct z3fold_pool *pool, test_bit(PAGE_CLAIMED, &page->private)) { z3fold_page_unlock(zhdr); zhdr =3D NULL; - put_cpu_ptr(pool->unbuddied); + local_unlock(&pool->llock); if (can_sleep) cond_resched(); goto lookup; @@ -934,7 +940,7 @@ static inline struct z3fold_header *__z3fold_alloc(st= ruct z3fold_pool *pool, kref_get(&zhdr->refcount); break; } - put_cpu_ptr(pool->unbuddied); + local_unlock(&pool->llock); =20 if (!zhdr) { int cpu; @@ -1005,6 +1011,7 @@ static struct z3fold_pool *z3fold_create_pool(const= char *name, gfp_t gfp, goto out_c; spin_lock_init(&pool->lock); spin_lock_init(&pool->stale_lock); + local_lock_init(&pool->llock); pool->unbuddied =3D __alloc_percpu(sizeof(struct list_head)*NCHUNKS, 2)= ; if (!pool->unbuddied) goto out_pool; ``` --=20 Oleksandr Natalenko (post-factum)