From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E811C433EF for ; Tue, 21 Jun 2022 14:29:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6B8F68E0001; Tue, 21 Jun 2022 10:29:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 668BB6B0073; Tue, 21 Jun 2022 10:29:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 558508E0001; Tue, 21 Jun 2022 10:29:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 428226B0072 for ; Tue, 21 Jun 2022 10:29:11 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 1A47A342E8 for ; Tue, 21 Jun 2022 14:29:11 +0000 (UTC) X-FDA: 79602475302.17.3DED6F6 Received: from mail-lf1-f42.google.com (mail-lf1-f42.google.com [209.85.167.42]) by imf23.hostedemail.com (Postfix) with ESMTP id A628014001A for ; Tue, 21 Jun 2022 14:29:10 +0000 (UTC) Received: by mail-lf1-f42.google.com with SMTP id f39so7064238lfv.3 for ; Tue, 21 Jun 2022 07:29:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:date:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=yFwOkX5u5xV89mKMUoEbIG06wYYhgJqYwetvw1ZUjd4=; b=owwv9fTp+ThPxppT/bUM8W4zSlE7eydJgrzcdiOBkgF/UzE786XjbBJFh4LNuqzwPV FcdKfRDCJwQtYpbddfkj4EJRnCGWNToze0MtV5AojiF0rwwu97Huq6j6JOC8ujvBii5A 5VRpxLArMhwGEr3fjmfnCXfZSr3a/LX7dswiaox80JZOQHE2zEJXA3Si6p/hfdeklbGZ NcCks14qr9KKQuGFmwDUt3jss/F9gqfJ1VdNorGx+lZXQKoR1xigsBvJxQobnYyZE2PS mLqZl1SdKlnttpDOp7mvn1w96B16y91XrBcdiUm1043ST2IjdFCqUnudDHurVGHavjM1 NGxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:date:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=yFwOkX5u5xV89mKMUoEbIG06wYYhgJqYwetvw1ZUjd4=; b=jl7hp/k3YGIhn4Ugh1BtdpG1fi10EPhySfYCup8CL+pLra4vNW+wmkTM/+NxSJqZOi 0ALhvnQxt16kFlNpvgfEYsESJ+g0u00cCK54oAKZrWlUt152o47YQ/Cknp7dgqHjR2E3 GOB6Vbr9gUQeDDl6sTxmDCvJDUCT+imAfHSeSGEsp3MlLCVN2DkioaLcYS0HTvdiyGD2 NtRZmCH+VjV/C8YQYiTjLLxR4q68L9bov76ClID2JGT/8vMF4V5IzdeVNp5u180GANLe UTrSTCtcS+NVI4etb6Q8RFrUQCjhmp2QwMXneOJaFEQiMQdc2xG+0j3eX3/FwHNpF+uu G94g== X-Gm-Message-State: AJIora8igvOedeAywLP94+yn2v6WuwhCRQjXuMs0qMGztYZYkTEk3Y06 iJH9ze1G9H0ABZa3B7I4cp7RK290D7hpJI+vjrs= X-Google-Smtp-Source: AGRyM1u9rwxeINwOLlaQMUTf5uUrbla4nc3wiEJsto1vaZZM6hRcm1M4u2etSfdlQY1qQOSHAUMcUw== X-Received: by 2002:a05:6512:b1c:b0:47d:df52:b5a9 with SMTP id w28-20020a0565120b1c00b0047ddf52b5a9mr15981152lfu.293.1655821748868; Tue, 21 Jun 2022 07:29:08 -0700 (PDT) Received: from pc638.lan ([155.137.26.201]) by smtp.gmail.com with ESMTPSA id n11-20020a2e878b000000b0025a6e47056csm850841lji.124.2022.06.21.07.29.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Jun 2022 07:29:08 -0700 (PDT) From: Uladzislau Rezki X-Google-Original-From: Uladzislau Rezki Date: Tue, 21 Jun 2022 16:29:06 +0200 To: Zhaoyang Huang Cc: Uladzislau Rezki , "zhaoyang.huang" , Andrew Morton , "open list:MEMORY MANAGEMENT" , LKML , Ke Wang , Christoph Hellwig Subject: Re: [PATCH] mm: fix racing of vb->va when kasan enabled Message-ID: References: <1653447164-15017-1-git-send-email-zhaoyang.huang@unisoc.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655821750; a=rsa-sha256; cv=none; b=OTFNN8kfG6qQkC0NKNoSBFChuwBRGC+Cj+n/iHo7qLSxg9hDzJBjafkoYw0qX7K11/f7Pe Hv+Iv9Rjy9RdEYxfCZxRxz4wFHLAFPdCld7qAyzGOtBg8zL8GchbIMWDVzn0wKW9tmDUG+ GSojlsjhvEIIOhEC+/6ChaED9vbGT+g= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=owwv9fTp; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf23.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.42 as permitted sender) smtp.mailfrom=urezki@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655821750; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yFwOkX5u5xV89mKMUoEbIG06wYYhgJqYwetvw1ZUjd4=; b=ImlbSIF0jKOoL33BsrMrmrxenb/0plAx8QHegR2dPAVC+RRod4/euAocdj0X61keXr70hf JaQ/T1SoK8vNMUPe7ERMdSxAvGys2ShpjLewloxorJlxTXTLg0vJ3ZFmACwRVNf77jNaHz 4HMYvK6tORutg/lnR9wXt+aDFJExPls= X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: A628014001A X-Stat-Signature: 999ofj1h37b3yr8acgz4zoo1rg3bjtzn X-Rspam-User: Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=owwv9fTp; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf23.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.42 as permitted sender) smtp.mailfrom=urezki@gmail.com X-HE-Tag: 1655821750-145391 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > On Tue, Jun 21, 2022 at 5:27 PM Uladzislau Rezki wrote: > > > > > On Mon, Jun 20, 2022 at 6:44 PM Uladzislau Rezki wrote: > > > > > > > > > > > > > > > > > Is it easy to reproduce? If so could you please describe the steps? As i see > > > > > > the freeing of the "vb" is RCU safe whereas vb->va is not. But from the first > > > > > > glance i do not see how it can accessed twice. Hm.. > > > > > It was raised from a monkey test on A13_k515 system and got 1/20 pcs > > > > > failed. IMO, vb->va which out of vmap_purge_lock protection could race > > > > > with a concurrent ra freeing within __purge_vmap_area_lazy. > > > > > > > > > Do you have exact steps how you run "monkey" test? > > > There are about 30+ kos inserted during startup which could be a > > > specific criteria for reproduction. Do you have doubts about the test > > > result or the solution? > > > > > > I do not have any doubt about your test results, so if you can trigger it > > then there is an issue at least on the 5.4.161-android12 kernel. > > > > 1. With your fix we get expanded mutex range, thus the worst case of vmalloc > > allocation can be increased when it fails and repeat. Because it also invokes > > the purge_vmap_area_lazy() that access the same mutex. > I am not sure I get your point. _vm_unmap_aliases calls > _purge_vmap_area_lazy instead of purge_vmap_area_lazy. Do you have any > other solutions? I really don't think my patch is the best way as I > don't have a full view of vmalloc mechanism. > Yep, but it holds the mutex: mutex_lock(&vmap_purge_lock); purge_fragmented_blocks_allcpus(); if (!__purge_vmap_area_lazy(start, end) && flush) flush_tlb_kernel_range(start, end); mutex_unlock(&vmap_purge_lock); I do not have a solution yet. I am trying still to figure out how you can trigger it. rcu_read_lock(); list_for_each_entry_rcu(vb, &vbq->free, free_list) { spin_lock(&vb->lock); if (vb->dirty && vb->dirty != VMAP_BBMAP_BITS) { unsigned long va_start = vb->va->va_start; so you say that "vb->va->va_start" can be accessed twice. I do not see how it can happen. The purge_fragmented_blocks() removes "vb" from the free_list and set vb->dirty to the VMAP_BBMAP_BITS to prevent purging it again. It is protected by the spin_lock(&vb->lock): spin_lock(&vb->lock); if (vb->free + vb->dirty == VMAP_BBMAP_BITS && vb->dirty != VMAP_BBMAP_BITS) { vb->free = 0; /* prevent further allocs after releasing lock */ vb->dirty = VMAP_BBMAP_BITS; /* prevent purging it again */ vb->dirty_min = 0; vb->dirty_max = VMAP_BBMAP_BITS; so the VMAP_BBMAP_BITS is set under spinlock. The _vm_unmap_aliases() checks it: list_for_each_entry_rcu(vb, &vbq->free, free_list) { spin_lock(&vb->lock); if (vb->dirty && vb->dirty != VMAP_BBMAP_BITS) { unsigned long va_start = vb->va->va_start; unsigned long s, e; if the "vb->dirty != VMAP_BBMAP_BITS". I am missing your point here? > > > > 2. You run 5.4.161-android12 kernel what is quite old. Could you please > > retest with latest kernel? I am asking because on the latest kernel with > > CONFIG_KASAN i am not able to reproduce it. > > > > I do a lot of: vm_map_ram()/vm_unmap_ram()/vmalloc()/vfree() in parallel > > by 64 kthreads on my 64 CPUs test system. > The failure generates at 20s from starting up, I think it is a rare timing. > > > > Could you please confirm that you can trigger an issue on the latest kernel? > Sorry, I don't have an available latest kernel for now. > Can you do: "gdb ./vmlinux", execute "l *_vm_unmap_aliases+0x164" and provide output? -- Uladzislau Rezki