From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BEB3CF649D for ; Mon, 30 Sep 2024 09:49:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F0C056B0110; Mon, 30 Sep 2024 05:49:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EBA606B0117; Mon, 30 Sep 2024 05:49:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3FA76B0119; Mon, 30 Sep 2024 05:49:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id B30ED6B0110 for ; Mon, 30 Sep 2024 05:49:49 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 164261A09BD for ; Mon, 30 Sep 2024 09:49:49 +0000 (UTC) X-FDA: 82620932898.26.86FC0FE Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) by imf03.hostedemail.com (Postfix) with ESMTP id 40B7320005 for ; Mon, 30 Sep 2024 09:49:47 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ei2eLVQL; spf=pass (imf03.hostedemail.com: domain of adrianhuang0701@gmail.com designates 209.85.218.54 as permitted sender) smtp.mailfrom=adrianhuang0701@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727689723; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4Y8xzZhNdiTm4MfGV2ugvdzV/xOWaN3/MM1HAjacBWE=; b=UMCKNFJTFWILf/bMX6ikkEbgAnNcEoK4cT1WJDCJMk9BybxVyNJuI5+hjsgEBgwZ+W0eBy agQdXX/poN6wZdbiQAU6hKISAUbeuA0JoskFjL4nullTnayICZgukFOXMw8VemxbUDKX8S SfgglV98co0kux9Pi++jT6X3NnwrZN4= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ei2eLVQL; spf=pass (imf03.hostedemail.com: domain of adrianhuang0701@gmail.com designates 209.85.218.54 as permitted sender) smtp.mailfrom=adrianhuang0701@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727689723; a=rsa-sha256; cv=none; b=S3kgFelQ7e1S/YyjuLLJr1J2RL//0qUyrkfd0Pg5tRX6clfwWjwMBWUbn9P/l1pEpXM6/t +dVZUpiyPohDTk+MJ48CUKFDRVQmRB78EDif1wIQ6V2gQTMkYjsDBdgieIycLFG1zV9Pkj O0NCDrLtaIBCx/UwjOapCnDDl8drzjI= Received: by mail-ej1-f54.google.com with SMTP id a640c23a62f3a-a8d2b24b7a8so954176266b.1 for ; Mon, 30 Sep 2024 02:49:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727689786; x=1728294586; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=4Y8xzZhNdiTm4MfGV2ugvdzV/xOWaN3/MM1HAjacBWE=; b=ei2eLVQLCJxkJaYelN7XXpIndOxUMSa4o0vCdG2fWHSVR7DuP7TJwrCihEzuuCyiku BNug5eZ7MuH9s6ULMJnD3VA8h1w2n865HJ8G+vyrsncdQYdk3TxBm3P2Ze6doJOL9gcN RUfs2cdtRLNkFYcndWx6OCcJW8ab4zWhSXnr8KOTCdGQFxwniff4/T9coHSbGxK/FIvy 29tGvsRYiCtwd0V6OtuFZYtB1pkHBQ0p8UjVXZOATcelY7QKQrSpuQXajtdchS4EtWk/ vBGE93/2E4hmbu1FiiV4bYjZdg/ZVttFr7vleA1OphbVadsCSPypSJ0mAD8aFTSlQ/Nl HvmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727689786; x=1728294586; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4Y8xzZhNdiTm4MfGV2ugvdzV/xOWaN3/MM1HAjacBWE=; b=XrYAXaKsL1V68YlPWu1/7uwcXLIXmGwv0+BaROh09KpoSdUsCwpb9RyjRENF74+FZw JbSeRtGOAMHmvgx6fYOQZMPtJVyQHKtw3CWTCfCA646PMeuUU6x4vsS9j14EaGrJzhdK ivvxwyVS4BLp/MNm8x3ddYH6eJv3Sa/5eUV4/UmO3UDSqNpBlP47O1Cfdph9NtJs+Qy/ 7kxPvZCqXgVIVMv8c3H+f+QV5ZeVmYrDongQvNOOzhLpkG/QyCsD46j0EeyanOL8Aymo wJeihdEG4wsrIKN3dgEucyXxEqUtjKun2u6E4yGyL8VIHUDL+MenpWiEpCW0KQOEU2/S kpBw== X-Forwarded-Encrypted: i=1; AJvYcCVqTWz6blRCdTRfguQsEurQ726grHsRkVvZ3Lj4QI4oHq5CfUVoaXEvl4rFt6h27lVtzq0Vl/3uLA==@kvack.org X-Gm-Message-State: AOJu0YxgJDwvdQkSwlNKeeLuJS1iXVK/rPtTcJgOG1aXi+6YOakaz47t cz0vV9zo9YL9xiFe85IDn2uQFPNnWS9kxTHqDc9cPxxi8psrkPMnxl759v5+66DJzJp0c3fTkG3 IqPShwAJZYqaCJ8KrubHHKGbb9A8= X-Google-Smtp-Source: AGHT+IGrukCaE6n3y/KlOYVIqcP8OuyMBkkIZLziiyGt5z7sqLX4iDXjZoDHipRmikKBdBZ8YrX1uOnpGdjavPiaMsM= X-Received: by 2002:a17:907:6d25:b0:a91:1699:f8eb with SMTP id a640c23a62f3a-a93c320e687mr1288321266b.28.1727689785361; Mon, 30 Sep 2024 02:49:45 -0700 (PDT) MIME-Version: 1.0 References: <20240925134732.24431-1-ahuang12@lenovo.com> <20240925134706.2a0c2717a41a338d938581ff@linux-foundation.org> In-Reply-To: From: Huang Adrian Date: Mon, 30 Sep 2024 17:49:33 +0800 Message-ID: Subject: Re: [PATCH 1/1] kasan, vmalloc: avoid lock contention when depopulating vmalloc To: Uladzislau Rezki Cc: Andrew Morton , Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Adrian Huang Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 40B7320005 X-Stat-Signature: k77fraqzzurhq4i5x5waj4pkk5o43xty X-HE-Tag: 1727689787-531667 X-HE-Meta: U2FsdGVkX1+iV9ozEDXToQ1+RNcTSDuuOAJ3LztoBMpQ07pbYZhumy9K0J1KMtt6OsKNK+1hZUhseP8vfVlWGcCR7qYBzRKoFd0LBZ02Kf6hx1IrUYLXE14w9TwXZ2pw4zLuDwpPOA61bVlhIxR9gqWwRFiT0K+qW1s+ur5ki9LzZ727lMJaDT1fFP6uwWDJqbGqMUWXmzZLpkC/HfDaa0OaxwjaMLvAGhH+fJbzI02BKsNAwT5vDmkztb2LTw/4fN7J4NRYBJ0nSaeOXFR3x9+eUQgE8vbHoGK8HZS1BEN9AaVxCNxmhCRC35nEeZ0CYh2N6REJ+t5hJ94neYexAZYukrC4mDLxQWTLl2Sy3ldhLKjysNDwwbwM3+4IX8LOtWgLjJiXFwuDk9VL6LPPrGa929SiimdW6JLLonEqqZFX2gFAoe1aNh+ty9aPxVVq2l1CYplbBb6oMGbYm7J4EtNfQznaxV0qnN3DTycyNFRx4NoZP1AU3kZdIKwuahUfSrInSzRG8guUsJoLnXDzrC3BQzl29PBukvupeenoK85H3D58W8E7yiT2luQXi7Q8LuZDxev0Es4EC/LlHa+C+IR54R6ZMm6KrB3NRr9jyV7A6RLBviga8I+7wXwNa/MyI5wRdza0FUv7WYe43+S3dR/xxsW2z+g7FH0XeAz2/S3o/f4uC/wowFC+I+0pcuGTlC3inJ62CsEqNNKWgIEbmR8vctm/tWLMEqX2kjOA19I4p6vShDTS11lElY8m+er83Mg+q12C+KeVBJynbFVdetalA3X8c7vuGgYM5gJk4wC8QwiruxZy5y9i9XftralOCMP+Y4j5RSqh9TVARFW2jOecO2L07gARDcbKCH7qLTNvXSlfXj+1nwtitPPVpRpplc14VcPvIWmq7brMfUydlPZUDh6ffjfdYsdpCFYVbIZgakg3nYofdFqhQ8kQz2E0uZM5yj+2f5IxNFl629N NvSwGR7h rSXLBcnJjqdAkQqRLTSd+X+lNjntW4iubp5uXvEcA9fHIIrQ282dhPQ4LR4LfkoDCO6BCQwFhAiNphc4wSBdmXM3G4VWIetHjt6a9MHWZgDl83nk+ZolIgWPfo+PElZns0Z1XjfEJQ04nENRG3Lz6cKu74DWg8mER/thFewlbH7Nm3pK/WjPK0IVfK7eqds+ZPC1xVlOMQjXQzb5iTBdneYUklM8HBSx3ygSzSTIxWGiL/Ci22kInV9QUHGOrOzIYf2hqOGq3Z/FleFZpgTfvj5a2Uq2cqLBP8uai12szUrRu3FxAUtexkBDKjtBOYztflnX/a///s1YVg4shl3+uYH23UpitCje1IWf9XTZhDhuqlsITVJE7xgve0BjzmhYskT0pZSgRfr+BnH4Pbwy4ZIa4FrA2PuAgai3ekt0KLlQ9+3lFLe4b1528iw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000017, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hello Uladzislau, On Fri, Sep 27, 2024 at 12:16=E2=80=AFAM Uladzislau Rezki wrote: > > Hello, Adrian! > > > > > > > > > From: Adrian Huang > > > > After re-visiting code path about setting the kasan ptep (pte point= er), > > > > it's unlikely that a kasan ptep is set and cleared simultaneously b= y > > > > different CPUs. So, use ptep_get_and_clear() to get rid of the spin= lock > > > > operation. > > > > > > "unlikely" isn't particularly comforting. We'd prefer to never corru= pt > > > pte's! > > > > > > I'm suspecting we need a more thorough solution here. > > > > > > btw, for a lame fix, did you try moving the spin_lock() into > > > kasan_release_vmalloc(), around the apply_to_existing_page_range() > > > call? That would at least reduce locking frequency a lot. Some > > > mitigation might be needed to avoid excessive hold times. > > > > I did try it before. That didn't help. In this case, each iteration in > > kasan_release_vmalloc_node() only needs to clear one pte. However, > > vn->purge_list is the long list under the heavy load: 128 cores (128 > > vmap_nodes) execute kasan_release_vmalloc_node() to clear the correspon= ding > > pte(s) while other cores allocate vmalloc space (populate the page tabl= e > > of the vmalloc address) and populate vmalloc shadow page table. Lots of > > cores contend init_mm.page_table_lock. > > > > For a lame fix, adding cond_resched() in the loop of > > kasan_release_vmalloc_node() is an option. > > > > Any suggestions and comments about this issue? > > > One question. Do you think that running a KASAN kernel and stressing > the vmalloc allocator is an issue here? It is a debug kernel, which > implies it is slow. Also, please note, the synthetic stress test is > not a real workload, it is tighten in a hard loop to stress it as much > as we can. Totally agree. > Can you trigger such splat using a real workload. For example running > stress-ng --fork XXX or any different workload? No, the issue could not be reproduced with stress-ng (over-weekend stress). So, please ignore it. Sorry for the noise. -- Adrian