From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FB78C4332F for ; Tue, 20 Dec 2022 18:45:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A196D8E0002; Tue, 20 Dec 2022 13:45:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9C9708E0001; Tue, 20 Dec 2022 13:45:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 86A9B8E0002; Tue, 20 Dec 2022 13:45:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 739068E0001 for ; Tue, 20 Dec 2022 13:45:26 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 308E1160D01 for ; Tue, 20 Dec 2022 18:45:26 +0000 (UTC) X-FDA: 80263562652.13.FF336E5 Received: from mail-lf1-f47.google.com (mail-lf1-f47.google.com [209.85.167.47]) by imf27.hostedemail.com (Postfix) with ESMTP id 52E8740006 for ; Tue, 20 Dec 2022 18:45:24 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=kysol0L3; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf27.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.47 as permitted sender) smtp.mailfrom=urezki@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1671561924; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GkTIGn2utZ7uNmGiVZXj3rWNJl0Nf4rSdYfRDQYhHv8=; b=Wgk5nQz3yvkrkh362d15Jt/1FJjj5DqDFGJhZm+FfO1bPOlxHaOt/8GNAqZyAMjoVnXE7M Eh+/XxAF8ZmO5fydJNgPBeapmU6t5yF0mSCQO1kW5ABuN5S7jEslrGZEoBwRcSqqyDc13q 4Y4YdBi70n4jyb2vW18jZbEW802eCxw= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=kysol0L3; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf27.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.47 as permitted sender) smtp.mailfrom=urezki@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1671561924; a=rsa-sha256; cv=none; b=2IYRHejHwuVgTnPt26u6ZdqXhF6AQtlK6pIXGmmA7jMtJ/ZF15pkyAPRhMWRomnss+Mc1E B67locVAWjpOihxskLGhsm6huFL4XAtWSIjeP5Sev20FfqgCl3K7NqDxIX/8T+SfTvuO99 p14rd10vJTx8GLERWZQyRs4Ye0kEkxo= Received: by mail-lf1-f47.google.com with SMTP id bf43so19955942lfb.6 for ; Tue, 20 Dec 2022 10:45:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:from:to:cc:subject:date:message-id:reply-to; bh=GkTIGn2utZ7uNmGiVZXj3rWNJl0Nf4rSdYfRDQYhHv8=; b=kysol0L3DxRu21ppajFTksrX+5/rSipbZx9Uj4dhXR8kYUqfCh7vp1asRkik7HLB+I X8KJa2Lxd6RwrO4WegvpfW8SSR2R/9kQkCU8aOzNLApwRW4Mc+Bx0cZlWDdH6PXVrW9X Nt8mlV94a49BwUkxIv/yocFJ7NG98H28id+EkDiWTbUxfIfHH8LtDEohw4z7175ZBZc9 l9FuHbmHNaEG1M2rgS6QqQtfJsh6AsRpT0g3IXEKte5ej2xG/x5OY0Jx4LrRD6sos+n0 QFCHd67/Rq9AkKOHzPcC5VZRVjl+h1Gi+H4CbwV4gSszQXPdM7T+Y2agPMbxRYuc7+X0 kYDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=GkTIGn2utZ7uNmGiVZXj3rWNJl0Nf4rSdYfRDQYhHv8=; b=W3XDshyvWe5ZPZkm+40RR25LFhCfDWSSqsuyst21SvCcXb1UOY2olbEICJ5Onii2WJ UxUTQWQpTCZMy7FX1KIO0RxIJRJTaQHAgEfukaJZzJNnIEzMvZpGIlU1tBhPIWhD9ua3 bfpvFYupskThScSkQjnE/Yc1DqgQv5eZD5mGssofoFnxBdshyLdABMNANebUm/NPAjFa ru8iac4eitMjtNDQaNwXopasKFLP9S75IzTnRw9NhU8/lvJeYoduVqnjdFMppW0PKjpW fEjj5TO18fke+7e0bFPJOuSpobqg16+hG3CzzV56Z8MJGFsV2L3moUf2EvQlEbb2OS/M hIPQ== X-Gm-Message-State: ANoB5pl8g+cVTJChqZBule9ITNpw1If7Sw+2IFvtX2FVG7gu4xMdStFE vnMqShQZmryzJLhoTMaZp+mExoM4q/s= X-Google-Smtp-Source: AA0mqf46AJ378ecB0DA/Jl0wnVpAMcCaDrio7nPDJXRa+L3FTIUqZN8TF8THNA2DKskl+FBC42oAmg== X-Received: by 2002:a05:6512:3c84:b0:4b5:8d45:7f32 with SMTP id h4-20020a0565123c8400b004b58d457f32mr18755184lfv.56.1671561922632; Tue, 20 Dec 2022 10:45:22 -0800 (PST) Received: from pc636 (host-90-233-218-120.mobileonline.telia.com. [90.233.218.120]) by smtp.gmail.com with ESMTPSA id o9-20020a05651205c900b00497aa190523sm1546271lfo.248.2022.12.20.10.45.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Dec 2022 10:45:22 -0800 (PST) From: Uladzislau Rezki X-Google-Original-From: Uladzislau Rezki Date: Tue, 20 Dec 2022 19:45:20 +0100 To: "Uladzislau Rezki (Sony)" Cc: Andrew Morton , linux-mm@kvack.org, LKML , Baoquan He , Christoph Hellwig , Matthew Wilcox , Nicholas Piggin , Oleksiy Avramchenko Subject: Re: [PATCH 1/2] mm: vmalloc: Avoid a double lookup of freed VA in a tree Message-ID: References: <20221220182704.181657-1-urezki@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20221220182704.181657-1-urezki@gmail.com> X-Rspamd-Queue-Id: 52E8740006 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: if3bmt4fqqsqwisxymmk4hgroshnh15e X-HE-Tag: 1671561924-433827 X-HE-Meta: U2FsdGVkX19zNwFoPae53Vlk8/BX6liJDSzbayRR/Ql00s3F7SjTZRsSQpC/ltl98j50R5dENIxw6b9VB6hxfOB1ySwFKMxoSANM+yYYREs8aA4Zn3OB9SoaW3t36se6yuKBJ8RS31+U0H/1x+hTe+zvnFd3gx/6sY0bjVvbmPlRCE1O2dXHH3ZSQ0k/aQwXWSLCw7oToiYRfvO5LjLyxYrHEph6cF1HklOxyMeCktBBmAjnd41LGhnw5ngM3zmThXsbDX+wBGJBqo3v2fTXq/0tp5Tir2BtSv9VbMzeEQaIZGz6O5FAX2DBsZ9mK/Dhe7sxgeL8xSimXYuORyHCdtFcCBfeWRH3/nuFSELJ1brNSBjQzSm1fdcq7A0d4V27u2vwpeRjO9mchrr4MBLhhyHtm8gFPbaj8FBCuqzh8jvCs3nCbjmO7SwAliLGwPwmFy1fWfCvyLa+aYl/mdmmfAdH/UUIK0q1z3+bFDQ2iVhyt1rbbRQENcZO0pVH8Ao3/ofw/yt9nYsceKnPi+XM6V1uHxwD5eqG/RXsFbb+euFgt93olhhp2jvwF09+5kMuAO8dZYN2IO531g11ar+dDhZs+1UYPNj2xcRmDOh/D1rlTRMvLt0bKpX73eRlS2qsRBnkW1hJweLQIKr6K8bLNGhjuVdqMLiUYT+82xomutZLtq46M9vnf9Muj7BeXirCChq8R2XYe/BT4oI8tWY/E7zPgWcryE/Rl94F4jSTneOBVds2+md4VpUB1MJiYKymff2hehOKFgJyZDhWx6jiqGY8FJfM/PtlZZqkNydovNEg9onLOqLHqlxLrSeeb8GTI3vbkxWQSyYXcdnNygZ8mFXh7qh6e30l/a4sJY5TdIh6JiXsLftz5fUcX5KCUkmHpHPsfYmivaM4F3FdGVt0FK3mzieZmmkbZ1NdywBCYxhQ8A/NPNVnkMwwEZA7q7hTbIz1kgoy52Ra/mnCcLe UbeKgmjE ZbdFtg5axezmSIBf2UewiyIgxBsfXp4jI0FEOVQx+MwbE3RizlNxrxp8gD8XCtBkan2+k4gQd+qPzaGkVYB2wM1+4Ag== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Dec 20, 2022 at 07:27:03PM +0100, Uladzislau Rezki (Sony) wrote: > When a VA is freed over a main path, for example by invoking > the vfree() function, a tree is accessed two times what is odd: > > vfree(): > __vunmap() > __find_vmap_area() > vm_remove_mappings() > remove_vm_area() > __find_vmap_area() > > __find_vmap_area() are called two times. Fix it by introducing > a find_unlink_vmap_area() helper that finds and un-links a VA > from a tree. > > Performance test results on a single CPU: > > - fix_size_alloc_test loops: 1000000 avg: 476847 usec > - full_fit_alloc_test loops: 1000000 avg: 806746 usec > - long_busy_list_alloc_test loops: 1000000 avg: 13552093 usec > - random_size_alloc_test loops: 1000000 avg: 7441322 usec > - fix_align_alloc_test loops: 1000000 avg: 1411132 usec > All test took worker0=87650866284 cycles > > - fix_size_alloc_test loops: 1000000 avg: 490713 usec > - full_fit_alloc_test loops: 1000000 avg: 579162 usec > - long_busy_list_alloc_test loops: 1000000 avg: 10485448 usec > - random_size_alloc_test loops: 1000000 avg: 5824449 usec > - fix_align_alloc_test loops: 1000000 avg: 984735 usec > All test took worker0=67952362802 cycles > > Signed-off-by: Uladzislau Rezki (Sony) > --- > mm/vmalloc.c | 40 ++++++++++++++++++++++++++++------------ > 1 file changed, 28 insertions(+), 12 deletions(-) > > diff --git a/mm/vmalloc.c b/mm/vmalloc.c > index 9e30f0b39203..0fc38c36e0df 100644 > --- a/mm/vmalloc.c > +++ b/mm/vmalloc.c > @@ -1825,9 +1825,14 @@ static void free_vmap_area_noflush(struct vmap_area *va) > unsigned long va_start = va->va_start; > unsigned long nr_lazy; > > - spin_lock(&vmap_area_lock); > - unlink_va(va, &vmap_area_root); > - spin_unlock(&vmap_area_lock); > + /* > + * A free_vmap_block() is left. It is NOT a main free path. > + */ > + if (!list_empty(&va->list)) { > + spin_lock(&vmap_area_lock); > + unlink_va(va, &vmap_area_root); > + spin_unlock(&vmap_area_lock); > + } > > nr_lazy = atomic_long_add_return((va->va_end - va->va_start) >> > PAGE_SHIFT, &vmap_lazy_nr); > @@ -1871,6 +1876,19 @@ struct vmap_area *find_vmap_area(unsigned long addr) > return va; > } > > +static struct vmap_area *find_unlink_vmap_area(unsigned long addr) > +{ > + struct vmap_area *va; > + > + spin_lock(&vmap_area_lock); > + va = __find_vmap_area(addr, &vmap_area_root); > + if (va) > + unlink_va(va, &vmap_area_root); > + spin_unlock(&vmap_area_lock); > + > + return va; > +} > + > /*** Per cpu kva allocator ***/ > > /* > @@ -2236,7 +2254,7 @@ void vm_unmap_ram(const void *mem, unsigned int count) > return; > } > > - va = find_vmap_area(addr); > + va = find_unlink_vmap_area(addr); > BUG_ON(!va); > debug_check_no_locks_freed((void *)va->va_start, > (va->va_end - va->va_start)); > @@ -2607,21 +2625,16 @@ struct vm_struct *remove_vm_area(const void *addr) > > might_sleep(); > > - spin_lock(&vmap_area_lock); > - va = __find_vmap_area((unsigned long)addr, &vmap_area_root); > - if (va && va->vm) { > + va = find_unlink_vmap_area((unsigned long) addr); > + if (va) { > struct vm_struct *vm = va->vm; > > - va->vm = NULL; > - spin_unlock(&vmap_area_lock); > - > kasan_free_module_shadow(vm); > free_unmap_vmap_area(va); > > return vm; > } > > - spin_unlock(&vmap_area_lock); > return NULL; > } > > @@ -2690,6 +2703,7 @@ static void vm_remove_mappings(struct vm_struct *area, int deallocate_pages) > static void __vunmap(const void *addr, int deallocate_pages) > { > struct vm_struct *area; > + struct vmap_area *va; > > if (!addr) > return; > @@ -2698,7 +2712,9 @@ static void __vunmap(const void *addr, int deallocate_pages) > addr)) > return; > > - area = find_vm_area(addr); > + va = find_unlink_vmap_area((unsigned long)addr); > + area = va->vm; > + > if (unlikely(!area)) { > WARN(1, KERN_ERR "Trying to vfree() nonexistent vm area (%p)\n", > addr); > -- > 2.30.2 > Will send a v2. -- Uladzislau Rezki