From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF88FC7618A for ; Mon, 20 Mar 2023 08:25:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 296FE6B0072; Mon, 20 Mar 2023 04:25:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2204C6B0074; Mon, 20 Mar 2023 04:25:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 099D36B0075; Mon, 20 Mar 2023 04:25:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E984B6B0072 for ; Mon, 20 Mar 2023 04:25:37 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id B0E118010F for ; Mon, 20 Mar 2023 08:25:37 +0000 (UTC) X-FDA: 80588592714.20.A5233A6 Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) by imf26.hostedemail.com (Postfix) with ESMTP id D6D37140005 for ; Mon, 20 Mar 2023 08:25:35 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=IJmshY7k; spf=pass (imf26.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.45 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679300735; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Lq9ByukcLzCd4kriwtF9/gxGSEg8VeFv6cJLw8F5Cs4=; b=iQddsM5FtfvCZtLyyJ/hTWbUdDvdiXUqhI3jUvdbeNnb+dFSv3ralzZvlnzXV1aLHPtEnf pf3eUNHPuspqc48eC29BUFAmq8Ngpo0CPiFxm1ry6enLj/2FNfKgsy9+5H1OSlvYpKaz2+ gaW+fkDC9Zc1JvhASp6et+o/UKUvpT0= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=IJmshY7k; spf=pass (imf26.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.45 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679300735; a=rsa-sha256; cv=none; b=jA9+AeIBzpZcwsSLvC/q9YZe2ciQbTc/Q6FS6bOOPWPfIin49/mxzWPt0dc3zY9G0orCOM dV/6BWxlhsEzVx5gGrGrnM4XmOMpA8JtWEvy9jpeFBOfBom6eSe7/eNLcaq8cu1VP03g1q JPTqBUo9UU1UWw3hYRGG+Sm16E9+qfo= Received: by mail-wm1-f45.google.com with SMTP id o32so481244wms.1 for ; Mon, 20 Mar 2023 01:25:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679300734; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Lq9ByukcLzCd4kriwtF9/gxGSEg8VeFv6cJLw8F5Cs4=; b=IJmshY7kOIZtz1stoAYNuTVmks50/4vFpi2fEKWTDNiLoLxZln6eaWVvG85lkMB1cd BcctcQI/JMb5ucLC+flnHuqr2J/myPMRO3gku/oPtKxcib56yqqEOq4pf2Di6SHMZgtB xG0bGt8navy8lZgLydyRUHr+pbEuM3glHWc5dcWGBymboc1pXuunrTiI1gnw8a2zGjwP N5kUALa0fFEuUrpCBXFHEujlcu7waD6BfOoMSiblLwrLTjvueQ+5fTDUXA1OeDuleYtt vC2kRV+BiUyX+UFXSSk4IqW05Op78qlCaP2LE55QFqEjnqVLY1FcjIO4/cxIFHwf/P1p U9nQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679300734; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Lq9ByukcLzCd4kriwtF9/gxGSEg8VeFv6cJLw8F5Cs4=; b=oKNWysSyZouskbSMWOizRwZL+VUzmn3GvqdsKlB/LCrKN0N/4rQhYx61CTnHUazbG1 9ccjon2d9YMOURFd9xlqpq42VR3SUco2cPmF8DDJsTzHoqOevgGQpwhPu/W5E0agz+K/ XSQEWIFSjrtghk8d9uVj3PNT0FFZDbNYVT5+82OlpC1pp3ncTSDZH22t6WJ6cH3sqAF6 tnkfil/GRaA0jcABn2JQHqGogSJ54aUpufvZ2FdbExodswPMgSmABYnJVZ7ycprPuk2B q2UtxWYvok5C0cGpF3qaOlW73ndOWCzkg0p+Tc0L8UdQacH328HBjo68ZA0fcQDdyJXR 5Ncw== X-Gm-Message-State: AO0yUKUDvM4Mx1U3UmT2GpK/sWpIQ0lwikTERi2/sVpttO32xPfAEWcY ydOiNtsRDylkWa0wvLv94Iw= X-Google-Smtp-Source: AK7set/KjExHxU/VRUundBhJwbAOn8f1vO3ZgnzhGHZgLDRty1RmkvUexWYNUQs9Tt5IKcmL51/a/A== X-Received: by 2002:a7b:c40b:0:b0:3ee:1084:aa79 with SMTP id k11-20020a7bc40b000000b003ee1084aa79mr359148wmi.20.1679300734123; Mon, 20 Mar 2023 01:25:34 -0700 (PDT) Received: from localhost (host86-146-209-214.range86-146.btcentralplus.com. [86.146.209.214]) by smtp.gmail.com with ESMTPSA id p17-20020adfcc91000000b002c71dd1109fsm8327072wrj.47.2023.03.20.01.25.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Mar 2023 01:25:33 -0700 (PDT) Date: Mon, 20 Mar 2023 08:25:32 +0000 From: Lorenzo Stoakes To: Uladzislau Rezki Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, Andrew Morton , Baoquan He , Matthew Wilcox , David Hildenbrand , Liu Shixin , Jiri Olsa Subject: Re: [PATCH v2 2/4] mm: vmalloc: use rwsem, mutex for vmap_area_lock and vmap_block->lock Message-ID: <413e0dfe-5a68-4cd9-9036-bed741e4cd22@lucifer.local> References: <6c7f1ac0aeb55faaa46a09108d3999e4595870d9.1679209395.git.lstoakes@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: khp3zc54uay9aieb3epuy8fi99qb3ysa X-Rspamd-Queue-Id: D6D37140005 X-HE-Tag: 1679300735-244057 X-HE-Meta: U2FsdGVkX19TUOCShVSmZTdMGL/KFY0QBoxIvRC631xDeD1bmREfajEpVQ3JcO4LB7YlWZsATSbTSh0haTduXZj9HDBeGAQb2nPB9J1bIYow05arUvIk3aDW2y+XXtsB4L6QZj9RQoFhMX26n062xgf2P9A40PyJcAZSuptjCR0HXB9xVHgh6WgtDIylMG4NFU7gydpFQD3WUmPL2UlT4ou3aEuowvSUrBes9GN/lhYFUKu/S//tK6ogdWoiUxEQ1L3AateakKme216T1Qkk4/VvanZhD1yOtVD9wuEjq57qcYnjtC89N5yCzJb2ld4xz8lBQ/YU02IF58Le9s0AFvxTHaZnxMUdyIg+2wuZ2+RETSQ2209gnwuZQ51UHxAspUS2lGfVZV7fH8LV1uhg/lL8hFdck93lTUJhC+rV7yOzQe+ZcyHUloEdm4liaurgDiomKyUlLgv/UnXfejvkS6MjAKuGu+BmVc//hBFii+2bC3BrX8JOibtn7OxFjYkQg9AIEbT82Lg4QeY10T2bE4Eh65HX1RDFrkRJON5xSDAnjOFK5MxtBBV0XWORZ/fUJFwehdTHLeuScmbrecCLD6BEc/AKo5RBbHpsLltquECfaPNINF0giHHU8zsNI6uUxYE8WqvrrBukbRM916ZbMw3HqpTPm7AwWl7rBAf0KcW3+NbV18PKvUeH6/4AtxitQqruhWYoEz39n3/PfOipADraU9CAxc/gM0c2iQWgTKjSV1RRQ39gxGseQg8+jtFJpBOA9fiPyeCoK9tDtQcWjiG1Yk6ypbCrRu9Gac59QktAYA2mxm3QNTNAU+QJrPlbXZf+oNs2XZ4OZniBNPtPpP6SNz06HSRO2HKKiVGYnCAecLtaEzaRJwUnha4EyvVpG9HP70KvxLfGbTMRA7vt2p8VaZjwp3HPdL3RGP2NNOgNHTI+7a20LmyUSKitRloaxckQOZ8t0bERQSpn1bw nR4Y0sLv Sne5Ya6hjN5lmSLyh5A94m1cWpbdVJG913sbccqlwt8O3oN7EbyuUqrP5AYZWW/ElwWgQ7JXQ0O3TKgFfQQex/llUvF/nhMDFs7bw8jU2FHDQX121nUJMCugLfs9oL5I9D+Q+cTzPPZDcHWqhWDq9M4L+VDMg3MazzyufXsA19B9egl27hI8A+HLOu0kcklOIJLVDry5jKWWd+SMgRB5jiuxrLefi8rb1vSB7PT2GOujFYhhFg17tgwz1qdGdBcDwKof8eWObS4/izgKemDIiBriHqpJwsBhYG3qodZA7vde4yUZu++SShQtGvGgZxHzRPzj9efiGk49Ngt6aot3bTlHQMK+qRq6CqwMUsd9cd/Glp0DnQQ/fPeMs53R8Qm3ZG31WZ1VkWoPem7WUxtZ81+g035Kcl0iTeMbU/X1pBcl4Hv7Sm9eJXOMinSBIcIlA5oHuF74GZhXryjM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Mar 20, 2023 at 08:54:33AM +0100, Uladzislau Rezki wrote: > > vmalloc() is, by design, not permitted to be used in atomic context and > > already contains components which may sleep, so avoiding spin locks is not > > a problem from the perspective of atomic context. > > > > The global vmap_area_lock is held when the red/black tree rooted in > > vmap_are_root is accessed and thus is rather long-held and under > > potentially high contention. It is likely to be under contention for reads > > rather than write, so replace it with a rwsem. > > > > Each individual vmap_block->lock is likely to be held for less time but > > under low contention, so a mutex is not an outrageous choice here. > > > > A subset of test_vmalloc.sh performance results:- > > > > fix_size_alloc_test 0.40% > > full_fit_alloc_test 2.08% > > long_busy_list_alloc_test 0.34% > > random_size_alloc_test -0.25% > > random_size_align_alloc_test 0.06% > > ... > > all tests cycles 0.2% > > > > This represents a tiny reduction in performance that sits barely above > > noise. > > > How important to have many simultaneous users of vread()? I do not see a > big reason to switch into mutexes due to performance impact and making it > less atomic. It's less about simultaneous users of vread() and more about being able to write direct to user memory rather than via a bounce buffer and not hold a spinlock over possible page faults. The performance impact is barely above noise (I got fairly widely varying results), so I don't think it's really much of a cost at all. I can't imagine there are many users critically dependent on a sub-single digit % reduction in speed in vmalloc() allocation. As I was saying to Willy, the code is already not atomic, or rather needs rework to become atomic-safe (there are a smattering of might_sleep()'s throughout) However, given your objection alongside Willy's, let me examine Willy's suggestion that we instead of doing this, prefault the user memory in advance of the vread call. > > So, how important for you to have this change? > Personally, always very important :) > -- > Uladzislau Rezki