From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DD9AC8303C for ; Wed, 9 Jul 2025 01:53:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E1CB36B0089; Tue, 8 Jul 2025 21:53:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DCCDA6B008A; Tue, 8 Jul 2025 21:53:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CBC5E6B0092; Tue, 8 Jul 2025 21:53:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id B5C966B0089 for ; Tue, 8 Jul 2025 21:53:09 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 2AE1E802C9 for ; Wed, 9 Jul 2025 01:53:09 +0000 (UTC) X-FDA: 83643053298.10.AF2794D Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by imf24.hostedemail.com (Postfix) with ESMTP id 4774C18000C for ; Wed, 9 Jul 2025 01:53:07 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=EUsOP3AC; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752025987; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=YVq8OliOKpTiMPXRZnkr313UvLWkoyiKBS7rnvUZNAg=; b=hGkb/h0BUyuFxZh4Rw84Mo+WUw9GrQwTisLGeDKMUhDnRyjd6QEI7JHSFeMQE9mObinrtJ YEgRu3RBi3n8ORtMGrXeRHPVd9n+Gfrlzsgx4aZx5ORvgNRPsbNlut5mPgBFYVhrdSXFx+ OiPV5xy2o9hbUjlayTtP5UwYq3UwQOQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752025987; a=rsa-sha256; cv=none; b=hPUyecObPdAppapRN9YB2MqZJBLhhavtTz0T4WBPCNhoXMe9XSI/cEDW0CsPM5PMhWgZ0Q INnH2pXKb8S0topr9XKdLk/1zcpvqlgn6FMz1WD7FH64MzZhPZ/0whVVzys0GwSs99XpCt whlnwQqSKjIQ91G+955PrVY3fn7hplM= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=EUsOP3AC; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-7399a2dc13fso5812452b3a.2 for ; Tue, 08 Jul 2025 18:53:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1752025986; x=1752630786; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=YVq8OliOKpTiMPXRZnkr313UvLWkoyiKBS7rnvUZNAg=; b=EUsOP3ACfgI6hNzitQLawFx4diK2ilinknFwx2l7pnQVZ7QN/x3JKPQBMQqQ9icAK3 g1frtmDdipD9jp8GWKds+h77rRyHn6iFzxLJ32LN+jgvz3DZcND9hlw5aWDCsaQX0+iz hhgF0Rl9HQbeX5aX143bZKdSswWr7xe4Op0o5gjCmumnyZrej7AxG8GLf8YuWT9eoF3y pMiIUj16t6SG/+Y3boyK7UVQQl7OUP0NvxKyM9E18hPCsG5Oe+VOTqhFeY9W9U818b7l ZFNFjXPusOIMM6ZHieJ79kl2ZtJa6X8x8nTEvMJBRGrUVJ1Njr2euIK+0JQIyVOQ4N7u K8Ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752025986; x=1752630786; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=YVq8OliOKpTiMPXRZnkr313UvLWkoyiKBS7rnvUZNAg=; b=hPj/uvJOZdrbFWHRUo3zqZq854whQE5jUhqLnEruIPc2H5tgF2sF3iRkN90b5Zl7EO mvTb9Q8sumsAk1ULYPalvybXjGDClghkH5R3+2PrMjSag+UWjrFbZQXJfPr7K0bl0tjQ ez6tdoIFdkf94cXGnDhszNb9jdxu8aB+HNwSU66SNPuF+A1fXSt30CdIrKgMcsc84QXy fCRQ4bcIbsuuFHNK2jTRuZBSDhAlwQJlmpfhwOqeqPtjSPkQpXMHsKk66T/xmbuScP0R R/nf0e3O8he9YqbpeaVGcJTKcHVDuzrbgDgqVD3hNp1eDBOPqUw6tORBufs/92w+Wl4q iHHw== X-Forwarded-Encrypted: i=1; AJvYcCW5KxVY9nLRqM4zUnwy8fAq3p/mb1740un2rcAxqf3eRJlgx2gnVGyuWvz1ezH+sBRykYs1SiFtoQ==@kvack.org X-Gm-Message-State: AOJu0YyLhJiiXBP686QJPiiEDCFQSQPRlZCufszhkV3Fg31z4sSHwrUE si2OUzpbchXJJm7kbyW9DQdbZrMLhpRVRU7DusUOaocX/PWLA0vcUwGH X-Gm-Gg: ASbGncupLtVHqIsl+Q6AzOSf40NqjuCvgacB1P+wtRmCXMESbDUIOewcd0dHYPafora LYcBqZrnU1Bd57d5vnsD52+aBFL7R4JOQ2RGgqdfkkn96v7dsforFRMUR8tZB3NcFosGadmj9gs pR9EbV2Jeuh4j/30SqE3C5H1OZPe1cGROsnHbtynYjk+hAaMkWqIMwpDjPOms3aRMkkyYt1pFQt E4FT0dRH5gjJ7wDTBEmNwz29nLkvikg96hwOoC1dFkW/TbGnO49ueRkiJwn4QATLMWyPu6v+6uP fJXJHBD5EnVMF4o1Ct7ou9byZutjr9Wp/nVXZyOmQA3eKN8djz9/SeoZ3J1cOhmLLVrI2kILS54 Sy+O2hIoWYouS/shDDI+KLRDUfEU= X-Google-Smtp-Source: AGHT+IHCph7Yg/NOpvmruFegQftJXZni6axyi8rZm/eKyQH1pYHkOEY8X87zl1eAyxvFY4CFbr7EhA== X-Received: by 2002:a05:6a00:3a1b:b0:740:9c57:3907 with SMTP id d2e1a72fcca58-74ea66d7ca3mr1151896b3a.19.1752025985900; Tue, 08 Jul 2025 18:53:05 -0700 (PDT) Received: from localhost.localdomain ([2001:558:600a:7:a83d:600f:32cc:235a]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-74ce35cc152sm12202908b3a.51.2025.07.08.18.53.04 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 08 Jul 2025 18:53:05 -0700 (PDT) From: Alexei Starovoitov To: bpf@vger.kernel.org, linux-mm@kvack.org Cc: vbabka@suse.cz, harry.yoo@oracle.com, shakeel.butt@linux.dev, mhocko@suse.com, bigeasy@linutronix.de, andrii@kernel.org, memxor@gmail.com, akpm@linux-foundation.org, peterz@infradead.org, rostedt@goodmis.org, hannes@cmpxchg.org Subject: [PATCH v2 0/6] slab: Re-entrant kmalloc_nolock() Date: Tue, 8 Jul 2025 18:52:57 -0700 Message-Id: <20250709015303.8107-1-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4774C18000C X-Stat-Signature: u73iotrqnuu81r3hzcnctsnqhbpbwrg8 X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1752025987-725271 X-HE-Meta: U2FsdGVkX1953GchSMdW6vC/H4OCu90qUq0OjEt/ck8UZ2I0EwpqfTbepaKNJ09EpShyN3PkvW2Hn8/JZf+8p+IZGUp4LMopOQQrRzoZgW3HpQEOYHROkkpvihagMilILD+4eHxgwJ5Ml9SvpGSvc+gBntL4daLc6ENy1wP3mPhp6sW4qgsNmi16fPkIaRW3OadN+b5ZaQnPYs5oJCF3T2C4PwQbuw1etmLrkB+WSeQJQ6PrHS+bxd0B9LTIMesOWm1aWg7hDR1/w4GNwHNv1FsYTDtw3O969uVm9pmWKjM10AFiQgQkdrHWSPVbp5NvfVuYsWayJ5L/languTob5mYStnkDYTY5/JreO5k++LxpNXmDbmlt3LI+fZYVGzrCz9f2alEkYQO8x6+5jCnmxW8acjYXTFzcMiyaIkDBFllfGxC6qEGg8CZ9M+XiyLcgK7I2Mvn9I+q89/7PTO4SzpMNvEnSPgrUsJwBbRb3qSMqh2GjUzTG2nq6qKujbn65ZTzXHMPVuBp8k7uONSyGb26ZN5KoyVOBiAcAJozzx3op1nyHl1W5mY+hPlsqbyTlHA20o20ajkhLZVvleAu7n06oTSQPBpQCx+V5AXc6ftaUUK53Ne8pnOBj3iAq0BYWefF+WauXHS7Q3MYXRAmOoJdRCthPfdY4ZHzYk2j1x2gn+PFnbtqQebKGOiu75XLnQl47a36GyQ+zegGa3wIrrpf3gZrHjSt/+/kLFfzyN6n7M/b7dDr/m1r30Uysh4z+0e3b2AHgjupVZEwgnRKrhkGhdAplAQZdAIAYZ5h1YEaTNNyRYi2YgPz3Wb5xs0lUf+7qKoF9+KhI+SRR7Ci7Ks9eLeP1WwUkeuPwQ9i9ab3A9WFEeH7jkIjB562lSfK0cPyj0CXyHaFJYsglMOZHqGNthQk143cLzRob8JyS0X6m/H6nNo3JGwCbP319HSWGby7KMPDriGk94/YDcTD 4zRk1PNT hpJQoSkF476p5xvzBE8L71qwliXOX/pO7GsL48aWCFZ8BfgcS55VACHaBdIGIctwlQlsnaqWZI2fRu044vBOJUpo5TYFR4k+NVrxUczXa7r3J8kUFNiJV1TpTu0iRg+WMJhq6d3HAo8V8kQ6mUEqBD+Zt7n+GyP6rYjTair9tkfOaJdF/PEAPdYuzyW2y30vO5hWWypyVrfHAh+JtASQgFVC8dPoI42/d4IgVNWrZFIgv/aZcsHv/rZ80mK29TDKb0PzODA/LXgu+fkc/V/mfEhAzOT3tZWGyl0vVUKWt3usN7m5mwp4MHlLriv0DsYbZTnR+jfKKTeKE3AzcVC0rM6pu86111qk7wzHBMJYAbKdwpctnxDWUDCW/gkq6FJlBDX9Nutode9PEQycJAjnxENARFUUZ5jTHnNTx6il/B0PGt4PwAswq6omrugQNF4b4wj4eooL0ez91fDJ7C8ttOaAUpVX5w797/9zDqHur56l9fdaGjt5ewPUVQTuvxAhbgmkVMZVH1tOUpuwBmUfdWfv1fqiQXzjhehyrpus3OlNgmG475+bI83E3Ew== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexei Starovoitov v1->v2: Added more comments for this non-trivial logic and addressed earlier comments. In particular: - Introduce alloc_frozen_pages_nolock() to avoid refcnt race - alloc_pages_nolock() defaults to GFP_COMP - Support SLUB_TINY - Added more variants to stress tester to discover that kfree_nolock() can OOM, because deferred per-slab llist won't be serviced if kfree_nolock() gets unlucky long enough. Scraped previous approach and switched to global per-cpu llist with immediate irq_work_queue() to process all object sizes. - Reentrant kmalloc cannot deactivate_slab(). In v1 the node hint was downgraded to NUMA_NO_NODE before calling slab_alloc(). Realized it's not good enough. There are odd cases that can trigger deactivate. Rewrote this part. - Struggled with SLAB_NO_CMPXCHG. Thankfully Harry had a great suggestion: https://lore.kernel.org/bpf/aFvfr1KiNrLofavW@hyeyoo/ which was adopted. So slab_debug works now. - In v1 I had to s/local_lock_irqsave/local_lock_irqsave_check/ in a bunch of places in mm/slub.c to avoid lockdep false positives. Came up with much cleaner approach to silence invalid lockdep reports without sacrificing lockdep coverage. See local_lock_lockdep_start/end(). v1: https://lore.kernel.org/bpf/20250501032718.65476-1-alexei.starovoitov@gmail.com/ Alexei Starovoitov (6): locking/local_lock: Expose dep_map in local_trylock_t. locking/local_lock: Introduce local_lock_is_locked(). locking/local_lock: Introduce local_lock_lockdep_start/end() mm: Allow GFP_ACCOUNT to be used in alloc_pages_nolock(). mm: Introduce alloc_frozen_pages_nolock() slab: Introduce kmalloc_nolock() and kfree_nolock(). include/linux/gfp.h | 2 +- include/linux/kasan.h | 13 +- include/linux/local_lock.h | 17 ++ include/linux/local_lock_internal.h | 16 +- include/linux/lockdep_types.h | 4 +- include/linux/rtmutex.h | 9 + include/linux/slab.h | 4 + kernel/bpf/syscall.c | 2 +- kernel/locking/lockdep.c | 4 + kernel/locking/rtmutex_common.h | 9 - mm/internal.h | 4 + mm/kasan/common.c | 5 +- mm/page_alloc.c | 54 +++-- mm/slub.c | 330 +++++++++++++++++++++++++--- 14 files changed, 402 insertions(+), 71 deletions(-) -- 2.47.1