From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A65AC52D71 for ; Tue, 6 Aug 2024 16:15:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 267106B0089; Tue, 6 Aug 2024 12:15:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 218006B008A; Tue, 6 Aug 2024 12:15:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0DF706B008C; Tue, 6 Aug 2024 12:15:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id E7FBF6B0089 for ; Tue, 6 Aug 2024 12:15:27 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 86E1040362 for ; Tue, 6 Aug 2024 16:15:27 +0000 (UTC) X-FDA: 82422320694.06.DB1E3BC Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf05.hostedemail.com (Postfix) with ESMTP id 5F4AF10001B for ; Tue, 6 Aug 2024 16:15:25 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=R3w6boAF; spf=pass (imf05.hostedemail.com: domain of longman@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722960876; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ocUO54UkX6UEu2Jsdmx7F3rzWjBkiKGcuGMRQDNpeXo=; b=CjOI9y5FfYCpOc8Y8TvaEv1/vw2xsmj6PvOGNuHeY/8zGusj926b3efDB8y4sWKW/6tpHB gSlWeHcQEHXnsZVYht26ZXnCD5aKlIXEZbQKDty1hBtMFWxvnk5zu+gNHdsQVC/8w+RHW+ 6H0LeSMSmkUb/JOygVr2Erx4qm0BR1U= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=R3w6boAF; spf=pass (imf05.hostedemail.com: domain of longman@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=longman@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722960876; a=rsa-sha256; cv=none; b=ffXOStEuIWelP8zdL2st4ECl+aZohPPpatA8PHgiErP3Ydy0UeTg+St10OTJOUZ+ojQ/uS +SxkKJjhaFfLmjO/KPwhZoQA8Dwkbagm90Tgobefp5wsbxY+tAqxCeQOqBVIO3dKbd6tUG uhY9wFYg5OFVCqD7QISndpYt7p8qV8w= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1722960924; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ocUO54UkX6UEu2Jsdmx7F3rzWjBkiKGcuGMRQDNpeXo=; b=R3w6boAFcdwlhVerNqBp7JwPV0eBQyTyml0W2/TUZKdirhzA4yHU4bJAwo6xVkhZNmm4GB rVqQf2uvtlLveUZKIlMsg0aXRVWJ7gSjoyP8fYDnwM2c6SEF2nWeZgyk6lMLwVkafYn6tl lUbQBaCs1TaB5RDBM/H9POr4Zs7i4/4= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-153-ttqLFjRfO3-g3vt4kA9zxQ-1; Tue, 06 Aug 2024 12:15:19 -0400 X-MC-Unique: ttqLFjRfO3-g3vt4kA9zxQ-1 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4AC5B1955D45; Tue, 6 Aug 2024 16:15:16 +0000 (UTC) Received: from [10.2.16.146] (unknown [10.2.16.146]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 3B73E300019B; Tue, 6 Aug 2024 16:15:13 +0000 (UTC) Message-ID: Date: Tue, 6 Aug 2024 12:15:12 -0400 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] mm/memory-failure: Use raw_spinlock_t in struct memory_failure_cpu To: Juri Lelli Cc: Andrew Morton , Miaohe Lin , Naoya Horiguchi , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Huang Ying , Len Brown References: <20240806142535.1033323-1-longman@redhat.com> Content-Language: en-US From: Waiman Long In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Stat-Signature: 7zphj7utf8umwjrei4o4beu3gfmjs4fn X-Rspam-User: X-Rspamd-Queue-Id: 5F4AF10001B X-Rspamd-Server: rspam02 X-HE-Tag: 1722960925-930461 X-HE-Meta: U2FsdGVkX1+FPRrLTjz9IFF+rVzlwR04HLuR7RhQDdcxcKS7kOIuj1jagtcDeVcKdcaDbe2+Yfu/79pJlTc6bxYarmoVQ60BX0eWOZTVl1INuT3FUp38+zqqhkeNeKA4A5lGAdPaj2NVznsvGstG45m0qERDFDT9VX/l9TUG4y0q9dUd71Y9B9FoB0/VinT4dX8mU/Oa7MHxit4+hEOgNSwe0D9iA/QkGpfa1TK+YWhwzt1oAymFuifJL0Pe4Ks8q0BfYFK+3XImj3Ppncxnr990mENUPoy1HtfWFKiLpoMJmW2OWc+RULGgjRYG/Jfr7bw4Jy5DDOaX0Aa6hI0yX/0/TaKdHZT1aKIzbjA2sI7HzNQr7vWi/Z4TyUU4oIB30ecbv95zbg7/9tUYbLElFB/OzO2Mfv2mlBQvIJ7rhOWnsZ9uVcfzE+u+QnwykoeYxZ3PueXb+RgFFHb20te7Z+qpNm1Lwcw3UpIwixY5FvBPuKuDfDi3KEyACm1fg2WA+jtWLHuhaDGvU/9+jhIZxevKdGWssuJhu+6a4NOSgnb1+hV3Pku7CCWH3ElX4peufwB6m7iDXv60gZS+WYlLjTUzl/Vglmy5Q6GNnPcL9DRTWrLZ89n71YfnMXgkHqKIYU937AUBnRQJPqBENR9L56eCPBhYqHGcJtX7+Tc4EWpJef9pA4yH7SAKoYxN0qMBPj128dWhB259BLu3bgOtdoXYvRG8qxlTMpUBmjtq6uz5ZSowQdfO5KR/fYMDsDmN2Z37MUUzNcfzkr3h29/6u5X4NSW0NhNbVy5XUbmmSrhGdj1MXFeuNsLn/CAv3BibZvZpOYNi6Pn07RaPHGL7yjwFEODvqISzm6CRBhbNtjg/vuSoygUY8kLdn3YXXMY/96dSf3IUPJfpHbJOKVTItG0T49gL6FavwIuXpMgTrWqHmjLuawXB10oQKRRor9NkOjTRznHmniUhNHdPrBT 7YEkcYVt jLMGQcRycqUIdqHV1I+HPuKw6VNd6fA7smoGZ7Jhd7ASYvLTLq7MPbvGd0fjf7gcSVkQV+8OrMyN4vc4NpHXVSsKLtGOUFy4CjuEpXcz6fFivH25iMpHogC2o+C0VjGhJ6VUv2KDKWF+Q3gTkCCPRRP2FC5g/jxk3pSX8QxEBfoeUmD3+lW64+u5p/UUfW1Upb3oyJPXpKxnIR/GolbSYIMRXIo8OVLHgjv+0X7Ua+FGWSm9xEckKkqEPUxezT/lNoxcRh8mjgfdhFahw4O57vUy82XWWBRMtWxkE3p2HKDrlRexhMPS1HTrBz6SCPJdu3xUU+tdzhfJ2dDR54s6IzM0m+ZaH0ZzjnjS825IfCXTMD38dZuBuRvUe5FzZ9SYZKxjqBnGwZDxqaO57k8WQlYhWikmh6gpgyzFnGDC1rG7wncuMlRjsUsZE++fNJjmAI/Bdg32bZMp15F7Z+E6iRfz9BIhoCf6PvV9WdXbhTVwwa+cbvWGjmStQWw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 8/6/24 11:53, Juri Lelli wrote: > Hi Waimain, > > On 06/08/24 10:25, Waiman Long wrote: >> The memory_failure_cpu structure is a per-cpu structure. Access to its >> content requires the use of get_cpu_var() to lock in the current CPU >> and disable preemption. The use of a regular spinlock_t for locking >> purpose is fine for a non-RT kernel. >> >> Since the integration of RT spinlock support into the v5.15 kernel, >> a spinlock_t in a RT kernel becomes a sleeping lock and taking a >> sleeping lock in a preemption disabled context is illegal resulting in >> the following kind of warning. >> >> [12135.732244] BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:48 >> [12135.732248] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 270076, name: kworker/0:0 >> [12135.732252] preempt_count: 1, expected: 0 >> [12135.732255] RCU nest depth: 2, expected: 2 >> : >> [12135.732420] Hardware name: Dell Inc. PowerEdge R640/0HG0J8, BIOS 2.10.2 02/24/2021 >> [12135.732423] Workqueue: kacpi_notify acpi_os_execute_deferred >> [12135.732433] Call Trace: >> [12135.732436] >> [12135.732450] dump_stack_lvl+0x57/0x81 >> [12135.732461] __might_resched.cold+0xf4/0x12f >> [12135.732479] rt_spin_lock+0x4c/0x100 >> [12135.732491] memory_failure_queue+0x40/0xe0 >> [12135.732503] ghes_do_memory_failure+0x53/0x390 >> [12135.732516] ghes_do_proc.constprop.0+0x229/0x3e0 >> [12135.732575] ghes_proc+0xf9/0x1a0 >> [12135.732591] ghes_notify_hed+0x6a/0x150 >> [12135.732602] notifier_call_chain+0x43/0xb0 >> [12135.732626] blocking_notifier_call_chain+0x43/0x60 >> [12135.732637] acpi_ev_notify_dispatch+0x47/0x70 >> [12135.732648] acpi_os_execute_deferred+0x13/0x20 >> [12135.732654] process_one_work+0x41f/0x500 >> [12135.732695] worker_thread+0x192/0x360 >> [12135.732715] kthread+0x111/0x140 >> [12135.732733] ret_from_fork+0x29/0x50 >> [12135.732779] >> >> Fix it by using a raw_spinlock_t for locking instead. > IIUC this is executed to recover a fault condition already, so maybe > latencies are of no interest at that point, but I wonder if something > like > > https://elixir.bootlin.com/linux/v6.10.1/source/Documentation/locking/locktypes.rst#L434 > > would still work and save us from introducing a raw_spinlock? > > Or maybe the critical section is anyway tiny and we don't care either? There are only 2 critical sections that makes use of this lock - memory_failure_queue() and memory_failure_work_func().  In memory_failure_queue(), there is a kfifo_put() and either schedule_work_on() or pr_err(). In memory_failure_work_func(), the critical section is just a kfifo_get(). kfifo_get() and kfifo_put() are not using loop and their run time, though not very short, shouldn't be long. The schedule_work_on() will take its own raw_spinlock_t to do its work anyway. So the only call that may have a long runtime is pr_err() before the printk rework lands. Fortunately, we can easily take the pr_err() call out of the critical section. As memory_failure_queue() is not a frequently called function and I doubt there will be much contention in the lock, I believe it is easier to understand to just use raw_spinlock_t than using migrate_disable() without using get_cpu_var(). Also if there is hardware issue leading to the call to memory_failure_queue(), a bit extra latency due to the use of raw_spinlock_t is not the most important concern anyway. I will post a v2 patch to move pr_err() call out of the lock critical section. Cheers, Longman