From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5A09C64ED6 for ; Fri, 17 Feb 2023 18:59:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5061D6B0073; Fri, 17 Feb 2023 13:59:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 48E196B0074; Fri, 17 Feb 2023 13:59:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 307916B0075; Fri, 17 Feb 2023 13:59:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 19F3A6B0073 for ; Fri, 17 Feb 2023 13:59:03 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id A884AA06A2 for ; Fri, 17 Feb 2023 18:59:02 +0000 (UTC) X-FDA: 80477696124.11.CC676CF Received: from mail-qv1-f43.google.com (mail-qv1-f43.google.com [209.85.219.43]) by imf15.hostedemail.com (Postfix) with ESMTP id DE8C9A0006 for ; Fri, 17 Feb 2023 18:58:59 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=HWHnbkjR; spf=pass (imf15.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.219.43 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1676660340; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=120DZvQoUAyoMne1gLDdZW6VztRdluB/SNtOtp0dBXE=; b=thUe4wVigDkIvwEDd1xc2m1RGHWpf4QxD0KxQpl2z1qlqq8wOmt9dgfM85nC+yAj/kdsLV kuVblRYeenygY3PUMCKpVYKwWI3ZGRz0CtcjkcopeNSTf/X9JOly/+eixUQD8Ciq1kWigR nScylIfQCt/Jnd42l5dDiBUBkHkwLqw= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=HWHnbkjR; spf=pass (imf15.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.219.43 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1676660340; a=rsa-sha256; cv=none; b=t/RG4OOurzkibw/97ClaMw7dT2dZohmLFitM93k6OiNlgjhD1hHdgUF8sELeF7vCOhrtP8 iIJ5aQknhwuicqOLhW9Wvpt3kHSYxn+5oXUCGpC5rGnAfStMYuL0Ty9HpJgNs28KIsv/rh 81Ic2Phb/mt7wp4aYwRIRpfDymsK7s0= Received: by mail-qv1-f43.google.com with SMTP id fn4so1357354qvb.12 for ; Fri, 17 Feb 2023 10:58:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; t=1676660339; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=120DZvQoUAyoMne1gLDdZW6VztRdluB/SNtOtp0dBXE=; b=HWHnbkjRQnNSEIxrvSuNgw015iDG0UYGSMRYvdvlCFyUG7IA88QJ/cgIw/BJiQkuqH SGBe/v9bzJ6x2dT13OGpv+bdX+oBZmtOWpzXvX0q3u/BfSOA8hNmPCw6EFJzppSBUhWh 583c+E/4UWXb9hHPC9vI/qkWrNYmafdPHp+OWYXj6eQIVU3LdUF5I/VKMGMdHoH7rre6 lBRNnsBDgBT7yJe2fBicS9PhitY1TBVqVywKHO5vvkm80sl6X0lQMrlE/o7yV+l0PSkp uHnaX0z9g1cVbP2Mh+pgohKxIiy7kaMlj75mfv5DgeiOpcUI3kbjbv55A9a/2z5pqjIu D8ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1676660339; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=120DZvQoUAyoMne1gLDdZW6VztRdluB/SNtOtp0dBXE=; b=R1O7UqLNP0BgUm4D2HaIoJzR6qjMPgelhn6eMJmwjeXP5T2ABefGNby8kR+hgYs0OW FRwJ97uonbBy5CHPlx7CYeUXq4xjDz8jUK3MaoFWTQeNrTQniV8eBMFog3sGEDNVGmFk a2hlQOHUUBR6PSpqt6Anik7hyW2hVD9Xc6ZvSN0m6dJA5Xyjcrg9Ic8i9hjQgoWrcd+m UI1aDUe/vxWiheRPU6ZEnmf6kvjhJmEh8uVdnQResQIwcIZNZQ743Szbabk6aSM0yEhN dr9RCwzHo3OqVnUMHR80VFEFBmYP56jV5JiT+fbxJI6p3AaPCPv6m+gPKmNcjjRVJBlG XMyw== X-Gm-Message-State: AO0yUKUTPC4zWf5xvheWYTtEA0woro1uhYmOz1//rbfNtqoijMKo2Nvr D2HvxNH5tZB/8cSFCDiacBXkAjwuBABQ8cVxpjd85g== X-Google-Smtp-Source: AK7set9sQXjvibYlOfpGG1s6aZBjsxvI/qhbl8oed0I1rQzHNLQ/EEclbb0BeC8mX0HIUmptFecbhFBs1chcqz59q4E= X-Received: by 2002:a0c:e006:0:b0:570:7e91:3927 with SMTP id j6-20020a0ce006000000b005707e913927mr389684qvk.76.1676660338988; Fri, 17 Feb 2023 10:58:58 -0800 (PST) MIME-Version: 1.0 References: <167653656244.3147810.5705900882794040229.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167653656244.3147810.5705900882794040229.stgit@dwillia2-xfh.jf.intel.com> From: Pasha Tatashin Date: Fri, 17 Feb 2023 13:58:23 -0500 Message-ID: Subject: Re: [PATCH] dax/kmem: Fix leak of memory-hotplug resources To: Dan Williams Cc: linux-cxl@vger.kernel.org, stable@vger.kernel.org, Oscar Salvador , David Hildenbrand , linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: cm6u1qekhg6zzpf3zwqdm5y74jsoj9h6 X-Rspam-User: X-Rspamd-Queue-Id: DE8C9A0006 X-Rspamd-Server: rspam06 X-HE-Tag: 1676660339-630025 X-HE-Meta: U2FsdGVkX1/As6TDRD0KJHmUTVb6lSQIWx0f22wuxzPTGmLVeoUPnPZG/MxXfuQ2ibwgoQ7EPA/c+LKchRdRCLV/0V9VLTa9b2vfo4YeR++O3uDsSk1Ms04IOJRH2e4rE33C9H7DD/v5nJd/cILE56RX+GbYfXn9JK5aQNgauLoDf2B/lsNOF6hxuyl5gMsaidg961wj8+I7BOwuoZ3xfJ97a8WwyqduLderFvoGrTHlYo0EYJWhHnNjIHzO6QzwVpio9lVIZtMO2WRZnGrnJfiYHteQASPBoXQzWmnum4INtqrTJ56++shnzGmO1O4/keluinG3HHRkXpxJYp6homEaEeKWIHpXWLsMM/jpN0gNoP1CDhERthWyjmqds7SfJJmMlY7Wq6V5WRaamcBUQAqD6+C4tRM0u3VuKlLI6lH89mauu7BVWWg14OFsPo5CS1S3/ojo4Cl9CQjLrUqTphL3uRp58Pmu2IzfiB5AHWfUOjVIuk0LURZexphw/7A3wO5c8011z0P7X+fgfU7jnPF3qqDLCzBUt9wVN2ufovh/KMhsa0+pBXcRqZ0o89/52F9bUsJwtLjTpa6EW0g9inreehsT67cVlvhi79yPeOmv/rchlXKeWSOHbykP24RMtRX+DD/GU/R9+umWvL9FX+xYkzzjaUgJLL5etkjkN1+KzExKTH2w2E4yOVOFdWAOo55PmJvXjJxaa4rZoxXsrUrTqIsBRXAVtp9rLWYyAzZwisptCxbSsD2UE59BXUMFg3+mIvopv782c/QIBbVSEMv2B7RGlY9+j1Nx5iAQmdnijIq/6DrKtmqhe4BK5I9oAja39uemfZsDBxODW1P0s2xD6lAeo5iI31PXJLyjoeF2x0fK0tMXDec6D9oP3Z5yYmejPDwrpzKLV1xNxm5qxicIA06euCvHTIqJX0GeZWX+MH1DKeYgNxAtz5UblMxAj9BtpFBap2e2kx0YyIY X/UH2NUS LdAQPyETDOS6KHVH+Ngh+gZHGxYDGCQZseuLgYMYoi65O4JJpj5UcpaiVxpqkx/iWlndQUD9efMmkPf9/NYvXIAazUiuDdWBWWqBlkvf7MJtSX70a5+CKTmQXxC6lBKgILN4Io0iie1PkFQ4SObr96eTVuLSoRnjhocmYIvL6rWYb5guOHmuMiB2ikeISc+9RuUHBOjXUjY2QIhwSeQjMI49BaVMMAL32lsn/sIdX/qIwQYBbU+VnwBNKHIV8+Aw7lzSrrpsCgdqDy+lmy3PKj2qawYTROSlwdgYEgzxOAOb+PZvuLn0+Fz3yLVasFtJkd3+ChWK6uink0sX8DH5NqjLP81NdoDfrrxaMiF7drBiKk8RkXeApmFqC0a8jX1+gWAFZ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Feb 16, 2023 at 3:36 AM Dan Williams wrote: > > While experimenting with CXL region removal the following corruption of > /proc/iomem appeared. > > Before: > f010000000-f04fffffff : CXL Window 0 > f010000000-f02fffffff : region4 > f010000000-f02fffffff : dax4.0 > f010000000-f02fffffff : System RAM (kmem) > > After (modprobe -r cxl_test): > f010000000-f02fffffff : **redacted binary garbage** > f010000000-f02fffffff : System RAM (kmem) > > ...and testing further the same is visible with persistent memory > assigned to kmem: > > Before: > 480000000-243fffffff : Persistent Memory > 480000000-57e1fffff : namespace3.0 > 580000000-243fffffff : dax3.0 > 580000000-243fffffff : System RAM (kmem) > > After (ndctl disable-region all): > 480000000-243fffffff : Persistent Memory > 580000000-243fffffff : ***redacted binary garbage*** > 580000000-243fffffff : System RAM (kmem) > > The corrupted data is from a use-after-free of the "dax4.0" and "dax3.0" > resources, and it also shows that the "System RAM (kmem)" resource is > not being removed. The bug does not appear after "modprobe -r kmem", it > requires the parent of "dax4.0" and "dax3.0" to be removed which > re-parents the leaked "System RAM (kmem)" instances. Those in turn > reference the freed resource as a parent. > > First up for the fix is release_mem_region_adjustable() needs to > reliably delete the resource inserted by add_memory_driver_managed(). > That is thwarted by a check for IORESOURCE_SYSRAM that predates the > dax/kmem driver, from commit: > > 65c78784135f ("kernel, resource: check for IORESOURCE_SYSRAM in release_mem_region_adjustable") > > That appears to be working around the behavior of HMM's > "MEMORY_DEVICE_PUBLIC" facility that has since been deleted. With that > check removed the "System RAM (kmem)" resource gets removed, but > corruption still occurs occasionally because the "dax" resource is not > reliably removed. > > The dax range information is freed before the device is unregistered, so > the driver can not reliably recall (another use after free) what it is > meant to release. Lastly if that use after free got lucky, the driver > was covering up the leak of "System RAM (kmem)" due to its use of > release_resource() which detaches, but does not free, child resources. > The switch to remove_resource() forces remove_memory() to be responsible > for the deletion of the resource added by add_memory_driver_managed(). > > Fixes: c2f3011ee697 ("device-dax: add an allocation interface for device-dax instances") > Cc: > Cc: Oscar Salvador > Cc: David Hildenbrand > Cc: Pavel Tatashin > Signed-off-by: Dan Williams Reviewed-by: Pasha Tatashin Thanks, Pasha