[PATCH -v3 0/3] resource: Fix region_intersects() vs add_memory_driver_managed()

[PATCH -v3 0/3] resource: Fix region_intersects() vs add_memory_driver_managed()

[Huang Ying]

The patchset fixes a bug of region_intersects() for systems with CXL
memory.  The details of the bug can be found in [1/3].  To avoid
similar bugs in the future.  A kunit test case for region_intersects()
is added in [3/3].  [2/3] is a preparation patch for [3/3].

Changelogs:

v3:

- Revise the patch subject, description and comments, Thanks Bjorn,
  Andy, David, and Dan!

- Added a kunit test case.

- Fixed several coding style issues, Thanks Andy!

- Link to v2: https://lore.kernel.org/linux-mm/20240819023413.1109779-1-ying.huang@intel.com/

v2:

- Fix a bug which occurs when descendant of a matched resource matches.

- Revise the patch description and comments to make the algorithm clearer.
  Thanks Dan and David's comments!

- Link to v1: https://lore.kernel.org/linux-mm/20240816020723.771196-1-ying.huang@intel.com/

Best Regards,
Huang, Ying

[PATCH -v3 1/3] resource: Fix region_intersects() vs add_memory_driver_managed()

[Huang Ying]

On a system with CXL memory, the resource tree (/proc/iomem) related
to CXL memory may look like something as follows.

490000000-50fffffff : CXL Window 0
  490000000-50fffffff : region0
    490000000-50fffffff : dax0.0
      490000000-50fffffff : System RAM (kmem)

Because drivers/dax/kmem.c calls add_memory_driver_managed() during
onlining CXL memory, which makes "System RAM (kmem)" a descendant of
"CXL Window X".  This confuses region_intersects(), which expects all
"System RAM" resources to be at the top level of iomem_resource.  This
can lead to bugs.

For example, when the following command line is executed to write some
memory in CXL memory range via /dev/mem,

 $ dd if=data of=/dev/mem bs=$((1 << 10)) seek=$((0x490000000 >> 10)) count=1
 dd: error writing '/dev/mem': Bad address
 1+0 records in
 0+0 records out
 0 bytes copied, 0.0283507 s, 0.0 kB/s

the command fails as expected.  However, the error code is wrong.  It
should be "Operation not permitted" instead of "Bad address".  More
seriously, the /dev/mem permission checking in devmem_is_allowed()
passes incorrectly.  Although the accessing is prevented later because
ioremap() isn't allowed to map system RAM, it is a potential security
issue.  During command executing, the following warning is reported in
the kernel log for calling ioremap() on system RAM.

 ioremap on RAM at 0x0000000490000000 - 0x0000000490000fff
 WARNING: CPU: 2 PID: 416 at arch/x86/mm/ioremap.c:216 __ioremap_caller.constprop.0+0x131/0x35d
 Call Trace:
  memremap+0xcb/0x184
  xlate_dev_mem_ptr+0x25/0x2f
  write_mem+0x94/0xfb
  vfs_write+0x128/0x26d
  ksys_write+0xac/0xfe
  do_syscall_64+0x9a/0xfd
  entry_SYSCALL_64_after_hwframe+0x4b/0x53

The details of command execution process are as follows.  In the above
resource tree, "System RAM" is a descendant of "CXL Window 0" instead
of a top level resource.  So, region_intersects() will report no
System RAM resources in the CXL memory region incorrectly, because it
only checks the top level resources.  Consequently,
devmem_is_allowed() will return 1 (allow access via /dev/mem) for CXL
memory region incorrectly.  Fortunately, ioremap() doesn't allow to
map System RAM and reject the access.

So, region_intersects() needs to be fixed to work correctly with the
resource tree with "System RAM" not at top level as above.  To fix it,
if we found a unmatched resource in the top level, we will continue to
search matched resources in its descendant resources.  So, we will not
miss any matched resources in resource tree anymore.

In the new implementation, an example resource tree

|------------- "CXL Window 0" ------------|
|-- "System RAM" --|

will behave similar as the following fake resource tree for
region_intersects(, IORESOURCE_SYSTEM_RAM, ),

|-- "System RAM" --||-- "CXL Window 0a" --|

Where "CXL Window 0a" is part of the original "CXL Window 0" that
isn't covered by "System RAM".

Fixes: c221c0b0308f ("device-dax: "Hotplug" persistent memory for use like normal RAM")
Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Alison Schofield <alison.schofield@intel.com>
Cc: Vishal Verma <vishal.l.verma@intel.com>
Cc: Ira Weiny <ira.weiny@intel.com>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Baoquan He <bhe@redhat.com>
---
 kernel/resource.c | 58 ++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 50 insertions(+), 8 deletions(-)

diff --git a/kernel/resource.c b/kernel/resource.c
index 14777afb0a99..235dc77f8add 100644
--- a/kernel/resource.c
+++ b/kernel/resource.c
@@ -540,20 +540,62 @@ static int __region_intersects(struct resource *parent, resource_size_t start,
 			       size_t size, unsigned long flags,
 			       unsigned long desc)
 {
-	struct resource res;
+	resource_size_t ostart, oend;
 	int type = 0; int other = 0;
-	struct resource *p;
+	struct resource *p, *dp;
+	bool is_type, covered;
+	struct resource res;
 
 	res.start = start;
 	res.end = start + size - 1;
 
 	for (p = parent->child; p ; p = p->sibling) {
-		bool is_type = (((p->flags & flags) == flags) &&
-				((desc == IORES_DESC_NONE) ||
-				 (desc == p->desc)));
-
-		if (resource_overlaps(p, &res))
-			is_type ? type++ : other++;
+		if (!resource_overlaps(p, &res))
+			continue;
+		is_type = (p->flags & flags) == flags &&
+			(desc == IORES_DESC_NONE || desc == p->desc);
+		if (is_type) {
+			type++;
+			continue;
+		}
+		/*
+		 * Continue to search in descendant resources as if the
+		 * matched descendant resources cover some ranges of 'p'.
+		 *
+		 * |------------- "CXL Window 0" ------------|
+		 * |-- "System RAM" --|
+		 *
+		 * will behave similar as the following fake resource
+		 * tree when searching "System RAM".
+		 *
+		 * |-- "System RAM" --||-- "CXL Window 0a" --|
+		 */
+		covered = false;
+		ostart = max(res.start, p->start);
+		oend = min(res.end, p->end);
+		for_each_resource(p, dp, false) {
+			if (!resource_overlaps(dp, &res))
+				continue;
+			is_type = (dp->flags & flags) == flags &&
+				(desc == IORES_DESC_NONE || desc == dp->desc);
+			if (is_type) {
+				type++;
+				/*
+				 * Range from 'ostart' to 'dp->start'
+				 * isn't covered by matched resource.
+				 */
+				if (dp->start > ostart)
+					break;
+				if (dp->end >= oend) {
+					covered = true;
+					break;
+				}
+				/* Remove covered range */
+				ostart = max(ostart, dp->end + 1);
+			}
+		}
+		if (!covered)
+			other++;
 	}
 
 	if (type == 0)
-- 
2.39.2

[PATCH -v3 2/3] resource: Make alloc_free_mem_region() works for iomem_resource

[Huang Ying]

During developing a kunit test case for region_intersects(), some fake
resources need to be inserted into iomem_resource.  To do that, a
resource hole needs to be found first in iomem_resource.

However, alloc_free_mem_region() cannot work for iomem_resource now.
Because the start address to check cannot be 0 to detect address
wrapping 0 in gfr_continue(), while iomem_resource.start == 0.  To
make alloc_free_mem_region() works for iomem_resource, gfr_start() is
changed to avoid to return 0 even if base->start == 0.  We don't need
to check 0 as start address.

Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Alison Schofield <alison.schofield@intel.com>
Cc: Vishal Verma <vishal.l.verma@intel.com>
Cc: Ira Weiny <ira.weiny@intel.com>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Baoquan He <bhe@redhat.com>
---
 kernel/resource.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/resource.c b/kernel/resource.c
index 235dc77f8add..035ef16c1a66 100644
--- a/kernel/resource.c
+++ b/kernel/resource.c
@@ -1873,7 +1873,7 @@ static resource_size_t gfr_start(struct resource *base, resource_size_t size,
 		return end - size + 1;
 	}
 
-	return ALIGN(base->start, align);
+	return ALIGN(max(base->start, align), align);
 }
 
 static bool gfr_continue(struct resource *base, resource_size_t addr,
-- 
2.39.2

[PATCH -v3 3/3] resource, kunit: Add test case for region_intersects()

[Huang Ying]

region_intersects() is important because it's used for /dev/mem
permission checking.  To avoid possible bug of region_intersects() in
the future, a kunit test case for region_intersects() is added.

Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Alison Schofield <alison.schofield@intel.com>
Cc: Vishal Verma <vishal.l.verma@intel.com>
Cc: Ira Weiny <ira.weiny@intel.com>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Baoquan He <bhe@redhat.com>
---
 kernel/resource.c       |  20 ++++--
 kernel/resource_kunit.c | 143 ++++++++++++++++++++++++++++++++++++++++
 lib/Kconfig.debug       |   1 +
 3 files changed, 158 insertions(+), 6 deletions(-)

diff --git a/kernel/resource.c b/kernel/resource.c
index 035ef16c1a66..e6d222fb0f84 100644
--- a/kernel/resource.c
+++ b/kernel/resource.c
@@ -1860,7 +1860,17 @@ EXPORT_SYMBOL(resource_list_free);
 #ifdef CONFIG_GET_FREE_REGION
 #define GFR_DESCENDING		(1UL << 0)
 #define GFR_REQUEST_REGION	(1UL << 1)
-#define GFR_DEFAULT_ALIGN (1UL << PA_SECTION_SHIFT)
+#ifdef PA_SECTION_SHIFT
+#define GFR_DEFAULT_ALIGN	(1UL << PA_SECTION_SHIFT)
+#else
+#define GFR_DEFAULT_ALIGN	PAGE_SIZE
+#endif
+
+#ifdef MAX_PHYSMEM_BITS
+#define MAX_PHYS_ADDR		((1ULL << MAX_PHYSMEM_BITS) - 1)
+#else
+#define MAX_PHYS_ADDR		(-1ULL)
+#endif
 
 static resource_size_t gfr_start(struct resource *base, resource_size_t size,
 				 resource_size_t align, unsigned long flags)
@@ -1868,8 +1878,7 @@ static resource_size_t gfr_start(struct resource *base, resource_size_t size,
 	if (flags & GFR_DESCENDING) {
 		resource_size_t end;
 
-		end = min_t(resource_size_t, base->end,
-			    (1ULL << MAX_PHYSMEM_BITS) - 1);
+		end = min_t(resource_size_t, base->end, MAX_PHYS_ADDR);
 		return end - size + 1;
 	}
 
@@ -1886,8 +1895,7 @@ static bool gfr_continue(struct resource *base, resource_size_t addr,
 	 * @size did not wrap 0.
 	 */
 	return addr > addr - size &&
-	       addr <= min_t(resource_size_t, base->end,
-			     (1ULL << MAX_PHYSMEM_BITS) - 1);
+		addr <= min_t(resource_size_t, base->end, MAX_PHYS_ADDR);
 }
 
 static resource_size_t gfr_next(resource_size_t addr, resource_size_t size,
@@ -2048,7 +2056,7 @@ struct resource *alloc_free_mem_region(struct resource *base,
 	return get_free_mem_region(NULL, base, size, align, name,
 				   IORES_DESC_NONE, flags);
 }
-EXPORT_SYMBOL_NS_GPL(alloc_free_mem_region, CXL);
+EXPORT_SYMBOL_GPL(alloc_free_mem_region);
 #endif /* CONFIG_GET_FREE_REGION */
 
 static int __init strict_iomem(char *str)
diff --git a/kernel/resource_kunit.c b/kernel/resource_kunit.c
index 0e509985a44a..42d2d8d20f5d 100644
--- a/kernel/resource_kunit.c
+++ b/kernel/resource_kunit.c
@@ -7,6 +7,8 @@
 #include <linux/ioport.h>
 #include <linux/kernel.h>
 #include <linux/string.h>
+#include <linux/sizes.h>
+#include <linux/mm.h>
 
 #define R0_START	0x0000
 #define R0_END		0xffff
@@ -137,9 +139,150 @@ static void resource_test_intersection(struct kunit *test)
 	} while (++i < ARRAY_SIZE(results_for_intersection));
 }
 
+/*
+ * The test resource tree for region_intersects() test:
+ *
+ * BASE-BASE+1M-1 : Test System RAM 0
+ *		  # hole 0 (BASE+1M-BASE+2M)
+ * BASE+2M-BASE+3M-1 : Test CXL Window 0
+ * BASE+3M-BASE+4M-1 : Test System RAM 1
+ * BASE+4M-BASE+7M-1 : Test CXL Window 1
+ *   BASE+4M-BASE+5M-1 : Test System RAM 2
+ *     BASE+4M+128K-BASE+4M+256K-1: Test Code
+ *   BASE+5M-BASE+6M-1 : Test System RAM 3
+ */
+#define RES_TEST_RAM0_OFFSET	0
+#define RES_TEST_RAM0_SIZE	SZ_1M
+#define RES_TEST_HOLE0_OFFSET	(RES_TEST_RAM0_OFFSET + RES_TEST_RAM0_SIZE)
+#define RES_TEST_HOLE0_SIZE	SZ_1M
+#define RES_TEST_WIN0_OFFSET	(RES_TEST_HOLE0_OFFSET + RES_TEST_HOLE0_SIZE)
+#define RES_TEST_WIN0_SIZE	SZ_1M
+#define RES_TEST_RAM1_OFFSET	(RES_TEST_WIN0_OFFSET + RES_TEST_WIN0_SIZE)
+#define RES_TEST_RAM1_SIZE	SZ_1M
+#define RES_TEST_WIN1_OFFSET	(RES_TEST_RAM1_OFFSET + RES_TEST_RAM1_SIZE)
+#define RES_TEST_WIN1_SIZE	(SZ_1M * 3)
+#define RES_TEST_RAM2_OFFSET	RES_TEST_WIN1_OFFSET
+#define RES_TEST_RAM2_SIZE	SZ_1M
+#define RES_TEST_CODE_OFFSET	(RES_TEST_RAM2_OFFSET + SZ_128K)
+#define RES_TEST_CODE_SIZE	SZ_128K
+#define RES_TEST_RAM3_OFFSET	(RES_TEST_RAM2_OFFSET + RES_TEST_RAM2_SIZE)
+#define RES_TEST_RAM3_SIZE	SZ_1M
+#define RES_TEST_TOTAL_SIZE	((RES_TEST_WIN1_OFFSET + RES_TEST_WIN1_SIZE))
+
+static void remove_free_resource(void *ctx)
+{
+	struct resource *res = (struct resource *)ctx;
+
+	remove_resource(res);
+	kfree(res);
+}
+
+static void resource_test_request_region(struct kunit *test, struct resource *parent,
+					 resource_size_t start, resource_size_t size,
+					 const char *name, unsigned long flags)
+{
+	struct resource *res;
+
+	res = __request_region(parent, start, size, name, flags);
+	KUNIT_ASSERT_NOT_NULL(test, res);
+	kunit_add_action_or_reset(test, remove_free_resource, res);
+}
+
+static void resource_test_insert_resource(struct kunit *test, struct resource *parent,
+					  resource_size_t start, resource_size_t size,
+					  const char *name, unsigned long flags)
+{
+	struct resource *res;
+
+	res = kzalloc(sizeof(*res), GFP_KERNEL);
+	KUNIT_ASSERT_NOT_NULL(test, res);
+
+	res->name = name;
+	res->start = start;
+	res->end = start + size - 1;
+	res->flags = flags;
+	if (insert_resource(parent, res)) {
+		kfree(res);
+		KUNIT_FAIL_AND_ABORT(test, "Fail to insert resource %pR\n", res);
+	}
+
+	kunit_add_action_or_reset(test, remove_free_resource, res);
+}
+
+static void resource_test_region_intersects(struct kunit *test)
+{
+	unsigned long flags =  IORESOURCE_SYSTEM_RAM | IORESOURCE_BUSY;
+	struct resource *parent;
+	resource_size_t start;
+
+	/* Find an iomem_resource hole to hold test resources */
+	parent = alloc_free_mem_region(&iomem_resource, RES_TEST_TOTAL_SIZE, SZ_1M,
+				       "test resources");
+	KUNIT_ASSERT_NOT_ERR_OR_NULL(test, parent);
+	start = parent->start;
+	kunit_add_action_or_reset(test, remove_free_resource, parent);
+
+	resource_test_request_region(test, parent, start + RES_TEST_RAM0_OFFSET,
+				     RES_TEST_RAM0_SIZE, "Test System RAM 0", flags);
+	resource_test_insert_resource(test, parent, start + RES_TEST_WIN0_OFFSET,
+				      RES_TEST_WIN0_SIZE, "Test CXL Window 0",
+				      IORESOURCE_MEM);
+	resource_test_request_region(test, parent, start + RES_TEST_RAM1_OFFSET,
+				     RES_TEST_RAM1_SIZE, "Test System RAM 1", flags);
+	resource_test_insert_resource(test, parent, start + RES_TEST_WIN1_OFFSET,
+				      RES_TEST_WIN1_SIZE, "Test CXL Window 1",
+				      IORESOURCE_MEM);
+	resource_test_request_region(test, parent, start + RES_TEST_RAM2_OFFSET,
+				     RES_TEST_RAM2_SIZE, "Test System RAM 2", flags);
+	resource_test_insert_resource(test, parent, start + RES_TEST_CODE_OFFSET,
+				      RES_TEST_CODE_SIZE, "Test Code", flags);
+	resource_test_request_region(test, parent, start + RES_TEST_RAM3_OFFSET,
+				     RES_TEST_RAM3_SIZE, "Test System RAM 3", flags);
+	kunit_release_action(test, remove_free_resource, parent);
+
+	KUNIT_EXPECT_EQ(test, REGION_INTERSECTS,
+			region_intersects(start + RES_TEST_RAM0_OFFSET, PAGE_SIZE,
+					  IORESOURCE_SYSTEM_RAM, IORES_DESC_NONE));
+	KUNIT_EXPECT_EQ(test, REGION_INTERSECTS,
+			region_intersects(start + RES_TEST_RAM0_OFFSET +
+					  RES_TEST_RAM0_SIZE - PAGE_SIZE, 2 * PAGE_SIZE,
+					  IORESOURCE_SYSTEM_RAM, IORES_DESC_NONE));
+	KUNIT_EXPECT_EQ(test, REGION_DISJOINT,
+			region_intersects(start + RES_TEST_HOLE0_OFFSET, PAGE_SIZE,
+					  IORESOURCE_SYSTEM_RAM, IORES_DESC_NONE));
+	KUNIT_EXPECT_EQ(test, REGION_DISJOINT,
+			region_intersects(start + RES_TEST_HOLE0_OFFSET +
+					  RES_TEST_HOLE0_SIZE - PAGE_SIZE, 2 * PAGE_SIZE,
+					  IORESOURCE_SYSTEM_RAM, IORES_DESC_NONE));
+	KUNIT_EXPECT_EQ(test, REGION_MIXED,
+			region_intersects(start + RES_TEST_WIN0_OFFSET +
+					  RES_TEST_WIN0_SIZE - PAGE_SIZE, 2 * PAGE_SIZE,
+					  IORESOURCE_SYSTEM_RAM, IORES_DESC_NONE));
+	KUNIT_EXPECT_EQ(test, REGION_INTERSECTS,
+			region_intersects(start + RES_TEST_RAM1_OFFSET +
+					  RES_TEST_RAM1_SIZE - PAGE_SIZE, 2 * PAGE_SIZE,
+					  IORESOURCE_SYSTEM_RAM, IORES_DESC_NONE));
+	KUNIT_EXPECT_EQ(test, REGION_INTERSECTS,
+			region_intersects(start + RES_TEST_RAM2_OFFSET +
+					  RES_TEST_RAM2_SIZE - PAGE_SIZE, 2 * PAGE_SIZE,
+					  IORESOURCE_SYSTEM_RAM, IORES_DESC_NONE));
+	KUNIT_EXPECT_EQ(test, REGION_INTERSECTS,
+			region_intersects(start + RES_TEST_CODE_OFFSET, PAGE_SIZE,
+					  IORESOURCE_SYSTEM_RAM, IORES_DESC_NONE));
+	KUNIT_EXPECT_EQ(test, REGION_INTERSECTS,
+			region_intersects(start + RES_TEST_RAM2_OFFSET,
+					  RES_TEST_RAM2_SIZE + PAGE_SIZE,
+					  IORESOURCE_SYSTEM_RAM, IORES_DESC_NONE));
+	KUNIT_EXPECT_EQ(test, REGION_MIXED,
+			region_intersects(start + RES_TEST_RAM3_OFFSET,
+					  RES_TEST_RAM3_SIZE + PAGE_SIZE,
+					  IORESOURCE_SYSTEM_RAM, IORES_DESC_NONE));
+}
+
 static struct kunit_case resource_test_cases[] = {
 	KUNIT_CASE(resource_test_union),
 	KUNIT_CASE(resource_test_intersection),
+	KUNIT_CASE(resource_test_region_intersects),
 	{}
 };
 
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index a30c03a66172..383453cbf4af 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -2616,6 +2616,7 @@ config RESOURCE_KUNIT_TEST
 	tristate "KUnit test for resource API" if !KUNIT_ALL_TESTS
 	depends on KUNIT
 	default KUNIT_ALL_TESTS
+	select GET_FREE_REGION
 	help
 	  This builds the resource API unit test.
 	  Tests the logic of API provided by resource.c and ioport.h.
-- 
2.39.2

Re: [PATCH -v3 1/3] resource: Fix region_intersects() vs add_memory_driver_managed()

[Andrew Morton]

On Fri,  6 Sep 2024 11:07:11 +0800 Huang Ying <ying.huang@intel.com> wrote:

> On a system with CXL memory, the resource tree (/proc/iomem) related
> to CXL memory may look like something as follows.
> 
> 490000000-50fffffff : CXL Window 0
>   490000000-50fffffff : region0
>     490000000-50fffffff : dax0.0
>       490000000-50fffffff : System RAM (kmem)
> 
> Because drivers/dax/kmem.c calls add_memory_driver_managed() during
> onlining CXL memory, which makes "System RAM (kmem)" a descendant of
> "CXL Window X".  This confuses region_intersects(), which expects all
> "System RAM" resources to be at the top level of iomem_resource.  This
> can lead to bugs.
> 
> ...
> 
> Fixes: c221c0b0308f ("device-dax: "Hotplug" persistent memory for use like normal RAM")

Do you believe this should be fixed in earlier (-stable) kernels?

Re: [PATCH -v3 1/3] resource: Fix region_intersects() vs add_memory_driver_managed()

[Huang, Ying]

Andrew Morton <akpm@linux-foundation.org> writes:

> On Fri,  6 Sep 2024 11:07:11 +0800 Huang Ying <ying.huang@intel.com> wrote:
>
>> On a system with CXL memory, the resource tree (/proc/iomem) related
>> to CXL memory may look like something as follows.
>> 
>> 490000000-50fffffff : CXL Window 0
>>   490000000-50fffffff : region0
>>     490000000-50fffffff : dax0.0
>>       490000000-50fffffff : System RAM (kmem)
>> 
>> Because drivers/dax/kmem.c calls add_memory_driver_managed() during
>> onlining CXL memory, which makes "System RAM (kmem)" a descendant of
>> "CXL Window X".  This confuses region_intersects(), which expects all
>> "System RAM" resources to be at the top level of iomem_resource.  This
>> can lead to bugs.
>> 
>> ...
>> 
>> Fixes: c221c0b0308f ("device-dax: "Hotplug" persistent memory for use like normal RAM")
>
> Do you believe this should be fixed in earlier (-stable) kernels?

Yes.  I think that we should fix this in -stable kernels.  Although no
severe bug will be triggered on x86 in the earlier kernels, it may be a
real bug for some other architectures.

--
Best Regards,
Huang, Ying

Re: [PATCH -v3 2/3] resource: Make alloc_free_mem_region() works for iomem_resource

[David Hildenbrand]

On 06.09.24 05:07, Huang Ying wrote:
> During developing a kunit test case for region_intersects(), some fake
> resources need to be inserted into iomem_resource.  To do that, a
> resource hole needs to be found first in iomem_resource.
> 
> However, alloc_free_mem_region() cannot work for iomem_resource now.
> Because the start address to check cannot be 0 to detect address
> wrapping 0 in gfr_continue(), while iomem_resource.start == 0.  To
> make alloc_free_mem_region() works for iomem_resource, gfr_start() is
> changed to avoid to return 0 even if base->start == 0.  We don't need
> to check 0 as start address.
> 
> Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Cc: David Hildenbrand <david@redhat.com>
> Cc: Davidlohr Bueso <dave@stgolabs.net>
> Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
> Cc: Dave Jiang <dave.jiang@intel.com>
> Cc: Alison Schofield <alison.schofield@intel.com>
> Cc: Vishal Verma <vishal.l.verma@intel.com>
> Cc: Ira Weiny <ira.weiny@intel.com>
> Cc: Alistair Popple <apopple@nvidia.com>
> Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Baoquan He <bhe@redhat.com>
> ---
>   kernel/resource.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/kernel/resource.c b/kernel/resource.c
> index 235dc77f8add..035ef16c1a66 100644
> --- a/kernel/resource.c
> +++ b/kernel/resource.c
> @@ -1873,7 +1873,7 @@ static resource_size_t gfr_start(struct resource *base, resource_size_t size,
>   		return end - size + 1;
>   	}
>   
> -	return ALIGN(base->start, align);

You should add a comment here. But I do find what you are doing here 
quite confusing.

Above you write: "We don't need to check 0 as start address." -- why? To 
make the code extra confusing? :)

/* Never return address 0, because XXX. */
if (!base->start)
	retrn align;
return ALIGN(base->start, align);


And i still haven't understood XXX. For whom exactly is address 0 a problem?

> +	return ALIGN(max(base->start, align), align);
>   }
>   
>   static bool gfr_continue(struct resource *base, resource_size_t addr,

-- 
Cheers,

David / dhildenb

Re: [PATCH -v3 2/3] resource: Make alloc_free_mem_region() works for iomem_resource

[Huang, Ying]

Hi, David,

David Hildenbrand <david@redhat.com> writes:

> On 06.09.24 05:07, Huang Ying wrote:
>> During developing a kunit test case for region_intersects(), some fake
>> resources need to be inserted into iomem_resource.  To do that, a
>> resource hole needs to be found first in iomem_resource.
>> However, alloc_free_mem_region() cannot work for iomem_resource now.
>> Because the start address to check cannot be 0 to detect address
>> wrapping 0 in gfr_continue(), while iomem_resource.start == 0.  To
>> make alloc_free_mem_region() works for iomem_resource, gfr_start() is
>> changed to avoid to return 0 even if base->start == 0.  We don't need
>> to check 0 as start address.
>> Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
>> Cc: Dan Williams <dan.j.williams@intel.com>
>> Cc: David Hildenbrand <david@redhat.com>
>> Cc: Davidlohr Bueso <dave@stgolabs.net>
>> Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
>> Cc: Dave Jiang <dave.jiang@intel.com>
>> Cc: Alison Schofield <alison.schofield@intel.com>
>> Cc: Vishal Verma <vishal.l.verma@intel.com>
>> Cc: Ira Weiny <ira.weiny@intel.com>
>> Cc: Alistair Popple <apopple@nvidia.com>
>> Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
>> Cc: Bjorn Helgaas <bhelgaas@google.com>
>> Cc: Baoquan He <bhe@redhat.com>
>> ---
>>   kernel/resource.c | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>> diff --git a/kernel/resource.c b/kernel/resource.c
>> index 235dc77f8add..035ef16c1a66 100644
>> --- a/kernel/resource.c
>> +++ b/kernel/resource.c
>> @@ -1873,7 +1873,7 @@ static resource_size_t gfr_start(struct resource *base, resource_size_t size,
>>   		return end - size + 1;
>>   	}
>>   -	return ALIGN(base->start, align);
>
> You should add a comment here. But I do find what you are doing here
> quite confusing.

Sure.  And sorry for confusing words.

> Above you write: "We don't need to check 0 as start address." -- why?
> To make the code extra confusing? :)

After the change, we will not return "0" from gfr_start().  So we cannot
check "0" as start address.  And I think nobody need to check "0", so it
should be OK to do that.

> /* Never return address 0, because XXX. */
> if (!base->start)
> 	retrn align;
> return ALIGN(base->start, align);
>
>
> And i still haven't understood XXX. For whom exactly is address 0 a problem?

Because the following lines in gfr_continue()

	/*
	 * In the ascend case be careful that the last increment by
	 * @size did not wrap 0.
	 */
	return addr > addr - size &&
	       addr <= min_t(resource_size_t, base->end,
			     (1ULL << MAX_PHYSMEM_BITS) - 1);

If addr == 0, then addr < addr - size.  gfr_continue() will return
false, and we will not check any address.

>> +	return ALIGN(max(base->start, align), align);
>>   }
>>     static bool gfr_continue(struct resource *base, resource_size_t
>> addr,

--
Best Regards,
Huang, Ying

Re: [PATCH -v3 2/3] resource: Make alloc_free_mem_region() works for iomem_resource

[David Hildenbrand]

On 09.09.24 09:07, Huang, Ying wrote:
> Hi, David,
> 
> David Hildenbrand <david@redhat.com> writes:
> 
>> On 06.09.24 05:07, Huang Ying wrote:
>>> During developing a kunit test case for region_intersects(), some fake
>>> resources need to be inserted into iomem_resource.  To do that, a
>>> resource hole needs to be found first in iomem_resource.
>>> However, alloc_free_mem_region() cannot work for iomem_resource now.
>>> Because the start address to check cannot be 0 to detect address
>>> wrapping 0 in gfr_continue(), while iomem_resource.start == 0.  To
>>> make alloc_free_mem_region() works for iomem_resource, gfr_start() is
>>> changed to avoid to return 0 even if base->start == 0.  We don't need
>>> to check 0 as start address.
>>> Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
>>> Cc: Dan Williams <dan.j.williams@intel.com>
>>> Cc: David Hildenbrand <david@redhat.com>
>>> Cc: Davidlohr Bueso <dave@stgolabs.net>
>>> Cc: Jonathan Cameron <jonathan.cameron@huawei.com>
>>> Cc: Dave Jiang <dave.jiang@intel.com>
>>> Cc: Alison Schofield <alison.schofield@intel.com>
>>> Cc: Vishal Verma <vishal.l.verma@intel.com>
>>> Cc: Ira Weiny <ira.weiny@intel.com>
>>> Cc: Alistair Popple <apopple@nvidia.com>
>>> Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
>>> Cc: Bjorn Helgaas <bhelgaas@google.com>
>>> Cc: Baoquan He <bhe@redhat.com>
>>> ---
>>>    kernel/resource.c | 2 +-
>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>> diff --git a/kernel/resource.c b/kernel/resource.c
>>> index 235dc77f8add..035ef16c1a66 100644
>>> --- a/kernel/resource.c
>>> +++ b/kernel/resource.c
>>> @@ -1873,7 +1873,7 @@ static resource_size_t gfr_start(struct resource *base, resource_size_t size,
>>>    		return end - size + 1;
>>>    	}
>>>    -	return ALIGN(base->start, align);
>>
>> You should add a comment here. But I do find what you are doing here
>> quite confusing.
> 
> Sure.  And sorry for confusing words.
> 
>> Above you write: "We don't need to check 0 as start address." -- why?
>> To make the code extra confusing? :)
> 
> After the change, we will not return "0" from gfr_start().  So we cannot
> check "0" as start address.  And I think nobody need to check "0", so it
> should be OK to do that.
> 
>> /* Never return address 0, because XXX. */
>> if (!base->start)
>> 	return align;
>> return ALIGN(base->start, align);
>>
>>
>> And i still haven't understood XXX. For whom exactly is address 0 a problem?
> 
> Because the following lines in gfr_continue()
> 
> 	/*
> 	 * In the ascend case be careful that the last increment by
> 	 * @size did not wrap 0.
> 	 */
> 	return addr > addr - size &&
> 	       addr <= min_t(resource_size_t, base->end,
> 			     (1ULL << MAX_PHYSMEM_BITS) - 1);
> 
> If addr == 0, then addr < addr - size.  gfr_continue() will return
> false, and we will not check any address.

Thanks, that makes things cleaner. I think it might be better to just rework
the retying logic, to detect wraps based on the old and new address. That
would require a bit more work, something like that should probably handle all
possible corner case. Dan wrote that code, so I'll leave it up to him to decide
how to handle that :)


diff --git a/kernel/resource.c b/kernel/resource.c
index 9f747bb7cd031..2cd054c8277e8 100644
--- a/kernel/resource.c
+++ b/kernel/resource.c
@@ -1819,40 +1819,48 @@ EXPORT_SYMBOL(resource_list_free);
  #define GFR_REQUEST_REGION	(1UL << 1)
  #define GFR_DEFAULT_ALIGN (1UL << PA_SECTION_SHIFT)
  
-static resource_size_t gfr_start(struct resource *base, resource_size_t size,
-				 resource_size_t align, unsigned long flags)
+static bool gfr_start(struct resource *base, resource_size_t *addr,
+		resource_size_t size, resource_size_t align, unsigned long flags)
  {
+	resource_size_t start_addr;
+
  	if (flags & GFR_DESCENDING) {
  		resource_size_t end;
  
  		end = min_t(resource_size_t, base->end,
  			    (1ULL << MAX_PHYSMEM_BITS) - 1);
-		return end - size + 1;
+		start_addr = end - size + 1;
+		if (start_addr > end || start_addr < base->start)
+			return false;
+	} else {
+		start_addr = ALIGN(base->start, align);
+		if (start_addr < base->start || start_addr > base->end)
+			return false;
  	}
-
-	return ALIGN(base->start, align);
+	*addr = start_addr;
+	return true;
  }
  
-static bool gfr_continue(struct resource *base, resource_size_t addr,
-			 resource_size_t size, unsigned long flags)
+static bool gfr_continue(struct resource *base, resource_size_t *addr,
+			 resource_size_t align, unsigned long flags)
  {
-	if (flags & GFR_DESCENDING)
-		return addr > size && addr >= base->start;
-	/*
-	 * In the ascend case be careful that the last increment by
-	 * @size did not wrap 0.
-	 */
-	return addr > addr - size &&
-	       addr <= min_t(resource_size_t, base->end,
-			     (1ULL << MAX_PHYSMEM_BITS) - 1);
-}
+	resource_size_t new_addr;
  
-static resource_size_t gfr_next(resource_size_t addr, resource_size_t size,
-				unsigned long flags)
-{
-	if (flags & GFR_DESCENDING)
-		return addr - size;
-	return addr + size;
+	if (flags & GFR_DESCENDING) {
+		new_addr = *addr - align;
+		if (new_addr > *addr || new_addr < base->start)
+			return false;
+	} else {
+		resource_size_t end;
+
+		end = min_t(resource_size_t, base->end,
+			    (1ULL << MAX_PHYSMEM_BITS) - 1);
+		new_addr = *addr + align;
+		if (new_addr < *addr || new_addr > end)
+			return false;
+	}
+	*addr = new_addr;
+	return true;
  }
  
  static void remove_free_mem_region(void *_res)
@@ -1893,9 +1901,9 @@ get_free_mem_region(struct device *dev, struct resource *base,
  	}
  
  	write_lock(&resource_lock);
-	for (addr = gfr_start(base, size, align, flags);
-	     gfr_continue(base, addr, align, flags);
-	     addr = gfr_next(addr, align, flags)) {
+	if (!gfr_start(base, &addr, size, align, flags))
+		goto unlock;
+	do {
  		if (__region_intersects(base, addr, size, 0, IORES_DESC_NONE) !=
  		    REGION_DISJOINT)
  			continue;
@@ -1939,7 +1947,8 @@ get_free_mem_region(struct device *dev, struct resource *base,
  		}
  
  		return res;
-	}
+	} while (gfr_continue(base, &addr, align, flags));
+unlock:
  	write_unlock(&resource_lock);
  
  	if (flags & GFR_REQUEST_REGION) {
-- 
2.46.0


-- 
Cheers,

David / dhildenb

Re: [PATCH -v3 3/3] resource, kunit: Add test case for region_intersects()

[Kees Bakker]

[-- Attachment #1: Type: text/plain, Size: 708 bytes --]

Op 06-09-2024 om 05:07 schreef Huang Ying:
> [...]
> +static void resource_test_insert_resource(struct kunit *test, struct resource *parent,
> +					  resource_size_t start, resource_size_t size,
> +					  const char *name, unsigned long flags)
> +{
> +	struct resource *res;
> +
> +	res = kzalloc(sizeof(*res), GFP_KERNEL);
> +	KUNIT_ASSERT_NOT_NULL(test, res);
> +
> +	res->name = name;
> +	res->start = start;
> +	res->end = start + size - 1;
> +	res->flags = flags;
> +	if (insert_resource(parent, res)) {
> +		kfree(res);
> +		KUNIT_FAIL_AND_ABORT(test, "Fail to insert resource %pR\n", res);
Isn't this a user-after-free?
> +	}
> +
> +	kunit_add_action_or_reset(test, remove_free_resource, res);
> +}
>

[-- Attachment #2: Type: text/html, Size: 1195 bytes --]

Re: [PATCH -v3 3/3] resource, kunit: Add test case for region_intersects()

[Huang, Ying]

Kees Bakker <kees@ijzerbout.nl> writes:

> Op 06-09-2024 om 05:07 schreef Huang Ying:
>> [...]
>> +static void resource_test_insert_resource(struct kunit *test, struct resource *parent,
>> +					  resource_size_t start, resource_size_t size,
>> +					  const char *name, unsigned long flags)
>> +{
>> +	struct resource *res;
>> +
>> +	res = kzalloc(sizeof(*res), GFP_KERNEL);
>> +	KUNIT_ASSERT_NOT_NULL(test, res);
>> +
>> +	res->name = name;
>> +	res->start = start;
>> +	res->end = start + size - 1;
>> +	res->flags = flags;
>> +	if (insert_resource(parent, res)) {
>> +		kfree(res);
>> +		KUNIT_FAIL_AND_ABORT(test, "Fail to insert resource %pR\n", res);
> Isn't this a user-after-free?

Good catch!  Thanks for pointing this out.  I should be more careful for
the error path.  
>> +	}
>> +
>> +	kunit_add_action_or_reset(test, remove_free_resource, res);

This may cause use-after-free if failed to allocate memory for
add_action.  Will fix this too.

>> +}
>>

--
Best Regards,
Huang, Ying

Re: [PATCH -v3 3/3] resource, kunit: Add test case for region_intersects()

[Kees Bakker]

Op 30-09-2024 om 02:53 schreef Huang, Ying:
> Kees Bakker <kees@ijzerbout.nl> writes:
>
>> Op 06-09-2024 om 05:07 schreef Huang Ying:
>>> [...]
>>> +static void resource_test_insert_resource(struct kunit *test, struct resource *parent,
>>> +					  resource_size_t start, resource_size_t size,
>>> +					  const char *name, unsigned long flags)
>>> +{
>>> +	struct resource *res;
>>> +
>>> +	res = kzalloc(sizeof(*res), GFP_KERNEL);
>>> +	KUNIT_ASSERT_NOT_NULL(test, res);
>>> +
>>> +	res->name = name;
>>> +	res->start = start;
>>> +	res->end = start + size - 1;
>>> +	res->flags = flags;
>>> +	if (insert_resource(parent, res)) {
>>> +		kfree(res);
>>> +		KUNIT_FAIL_AND_ABORT(test, "Fail to insert resource %pR\n", res);
>> Isn't this a user-after-free?
> Good catch!  Thanks for pointing this out.  I should be more careful for
> the error path.
I have to honest, it wasn't me who found this. It was Coverity.
>>> +	}
>>> +
>>> +	kunit_add_action_or_reset(test, remove_free_resource, res);
> This may cause use-after-free if failed to allocate memory for
> add_action.  Will fix this too.
>
>>> +}
>>>
> --
> Best Regards,
> Huang, Ying