[PATCH v2] alloc_tag: handle incomplete bulk allocations in vm_module_tags_populate

[PATCH v2] alloc_tag: handle incomplete bulk allocations in vm_module_tags_populate

[T.J. Mercier]

alloc_pages_bulk_node may partially succeed and allocate fewer than the
requested nr_pages. There are several conditions under which this can
occur, but we have encountered the case where CONFIG_PAGE_OWNER is
enabled causing all bulk allocations to always fallback to single page
allocations due to commit 187ad460b841 ("mm/page_alloc: avoid page
allocator recursion with pagesets.lock held").

Currently vm_module_tags_populate immediately fails when
alloc_pages_bulk_node returns fewer than the requested number of pages.
When this happens memory allocation profiling gets disabled, for example

[   14.297583] [9:       modprobe:  465] Failed to allocate memory for allocation tags in the module scsc_wlan. Memory allocation profiling is disabled!
[   14.299339] [9:       modprobe:  465] modprobe: Failed to insmod '/vendor/lib/modules/scsc_wlan.ko' with args '': Out of memory

This patch causes vm_module_tags_populate to retry bulk allocations for
the remaining memory instead of failing immediately which will avoid the
disablement of memory allocation profiling.

Reported-by: Janghyuck Kim <janghyuck.kim@samsung.com>
Fixes: 0f9b685626da ("alloc_tag: populate memory for module tags as needed")
Signed-off-by: T.J. Mercier <tjmercier@google.com>
Acked-by: Suren Baghdasaryan <surenb@google.com>
---
 lib/alloc_tag.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c
index 1d893e313614..25ecc1334b67 100644
--- a/lib/alloc_tag.c
+++ b/lib/alloc_tag.c
@@ -422,11 +422,20 @@ static int vm_module_tags_populate(void)
 		unsigned long old_shadow_end = ALIGN(phys_end, MODULE_ALIGN);
 		unsigned long new_shadow_end = ALIGN(new_end, MODULE_ALIGN);
 		unsigned long more_pages;
-		unsigned long nr;
+		unsigned long nr = 0;
 
 		more_pages = ALIGN(new_end - phys_end, PAGE_SIZE) >> PAGE_SHIFT;
-		nr = alloc_pages_bulk_node(GFP_KERNEL | __GFP_NOWARN,
-					   NUMA_NO_NODE, more_pages, next_page);
+		while (nr < more_pages) {
+			unsigned long allocated;
+
+			allocated = alloc_pages_bulk_node(GFP_KERNEL | __GFP_NOWARN,
+				NUMA_NO_NODE, more_pages - nr, next_page + nr);
+
+			if (!allocated)
+				break;
+			nr += allocated;
+		}
+
 		if (nr < more_pages ||
 		    vmap_pages_range(phys_end, phys_end + (nr << PAGE_SHIFT), PAGE_KERNEL,
 				     next_page, PAGE_SHIFT) < 0) {
-- 
2.49.0.504.g3bcea36a83-goog

Re: [PATCH v2] alloc_tag: handle incomplete bulk allocations in vm_module_tags_populate

[Andrew Morton]

On Wed,  9 Apr 2025 22:51:11 +0000 "T.J. Mercier" <tjmercier@google.com> wrote:

> alloc_pages_bulk_node may partially succeed and allocate fewer than the
> requested nr_pages. There are several conditions under which this can
> occur, but we have encountered the case where CONFIG_PAGE_OWNER is
> enabled causing all bulk allocations to always fallback to single page
> allocations due to commit 187ad460b841 ("mm/page_alloc: avoid page
> allocator recursion with pagesets.lock held").
> 
> Currently vm_module_tags_populate immediately fails when
> alloc_pages_bulk_node returns fewer than the requested number of pages.
> When this happens memory allocation profiling gets disabled, for example
> 
> [   14.297583] [9:       modprobe:  465] Failed to allocate memory for allocation tags in the module scsc_wlan. Memory allocation profiling is disabled!
> [   14.299339] [9:       modprobe:  465] modprobe: Failed to insmod '/vendor/lib/modules/scsc_wlan.ko' with args '': Out of memory
> 
> This patch causes vm_module_tags_populate to retry bulk allocations for
> the remaining memory instead of failing immediately which will avoid the
> disablement of memory allocation profiling.
> 

Thanks.  I'm assuming we want cc:stable on this?

btw, it looks like the "Clean up and error out" code in
vm_module_tags_populate() could use release_pages().

Re: [PATCH v2] alloc_tag: handle incomplete bulk allocations in vm_module_tags_populate

[Suren Baghdasaryan]

On Thu, Apr 10, 2025 at 12:12 AM Andrew Morton
<akpm@linux-foundation.org> wrote:
>
> On Wed,  9 Apr 2025 22:51:11 +0000 "T.J. Mercier" <tjmercier@google.com> wrote:
>
> > alloc_pages_bulk_node may partially succeed and allocate fewer than the
> > requested nr_pages. There are several conditions under which this can
> > occur, but we have encountered the case where CONFIG_PAGE_OWNER is
> > enabled causing all bulk allocations to always fallback to single page
> > allocations due to commit 187ad460b841 ("mm/page_alloc: avoid page
> > allocator recursion with pagesets.lock held").
> >
> > Currently vm_module_tags_populate immediately fails when
> > alloc_pages_bulk_node returns fewer than the requested number of pages.
> > When this happens memory allocation profiling gets disabled, for example
> >
> > [   14.297583] [9:       modprobe:  465] Failed to allocate memory for allocation tags in the module scsc_wlan. Memory allocation profiling is disabled!
> > [   14.299339] [9:       modprobe:  465] modprobe: Failed to insmod '/vendor/lib/modules/scsc_wlan.ko' with args '': Out of memory
> >
> > This patch causes vm_module_tags_populate to retry bulk allocations for
> > the remaining memory instead of failing immediately which will avoid the
> > disablement of memory allocation profiling.
> >
>
> Thanks.  I'm assuming we want cc:stable on this?
>
> btw, it looks like the "Clean up and error out" code in
> vm_module_tags_populate() could use release_pages().

True. I'll add that into my TODO list. Thanks!

>

Re: [PATCH v2] alloc_tag: handle incomplete bulk allocations in vm_module_tags_populate

[Yunsheng Lin]

On 2025/4/10 9:44, Suren Baghdasaryan wrote:
> On Thu, Apr 10, 2025 at 12:12 AM Andrew Morton
> <akpm@linux-foundation.org> wrote:
>>
>> On Wed,  9 Apr 2025 22:51:11 +0000 "T.J. Mercier" <tjmercier@google.com> wrote:
>>
>>> alloc_pages_bulk_node may partially succeed and allocate fewer than the
>>> requested nr_pages. There are several conditions under which this can
>>> occur, but we have encountered the case where CONFIG_PAGE_OWNER is
>>> enabled causing all bulk allocations to always fallback to single page
>>> allocations due to commit 187ad460b841 ("mm/page_alloc: avoid page
>>> allocator recursion with pagesets.lock held").
>>>
>>> Currently vm_module_tags_populate immediately fails when
>>> alloc_pages_bulk_node returns fewer than the requested number of pages.
>>> When this happens memory allocation profiling gets disabled, for example
>>>
>>> [   14.297583] [9:       modprobe:  465] Failed to allocate memory for allocation tags in the module scsc_wlan. Memory allocation profiling is disabled!
>>> [   14.299339] [9:       modprobe:  465] modprobe: Failed to insmod '/vendor/lib/modules/scsc_wlan.ko' with args '': Out of memory
>>>
>>> This patch causes vm_module_tags_populate to retry bulk allocations for
>>> the remaining memory instead of failing immediately which will avoid the
>>> disablement of memory allocation profiling.
>>>
>>
>> Thanks.  I'm assuming we want cc:stable on this?
>>
>> btw, it looks like the "Clean up and error out" code in
>> vm_module_tags_populate() could use release_pages().

For the 'Clean up and error out' part:
next_page[] array might need to be reset to NULL if user is able to
reenable the memory allocation profiling when the above happens as the
current page bulk alloc API are only populating NULL elements.

> 
> True. I'll add that into my TODO list. Thanks!
> 
>>
> 

Re: [PATCH v2] alloc_tag: handle incomplete bulk allocations in vm_module_tags_populate

[Suren Baghdasaryan]

On Wed, Apr 9, 2025 at 7:52 PM Yunsheng Lin <linyunsheng@huawei.com> wrote:
>
> On 2025/4/10 9:44, Suren Baghdasaryan wrote:
> > On Thu, Apr 10, 2025 at 12:12 AM Andrew Morton
> > <akpm@linux-foundation.org> wrote:
> >>
> >> On Wed,  9 Apr 2025 22:51:11 +0000 "T.J. Mercier" <tjmercier@google.com> wrote:
> >>
> >>> alloc_pages_bulk_node may partially succeed and allocate fewer than the
> >>> requested nr_pages. There are several conditions under which this can
> >>> occur, but we have encountered the case where CONFIG_PAGE_OWNER is
> >>> enabled causing all bulk allocations to always fallback to single page
> >>> allocations due to commit 187ad460b841 ("mm/page_alloc: avoid page
> >>> allocator recursion with pagesets.lock held").
> >>>
> >>> Currently vm_module_tags_populate immediately fails when
> >>> alloc_pages_bulk_node returns fewer than the requested number of pages.
> >>> When this happens memory allocation profiling gets disabled, for example
> >>>
> >>> [   14.297583] [9:       modprobe:  465] Failed to allocate memory for allocation tags in the module scsc_wlan. Memory allocation profiling is disabled!
> >>> [   14.299339] [9:       modprobe:  465] modprobe: Failed to insmod '/vendor/lib/modules/scsc_wlan.ko' with args '': Out of memory
> >>>
> >>> This patch causes vm_module_tags_populate to retry bulk allocations for
> >>> the remaining memory instead of failing immediately which will avoid the
> >>> disablement of memory allocation profiling.
> >>>
> >>
> >> Thanks.  I'm assuming we want cc:stable on this?
> >>
> >> btw, it looks like the "Clean up and error out" code in
> >> vm_module_tags_populate() could use release_pages().
>
> For the 'Clean up and error out' part:
> next_page[] array might need to be reset to NULL if user is able to
> reenable the memory allocation profiling when the above happens as the
> current page bulk alloc API are only populating NULL elements.

We shouldn't be able to re-enable memory allocation profiling once
vm_module_tags_populate() fails. In that case shutdown_mem_profiling()
call disables memory allocation profiling and sets
mem_profiling_support=false. I might have to modify
memory_allocation_profiling_sysctls to prevent users from trying to
re-enable profiling via sysfs if mem_profiling_support is set to
false. Will take a closer look at that but regarding your comment,
re-enabling profiling once it's shut down is not a valid usecase.

>
> >
> > True. I'll add that into my TODO list. Thanks!
> >
> >>
> >