* Re: [Bug 210023] New: Crash when allocating > 2 TB memory [not found] <bug-210023-27@https.bugzilla.kernel.org/> @ 2020-11-04 0:27 ` Andrew Morton 2020-11-04 0:47 ` Matthew Wilcox 2020-11-04 13:55 ` Hsin-Hui Wu 0 siblings, 2 replies; 5+ messages in thread From: Andrew Morton @ 2020-11-04 0:27 UTC (permalink / raw) To: hsinhuiwu; +Cc: bugzilla-daemon, linux-mm (switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface). On Tue, 03 Nov 2020 18:50:07 +0000 bugzilla-daemon@bugzilla.kernel.org wrote: > https://bugzilla.kernel.org/show_bug.cgi?id=210023 > > Bug ID: 210023 > Summary: Crash when allocating > 2 TB memory > Product: Memory Management > Version: 2.5 > Kernel Version: 3.10.0-957.27.2.el7.x86_64 > Hardware: All > OS: Linux > Tree: Mainline > Status: NEW > Severity: blocking > Priority: P1 > Component: Slab Allocator > Assignee: akpm@linux-foundation.org > Reporter: hsinhuiwu@gmail.com > Regression: No > > With a machine with 3 TB (more than 2 TB memory). If you use vmalloc to > allocate > 2 TB memory, the array_size below will be overflowed. How was this observed? Is there any know userspace operation which causes the kernel to try to vmalloc such a large hunk of memory? > The array_size is an unsigned int and can only be used to allocate less than 2 > TB memory. If you pass 2*1028*1028*1024*1024 = 2 * 2^40 in the argument of > vmalloc. The array_size will become 2*2^31 = 2^32. The 2^32 cannot be store > with a 32 bit integer. > > The fix is to change the type of array_size to unsigned long. > > vmalloc.c > > 1762 void *vmalloc(unsigned long size) > 1763 { > 1764 return __vmalloc_node_flags(size, NUMA_NO_NODE, > 1765 GFP_KERNEL | __GFP_HIGHMEM); > 1766 } OK, thanks. Against current mainline your proposed change would look like this, yes? --- a/mm/vmalloc.c~a +++ a/mm/vmalloc.c @@ -2461,9 +2461,11 @@ static void *__vmalloc_area_node(struct { const gfp_t nested_gfp = (gfp_mask & GFP_RECLAIM_MASK) | __GFP_ZERO; unsigned int nr_pages = get_vm_area_size(area) >> PAGE_SHIFT; - unsigned int array_size = nr_pages * sizeof(struct page *), i; + unsigned long array_size + unsigned int i; struct page **pages; + array_size = (unsigned long)nr_pages * sizeof(struct page *); gfp_mask |= __GFP_NOWARN; if (!(gfp_mask & (GFP_DMA | GFP_DMA32))) gfp_mask |= __GFP_HIGHMEM; _ ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Bug 210023] New: Crash when allocating > 2 TB memory 2020-11-04 0:27 ` [Bug 210023] New: Crash when allocating > 2 TB memory Andrew Morton @ 2020-11-04 0:47 ` Matthew Wilcox 2020-11-04 13:55 ` Hsin-Hui Wu 1 sibling, 0 replies; 5+ messages in thread From: Matthew Wilcox @ 2020-11-04 0:47 UTC (permalink / raw) To: Andrew Morton; +Cc: hsinhuiwu, bugzilla-daemon, linux-mm On Tue, Nov 03, 2020 at 04:27:40PM -0800, Andrew Morton wrote: > const gfp_t nested_gfp = (gfp_mask & GFP_RECLAIM_MASK) | __GFP_ZERO; > unsigned int nr_pages = get_vm_area_size(area) >> PAGE_SHIFT; > - unsigned int array_size = nr_pages * sizeof(struct page *), i; > + unsigned long array_size > + unsigned int i; > struct page **pages; > > + array_size = (unsigned long)nr_pages * sizeof(struct page *); This is only pushing the problem out ~5 years. If somebody tries to allocate a 16TB area, 16TB / 4kB is 4GB and nr_pages overflows. That's only 3 doublings of RAM away. I think we need to change vm_struct's nr_pages from an unsigned int to an unsigned long. It won't take up any more room because it's sandwiched between a pointer and a phys_addr_t. ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Bug 210023] New: Crash when allocating > 2 TB memory 2020-11-04 0:27 ` [Bug 210023] New: Crash when allocating > 2 TB memory Andrew Morton 2020-11-04 0:47 ` Matthew Wilcox @ 2020-11-04 13:55 ` Hsin-Hui Wu 2020-11-04 18:19 ` Andrew Morton 1 sibling, 1 reply; 5+ messages in thread From: Hsin-Hui Wu @ 2020-11-04 13:55 UTC (permalink / raw) To: Andrew Morton; +Cc: bugzilla-daemon, linux-mm [-- Attachment #1: Type: text/plain, Size: 3370 bytes --] > With a machine with 3 TB (more than 2 TB memory). If you use vmalloc to > allocate > 2 TB memory, the array_size below will be overflowed. How was this observed? Is there any know userspace operation which causes the kernel to try to vmalloc such a large hunk of memory? [Frank] The Dell PowerEdge R740/R940 can have up to 3TB/6TB memory. installed. Our application requires reserve consecutive memory in the kernel space and protected from userspace programs. ---------------------------------------------------------------------------------------------------- OK, thanks. Against current mainline your proposed change would look like this, yes? [Frank] Yes. This will support up to less than 16 TB. If you want to support more than 16 TB, we need to expand nr_pages to unsigned long as Matthew pointed out. Will it be possible to add this to kernel 3.10.0-957.27.2.el7.x86_64? Thanks, Frank On Tue, Nov 3, 2020 at 7:27 PM Andrew Morton <akpm@linux-foundation.org> wrote: > (switched to email. Please respond via emailed reply-to-all, not via the > bugzilla web interface). > > On Tue, 03 Nov 2020 18:50:07 +0000 bugzilla-daemon@bugzilla.kernel.org > wrote: > > > https://bugzilla.kernel.org/show_bug.cgi?id=210023 > > > > Bug ID: 210023 > > Summary: Crash when allocating > 2 TB memory > > Product: Memory Management > > Version: 2.5 > > Kernel Version: 3.10.0-957.27.2.el7.x86_64 > > Hardware: All > > OS: Linux > > Tree: Mainline > > Status: NEW > > Severity: blocking > > Priority: P1 > > Component: Slab Allocator > > Assignee: akpm@linux-foundation.org > > Reporter: hsinhuiwu@gmail.com > > Regression: No > > > > With a machine with 3 TB (more than 2 TB memory). If you use vmalloc to > > allocate > 2 TB memory, the array_size below will be overflowed. > > How was this observed? > > Is there any know userspace operation which causes the kernel to try to > vmalloc such a large hunk of memory? > > > The array_size is an unsigned int and can only be used to allocate less > than 2 > > TB memory. If you pass 2*1028*1028*1024*1024 = 2 * 2^40 in the argument > of > > vmalloc. The array_size will become 2*2^31 = 2^32. The 2^32 cannot be > store > > with a 32 bit integer. > > > > The fix is to change the type of array_size to unsigned long. > > > > vmalloc.c > > > > 1762 void *vmalloc(unsigned long size) > > 1763 { > > 1764 return __vmalloc_node_flags(size, NUMA_NO_NODE, > > 1765 GFP_KERNEL | __GFP_HIGHMEM); > > 1766 } > > OK, thanks. Against current mainline your proposed change would look > like this, yes? > > --- a/mm/vmalloc.c~a > +++ a/mm/vmalloc.c > @@ -2461,9 +2461,11 @@ static void *__vmalloc_area_node(struct > { > const gfp_t nested_gfp = (gfp_mask & GFP_RECLAIM_MASK) | > __GFP_ZERO; > unsigned int nr_pages = get_vm_area_size(area) >> PAGE_SHIFT; > - unsigned int array_size = nr_pages * sizeof(struct page *), i; > + unsigned long array_size > + unsigned int i; > struct page **pages; > > + array_size = (unsigned long)nr_pages * sizeof(struct page *); > gfp_mask |= __GFP_NOWARN; > if (!(gfp_mask & (GFP_DMA | GFP_DMA32))) > gfp_mask |= __GFP_HIGHMEM; > _ > > [-- Attachment #2: Type: text/html, Size: 4788 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Bug 210023] New: Crash when allocating > 2 TB memory 2020-11-04 13:55 ` Hsin-Hui Wu @ 2020-11-04 18:19 ` Andrew Morton [not found] ` <CANpob6LY6oXqTG0GTJk-Z9qJtp9kpCCdoDAsbOJgTA8QfPCe2Q@mail.gmail.com> 0 siblings, 1 reply; 5+ messages in thread From: Andrew Morton @ 2020-11-04 18:19 UTC (permalink / raw) To: Hsin-Hui Wu; +Cc: bugzilla-daemon, linux-mm On Wed, 4 Nov 2020 08:55:11 -0500 Hsin-Hui Wu <hsinhuiwu@gmail.com> wrote: > > With a machine with 3 TB (more than 2 TB memory). If you use vmalloc to > > allocate > 2 TB memory, the array_size below will be overflowed. > > How was this observed? > > Is there any know userspace operation which causes the kernel to try to > vmalloc such a large hunk of memory? > > [Frank] The Dell PowerEdge R740/R940 can have up to 3TB/6TB memory. > installed. Our application requires reserve consecutive memory in the kernel > space and protected from userspace programs. Did this require custom kernel changes? If not, precisely which system calls were used to cause this allocation attempt? > ---------------------------------------------------------------------------------------------------- > OK, thanks. Against current mainline your proposed change would look > like this, yes? > > [Frank] Yes. This will support up to less than 16 TB. If you want to support > more than 16 TB, we need to expand nr_pages to unsigned long as > Matthew pointed out. > > Will it be possible to add this to kernel 3.10.0-957.27.2.el7.x86_64? That is up to Red Hat to decide. ^ permalink raw reply [flat|nested] 5+ messages in thread
[parent not found: <CANpob6LY6oXqTG0GTJk-Z9qJtp9kpCCdoDAsbOJgTA8QfPCe2Q@mail.gmail.com>]
[parent not found: <20201104163822.95df9ee4e42ad5ec45852a19@linux-foundation.org>]
* Re: [Bug 210023] New: Crash when allocating > 2 TB memory [not found] ` <20201104163822.95df9ee4e42ad5ec45852a19@linux-foundation.org> @ 2020-11-05 3:52 ` Hsin-Hui Wu 0 siblings, 0 replies; 5+ messages in thread From: Hsin-Hui Wu @ 2020-11-05 3:52 UTC (permalink / raw) To: Andrew Morton; +Cc: bugzilla-daemon, linux-mm [-- Attachment #1: Type: text/plain, Size: 1943 bytes --] We don't use userspace program to allocate memory. We use kernel module to do so. Here is the sample kernel program you can use to test on a machine that has more than 2 TB memory. Use "insmod memtest.ko mb=2100000" to allocate more than 2 TB memory. Use "rmmod memtest" to free the memory. memtest.c ============================================================= #include <linux/version.h> #include <linux/init.h> #include <linux/module.h> #include <linux/fs.h> #include <linux/cdev.h> #include <linux/mm.h> #include <linux/vmalloc.h> #ifdef MODVERSIONS #include <linux/modversions.h> #endif #include <asm/io.h> static char *mem_ptr; MODULE_LICENSE("GPL"); int mb = 0; MODULE_PARM_DESC(mb, "alloc MB memory"); module_param(mb, int, S_IWUSR | S_IRUGO); static int __init memtest_init(void) { mem_ptr = NULL; if (mb > 0) { unsigned long alloc_len; alloc_len = mb * 1024 * 1024; mem_ptr = vmalloc(alloc_len); if (mem_ptr) { printk("memtest: vmalloc for size %u MB.\n", mb); } else { printk("memtest: failed to vmalloc for bytes %u MB.\n", mb); } } else { printk("memtest len_MB\n"); } return(0); } static void __exit memtest_exit(void) { if (mem_ptr) { vfree(mem_ptr); printk("memtest: vfree for size %u MB.\n", mb); mem_ptr = NULL; } } module_init(memtest_init); module_exit(memtest_exit); On Wed, Nov 4, 2020 at 7:38 PM Andrew Morton <akpm@linux-foundation.org> wrote: > On Wed, 4 Nov 2020 15:53:36 -0500 Hsin-Hui Wu <hsinhuiwu@gmail.com> wrote: > > > Did this require custom kernel changes? If not, precisely which system > > calls were used to cause this allocation attempt? > > > > [Frank] No. vmalloc(). > > > > vmalloc() is not a system call. > > Please fully describe how vmalloc() came to be called with such a large > request size. Fully. Are you able to provide example userspace code? > > [-- Attachment #2: Type: text/html, Size: 2803 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2020-11-05 3:52 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <bug-210023-27@https.bugzilla.kernel.org/>
2020-11-04 0:27 ` [Bug 210023] New: Crash when allocating > 2 TB memory Andrew Morton
2020-11-04 0:47 ` Matthew Wilcox
2020-11-04 13:55 ` Hsin-Hui Wu
2020-11-04 18:19 ` Andrew Morton
[not found] ` <CANpob6LY6oXqTG0GTJk-Z9qJtp9kpCCdoDAsbOJgTA8QfPCe2Q@mail.gmail.com>
[not found] ` <20201104163822.95df9ee4e42ad5ec45852a19@linux-foundation.org>
2020-11-05 3:52 ` Hsin-Hui Wu
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox