Re: [Xen-devel] [PATCH 2/2] x86/xen: dont add memory above max allowed allocation

Re: [Xen-devel] [PATCH 2/2] x86/xen: dont add memory above max allowed allocation

[Juergen Gross]

On 22/01/2019 09:52, Jan Beulich wrote:
>>>> On 22.01.19 at 09:06, <jgross@suse.com> wrote:
>> Don't allow memory to be added above the allowed maximum allocation
>> limit set by Xen.
> 
> This reads as if the hypervisor was imposing a limit here, but looking at
> xen_get_max_pages(), xen_foreach_remap_area(), and
> xen_count_remap_pages() I take it that it's a restriction enforced by
> the Xen subsystem in Linux. Furthermore from the cover letter I imply
> that the observed issue was on a Dom0, yet xen_get_max_pages()'s
> use of XENMEM_maximum_reservation wouldn't impose any limit there
> at all (without use of the hypervisor option "dom0_mem=max:..."),
> would it?

Oh yes, you are right, of course!

I need to check the current reservation and adjust the allowed limit
in case of ballooning and/or memory hotplug.

Thanks for noticing that!


Juergen

[PATCH 0/2] x86: respect memory size limits

[Juergen Gross]

On a customer system running Xen a boot problem was observed due to
the kernel not respecting the memory size limit imposed by the Xen
hypervisor.

During analysis I found the same problem should be able to occur on
bare metal in case the memory would be limited via the "mem=" boot
parameter.

The system this problem has been observed on has tons of memory
added via PCI. So while in the E820 map the not to be used memory has
been wiped out the additional PCI memory is detected during ACPI scan
and it is added via __add_memory().

This small series tries to repair the issue by testing the imposed
memory limit during the memory hotplug process and refusing to add it
in case the limit is being violated.

I've chosen to refuse adding the complete memory chunk in case the
limit is reached instead of adding only some of the memory, as I
thought this would result in less problems (e.g. avoiding to add
only parts of a 128MB memory bar which might be difficult to remove
later).


Juergen Gross (2):
  x86: respect memory size limiting via mem= parameter
  x86/xen: dont add memory above max allowed allocation

 arch/x86/kernel/e820.c         | 5 +++++
 arch/x86/xen/setup.c           | 5 +++++
 include/linux/memory_hotplug.h | 2 ++
 mm/memory_hotplug.c            | 6 ++++++
 4 files changed, 18 insertions(+)

-- 
2.16.4

[PATCH 2/2] x86/xen: dont add memory above max allowed allocation

[Juergen Gross]

Don't allow memory to be added above the allowed maximum allocation
limit set by Xen.

Trying to do so would result in cases like the following:

[  584.559652] ------------[ cut here ]------------
[  584.564897] WARNING: CPU: 2 PID: 1 at ../arch/x86/xen/multicalls.c:129 xen_alloc_pte+0x1c7/0x390()
[  584.575151] Modules linked in:
[  584.578643] Supported: Yes
[  584.581750] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 4.4.120-92.70-default #1
[  584.590000] Hardware name: Cisco Systems Inc UCSC-C460-M4/UCSC-C460-M4, BIOS C460M4.4.0.1b.0.0629181419 06/29/2018
[  584.601862]  0000000000000000 ffffffff813175a0 0000000000000000 ffffffff8184777c
[  584.610200]  ffffffff8107f4e1 ffff880487eb7000 ffff8801862b79c0 ffff88048608d290
[  584.618537]  0000000000487eb7 ffffea0000000201 ffffffff81009de7 ffffffff81068561
[  584.626876] Call Trace:
[  584.629699]  [<ffffffff81019ad9>] dump_trace+0x59/0x340
[  584.635645]  [<ffffffff81019eaa>] show_stack_log_lvl+0xea/0x170
[  584.642391]  [<ffffffff8101ac51>] show_stack+0x21/0x40
[  584.648238]  [<ffffffff813175a0>] dump_stack+0x5c/0x7c
[  584.654085]  [<ffffffff8107f4e1>] warn_slowpath_common+0x81/0xb0
[  584.660932]  [<ffffffff81009de7>] xen_alloc_pte+0x1c7/0x390
[  584.667289]  [<ffffffff810647f0>] pmd_populate_kernel.constprop.6+0x40/0x80
[  584.675241]  [<ffffffff815ecfe8>] phys_pmd_init+0x210/0x255
[  584.681587]  [<ffffffff815ed207>] phys_pud_init+0x1da/0x247
[  584.687931]  [<ffffffff815edb3b>] kernel_physical_mapping_init+0xf5/0x1d4
[  584.695682]  [<ffffffff815e9bdd>] init_memory_mapping+0x18d/0x380
[  584.702631]  [<ffffffff81064699>] arch_add_memory+0x59/0xf0

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 arch/x86/xen/setup.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c
index d5f303c0e656..5929a6ba5c25 100644
--- a/arch/x86/xen/setup.c
+++ b/arch/x86/xen/setup.c
@@ -12,6 +12,7 @@
 #include <linux/memblock.h>
 #include <linux/cpuidle.h>
 #include <linux/cpufreq.h>
+#include <linux/memory_hotplug.h>
 
 #include <asm/elf.h>
 #include <asm/vdso.h>
@@ -785,6 +786,10 @@ char * __init xen_memory_setup(void)
 	/* How many extra pages do we need due to remapping? */
 	max_pages += xen_foreach_remap_area(max_pfn, xen_count_remap_pages);
 
+#ifdef CONFIG_MEMORY_HOTPLUG
+	max_mem_size = PFN_PHYS(max_pages);
+#endif
+
 	if (max_pages > max_pfn)
 		extra_pages += max_pages - max_pfn;
 
-- 
2.16.4

Re: [Xen-devel] [PATCH 2/2] x86/xen: dont add memory above max allowed allocation

[Jan Beulich]

>>> On 22.01.19 at 09:06, <jgross@suse.com> wrote:
> Don't allow memory to be added above the allowed maximum allocation
> limit set by Xen.

This reads as if the hypervisor was imposing a limit here, but looking at
xen_get_max_pages(), xen_foreach_remap_area(), and
xen_count_remap_pages() I take it that it's a restriction enforced by
the Xen subsystem in Linux. Furthermore from the cover letter I imply
that the observed issue was on a Dom0, yet xen_get_max_pages()'s
use of XENMEM_maximum_reservation wouldn't impose any limit there
at all (without use of the hypervisor option "dom0_mem=max:..."),
would it?

Jan

Re: [Xen-devel] [PATCH 2/2] x86/xen: dont add memory above max allowed allocation

[Jan Beulich]

>>> On 22.01.19 at 09:06, <jgross@suse.com> wrote:
> Don't allow memory to be added above the allowed maximum allocation
> limit set by Xen.

This reads as if the hypervisor was imposing a limit here, but looking at
xen_get_max_pages(), xen_foreach_remap_area(), and
xen_count_remap_pages() I take it that it's a restriction enforced by
the Xen subsystem in Linux. Furthermore from the cover letter I imply
that the observed issue was on a Dom0, yet xen_get_max_pages()'s
use of XENMEM_maximum_reservation wouldn't impose any limit there
at all (without use of the hypervisor option "dom0_mem=max:..."),
would it?

Jan