From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-vc0-f180.google.com (mail-vc0-f180.google.com [209.85.220.180]) by kanga.kvack.org (Postfix) with ESMTP id E4A346B0039 for ; Fri, 18 Jul 2014 13:48:05 -0400 (EDT) Received: by mail-vc0-f180.google.com with SMTP id ij19so8055763vcb.39 for ; Fri, 18 Jul 2014 10:48:05 -0700 (PDT) Received: from mail-vc0-x230.google.com (mail-vc0-x230.google.com [2607:f8b0:400c:c03::230]) by mx.google.com with ESMTPS id ak16si6549950vdc.93.2014.07.18.10.48.04 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 18 Jul 2014 10:48:05 -0700 (PDT) Received: by mail-vc0-f176.google.com with SMTP id id10so3336873vcb.7 for ; Fri, 18 Jul 2014 10:48:04 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: References: <1405064267-11678-1-git-send-email-jiang.liu@linux.intel.com> <20140711082956.GC20603@laptop.programming.kicks-ass.net> <20140711153314.GA6155@kroah.com> Date: Fri, 18 Jul 2014 10:48:04 -0700 Message-ID: Subject: Re: [RFC Patch V1 00/30] Enable memoryless node on x86 platforms From: Nish Aravamudan Content-Type: multipart/alternative; boundary=001a11c22fb401c56804fe7b5c75 Sender: owner-linux-mm@kvack.org List-ID: To: David Rientjes Cc: Jiri Kosina , Greg KH , Jiang Liu , Peter Zijlstra , Andrew Morton , Mel Gorman , Mike Galbraith , "Rafael J . Wysocki" , Tony Luck , Nishanth Aravamudan , Linux Memory Management List , linux-hotplug@vger.kernel.org, "linux-kernel@vger.kernel.org" --001a11c22fb401c56804fe7b5c75 Content-Type: text/plain; charset=UTF-8 Hi David, On Mon, Jul 14, 2014 at 6:19 PM, David Rientjes wrote: > > On Sat, 12 Jul 2014, Jiri Kosina wrote: > > > I am pretty sure I've seen ppc64 machine with memoryless NUMA node. > > > > Yes, Nishanth Aravamudan (now cc'd) has been working diligently on the > problems that have been encountered, including problems in generic kernel > code, on powerpc with memoryless nodes. Thanks for Cc'ing me on this discussion. I'm going to review Jiang's patchset now, as best I can, but yes I can confirm we see memoryless nodes somewhat frequently on powerpc under PowerVM, due to presumably hypervisor fragmentation (the reason isn't clear to an LPAR, as it's just given a topology). I agree with Dave Hansen that this seems like a "good thing" to try and figure out, unless KVM decides it's going to hide the underlying topology of a guest's memory from the guest -- which I think could lead (eventually) to confusing performance results. I believe I have also seen them in hardware on ia64 (cpu-only and memory-only drawers), but not sure if those specific models are in production still. Finally, I will say that in working on supporting memoryless nodes, I've come across what look like bugs in the NUMA code. Or more accurately, assumptions which aren't always true. So it's a useful exercise for that reason to. Thanks, Nish --001a11c22fb401c56804fe7b5c75 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi David,

On Mon, Jul 14, 2014 at 6:19 PM, David Ri= entjes <rientjes@google.com&g= t; wrote:
>
> On Sat, 12 Jul 2014, Jiri Kosina wrote:
> > > I am pretty sure I've seen ppc64 machine with memoryless NUMA= node.
> >
>
> Yes, Nishanth Aravamudan (now cc'd)= has been working diligently on the
> problems that have been encount= ered, including problems in generic kernel
> code, on powerpc with memoryless nodes.

Thanks for Cc'ing m= e on this discussion. I'm going to review Jiang's patchset now, as = best I can, but yes I can confirm we see memoryless nodes somewhat frequent= ly on powerpc under PowerVM, due to presumably hypervisor fragmentation (th= e reason isn't clear to an LPAR, as it's just given a topology).
I agree with Dave Hansen that this seems like a "good thing" = to try and figure out, unless KVM decides it's going to hide the underl= ying topology of a guest's memory from the guest -- which I think could= lead (eventually) to confusing performance results.

I believe I have also seen them in hardware on ia64 (cpu-only and memor= y-only drawers), but not sure if those specific models are in production st= ill.

Finally, I will say that in working on supporting memoryless no= des, I've come across what look like bugs in the NUMA code. Or more acc= urately, assumptions which aren't always true. So it's a useful exe= rcise for that reason to.

Thanks,
Nish
--001a11c22fb401c56804fe7b5c75-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org