From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by kanga.kvack.org (Postfix) with ESMTP id 927178E0002 for ; Thu, 20 Dec 2018 08:17:15 -0500 (EST) Received: by mail-qk1-f197.google.com with SMTP id r145so1715890qke.20 for ; Thu, 20 Dec 2018 05:17:15 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id i66si2486272qkc.207.2018.12.20.05.17.13 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Dec 2018 05:17:14 -0800 (PST) Subject: Re: [PATCH RFCv2 0/4] mm/memory_hotplug: Introduce memory block types References: <20181130175922.10425-1-david@redhat.com> <1b4afb6a-5f91-407d-6e6e-6a89b8cf5d56@redhat.com> <20181220130832.GH9104@dhcp22.suse.cz> From: David Hildenbrand Message-ID: <872b5496-7227-9171-fb3c-ec03cf190302@redhat.com> Date: Thu, 20 Dec 2018 14:16:58 +0100 MIME-Version: 1.0 In-Reply-To: <20181220130832.GH9104@dhcp22.suse.cz> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Michal Hocko Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-ia64@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, linux-acpi@vger.kernel.org, devel@linuxdriverproject.org, xen-devel@lists.xenproject.org, x86@kernel.org, Andrew Banman , Andrew Morton , Andy Lutomirski , Arun KS , Balbir Singh , Benjamin Herrenschmidt , Borislav Petkov , Boris Ostrovsky , Christophe Leroy , Dan Williams , Dave Hansen , Dave Jiang , Fenghua Yu , Greg Kroah-Hartman , Haiyang Zhang , Heiko Carstens , "H. Peter Anvin" , Ingo Molnar , Ingo Molnar , =?UTF-8?Q?Jan_H=2e_Sch=c3=b6nherr?= , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , =?UTF-8?Q?Jonathan_Neusch=c3=a4fer?= , Joonsoo Kim , Juergen Gross , "Kirill A. Shutemov" , "K. Y. Srinivasan" , Len Brown , Logan Gunthorpe , Martin Schwidefsky , Mathieu Malaterre , Matthew Wilcox , Mauricio Faria de Oliveira , Michael Ellerman , Michael Neuling , =?UTF-8?Q?Michal_Such=c3=a1nek?= , Mike Rapoport , "mike.travis@hpe.com" , Nathan Fontenot , Nicholas Piggin , Oscar Salvador , Oscar Salvador , Paul Mackerras , Pavel Tatashin , Pavel Tatashin , Pavel Tatashin , Peter Zijlstra , "Rafael J. Wysocki" , "Rafael J. Wysocki" , Rashmica Gupta , Rich Felker , Rob Herring , Stefano Stabellini , Stephen Hemminger , Stephen Rothwell , Thomas Gleixner , Tony Luck , Vasily Gorbik , Vitaly Kuznetsov , Wei Yang , Yoshinori Sato , YueHaibing On 20.12.18 14:08, Michal Hocko wrote: > On Thu 20-12-18 13:58:16, David Hildenbrand wrote: >> On 30.11.18 18:59, David Hildenbrand wrote: >>> This is the second approach, introducing more meaningful memory block >>> types and not changing online behavior in the kernel. It is based on >>> latest linux-next. >>> >>> As we found out during dicussion, user space should always handle onlining >>> of memory, in any case. However in order to make smart decisions in user >>> space about if and how to online memory, we have to export more information >>> about memory blocks. This way, we can formulate rules in user space. >>> >>> One such information is the type of memory block we are talking about. >>> This helps to answer some questions like: >>> - Does this memory block belong to a DIMM? >>> - Can this DIMM theoretically ever be unplugged again? >>> - Was this memory added by a balloon driver that will rely on balloon >>> inflation to remove chunks of that memory again? Which zone is advised? >>> - Is this special standby memory on s390x that is usually not automatically >>> onlined? >>> >>> And in short it helps to answer to some extend (excluding zone imbalances) >>> - Should I online this memory block? >>> - To which zone should I online this memory block? >>> ... of course special use cases will result in different anwers. But that's >>> why user space has control of onlining memory. >>> >>> More details can be found in Patch 1 and Patch 3. >>> Tested on x86 with hotplugged DIMMs. Cross-compiled for PPC and s390x. >>> >>> >>> Example: >>> $ udevadm info -q all -a /sys/devices/system/memory/memory0 >>> KERNEL=="memory0" >>> SUBSYSTEM=="memory" >>> DRIVER=="" >>> ATTR{online}=="1" >>> ATTR{phys_device}=="0" >>> ATTR{phys_index}=="00000000" >>> ATTR{removable}=="0" >>> ATTR{state}=="online" >>> ATTR{type}=="boot" >>> ATTR{valid_zones}=="none" >>> $ udevadm info -q all -a /sys/devices/system/memory/memory90 >>> KERNEL=="memory90" >>> SUBSYSTEM=="memory" >>> DRIVER=="" >>> ATTR{online}=="1" >>> ATTR{phys_device}=="0" >>> ATTR{phys_index}=="0000005a" >>> ATTR{removable}=="1" >>> ATTR{state}=="online" >>> ATTR{type}=="dimm" >>> ATTR{valid_zones}=="Normal" >>> >>> >>> RFC -> RFCv2: >>> - Now also taking care of PPC (somehow missed it :/ ) >>> - Split the series up to some degree (some ideas on how to split up patch 3 >>> would be very welcome) >>> - Introduce more memory block types. Turns out abstracting too much was >>> rather confusing and not helpful. Properly document them. >>> >>> Notes: >>> - I wanted to convert the enum of types into a named enum but this >>> provoked all kinds of different errors. For now, I am doing it just like >>> the other types (e.g. online_type) we are using in that context. >>> - The "removable" property should never have been named like that. It >>> should have been "offlinable". Can we still rename that? E.g. boot memory >>> is sometimes marked as removable ... >>> >> >> >> Any feedback regarding the suggested block types would be very much >> appreciated! > > I still do not like this much to be honest. I just didn't get to think > through this properly. My fear is that this is conflating an actual API > with the current implementation and as such will cause problems in > future. But I haven't really looked into your patches closely so I might > be wrong. Anyway I won't be able to look into it by the end of year. > I guess as long as we have memory block devices and we expect user space to make a decision we will have this API and the involved problems. I am open for alternatives, and as I said, any feedback on how to sort this out will be highly appreciated. I'll be on vacation for the next two weeks, so this can wait. Just wanted to note that I am still interested in feedback :) -- Thanks, David / dhildenb