From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53A39C433E8 for ; Thu, 23 Jul 2020 16:04:12 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 16584207BB for ; Thu, 23 Jul 2020 16:04:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="WpXEjNsM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 16584207BB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8D0F96B0027; Thu, 23 Jul 2020 12:04:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 881846B0029; Thu, 23 Jul 2020 12:04:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 770F08D0001; Thu, 23 Jul 2020 12:04:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0182.hostedemail.com [216.40.44.182]) by kanga.kvack.org (Postfix) with ESMTP id 63CBA6B0027 for ; Thu, 23 Jul 2020 12:04:11 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 230091803107A for ; Thu, 23 Jul 2020 16:04:11 +0000 (UTC) X-FDA: 77069812302.25.brain59_3b158b226f40 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin25.hostedemail.com (Postfix) with ESMTP id D70031804EE3A for ; Thu, 23 Jul 2020 16:04:04 +0000 (UTC) X-HE-Tag: brain59_3b158b226f40 X-Filterd-Recvd-Size: 7912 Received: from esa1.hc3370-68.iphmx.com (esa1.hc3370-68.iphmx.com [216.71.145.142]) by imf11.hostedemail.com (Postfix) with ESMTP for ; Thu, 23 Jul 2020 16:03:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1595520245; h=subject:to:cc:references:from:message-id:date: mime-version:in-reply-to:content-transfer-encoding; bh=EvxeNc47Aj2ifX7I2dOEp/A9QT4H2xQ/u9lwoz5shzI=; b=WpXEjNsMoNtCOeRCdlKGVpuT6RjL4ydtKR+Z8cPLOisVP5w47GTwfE/B LVLyKmLKHFlwTD5xNx4wC7o4XHF7aOEWUJd6IvecaIPvfkJbyMj3Onwvb DbaCgyLnijHocom8YVjrb1hcQVmQPtlBFH7ISSYWlOo8waWV1tuSoydal M=; Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none IronPort-SDR: n9UugflWA/BE40d8h8v7T+t8Hk1PRLHxDefttB491tL3qzQz3blSONoIq24jbkc/6Wnn0r8rBU XFHH8vvUaGjTEdO/ipfRj37LAZhDf9pbv6oUTkGuIyngH+RK9dyfF9tk2uYHnmxSWpTXoa7asc IBXjTlfQNJ/KhIAyoELwzVB14fs1Vz5O378JuSI6x4xoewXdIcNpfbGrFnN8gjSndsCQUQ7ugn YZEXeyeQRDOq82nNDfUvVnaeW3j6P0U07RuAPj70IcRn4ncCgk1QVJ3TTv391AQZEHGbukVhpD oEo= X-SBRS: 2.7 X-MesageID: 23390074 X-Ironport-Server: esa1.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.75,387,1589256000"; d="scan'208";a="23390074" Subject: Re: [PATCH 3/3] memory: introduce an option to force onlining of hotplug memory To: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= , =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= , David Hildenbrand CC: Stefano Stabellini , , , , Boris Ostrovsky , Andrew Morton References: <20200723084523.42109-1-roger.pau@citrix.com> <20200723084523.42109-4-roger.pau@citrix.com> <21490d49-b2cf-a398-0609-8010bdb0b004@redhat.com> <20200723122300.GD7191@Air-de-Roger> <20200723135930.GH7191@Air-de-Roger> <82b131f4-8f50-cd49-65cf-9a87d51b5555@suse.com> From: Andrew Cooper Message-ID: Date: Thu, 23 Jul 2020 17:03:47 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <82b131f4-8f50-cd49-65cf-9a87d51b5555@suse.com> Content-Type: text/plain; charset="utf-8" Content-Language: en-GB X-ClientProxiedBy: AMSPEX02CAS02.citrite.net (10.69.22.113) To AMSPEX02CL02.citrite.net (10.69.22.126) X-Rspamd-Queue-Id: D70031804EE3A X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 23/07/2020 16:10, J=C3=BCrgen Gro=C3=9F wrote: > On 23.07.20 15:59, Roger Pau Monn=C3=A9 wrote: >> On Thu, Jul 23, 2020 at 03:22:49PM +0200, David Hildenbrand wrote: >>> On 23.07.20 14:23, Roger Pau Monn=C3=A9 wrote: >>>> On Thu, Jul 23, 2020 at 01:37:03PM +0200, David Hildenbrand wrote: >>>>> On 23.07.20 10:45, Roger Pau Monne wrote: >>>>>> Add an extra option to add_memory_resource that overrides the memo= ry >>>>>> hotplug online behavior in order to force onlining of memory from >>>>>> add_memory_resource unconditionally. >>>>>> >>>>>> This is required for the Xen balloon driver, that must run the >>>>>> online page callback in order to correctly process the newly added >>>>>> memory region, note this is an unpopulated region that is used by >>>>>> Linux >>>>>> to either hotplug RAM or to map foreign pages from other domains, >>>>>> and >>>>>> hence memory hotplug when running on Xen can be used even without >>>>>> the >>>>>> user explicitly requesting it, as part of the normal operations >>>>>> of the >>>>>> OS when attempting to map memory from a different domain. >>>>>> >>>>>> Setting a different default value of memhp_default_online_type whe= n >>>>>> attaching the balloon driver is not a robust solution, as the >>>>>> user (or >>>>>> distro init scripts) could still change it and thus break the Xen >>>>>> balloon driver. >>>>> >>>>> I think we discussed this a couple of times before (even triggered >>>>> by my >>>>> request), and this is responsibility of user space to configure. >>>>> Usually >>>>> distros have udev rules to online memory automatically. >>>>> Especially, user >>>>> space should eb able to configure *how* to online memory. >>>> >>>> Note (as per the commit message) that in the specific case I'm >>>> referring to the memory hotplugged by the Xen balloon driver will be >>>> an unpopulated range to be used internally by certain Xen subsystems= , >>>> like the xen-blkback or the privcmd drivers. The addition of such >>>> blocks of (unpopulated) memory can happen without the user explicitl= y >>>> requesting it, and hence not even aware such hotplug process is taki= ng >>>> place. To be clear: no actual RAM will be added to the system. >>> >>> Okay, but there is also the case where XEN will actually hotplug memo= ry >>> using this same handler IIRC (at least I've read papers about it). Bo= th >>> are using the same handler, correct? >> >> Yes, it's used by this dual purpose, which I have to admit I don't >> like that much either. >> >> One set of pages should be clearly used for RAM memory hotplug, and >> the other to map foreign pages that are not related to memory hotplug, >> it's just that we happen to need a physical region with backing struct >> pages. >> >>>> >>>>> It's the admin/distro responsibility to configure this properly. >>>>> In case >>>>> this doesn't happen (or as you say, users change it), bad luck. >>>>> >>>>> E.g., virtio-mem takes care to not add more memory in case it is no= t >>>>> getting onlined. I remember hyper-v has similar code to at least >>>>> wait a >>>>> bit for memory to get onlined. >>>> >>>> I don't think VirtIO or Hyper-V use the hotplug system in the same w= ay >>>> as Xen, as said this is done to add unpopulated memory regions that >>>> will be used to map foreign memory (from other domains) by Xen drive= rs >>>> on the system. >>> >>> Indeed, if the memory is never exposed to the buddy (and all you >>> need is >>> struct pages +=C2=A0 a kernel virtual mapping), I wonder if >>> memremap/ZONE_DEVICE is what you want? >> >> I'm certainly not familiar with the Linux memory subsystem, but if >> that gets us a backing struct page and a kernel mapping then I would >> say yes. >> >>> Then you won't have user-visible >>> memory blocks created with unclear online semantics, partially >>> involving >>> the buddy. >> >> Seems like a fine solution. >> >> Juergen: would you be OK to use a separate page-list for >> alloc_xenballooned_pages on HVM/PVH using the logic described by >> David? >> >> I guess I would leave PV as-is, since it already has this reserved >> region to map foreign pages. > > I would really like a common solution, especially as it would enable > pv driver domains to use that feature, too. > > And finding a region for this memory zone in PVH dom0 should be common > with PV dom0 after all. We don't want to collide with either PCI space > or hotplug memory. While I agree with goal here, these are two very different things, due to the completely different nature of PV and HVM/PVH guests. HVM/PVH guests have a concrete guest physical address space.=C2=A0 Linux needs to pick some gfn's to use which aren't used by anything else (and Xen's behaviour of not providing any help here is deeply unhelpful, and needs fixing), and get struct page_info's for them. PV is totally different.=C2=A0 Linux still needs page_info's for them, bu= t there is no concept of a guest physical address space.=C2=A0 You can literally gain access to foreign mappings or grant maps by asking Xen to modify a PTE.=C2=A0 For convenience with the core code, Linux tries to ma= p this concept back into a 1:1 pfn space, but it is quite fictitious. ~Andrew