From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F84CC433E0 for ; Wed, 27 Jan 2021 03:41:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AE39F205CA for ; Wed, 27 Jan 2021 03:41:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AE39F205CA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E72466B0005; Tue, 26 Jan 2021 22:41:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E24666B0006; Tue, 26 Jan 2021 22:41:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3AC56B0007; Tue, 26 Jan 2021 22:41:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id BE3E26B0005 for ; Tue, 26 Jan 2021 22:41:45 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 77A4D1EE6 for ; Wed, 27 Jan 2021 03:41:45 +0000 (UTC) X-FDA: 77750155770.15.table14_180997e27593 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin15.hostedemail.com (Postfix) with ESMTP id 3FDAE1814B0C7 for ; Wed, 27 Jan 2021 03:41:45 +0000 (UTC) X-HE-Tag: table14_180997e27593 X-Filterd-Recvd-Size: 7763 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Wed, 27 Jan 2021 03:41:44 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7FFCE1042; Tue, 26 Jan 2021 19:41:43 -0800 (PST) Received: from [192.168.0.130] (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E46273F68F; Tue, 26 Jan 2021 19:41:36 -0800 (PST) From: Anshuman Khandual Subject: Re: [PATCH V4 4/4] virtio-mem: check against mhp_get_pluggable_range() which memory we can hotplug To: David Hildenbrand , linux-mm@kvack.org, akpm@linux-foundation.org, hca@linux.ibm.com, catalin.marinas@arm.com Cc: Oscar Salvador , Vasily Gorbik , Will Deacon , Ard Biesheuvel , Mark Rutland , linux-arm-kernel@lists.infradead.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, "Michael S. Tsirkin" , Jason Wang , Pankaj Gupta , Wei Yang , teawater , Pankaj Gupta , Jonathan Cameron , Michal Hocko References: <1611543532-18698-1-git-send-email-anshuman.khandual@arm.com> <1611543532-18698-5-git-send-email-anshuman.khandual@arm.com> Message-ID: <95e86a10-16db-f5ca-6aba-282ea4ead717@arm.com> Date: Wed, 27 Jan 2021 09:12:01 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 1/25/21 5:31 PM, David Hildenbrand wrote: > On 25.01.21 03:58, Anshuman Khandual wrote: >> From: David Hildenbrand >> >> Right now, we only check against MAX_PHYSMEM_BITS - but turns out there >> are more restrictions of which memory we can actually hotplug, especially >> om arm64 or s390x once we support them: we might receive something like >> -E2BIG or -ERANGE from add_memory_driver_managed(), stopping device >> operation. >> >> So, check right when initializing the device which memory we can add, >> warning the user. Try only adding actually pluggable ranges: in the worst >> case, no memory provided by our device is pluggable. >> >> In the usual case, we expect all device memory to be pluggable, and in >> corner cases only some memory at the end of the device-managed memory >> region to not be pluggable. >> >> Cc: "Michael S. Tsirkin" >> Cc: Jason Wang >> Cc: Pankaj Gupta >> Cc: Oscar Salvador >> Cc: Wei Yang >> Cc: Andrew Morton >> Cc: catalin.marinas@arm.com >> Cc: teawater >> Cc: Anshuman Khandual >> Cc: Pankaj Gupta >> Cc: Jonathan Cameron >> Cc: hca@linux.ibm.com >> Cc: Vasily Gorbik >> Cc: Will Deacon >> Cc: Ard Biesheuvel >> Cc: Mark Rutland >> Cc: Heiko Carstens >> Cc: Michal Hocko >> Signed-off-by: David Hildenbrand >> Signed-off-by: Anshuman Khandual >> --- >> drivers/virtio/virtio_mem.c | 40 +++++++++++++++++++++++++------------ >> 1 file changed, 27 insertions(+), 13 deletions(-) >> >> diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c >> index 9fc9ec4a25f5..14c17c5c1695 100644 >> --- a/drivers/virtio/virtio_mem.c >> +++ b/drivers/virtio/virtio_mem.c >> @@ -2222,7 +2222,7 @@ static int virtio_mem_unplug_pending_mb(struct virtio_mem *vm) >> */ >> static void virtio_mem_refresh_config(struct virtio_mem *vm) >> { >> - const uint64_t phys_limit = 1UL << MAX_PHYSMEM_BITS; >> + const struct range pluggable_range = mhp_get_pluggable_range(true); >> uint64_t new_plugged_size, usable_region_size, end_addr; >> >> /* the plugged_size is just a reflection of what _we_ did previously */ >> @@ -2234,15 +2234,25 @@ static void virtio_mem_refresh_config(struct virtio_mem *vm) >> /* calculate the last usable memory block id */ >> virtio_cread_le(vm->vdev, struct virtio_mem_config, >> usable_region_size, &usable_region_size); >> - end_addr = vm->addr + usable_region_size; >> - end_addr = min(end_addr, phys_limit); >> + end_addr = min(vm->addr + usable_region_size - 1, >> + pluggable_range.end); >> >> - if (vm->in_sbm) >> - vm->sbm.last_usable_mb_id = >> - virtio_mem_phys_to_mb_id(end_addr) - 1; >> - else >> - vm->bbm.last_usable_bb_id = >> - virtio_mem_phys_to_bb_id(vm, end_addr) - 1; >> + if (vm->in_sbm) { >> + vm->sbm.last_usable_mb_id = virtio_mem_phys_to_mb_id(end_addr); >> + if (!IS_ALIGNED(end_addr + 1, memory_block_size_bytes())) >> + vm->sbm.last_usable_mb_id--; >> + } else { >> + vm->bbm.last_usable_bb_id = virtio_mem_phys_to_bb_id(vm, >> + end_addr); >> + if (!IS_ALIGNED(end_addr + 1, vm->bbm.bb_size)) >> + vm->bbm.last_usable_bb_id--; >> + } >> + /* >> + * If we cannot plug any of our device memory (e.g., nothing in the >> + * usable region is addressable), the last usable memory block id will >> + * be smaller than the first usable memory block id. We'll stop >> + * attempting to add memory with -ENOSPC from our main loop. >> + */ >> >> /* see if there is a request to change the size */ >> virtio_cread_le(vm->vdev, struct virtio_mem_config, requested_size, >> @@ -2364,6 +2374,7 @@ static int virtio_mem_init_vq(struct virtio_mem *vm) >> >> static int virtio_mem_init(struct virtio_mem *vm) >> { >> + const struct range pluggable_range = mhp_get_pluggable_range(true); >> const uint64_t phys_limit = 1UL << MAX_PHYSMEM_BITS; > > Sorry, forgot to drop ^ (phys_limit), otherwise ther is a friendly > warning from the compiler. We have to drop that line. Okay sure, will drop. > > > > Apart from that, at least on x86-64 it does what it's supposed to do. I > temporarily changed MAX_PHYSMEM_BITS to 35 bits and added a virtio-mem > device that crosses the 32 GiB address limit. > > > [ 0.572084] virtio_mem virtio1: Some device memory is not > addressable/pluggable. This can make some memory unusable. > [ 0.573013] virtio_mem virtio1: start address: 0x740000000 > [ 0.573497] virtio_mem virtio1: region size: 0x500000000 > > > And virtio-mem won't add any memory exceeding that: > > (qemu) qom-set vmem0 requested-size 20G > (qemu) info memory-devices > Memory device [virtio-mem]: "vmem0" > [...] > Memory device [virtio-mem]: "vmem1" > memaddr: 0x740000000 > node: 1 > requested-size: 21474836480 > size: 3221225472 > max-size: 21474836480 > block-size: 2097152 > memdev: /objects/mem1 > > I adds all memory up to the 32GiB address limit (35 bits) and stops. > > LGTM (arm64 to be tested in the future once supported). > I will respin the series with the above minor change unless something else comes up in the meantime. But once this virtio-mem change gets some more reviews, I guess the series should be complete in itself.