From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47ECAC433DB for ; Mon, 25 Jan 2021 12:01:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9B59F22DD6 for ; Mon, 25 Jan 2021 12:01:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9B59F22DD6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B9E788D0005; Mon, 25 Jan 2021 07:01:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B27FD8D0001; Mon, 25 Jan 2021 07:01:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9EE808D0005; Mon, 25 Jan 2021 07:01:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0125.hostedemail.com [216.40.44.125]) by kanga.kvack.org (Postfix) with ESMTP id 843578D0001 for ; Mon, 25 Jan 2021 07:01:27 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 4CAA0180ACC5A for ; Mon, 25 Jan 2021 12:01:27 +0000 (UTC) X-FDA: 77744157414.11.alley18_3203eb927585 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin11.hostedemail.com (Postfix) with ESMTP id 2B78D180F8B81 for ; Mon, 25 Jan 2021 12:01:27 +0000 (UTC) X-HE-Tag: alley18_3203eb927585 X-Filterd-Recvd-Size: 8379 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Mon, 25 Jan 2021 12:01:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1611576086; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xtg7PDUV7D5/4c+APnQgvAcp6L1BanPgaK8SorUjGaE=; b=VH8ashk+ZN13c4b017gQGgIE6N7WJHZ35pw6wIiL75JS6jbMQMZevgIbm88iJlKNZGJGxX bgs3YS0+DarBBNNTv1ZnNJVmnObhRcNY7BOJb9vX/9+FwGCNrP3wbhgkqVT4LcKuWmA/gi rH9+AniwXTWRkI56QQNzTqQKSNbe7/c= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-593-J0GIFzHXM_Cx0I_wguN0Sg-1; Mon, 25 Jan 2021 07:01:24 -0500 X-MC-Unique: J0GIFzHXM_Cx0I_wguN0Sg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DFC4B192D786; Mon, 25 Jan 2021 12:01:19 +0000 (UTC) Received: from [10.36.115.13] (ovpn-115-13.ams2.redhat.com [10.36.115.13]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2773910013BD; Mon, 25 Jan 2021 12:01:14 +0000 (UTC) Subject: Re: [PATCH V4 4/4] virtio-mem: check against mhp_get_pluggable_range() which memory we can hotplug To: Anshuman Khandual , linux-mm@kvack.org, akpm@linux-foundation.org, hca@linux.ibm.com, catalin.marinas@arm.com Cc: Oscar Salvador , Vasily Gorbik , Will Deacon , Ard Biesheuvel , Mark Rutland , linux-arm-kernel@lists.infradead.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, "Michael S. Tsirkin" , Jason Wang , Pankaj Gupta , Wei Yang , teawater , Pankaj Gupta , Jonathan Cameron , Michal Hocko References: <1611543532-18698-1-git-send-email-anshuman.khandual@arm.com> <1611543532-18698-5-git-send-email-anshuman.khandual@arm.com> From: David Hildenbrand Organization: Red Hat GmbH Message-ID: Date: Mon, 25 Jan 2021 13:01:14 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.5.0 MIME-Version: 1.0 In-Reply-To: <1611543532-18698-5-git-send-email-anshuman.khandual@arm.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 25.01.21 03:58, Anshuman Khandual wrote: > From: David Hildenbrand > > Right now, we only check against MAX_PHYSMEM_BITS - but turns out there > are more restrictions of which memory we can actually hotplug, especially > om arm64 or s390x once we support them: we might receive something like > -E2BIG or -ERANGE from add_memory_driver_managed(), stopping device > operation. > > So, check right when initializing the device which memory we can add, > warning the user. Try only adding actually pluggable ranges: in the worst > case, no memory provided by our device is pluggable. > > In the usual case, we expect all device memory to be pluggable, and in > corner cases only some memory at the end of the device-managed memory > region to not be pluggable. > > Cc: "Michael S. Tsirkin" > Cc: Jason Wang > Cc: Pankaj Gupta > Cc: Oscar Salvador > Cc: Wei Yang > Cc: Andrew Morton > Cc: catalin.marinas@arm.com > Cc: teawater > Cc: Anshuman Khandual > Cc: Pankaj Gupta > Cc: Jonathan Cameron > Cc: hca@linux.ibm.com > Cc: Vasily Gorbik > Cc: Will Deacon > Cc: Ard Biesheuvel > Cc: Mark Rutland > Cc: Heiko Carstens > Cc: Michal Hocko > Signed-off-by: David Hildenbrand > Signed-off-by: Anshuman Khandual > --- > drivers/virtio/virtio_mem.c | 40 +++++++++++++++++++++++++------------ > 1 file changed, 27 insertions(+), 13 deletions(-) > > diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c > index 9fc9ec4a25f5..14c17c5c1695 100644 > --- a/drivers/virtio/virtio_mem.c > +++ b/drivers/virtio/virtio_mem.c > @@ -2222,7 +2222,7 @@ static int virtio_mem_unplug_pending_mb(struct virtio_mem *vm) > */ > static void virtio_mem_refresh_config(struct virtio_mem *vm) > { > - const uint64_t phys_limit = 1UL << MAX_PHYSMEM_BITS; > + const struct range pluggable_range = mhp_get_pluggable_range(true); > uint64_t new_plugged_size, usable_region_size, end_addr; > > /* the plugged_size is just a reflection of what _we_ did previously */ > @@ -2234,15 +2234,25 @@ static void virtio_mem_refresh_config(struct virtio_mem *vm) > /* calculate the last usable memory block id */ > virtio_cread_le(vm->vdev, struct virtio_mem_config, > usable_region_size, &usable_region_size); > - end_addr = vm->addr + usable_region_size; > - end_addr = min(end_addr, phys_limit); > + end_addr = min(vm->addr + usable_region_size - 1, > + pluggable_range.end); > > - if (vm->in_sbm) > - vm->sbm.last_usable_mb_id = > - virtio_mem_phys_to_mb_id(end_addr) - 1; > - else > - vm->bbm.last_usable_bb_id = > - virtio_mem_phys_to_bb_id(vm, end_addr) - 1; > + if (vm->in_sbm) { > + vm->sbm.last_usable_mb_id = virtio_mem_phys_to_mb_id(end_addr); > + if (!IS_ALIGNED(end_addr + 1, memory_block_size_bytes())) > + vm->sbm.last_usable_mb_id--; > + } else { > + vm->bbm.last_usable_bb_id = virtio_mem_phys_to_bb_id(vm, > + end_addr); > + if (!IS_ALIGNED(end_addr + 1, vm->bbm.bb_size)) > + vm->bbm.last_usable_bb_id--; > + } > + /* > + * If we cannot plug any of our device memory (e.g., nothing in the > + * usable region is addressable), the last usable memory block id will > + * be smaller than the first usable memory block id. We'll stop > + * attempting to add memory with -ENOSPC from our main loop. > + */ > > /* see if there is a request to change the size */ > virtio_cread_le(vm->vdev, struct virtio_mem_config, requested_size, > @@ -2364,6 +2374,7 @@ static int virtio_mem_init_vq(struct virtio_mem *vm) > > static int virtio_mem_init(struct virtio_mem *vm) > { > + const struct range pluggable_range = mhp_get_pluggable_range(true); > const uint64_t phys_limit = 1UL << MAX_PHYSMEM_BITS; Sorry, forgot to drop ^ (phys_limit), otherwise ther is a friendly warning from the compiler. We have to drop that line. Apart from that, at least on x86-64 it does what it's supposed to do. I temporarily changed MAX_PHYSMEM_BITS to 35 bits and added a virtio-mem device that crosses the 32 GiB address limit. [ 0.572084] virtio_mem virtio1: Some device memory is not addressable/pluggable. This can make some memory unusable. [ 0.573013] virtio_mem virtio1: start address: 0x740000000 [ 0.573497] virtio_mem virtio1: region size: 0x500000000 And virtio-mem won't add any memory exceeding that: (qemu) qom-set vmem0 requested-size 20G (qemu) info memory-devices Memory device [virtio-mem]: "vmem0" [...] Memory device [virtio-mem]: "vmem1" memaddr: 0x740000000 node: 1 requested-size: 21474836480 size: 3221225472 max-size: 21474836480 block-size: 2097152 memdev: /objects/mem1 I adds all memory up to the 32GiB address limit (35 bits) and stops. LGTM (arm64 to be tested in the future once supported). -- Thanks, David / dhildenb