From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46DC1C001DB for ; Mon, 7 Aug 2023 18:36:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C55886B0072; Mon, 7 Aug 2023 14:36:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C06306B0074; Mon, 7 Aug 2023 14:36:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA74F6B0075; Mon, 7 Aug 2023 14:36:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 9CF296B0072 for ; Mon, 7 Aug 2023 14:36:00 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 1F59A40530 for ; Mon, 7 Aug 2023 18:36:00 +0000 (UTC) X-FDA: 81098162880.28.461894F Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf02.hostedemail.com (Postfix) with ESMTP id D06548002A for ; Mon, 7 Aug 2023 18:35:56 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Ic6rEStz; spf=pass (imf02.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1691433356; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nSzDv346kgN2CfjlYQS29gdaOShLy0rGapky8ZA9EFo=; b=DAPSRjC+JgBYvHAtyfE3x7e5v5/uzcNCGMjsZ0cP4HQLYrB/Y28l3Cz9+1Ww9T87rC8Tqh a+NajPPtrCbWrHI+x+SMXB1jwiUFArRBefpsHTIZR0dsBEkS3yo/UWgdO2uRS1V84EJvxQ +tou0c0txXrBrEZHmmyfdf0gbR8t8Oo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1691433356; a=rsa-sha256; cv=none; b=DsKs9rKnW4EhDpzunnHas/zoi0sd6NnUaO2vABYy5oWvvpSjb5pg96fh4sgJfEvQg/wg6q h9PHrqLAHHlpjfY5kHyK/Y9O2RcE439AxCX4KP+4mjXAgYgoenVIRk7fbivsp9qw3sN2Ad uvmz1zqgKg08Js3YpoFFHdlwP7iyzEI= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Ic6rEStz; spf=pass (imf02.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691433356; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nSzDv346kgN2CfjlYQS29gdaOShLy0rGapky8ZA9EFo=; b=Ic6rEStzYA9DAQXEfE21281RA6+KHyfhLOV5ProngVTwOxlILSxxqPTBSwMr6xNMZAGPA7 E2UtjDqhStFKtazEfn9ASAyxm9+mliK/LYN2bOQGeT/54yp9S5GlS/B2iSqzsXQ3dFVavT v625PlHCe8KZ1LqStk05hfHQjT3hTPY= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-523-eBli4bcsM7SQnEk54u7Rxw-1; Mon, 07 Aug 2023 14:35:52 -0400 X-MC-Unique: eBli4bcsM7SQnEk54u7Rxw-1 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-317421b94a4so2144676f8f.3 for ; Mon, 07 Aug 2023 11:35:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691433351; x=1692038151; h=content-transfer-encoding:in-reply-to:subject:organization :references:cc:to:from:content-language:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=nSzDv346kgN2CfjlYQS29gdaOShLy0rGapky8ZA9EFo=; b=QxtZLXX3w0sxp8HQmNtMDeQ5S1J/aHCYRSe8gqWuAI22hzsehvU9CNbVkFqabLDtnm 5JaT2R1DGrdUc5f7MyCwe5O/Vz4oKKGfz6XJeMcnF+X6oyth8VPoebrwsJA/KJifpCbn y1Aeq99J7eTkW8UdzEzT+wTylX5C+zRZy1wJbMNytFVJzq9dLW+96Fs/Htjdacc7aP4R yzAPQILkZw4dz5CoSkz57sHAEqM24f0zfbrz2PxdshDx5AgVrN3i+Sm7ae3x+IDm5MIA 3GKgGSfhCzoAT7YNHLd6WyXvH2AFppTEPmnr2LTVqOXbF9Z2kq9+qyvYHwVzpOT8ZE+x kWog== X-Gm-Message-State: AOJu0YzLtiG3rFyLlo5W1KPmw5LgQerNpqdG6e2Ky+sqzK7meHFCvfd+ L3nKd0GUGV0dpjswaVBT3og3S0lXWprpACocVuqIkM97ZnRBq1dY8rFQ/3GVCKLfTqOOFIu2suH ZGNdzLbSvKlA= X-Received: by 2002:adf:ef8e:0:b0:317:54de:7315 with SMTP id d14-20020adfef8e000000b0031754de7315mr6464149wro.61.1691433351433; Mon, 07 Aug 2023 11:35:51 -0700 (PDT) X-Google-Smtp-Source: AGHT+IG4AgrI9SjrzPcGzpTnCQBzDx5r18Q61Uok0+tcU8/asAI8Gn40tuost0P4KHvDEDEzvNL1QA== X-Received: by 2002:adf:ef8e:0:b0:317:54de:7315 with SMTP id d14-20020adfef8e000000b0031754de7315mr6464137wro.61.1691433351055; Mon, 07 Aug 2023 11:35:51 -0700 (PDT) Received: from ?IPV6:2003:cb:c740:5d00:5143:1cd2:a300:ceff? (p200300cbc7405d0051431cd2a300ceff.dip0.t-ipconnect.de. [2003:cb:c740:5d00:5143:1cd2:a300:ceff]) by smtp.gmail.com with ESMTPSA id m9-20020adfe0c9000000b003145559a691sm11454099wri.41.2023.08.07.11.35.49 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 07 Aug 2023 11:35:50 -0700 (PDT) Message-ID: Date: Mon, 7 Aug 2023 20:35:49 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 From: David Hildenbrand To: Michal Hocko , Aneesh Kumar K V Cc: linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu, Oscar Salvador , Vishal Verma References: <31305ab7-1e65-80aa-ee91-9190c8f67430@redhat.com> <1c6a74f0-85e9-5299-1520-9068e842b1a5@redhat.com> Organization: Red Hat Subject: Re: [PATCH v7 7/7] mm/memory_hotplug: Enable runtime update of memmap_on_memory parameter In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Stat-Signature: xa6ony6ft4dg8wwmt9rop893c1ome6gb X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: D06548002A X-Rspam-User: X-HE-Tag: 1691433356-928358 X-HE-Meta: U2FsdGVkX18Z3Nj+Yq9FomEti+Tc6hD97x2xjgQL2+Zz1AbpnmdqIeRRnaX5S9bIMchyU/M/DlMKScP14AHasTdxn5nmJfJeOLr8yjGLewP34geT2SS10UOkYNgSgkgMZ94+rvXaB/ORtu+tBAOoQhAOnNOy9VsNp031RkgT4SbmYSoMPiOHaWTTmcI2Hy4Xx7wfkywT2lR1Ax9RWSnAfOuPRVJzMP6r97jjAP6aF8rZH+LsdVNI5fHTUBuNLDVbcSH0jEtF4DR+V3JCZdi6I3rzQpYnjVzjhr4GRy8+OvMTJlSsRiNiPbGMXVqv0ZZ6qAivGDsJV5HQTFRrPqqVc/A55gSiH2mPyE8wZSff2BCIiEwzLSTNvchaS2UCnNQekBGiPPkDFWIP2dpRusKRQXRsgIueFXO/rCD70dm+K9qQXOQhKFnCnImUbNP7l1U1Hi+A5wB0KCM+5FaSQzt8lck004JqWbssi3WWLI0LuADug+FhBD/XmSkIV18KswTiXWIJjKRjCvuZ4hllQzehkdCjIkebnro3KCZ+nTK/tvIXD1rdOXHotAWdWOzcHeEoyxFRiy6S5awid6E3nMKdGUPNv3qON6k6lWQoOqi+yafx2iJ816wm4kksWvh6JgXxO0IvQrsWSB6vES5it4volPXOYfVCoSN2ht0wU4D9TGE1BYDBgenQVQIzNB6SXEknyLFT3TbdGqZNio85x7QMxGGSkgOymInEF1osGBC3/MJSvkqbuv1MBCjYCtcGfwA956/HijCFl7tY1E2zwdLh7u8IxRFfvcQS/nXAFSMi+3vFcRqykBRKRoRV0GeI65VBGKjjPipoLpfJzcWVC90MrK3gStfH6l1lSfAsIC0j7g7J/+UPpPyMllx62246aViAVguMt85NVWprIwy3UoBkNoRsa5naXtRZ3ehpPc5H3QKdQXTxHG2ypWm8sIGMi/v+i3VL4J7cqf1SKoUeyqF /oNSE0ZF 8oM7cbq+CBMkQGI4SRbND68ysWOaT6y+my9WtlNHGDYQXZLCdmCappo28/peJfvfbJ/b2xKLlfQGUlNo2ASs7ZSIDsY2mSbqzkM35ypnt+wjGkWJweLp23Bt+Nh1qA6ip4+mJ7p58QvN3435gnic1jZ6MK/C96XxsCkAcSmmBd0xrNfUEgjKnyHRDeWLrPKekZrVLzL27z7NkPu88Z/jXlYr42n9xlvEIQPZapGtUYqrM7q7IeJ04n7MjQHPyNbREBiCOKaHZYLIiHgV45QY6hf4xFPerUyFB2q90sCQct8skViWV2SQ7BcvVhBI5+Nn+l8OR09P2KbPkShrcT3aOdU23oPQOyXXTuQxqJUiBMg9bWvnw4Ln3GIXGwJ7UBo/eZNtM8/tRjMZSpfefewcV3SD3TNAaHG7tPTdajH/S6vhQIImzWO0nJJJ05A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 07.08.23 14:41, David Hildenbrand wrote: > On 07.08.23 14:27, Michal Hocko wrote: >> On Sat 05-08-23 19:54:23, Aneesh Kumar K V wrote: >> [...] >>> Do you see a need for firmware-managed memory to be hotplugged in with >>> different memory block sizes? >> >> In short. Yes. Slightly longer, a fixed size memory block semantic is >> just standing in the way and I would even argue it is actively harmful. >> Just have a look at ridicously small memory blocks on ppc. I do >> understand that it makes some sense to be aligned to the memory model >> (so sparsmem section aligned). In an ideal world, memory hotplug v2 >> interface (if we ever go that path) should be physical memory range based. > > Yes, we discussed that a couple of times already (and so far nobody > cared to implement any of that). > > Small memory block sizes are very beneficial for use cases like PPC > dlar, virtio-mem, hyperv-balloon, ... essentially in most virtual > environments where you might want to add/remove memory in very small > granularity. I don't see that changing any time soon. Rather the opposite. > > Small memory block sizes are suboptimal for large machines where you > might never end up removing such memory (boot memory), or when dealing > with devices that can only be removed in one piece (DIMM/kmem). We > already have memory groups in place to model that. > > For the latter it might be beneficial to have memory blocks of larger > size that correspond to the physical memory ranges. That might also make > a memmap (re-)configuration easier. > > Not sure if that is standing in any way or is harmful, though. > Just because I thought of something right now, I'll share it, maybe it makes sense. Assume when we get add_memory*(MHP_MEMMAP_ON_MEMORY) and it is enabled by the admin: 1) We create a single altmap at the beginning of the memory 2) We create the existing fixed-size memory block devices, but flag them to be part of a single "altmap" unit. 3) Whenever we trigger offlining of a single such memory block, we offline *all* memory blocks belonging to that altmap, essentially using a single offline_pages() call and updating all memory block states accordingly. 4) Whenever we trigger onlining of a single such memory block, we online *all* memory blocks belonging to that altmap, using a single online_pages() call. 5) We fail remove_memory() if it doesn't cover the same (altmap) range. So we can avoid having a memory block v2 (and all that comes with that ...) for now and still get that altmap stuff sorted out. As that altmap behavior can be controlled by the admin, we should be fine for now. I think all memory notifiers should already be able to handle bigger granularity, but it would be easy to check. Some internal things might require a bit of tweaking. Just a thought. -- Cheers, David / dhildenb