From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25011E95A95 for ; Mon, 9 Oct 2023 15:15:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A77508D0080; Mon, 9 Oct 2023 11:15:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A27B38D0031; Mon, 9 Oct 2023 11:15:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8EF6D8D0080; Mon, 9 Oct 2023 11:15:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 7F7778D0031 for ; Mon, 9 Oct 2023 11:15:47 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 47D5A16030B for ; Mon, 9 Oct 2023 15:15:47 +0000 (UTC) X-FDA: 81326272734.19.3C14D39 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf28.hostedemail.com (Postfix) with ESMTP id 02A15C002E for ; Mon, 9 Oct 2023 15:15:44 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Pq8u1uaA; spf=pass (imf28.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696864545; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lxbkxsdepLAwI//Nw1rXTrNsOKPrZ+J+qWsPA89lw+E=; b=njoIRGtbLd4dnc1Cp3Ipkbr2yybsz00AKpNrFWr5atp7YuFCpWApWhskwpGNmdkRgIU5Kg 93hH0UuboNzEXFw3S5W05S3M66t7ZozuhPnepUusGzKKvdi9rTNFSxdEbCHNYcreMM3Qgv 8o4sAUzkvVjxFmWfJTHon0TUr6PyEcU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696864545; a=rsa-sha256; cv=none; b=42ZZMh28/qwQbeamD5I+Z0jwbuzn5XNrlTgZSyq1f6hW6EeWLJX4arBF9J8BziEppMn8uF bVGo5JrVPbie5X4wVIT06FNE/eTmv7mmSPlPKBS+hqQSBpsP27I4ffg/rOHZvtDw9sLQUO Rf6CfT7Q4ZqPoSh5c3uaMaueJD0nIG0= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Pq8u1uaA; spf=pass (imf28.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696864544; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lxbkxsdepLAwI//Nw1rXTrNsOKPrZ+J+qWsPA89lw+E=; b=Pq8u1uaAnEyYgDtjXR1dYf88jSGrr1xvGw4s1hmffEdauiBXJLj6BvdQGGM6od4wDxtk7x T7MfbCyv0BqhO+kBActGiZpNxWYsP9tb+qYZMlKc0GsOT/xW2/JhquRTnpacRcCALC4X/q gI7etjNeeiyqeuYxrTTogdvDaKJxI5M= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-193-lpiHQVdJPgiqmCjf82FVkg-1; Mon, 09 Oct 2023 11:15:43 -0400 X-MC-Unique: lpiHQVdJPgiqmCjf82FVkg-1 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-313c930ee0eso2918155f8f.0 for ; Mon, 09 Oct 2023 08:15:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696864541; x=1697469341; h=content-transfer-encoding:in-reply-to:subject:organization:from :references:cc:to:content-language:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=lxbkxsdepLAwI//Nw1rXTrNsOKPrZ+J+qWsPA89lw+E=; b=DPnq1kEnC0vsCmVq6Yqhqwq9ePsrDF1Um8HhL7d0Kw05mAGc5sXqH4OJqdksF9pwUz cK9bOhjUuakHgABY5lKr88ar3j3KPhV3BZyWWxYjnnIOp6tPb9mBLL7hCQtS/5Nygl1m WeDWwpUyyLfnKdVhEp756YLxkOeatdLDAMEakq5TDgTnp7QtZz7iXodTD1VprdxfeaHz 3RcfKzzEnfeo5/a3i8pnwOqvy4uXJMMMM/KA++LrWu/APOTnm8LPqwg6hLW/lB1/yXQf zVaw01rwIQR9Jb01I3J3M1a9LfPsW15hP9Ar9F8YMBU5eK4aZV/HPF8gWifb7JrtTcBc +Fqg== X-Gm-Message-State: AOJu0YwyoQjGldb3q54dlofuaK1HB0cerddRm9sY5BDHgapfo4MGm0sx 8ILxgRcm3TVpisViVewh+dEhfjaEiTRRcC/dni46qHvaGMALMOUjDeq9MmMyE+LLfcU2VsxfLQB mfuUKC+47ErQ= X-Received: by 2002:a5d:4942:0:b0:323:1a0c:a5e0 with SMTP id r2-20020a5d4942000000b003231a0ca5e0mr13395355wrs.13.1696864541281; Mon, 09 Oct 2023 08:15:41 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF7b2lwzlp+uSM9hiBU+z/Mq/3Q8Bio+XwSc3ICD2i4PuZSt2i99eOmikwZ+NaXx3+BddMdfQ== X-Received: by 2002:a5d:4942:0:b0:323:1a0c:a5e0 with SMTP id r2-20020a5d4942000000b003231a0ca5e0mr13395322wrs.13.1696864540855; Mon, 09 Oct 2023 08:15:40 -0700 (PDT) Received: from ?IPV6:2003:cb:c733:6400:ae10:4bb7:9712:8548? (p200300cbc7336400ae104bb797128548.dip0.t-ipconnect.de. [2003:cb:c733:6400:ae10:4bb7:9712:8548]) by smtp.gmail.com with ESMTPSA id j13-20020adfe50d000000b003196b1bb528sm9921911wrm.64.2023.10.09.08.15.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 09 Oct 2023 08:15:40 -0700 (PDT) Message-ID: <0bc29e93-7efe-4bce-ee3f-1fdf76104843@redhat.com> Date: Mon, 9 Oct 2023 17:15:39 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.15.1 To: "Verma, Vishal L" , "Williams, Dan J" , "Jiang, Dave" , "osalvador@suse.de" , "akpm@linux-foundation.org" Cc: "dave.hansen@linux.intel.com" , "Huang, Ying" , "linux-mm@kvack.org" , "aneesh.kumar@linux.ibm.com" , "linux-kernel@vger.kernel.org" , "linux-cxl@vger.kernel.org" , "Hocko, Michal" , "nvdimm@lists.linux.dev" , "jmoyer@redhat.com" , "Jonathan.Cameron@Huawei.com" References: <20231005-vv-kmem_memmap-v5-0-a54d1981f0a3@intel.com> <20231005-vv-kmem_memmap-v5-1-a54d1981f0a3@intel.com> <4ad40b9b-086b-e31f-34bd-c96550bb73e9@redhat.com> <45cfd268da63eeddb741e9c9c3026b0e15eade4b.camel@intel.com> From: David Hildenbrand Organization: Red Hat Subject: Re: [PATCH v5 1/2] mm/memory_hotplug: split memmap_on_memory requests across memblocks In-Reply-To: <45cfd268da63eeddb741e9c9c3026b0e15eade4b.camel@intel.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 02A15C002E X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 34nxnxzo18jdhyrqfc6mgicnga1hc4bt X-HE-Tag: 1696864544-307173 X-HE-Meta: U2FsdGVkX19nXWX3flLGqrP1KpBGrs6DD3fKjtq2zQr1/4wtX4pFadd9Ly4Xky+0f5YsYz4s9xJVymrUuX/7BvOjg8+eyxEXw2bv/2xanR8GIn90xQJC+XIz+kF4QvT4x5YVujIcYnOs1uzvPa8uDUbzPwN/3UCr8bs2NANpXobcu1TpJ0EseONS3bq2T6ssaUSz8drmUmbCpsyx5ld3Msm/pMFXgd8atYcKANKkFynwfIWb7ZDUHUVAZLQ/UFkGy8gEp4DAOs6i4hrF6AaSYXK0N+7UkKZOvygaDFCa4C/vAwqPuqqdSQQY6mBrAAw9/siAqUgSCFspgN4pX+pPZc1MlDOlB7wa84ZoDbp62zcvYWBVyHoO12cs92W/PkA2cZkrp/zV1cxZRy77ZlZmbqQm999qhDMDwYfekmlSGX1byXbv3h8vjsOM89Cuqj/m1tmIzYDVOH0kFGe1B9qm3XoA7hZuR7xMj/9HCytQ5J4wyPV2hVPKkWh1Kx6jktmalju2beCYeF6cyQSOqIfuFB3LS2bFx4859lTz8tGJvBH2elR4cK8AqgnZOnK38JZJQM7CczLAkGFvAYnqVngdHnGO+ISdtcsoj7Uju1Aa5Jl5RaPZr/JGbU/tgJ+9qZwRMlbFgxtOFDg1M3SzuJ9JU8kLNyQcytHMnM9JBCzyVBWXMN23dufrLNRS5/ll9VU/HGotcD7tZHx8zXemv/mh5761OGGWto1TTyg2rCd0b4TJ8dWHiJtK+S/XZctJMTLHFSbiMqyvJQX9PRductEhrUqh7/sV5dDeks2oneqY0ehFA3Q2/huYHEvqIV60rkycgO8vhucnaoSlaDDrWeMVkQyiXs74wW/zBfAOEqegSItNjHk2hn+IEIuwNNqlrAyH2/UTomI1+OrLX2qcMfTImGOD0aFeoAChui1NNyvwR13lhNSrb/e+QL7f+OqAcb3LGppEDYKzaXsDzWaHUm0 45E/HiM2 HZigbPUy7A+clkcRdlQCFWOfaBbvElqa+t7/0dHD+AFXtcdQ3PjHf+OC5y8pyy5NIRlnlA+Ov+53SFtTunFZ3Gom6/w2PRqPDiy8m0zbaMqRLaOevcobrcmF2dVnrLmhLC4ehC1NtFcvy2m0eC4k62XcwcvAh0Jy4MsrWEdANv78d3eP+st7YSBHiJsZIs9iIAqX6EiFk6Z2FV4OV57OSLfT0z/vhyOcAxSr+zGXMU+SNTyYk5sr1URbwx0QcATGFEND+Ue+yyZPqgzQKT1kXxBeTgD5F97LXx1nbv1aUIj8j+jEK6RJAqWk3LGxG3qZleFkhKHyD03HnAXCDFZzxldL4qKJMp6S3VcG5iDk91XteCE3vjQ+YaK6V+ohXsBeYEvFcKK90cc8mxULZX6QY9k+5kKjei8X69Rg9xStC/OyTSZxM+2xGPETPB7s7a/gC1B75lN/nQm8tN3bAbVVfPBcLMa9xRP1Kc+Rz X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 07.10.23 00:01, Verma, Vishal L wrote: > On Fri, 2023-10-06 at 14:52 +0200, David Hildenbrand wrote: >> On 05.10.23 20:31, Vishal Verma wrote: >>> > <..> >>> @@ -2167,47 +2221,28 @@ static int __ref try_remove_memory(u64 start, u64 size) >>>         if (rc) >>>                 return rc; >>> >>> +       mem_hotplug_begin(); >>> + >>>         /* >>> -        * We only support removing memory added with MHP_MEMMAP_ON_MEMORY in >>> -        * the same granularity it was added - a single memory block. >>> +        * For memmap_on_memory, the altmaps could have been added on >>> +        * a per-memblock basis. Loop through the entire range if so, >>> +        * and remove each memblock and its altmap. >>>          */ >>>         if (mhp_memmap_on_memory()) { >>> -               rc = walk_memory_blocks(start, size, &mem, test_has_altmap_cb); >>> -               if (rc) { >>> -                       if (size != memory_block_size_bytes()) { >>> -                               pr_warn("Refuse to remove %#llx - %#llx," >>> -                                       "wrong granularity\n", >>> -                                       start, start + size); >>> -                               return -EINVAL; >>> -                       } >>> -                       altmap = mem->altmap; >>> -                       /* >>> -                        * Mark altmap NULL so that we can add a debug >>> -                        * check on memblock free. >>> -                        */ >>> -                       mem->altmap = NULL; >>> -               } >>> +               unsigned long memblock_size = memory_block_size_bytes(); >>> +               u64 cur_start; >>> + >>> +               for (cur_start = start; cur_start < start + size; >>> +                    cur_start += memblock_size) >>> +                       remove_memory_block_and_altmap(nid, cur_start, >>> +                                                      memblock_size); >>> +       } else { >>> +               remove_memory_block_and_altmap(nid, start, size); >> >> Better call remove_memory_block_devices() and arch_remove_memory(start, >> size, altmap) here explicitly instead of using >> remove_memory_block_and_altmap() that really can only handle a single >> memory block with any inputs. >> > I'm not sure I follow. Even in the non memmap_on_memory case, we'd have > to walk_memory_blocks() to get to the memory_block->altmap, right? See my other reply to, at least with mhp_memmap_on_memory()==false, we don't have to worry about the altmap. > > Or is there a more direct way? If we have to walk_memory_blocks, what's > the advantage of calling those directly instead of calling the helper > created above? I think we have two cases to handle 1) All have an altmap. Remove them block-by-block. Probably we should call a function remove_memory_blocks(altmap=true) [or alternatively remove_memory_blocks_and_altmaps()] and just handle iterating internally. 2) All don't have an altmap. We can remove them in one go. Probably we should call that remove_memory_blocks(altmap=false) [or alternatively remove_memory_blocks_no_altmaps()]. I guess it's best to do a walk upfront to make sure either all have an altmap or none has one. Then we can branch off to the right function knowing whether we have to process altmaps or not. The existing if (mhp_memmap_on_memory()) { ... } Can be extended for that case. Please let me know if I failed to express what I mean, then I can briefly prototype it on top of your changes. -- Cheers, David / dhildenb