From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB41CC7619A for ; Wed, 12 Apr 2023 08:38:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5183A900006; Wed, 12 Apr 2023 04:38:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4C86B900004; Wed, 12 Apr 2023 04:38:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 390AA900006; Wed, 12 Apr 2023 04:38:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 29FC7900004 for ; Wed, 12 Apr 2023 04:38:19 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 0BE69A1D79 for ; Wed, 12 Apr 2023 08:38:12 +0000 (UTC) X-FDA: 80672086824.03.B4461F9 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf01.hostedemail.com (Postfix) with ESMTP id CD19D4001F for ; Wed, 12 Apr 2023 08:38:09 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=OCSdn8Xl; spf=pass (imf01.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681288689; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ti3j3d9k9vuAnWKNJ1m/mxjRXId1YNvSwaENJe/WWtQ=; b=45NzaPzkzLWNwCvY3lFTLzF4wKsmpVpa8wS2qnLT4FM8xx+N4lp/oinht72uHR9gByCNM4 7bjpuZoC6q3CUPvZTSYOMMo8LC6Pe/+qelTnjhhSdK2EVkOWlOxyjj7CkLskmhv36gV1B1 rGHkxz3zQBiLfhAKMd7HvSIkFoBW/4k= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=OCSdn8Xl; spf=pass (imf01.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681288689; a=rsa-sha256; cv=none; b=Rdt9mDs3YjZ18uBnJ3HqxFa1jXJhWYDuExyD6/HKbfYmO3fvX6Z8sKCPHWdxDUibLY3f96 mB/ZgipkaEc/sVh/4Ixe5+SLyu92nzpXQ1uVZI3QEizlI0QrGYYbj8GEQq+T/tlLvhQXjC 0wqp26KX/+wKi0g0Tr+924ufh46mF1Y= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1681288688; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ti3j3d9k9vuAnWKNJ1m/mxjRXId1YNvSwaENJe/WWtQ=; b=OCSdn8Xlsvw4yrIdbVWNUryvx7uExTYmAaObPmwoE+XEXHe+n+kWg2q/XpPfeYfd6mdvQU BpndXTiiWK/atfJ7PdcFTI2Uc/NIN0jJ2QHJjLdGLDP43x6hMpj8RFL0VPclIJU/T/D9uo hUg9jnX2jWNsF4bWHP9dTGYKDU1RLyc= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-583-TJf-j8s8Nl6nKKHC5vm1DQ-1; Wed, 12 Apr 2023 04:38:07 -0400 X-MC-Unique: TJf-j8s8Nl6nKKHC5vm1DQ-1 Received: by mail-wm1-f72.google.com with SMTP id bh18-20020a05600c3d1200b003f05a99b571so8331149wmb.8 for ; Wed, 12 Apr 2023 01:38:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1681288686; x=1683880686; h=content-transfer-encoding:in-reply-to:subject:organization:from :references:cc:to:content-language:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ti3j3d9k9vuAnWKNJ1m/mxjRXId1YNvSwaENJe/WWtQ=; b=E+NEwQZfuEZqPYcWHL2Ms52cd3wCsbrwFJbYHzv23z6hKUq65BgGIvRA2ZGMm6gmVj vWQUMq77mMdl54W/rAVnieqQ1RSyhu3dTZE+ZM67UMzf4cFNfXduW+6fquWNrWmLisbL k+WDzySkMk8SsI/3/XdHy7dGloHVxxWN3GR/LT1IhUHRFE8G2t3C483F8sU4uL/eomsL Lv5j2o/WNo8pa2xF595vnFu+5givZaKeIHBd+owbPAdXFPJfvOQgwp6aC2CyCxe75sUz yCf+zup21Goo00sym7rvvHzh79sibn8zvcDrigT6ntq2mm7kKFSci1+N1euZDNT/Npcq XQTw== X-Gm-Message-State: AAQBX9djwtR3ZA53rinnvhlg4MJJXqpY/vzrcyAqLJj23AldUDCCnqYQ bou0bB9say2YMi/2rhchnSiBxL44CMLqPGmJcHoAuSxNwwoO8wXW3zg7ltj/uoj+D0wpHzt9TnV yiBf7H1bysGE= X-Received: by 2002:a7b:c386:0:b0:3ed:8780:f265 with SMTP id s6-20020a7bc386000000b003ed8780f265mr12154013wmj.21.1681288686324; Wed, 12 Apr 2023 01:38:06 -0700 (PDT) X-Google-Smtp-Source: AKy350a6zjmQhTioVzjwahDEwgvxsQMm7XPDrFYXNSLu8OiEDGDN4wH+L1RNsifRWzHOhrdD4rdeRA== X-Received: by 2002:a7b:c386:0:b0:3ed:8780:f265 with SMTP id s6-20020a7bc386000000b003ed8780f265mr12154002wmj.21.1681288685934; Wed, 12 Apr 2023 01:38:05 -0700 (PDT) Received: from ?IPV6:2003:cb:c702:4b00:c6fa:b613:dbdc:ab? (p200300cbc7024b00c6fab613dbdc00ab.dip0.t-ipconnect.de. [2003:cb:c702:4b00:c6fa:b613:dbdc:ab]) by smtp.gmail.com with ESMTPSA id o3-20020a05600c4fc300b003ee1e07a14asm1548735wmq.45.2023.04.12.01.38.04 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 12 Apr 2023 01:38:05 -0700 (PDT) Message-ID: Date: Wed, 12 Apr 2023 10:38:04 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.9.1 To: "Huang, Ying" , Gregory Price Cc: Dragan Stancevic , lsf-pc@lists.linux-foundation.org, nil-migration@lists.linux.dev, linux-cxl@vger.kernel.org, linux-mm@kvack.org References: <5d1156eb-02ae-a6cc-54bb-db3df3ca0597@stancevic.com> <87v8i22abl.fsf@yhuang6-desk2.ccr.corp.intel.com> <87bkjtzu7e.fsf@yhuang6-desk2.ccr.corp.intel.com> From: David Hildenbrand Organization: Red Hat Subject: =?UTF-8?Q?Re=3a_=5bLSF/MM/BPF_TOPIC=5d_BoF_VM_live_migration_over_C?= =?UTF-8?B?WEwgbWVtb3J54oCL?= In-Reply-To: <87bkjtzu7e.fsf@yhuang6-desk2.ccr.corp.intel.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: CD19D4001F X-Rspam-User: X-Stat-Signature: b6nh7m14u7xdgacmrq1qye1dns89yyub X-HE-Tag: 1681288689-663553 X-HE-Meta: U2FsdGVkX1/Suj8aG5cEyYZMXYVyK+9USCv3qbqibPkdh42hQylOAgp7ESg2GwaWF+FgwTUF4kRHYXEInfKAWqSplLU5MmyZfL4iV7e3AOAZ5ZdX4/clYTCoCYZfk1kMJeSch7YLuors8JfrTliQCsHYfCZIbv+iuqsMJ4A9cQk/ICRF8RTSc7hDEt7Ffooyc8n7WoFAUPD7pkAF0NUNn++Yb5/qEYXhw9YYNhV8XqJJjnrMPo3Za3fpizvWDQCIt/onI/q+pJMmQcu3PYkKHyZrdAj3tu0jwMmj6OEA0Dx5UEJaNqKwWQo7KD8MaMKlGaOPz9Tljb5fhSansU1Ny1zhPevJX6nZLYp8jdVM6XJo9ifK+WFUu5sa/V7Rv9ODBI6VDItBMGErjTdAFCGrnHCQ3Q8b6YgiAUss7KpxfG6H6vYdvzSIDkwilbMHQ5RQZ4G+sr7i0oLcXmpO8GYHCB57CdQ4LXNKAmhgtztN+o6tXp2g9mrij2WCDrOEic/NFC0YxumLOTluarh8K/YlvHtGFI/Zf7qtZQbP2V9G3iY24VcyBMHE1IkUjI/7PX5XF8g0nKJErb9pzXZTue+6BHW6xXYhgI6RLPwuERmPMqSlZEO58NTMOMsLxTVAEUme01ZT7Khu+OV7it13SiLIEyGeNAhkHJbWrEDS6cOhjUHgedAEkNOF1c0xuuF56K6m75KN/YW5Xt193bEwkrR1MK7dI74JRoDpKD0ZaCIoPw5QwT9VdHiNd/dFaYr1Cf3vfRsorLFeBjFUcH9p8+fP1M8yRy4P1sjcBMaoWC07NZeJqtgSXnrAfvIqAeWiyP7mMg6ZcklrcqRGg+WFaKNPY5fUTUn3YfnuWbq79NNyJx6iR5Py8APb6f4TqWvgEhp6WYqZWjul2mWUA+/UfLuO/HkYYXRjgrzUxxIh7KzGnHmVqSl1PUDOxSVZJ3+GhUe4/tIYE0WZt13CVjc3uAn KaxDDpFd Trk1HzxcIpKVhYU/eOAtLpJifw6PH6mT7WyIQZ7IO4UlfVvFKrc8CoRDH4gkdvv6SfzoVRB0odRbXwAnmTfObOcSSQ/EV1+a846RDsUyE0VOYcgx/zxsZVCtXOtAyMBD0Hv0skKW2/XPzawLWBruXujDa3sdAmPPZFHs43jfZLI5y6oba2IZpdbz3AMoZeW/MTweSafWz4cpzRDamWnnCTWqyL0QqK7eYtvyykYp6dbVbmgrbJFHY4kx5KCFNldFdJqkmuPgaz/P8V/PmAHs4ozuHMzVF0LE4j4aiMG7B8PpG5fozFpV11Y2oLnpIfUIGc+YfCQRR+ibjS+nAoe7tK9nXTKY/KjnxTEkOJtmWCdVLEekrPo+aGFw0SoOx28grvWo07+kj7Sr4Oye3DJ32ZJv9bPNrgjwxaRybeKazZGdhPmaAUhHveDCGmyOfBUMqvwkH+B0qn8r06Yzq+PdhI8rFTrW81LBCMed2i5E+VyMOA4oycx6PHRxHw2T0+IC8ncMP X-Bogosity: Ham, tests=bogofilter, spamicity=0.091495, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 12.04.23 04:54, Huang, Ying wrote: > Gregory Price writes: > >> On Tue, Apr 11, 2023 at 02:37:50PM +0800, Huang, Ying wrote: >>> Gregory Price writes: >>> >>> [snip] >>> >>>> 2. During the migration process, the memory needs to be forced not to be >>>> migrated to another node by other means (tiering software, swap, >>>> etc). The obvious way of doing this would be to migrate and >>>> temporarily pin the page... but going back to problem #1 we see that >>>> ZONE_MOVABLE and Pinning are mutually exclusive. So that's >>>> troublesome. >>> >>> Can we use memory policy (cpusets, mbind(), set_mempolicy(), etc.) to >>> avoid move pages out of CXL.mem node? Now, there are gaps in tiering, >>> but I think it is fixable. >>> >>> Best Regards, >>> Huang, Ying >>> >>> [snip] >> >> That feels like a hack/bodge rather than a proper solution to me. >> >> Maybe this is an affirmative argument for the creation of an EXMEM >> zone. > > Let's start with requirements. What is the requirements for a new zone > type? I'm stills scratching my head regarding this. I keep hearing all different kind of statements that just add more confusions "we want it to be hotunpluggable" "we want to allow for long-term pinning memory" "but we still want it to be movable" "we want to place some unmovable allocations on it". Huh? Just to clarify: ZONE_MOVABLE allows for pinning. It just doesn't allow for long-term pinning of memory. For good reason, because long-term pinning of memory is just the worst (memory waste, fragmentation, overcommit) and instead of finding new ways to *avoid* long-term pinnings, we're coming up with advanced concepts to work-around the fundamental property of long-term pinnings. We want all memory to be long-term pinnable and we want all memory to be movable/hotunpluggable. That's not going to work. If you'd ask me today, my prediction is that ZONE_EXMEM is not going to happen. -- Thanks, David / dhildenb