From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8787C76196 for ; Fri, 31 Mar 2023 15:57:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6E58A6B0072; Fri, 31 Mar 2023 11:57:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 66F736B0074; Fri, 31 Mar 2023 11:57:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 542036B0075; Fri, 31 Mar 2023 11:57:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 3EA4D6B0072 for ; Fri, 31 Mar 2023 11:57:10 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 143F180367 for ; Fri, 31 Mar 2023 15:57:10 +0000 (UTC) X-FDA: 80629647420.26.2064774 Received: from mail-ed1-f54.google.com (mail-ed1-f54.google.com [209.85.208.54]) by imf10.hostedemail.com (Postfix) with ESMTP id 2622DC001F for ; Fri, 31 Mar 2023 15:57:06 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="bkZ4l/r9"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf10.hostedemail.com: domain of fvdl@google.com designates 209.85.208.54 as permitted sender) smtp.mailfrom=fvdl@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680278227; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QUqPxRAuAHrc1y6VCqhIsZPLfTNQYcDlCWZh8QFbAXg=; b=wcZQMmfdZG/62PHhz6950e+Ou+UUznKGTD1Dq2GgSRe9WmptkEkuutAp5IUYznm45aOOp4 ZPio/g3b5m8YSMg12ZUV0s+U7LCbe+BWeS0cAi/kW82Sov6ESnuh6ey1BzJYjv4dfLCqIs xYXd1aH3XTIL3TqK2tkNPcTDOoxWDCA= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b="bkZ4l/r9"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf10.hostedemail.com: domain of fvdl@google.com designates 209.85.208.54 as permitted sender) smtp.mailfrom=fvdl@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680278227; a=rsa-sha256; cv=none; b=gy4KhFs0w30cjGY/rs4KysI1wD4Grj/+be47qYPsjCuceKu8iPbSRfjyvmYe36rOWJCFkH 4jh/RnyIushiB3/QDCx/W1e9YFCCrmNOfDu1H8AM4nkBq5Z279tZTziQkrELstUJHq/gdz X47bhM0wra77F15/FTndjOnVBWAtCSk= Received: by mail-ed1-f54.google.com with SMTP id ew6so91394893edb.7 for ; Fri, 31 Mar 2023 08:57:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1680278225; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=QUqPxRAuAHrc1y6VCqhIsZPLfTNQYcDlCWZh8QFbAXg=; b=bkZ4l/r93KAdsfu5vhV8HHA7H+tUt8+RCUdzBWrzIs7sG+RCRM8k4Y3YH7QTYnsH/W 9oXe2S3FFnucf0XXTHbmngcR/JOQsMaGTcztEr+Rz83Zi7jLjQ01txfmjjQ3K2d2NCYJ vKjMYcQFsFhfJjhZf5QxUh5bcJSbEGuqESthyKBchb5q/IWM4c2k7EHOd4IRAK3T4YPy M2DgTeRmyJpBvjyFLsScUlylCNVe+bxzq2LlDxmvBCawObRb5Z4U1gEB3nPmVRpTN9TF N/wvzvo4hcCvNpviCA8V4IbPZ27nOJBomb9wTvpV6D6RQCvluq7bpN7aPhjhiGMexceG wQng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680278225; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QUqPxRAuAHrc1y6VCqhIsZPLfTNQYcDlCWZh8QFbAXg=; b=Q5ZG7qz5sLKisO74eD1L6FjFu8CtJzvh4rpnnUJetRpNoUmmh1yhP5gEtEQPLe/wyv ff/2x0pWzISm6y/W58GlUy1BTWj6LHltmR93pY7ZYxG6XJqj7IIAbWAK0+hq+WDCGDCQ 2WJu0wZ0D4U2eMXXFBmyFmOMZm4LMWet0EdT036W71hK3T8YqFgODjytPrCjh/jdS6Yh xdEadBXPxUtWLc62meTvAyqBiImxZXkMMMphGBWtu/TdN2ZZp55+hARHHCjJ9vZRxIL2 Dc3rXFqUYxKc5bYUIHSbc0RXTH+jmkBcVgPRNFvgTwsxxPE2qiRWDIc6aQY+78wkFEdu jj9Q== X-Gm-Message-State: AAQBX9fZMUqfueIvQzoSZ/NwxTXmbX9HKD7znW4BCLgk2UZDzU9uI9e+ x/hF0pHlsW8okZKRZ5V5D+CpbpCADECNUuoHGSRL7Q== X-Google-Smtp-Source: AKy350YeU3SBnfFyD4Pfud1dHmqGIMxXnNo9n3lah5R3ayWKJr0l5EdZnUMm+T5gAOWjW7sp5RniVOOJKfQVkRGy9IU= X-Received: by 2002:a50:ce58:0:b0:502:6d4b:40f5 with SMTP id k24-20020a50ce58000000b005026d4b40f5mr3720859edj.7.1680278225487; Fri, 31 Mar 2023 08:57:05 -0700 (PDT) MIME-Version: 1.0 References: <7c7933df-43da-24e3-2144-0551cde05dcd@redhat.com> <20230331114220.400297-1-ks0204.kim@samsung.com> In-Reply-To: From: Frank van der Linden Date: Fri, 31 Mar 2023 08:56:54 -0700 Message-ID: Subject: Re: RE: FW: [LSF/MM/BPF TOPIC] SMDK inspired MM changes for CXL To: Matthew Wilcox Cc: Kyungsan Kim , david@redhat.com, lsf-pc@lists.linux-foundation.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-cxl@vger.kernel.org, a.manzanares@samsung.com, viacheslav.dubeyko@bytedance.com, dan.j.williams@intel.com, seungjun.ha@samsung.com, wj28.lee@samsung.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 2622DC001F X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: kpx1bn3ynffqfg7h1dfs45jxeogprwcr X-HE-Tag: 1680278226-567155 X-HE-Meta: U2FsdGVkX1+R1N4S/6k6+IFrKr+mIfbqbUiav+15XmoDYPnSDUALAcEn5kx1JYEnn6dr3qARQbupg9hM0MrvUE1brBLPhzk3slCmlrB8hoAdWu8x5FWim9gbKtzK/Pcix+wcS3jTvfNkXazTZ87MNdBRa8sCGrn285GpkgVTKX7e10ilarDtmwUOWa04rYn0E72Xg5vcIGGSa+py+9hR0CA0SYpgwoH0eyLxFcgvBrnxlFbGw+NsJOyxq3sc6OD4OJKWHEfWCCfOLv7MsrlXwFeMCTN2nRa8YWlkdPnFG1g9sBnoGSNGPwRYHVf7D4q6DM9lFxWUZ+g8Ep+gnPLk34eQrWrScAplkWEBstDkz+ZV2rgqCcPhfR8jmyJqiVKv/V/v4RRDZN+4ocSc3KI0QRBUldWsmWBOAaGut6kwDik/PlYekgfrBvOlQhtc4XkRtpEWsJHL86S1jImTXN19eV33mZ6Iyz5evoI0wpmfhAAxGehcOdpo0iYliYCNIRtOhtKsUL0EmIMk0vylRUDq3U27/unQzRUdJ85rIaCWVlRLfz0tCwy+DZhGlm28xDoQw0wfXZ53M/57lR+dXDMLsrz0N1oKOXqhRduk+0ppPKx13Py4hJti4879FiySEawGygymcwsbrDGq2LFjOlbWHshXL/Si6+944gkp06Te0vaXlgY7W0h++TLlqqVS3dk12erkLe576OoSMfGQt3yzBYnG2WCG/alyE6l0DTrqdFeDUN6RPVXkiQNuvs1pIjP6H30N4v8ptkbhFDbh/43m1GvUV+m4dy3FNGaqcDmqMc3O90JG93a+BHYkEXo8yRm5aWr5r68PDD+3CkR4QoS2Xj+5jMtwcCGTOkRU8fb0WnyZPI79pTpG7h8kD+lrbvuzz5A65KRowVHxjXoGu/d/AJU4Xy+SvZ/hrirHmj3Ex/x1Gpx7b2GJdUl8QWiR/vOeqeFIgbXTHkn+q5zYvAe v4MbbGQ2 1LrO079Ksa95d+y7M0BwmFC6fbxUML3BFfB5gTP5R2AXDGPYNwg7iZp2xcxcPOEj0xz8lkfLnPwpF7SBVVc27z1DHrJBB6L5WE5glfh0w46FQRcMWVmKfuSbhmYaFgnUmaYjCnikVGtANyuVwvrEw+BJiK3x+MHeDUJbg5wr0Zhtworl754ZUlfbEfeA5fNbmc28I398lgKS7JGJaTdokZVa9tYTwj2WrCj/PxtVUgcXP+lpgDPw278O5tuhiw2nPkfcyQ8ssP77Bvb3O1iSbExeEIwU3LOg+oVugKnd05ZRnKGzEL04lWHTCEHL1fJmL7xCks+wgOj/lCqQ8gmEU5BN+mkp/5kE5lJfLgzRDNydx/GHF/sDHPfGevVFTp5UD5aHCiiLlYpA/p1gzv0rBtWFkTR0HRj/8oy81hF44o5yG4sw9yuvi5OSeoA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Mar 31, 2023 at 6:42=E2=80=AFAM Matthew Wilcox wrote: > > On Fri, Mar 31, 2023 at 08:42:20PM +0900, Kyungsan Kim wrote: > > Given our experiences/design and industry's viewpoints/inquiries, > > I will prepare a few slides in the session to explain > > 1. Usecase - user/kernespace memory tiering for near/far placement, m= emory virtualization between hypervisor/baremetal OS > > 2. Issue - movability(movable/unmovable), allocation(explicit/implici= t), migration(intented/unintended) > > 3. HW - topology(direct, switch, fabric), feature(pluggability,error-= handling,etc) > > I think you'll find everybody else in the room understands these issues > rather better than you do. This is hardly the first time that we've > talked about CXL, and CXL is not the first time that people have > proposed disaggregated memory, nor heterogenous latency/bandwidth > systems. All the previous attempts have failed, and I expect this > one to fail too. Maybe there's something novel that means this time > it really will work, so any slides you do should focus on that. > > A more profitable discussion might be: > > 1. Should we have the page allocator return pages from CXL or should > CXL memory be allocated another way? > 2. Should there be a way for userspace to indicate that it prefers CXL > memory when it calls mmap(), or should it always be at the discretion > of the kernel? > 3. Do we continue with the current ZONE_DEVICE model, or do we come up > with something new? > > Point 2 is what I proposed talking about here: https://lore.kernel.org/linux-mm/a80a4d4b-25aa-a38a-884f-9f119c03a1da@googl= e.com/T/ With the current cxl-as-numa-node model, an application can express a preference through mbind(). But that also means that mempolicy and madvise (e.g. MADV_COLD) are starting to overlap if the intention is to use cxl as a second tier for colder memory. Are these the right abstractions? Might it be more flexible to attach properties to memory ranges, and have applications hint which properties they prefer? It's an interesting discussion, and I hope it'll be touched on at LSF/MM, happy to participate there. - Frank