From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 719BAC43334 for ; Fri, 17 Jun 2022 09:40:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E7D5D6B0073; Fri, 17 Jun 2022 05:40:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E2D256B0074; Fri, 17 Jun 2022 05:40:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CF5DF6B0075; Fri, 17 Jun 2022 05:40:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C273C6B0073 for ; Fri, 17 Jun 2022 05:40:44 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 95C2A60A51 for ; Fri, 17 Jun 2022 09:40:44 +0000 (UTC) X-FDA: 79587233208.01.88D7527 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf22.hostedemail.com (Postfix) with ESMTP id 0BB5FC00A4 for ; Fri, 17 Jun 2022 09:40:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1655458843; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H2YIQuKwekY0WiBRZ5F8WSRdemt7ajPawZ+d5zPsp/4=; b=cAXYms0CzOZPvbN82uRWBMX1L7gTmobv9vIb2pT6xVS3MjOCDFAfWX40X3w6NasEp1BrH0 9H+SZHjwu+1e0IJp41NBzTI236K59G8XYN1NPcqPgrzBpyoO7BKZ4yavRmI0H7ixVmhCev FsSZf6+r50HgckCA7jFoRLIgZLfre7k= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-36-BPGXh7O7OKSrrZSCP5N7ow-1; Fri, 17 Jun 2022 05:40:42 -0400 X-MC-Unique: BPGXh7O7OKSrrZSCP5N7ow-1 Received: by mail-wr1-f69.google.com with SMTP id e21-20020adfa455000000b0021b80b12356so294960wra.5 for ; Fri, 17 Jun 2022 02:40:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent :content-language:to:cc:references:from:organization:subject :in-reply-to:content-transfer-encoding; bh=H2YIQuKwekY0WiBRZ5F8WSRdemt7ajPawZ+d5zPsp/4=; b=OuQhjh+U3yMavRJt/yGZM5GN/D5VRXSbqp8IDLOoRawbvFJCV09mtyl4PSWoBBRbhX fREysVq96+xWLavNG8bYssFkdpKE78jVbGK8UEUjWlbV/JlqAzAB6WtAdFiDIDdyMsFX CGMRJgz79kxsiKlbZehau52Vx9Kft2K8fh+mgZ7pVavbdZK4OYP2aZYD4xApcnagNKsi bgzkMqYGp9gA1fxPZdi0FUq1SY0R8X+zvoe31DIS/TziSVbGlm5RepS3b4ryKS+nUyOZ Sy1zC6w5FEuR8V/f1Fn/NVrElPm9Ojh//BpPLRpSg++fwzQ6KPhbIg81EreT+s5Qsu83 Ur1Q== X-Gm-Message-State: AJIora8aCSefIXB8b1TdL8AEo2XgRCQBr7Cq0Xa1S5XLxvqevUYXltfn KKrfAOL+6Ggiaf/0019DeFVHbeBROscqdyPO2dRCTZF2rK3GtttrDbGFyQmKhQis0oMX0wZ+q6o ekt9AR+S1t+o= X-Received: by 2002:adf:ea90:0:b0:215:a11d:3329 with SMTP id s16-20020adfea90000000b00215a11d3329mr8273280wrm.709.1655458841236; Fri, 17 Jun 2022 02:40:41 -0700 (PDT) X-Google-Smtp-Source: AGRyM1tcyIDstW1UoHx515Nm3zkT6PwFo4LTcTooQfD0VdaRVMK+nlfneS0G9f2+zYKCej73GtUeFg== X-Received: by 2002:adf:ea90:0:b0:215:a11d:3329 with SMTP id s16-20020adfea90000000b00215a11d3329mr8273248wrm.709.1655458840923; Fri, 17 Jun 2022 02:40:40 -0700 (PDT) Received: from ?IPV6:2003:cb:c70a:7e00:bb5b:b526:5b76:5824? (p200300cbc70a7e00bb5bb5265b765824.dip0.t-ipconnect.de. [2003:cb:c70a:7e00:bb5b:b526:5b76:5824]) by smtp.gmail.com with ESMTPSA id y14-20020a5d614e000000b0020d09f0b766sm4082674wrt.71.2022.06.17.02.40.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 17 Jun 2022 02:40:40 -0700 (PDT) Message-ID: <3ac89358-2ce0-7d0d-8b9c-8b0e5cc48945@redhat.com> Date: Fri, 17 Jun 2022 11:40:39 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.9.0 To: Alex Sierra , jgg@nvidia.com Cc: Felix.Kuehling@amd.com, linux-mm@kvack.org, rcampbell@nvidia.com, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, hch@lst.de, jglisse@redhat.com, apopple@nvidia.com, willy@infradead.org, akpm@linux-foundation.org References: <20220531200041.24904-1-alex.sierra@amd.com> <20220531200041.24904-2-alex.sierra@amd.com> From: David Hildenbrand Organization: Red Hat Subject: Re: [PATCH v5 01/13] mm: add zone device coherent type memory support In-Reply-To: <20220531200041.24904-2-alex.sierra@amd.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655458844; a=rsa-sha256; cv=none; b=Lb8obEtlRnoryAIOKA+EBxQkqsmwkEpKEy1RqPqeUUoMaezVjkFjmbgPXcy6vrdR638QNs 6HfCwiqWmcieLqNR4+MQD7g+Mt+dLt+JOEQvirHwiLpCjhSiOEP/kGKP/d4h46JRTPp/u4 uD3v+5zClU4oht0atLtkevrYU2v0vtk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655458844; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=H2YIQuKwekY0WiBRZ5F8WSRdemt7ajPawZ+d5zPsp/4=; b=1Bt9wKK6czoX/D+5vimGwYs0TbszeKzIcvztV9XvV9BKVlBtagvc9WOmbDqUDwRB6KOb5j kCtlZJgqTaSrbpH97KrPsMNJ2uNXGlz26mvl76nVgzdqqhgOioTrU32Mj5go9LK/2mffL7 fImlxAmWZGJf154LURLtMlEe/dasCzQ= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=cAXYms0C; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf22.hostedemail.com: domain of david@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=david@redhat.com Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=cAXYms0C; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf22.hostedemail.com: domain of david@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=david@redhat.com X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 0BB5FC00A4 X-Stat-Signature: ory98kizj4r4j1urrd5poiqt9nnugmn9 X-HE-Tag: 1655458843-859074 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 31.05.22 22:00, Alex Sierra wrote: > Device memory that is cache coherent from device and CPU point of view. > This is used on platforms that have an advanced system bus (like CAPI > or CXL). Any page of a process can be migrated to such memory. However, > no one should be allowed to pin such memory so that it can always be > evicted. > > Signed-off-by: Alex Sierra > Acked-by: Felix Kuehling > Reviewed-by: Alistair Popple > [hch: rebased ontop of the refcount changes, > removed is_dev_private_or_coherent_page] > Signed-off-by: Christoph Hellwig > --- > include/linux/memremap.h | 19 +++++++++++++++++++ > mm/memcontrol.c | 7 ++++--- > mm/memory-failure.c | 8 ++++++-- > mm/memremap.c | 10 ++++++++++ > mm/migrate_device.c | 16 +++++++--------- > mm/rmap.c | 5 +++-- > 6 files changed, 49 insertions(+), 16 deletions(-) > > diff --git a/include/linux/memremap.h b/include/linux/memremap.h > index 8af304f6b504..9f752ebed613 100644 > --- a/include/linux/memremap.h > +++ b/include/linux/memremap.h > @@ -41,6 +41,13 @@ struct vmem_altmap { > * A more complete discussion of unaddressable memory may be found in > * include/linux/hmm.h and Documentation/vm/hmm.rst. > * > + * MEMORY_DEVICE_COHERENT: > + * Device memory that is cache coherent from device and CPU point of view. This > + * is used on platforms that have an advanced system bus (like CAPI or CXL). A > + * driver can hotplug the device memory using ZONE_DEVICE and with that memory > + * type. Any page of a process can be migrated to such memory. However no one Any page might not be right, I'm pretty sure. ... just thinking about special pages like vdso, shared zeropage, ... pinned pages ... > + * should be allowed to pin such memory so that it can always be evicted. > + * > * MEMORY_DEVICE_FS_DAX: > * Host memory that has similar access semantics as System RAM i.e. DMA > * coherent and supports page pinning. In support of coordinating page > @@ -61,6 +68,7 @@ struct vmem_altmap { > enum memory_type { > /* 0 is reserved to catch uninitialized type fields */ > MEMORY_DEVICE_PRIVATE = 1, > + MEMORY_DEVICE_COHERENT, > MEMORY_DEVICE_FS_DAX, > MEMORY_DEVICE_GENERIC, > MEMORY_DEVICE_PCI_P2PDMA, > @@ -143,6 +151,17 @@ static inline bool folio_is_device_private(const struct folio *folio) In general, this LGTM, and it should be correct with PageAnonExclusive I think. However, where exactly is pinning forbidden? -- Thanks, David / dhildenb