From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96B2FC433F5 for ; Tue, 15 Feb 2022 12:16:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 296CE6B0078; Tue, 15 Feb 2022 07:16:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2456A6B007B; Tue, 15 Feb 2022 07:16:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0C0C76B007D; Tue, 15 Feb 2022 07:16:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id F263D6B0078 for ; Tue, 15 Feb 2022 07:16:48 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id B5B5C181AC9C6 for ; Tue, 15 Feb 2022 12:16:48 +0000 (UTC) X-FDA: 79144912896.06.06368B8 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf30.hostedemail.com (Postfix) with ESMTP id 2F4D180007 for ; Tue, 15 Feb 2022 12:16:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1644927407; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Pt3kaDq1aQDkXB/hX2rUZYURxbJS3EoGhO0bUKlqTws=; b=JDfCjqg2zoroNThqR/XA9QKeAjdFenvQ7BDKXd9ZqqocagWyZ7nk3EuCgv8Ak3xJ1MMwKu CPefNfm22rty+4gWV3/OpKKdgM4Nthjj+SpnHysi/q0q1nXKoIn3MBflEJrDIMIXV8/tBJ FY43NQrJlFN7t+kIiN+CpwmqiT59eiU= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-7-_lvZcaKiOberG6bhy1sUrQ-1; Tue, 15 Feb 2022 07:16:46 -0500 X-MC-Unique: _lvZcaKiOberG6bhy1sUrQ-1 Received: by mail-wm1-f70.google.com with SMTP id h82-20020a1c2155000000b003552c13626cso1307461wmh.3 for ; Tue, 15 Feb 2022 04:16:46 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:organization:in-reply-to :content-transfer-encoding; bh=Pt3kaDq1aQDkXB/hX2rUZYURxbJS3EoGhO0bUKlqTws=; b=yQsifdvdlGJiSPZ97hMmTC58l5H0mBgYCAc1EPrCyLRSPNgQkWiT0IbKAYjncI+sTS xfhYQuKitjMalLRZYAwirpjwrNWVo0lhYhXBrHQw71q8LOVrt1PdedyTj4zVCq5v36/b MtndgCJ0XP2gB6RY6GE/YU0NgblrrHo35U6QWzb13QUUWoCr+n13CPkmGAyG5xgKHZfQ 84aZ+IbYrdKv/e/3XCNm4qZmpRDdAE/rHpxQEux2vhUuAqf48ol/N0XxZde/SO0mPkFk Ae0mhejbRJpxFs1vhM3ojycdLFaHEAo57XYVjzC4APdwW1SOJCnYFlePOwqDIbbxjZmk ePQw== X-Gm-Message-State: AOAM531UEuJuW9E5nSjG++ce2MLYNbwG1rarZ83AAOqb91YpPnodIhl9 Z9FbOsG1nhDj9wN8w84ENQhQvxt86ZXyk9aHrhR5IkU9oCNitHvTrDunxbLBZUnGlg+WuTv54Tc b1m6SdbbZemU= X-Received: by 2002:a05:6000:156c:: with SMTP id 12mr2943168wrz.387.1644927405395; Tue, 15 Feb 2022 04:16:45 -0800 (PST) X-Google-Smtp-Source: ABdhPJwsEhDLYb4heqpDoCuYC8NeTX+i0m6l8MCzsccoJvk2OwBqm/N5306LsnwqqqFvXB+oF92GrA== X-Received: by 2002:a05:6000:156c:: with SMTP id 12mr2943150wrz.387.1644927405198; Tue, 15 Feb 2022 04:16:45 -0800 (PST) Received: from ?IPV6:2003:cb:c70e:3700:9260:2fb2:742d:da3e? (p200300cbc70e370092602fb2742dda3e.dip0.t-ipconnect.de. [2003:cb:c70e:3700:9260:2fb2:742d:da3e]) by smtp.gmail.com with ESMTPSA id y6sm7829264wrd.30.2022.02.15.04.16.43 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 15 Feb 2022 04:16:44 -0800 (PST) Message-ID: <078dd84e-ebbc-5c89-0407-f5ecc2ca3ebf@redhat.com> Date: Tue, 15 Feb 2022 13:16:43 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.4.0 Subject: Re: [PATCH v6 01/10] mm: add zone device coherent type memory support To: Felix Kuehling , Alex Sierra , akpm@linux-foundation.org, linux-mm@kvack.org, rcampbell@nvidia.com, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org Cc: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, hch@lst.de, jgg@nvidia.com, jglisse@redhat.com, apopple@nvidia.com, willy@infradead.org References: <20220201154901.7921-1-alex.sierra@amd.com> <20220201154901.7921-2-alex.sierra@amd.com> From: David Hildenbrand Organization: Red Hat In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JDfCjqg2; spf=none (imf30.hostedemail.com: domain of david@redhat.com has no SPF policy when checking 170.10.129.124) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam07 X-Rspam-User: X-Rspamd-Queue-Id: 2F4D180007 X-Stat-Signature: 1wwzrykb3x4e5crz9yjdca8rjoxj166h X-HE-Tag: 1644927408-430646 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 11.02.22 18:07, Felix Kuehling wrote: > > Am 2022-02-11 um 11:39 schrieb David Hildenbrand: >> On 11.02.22 17:15, David Hildenbrand wrote: >>> On 01.02.22 16:48, Alex Sierra wrote: >>>> Device memory that is cache coherent from device and CPU point of view. >>>> This is used on platforms that have an advanced system bus (like CAPI >>>> or CXL). Any page of a process can be migrated to such memory. However, >>>> no one should be allowed to pin such memory so that it can always be >>>> evicted. >>>> >>>> Signed-off-by: Alex Sierra >>>> Acked-by: Felix Kuehling >>>> Reviewed-by: Alistair Popple >>> So, I'm currently messing with PageAnon() pages and CoW semantics ... >>> all these PageAnon() ZONE_DEVICE variants don't necessarily make my life >>> easier but I'm not sure yet if they make my life harder. I hope you can >>> help me understand some of that stuff. >>> >>> 1) What are expected CoW semantics for DEVICE_COHERENT? >>> >>> I assume we'll share them just like other PageAnon() pages during fork() >>> readable, and the first sharer writing to them receives an "ordinary" >>> !ZONE_DEVICE copy. >>> >>> So this would be just like DEVICE_EXCLUSIVE CoW handling I assume, just >>> that we don't have to go through the loop of restoring a device >>> exclusive entry? >>> >>> 2) How are these pages freed to clear/invalidate PageAnon() ? >>> >>> I assume for PageAnon() ZONE_DEVICE pages we'll always for via >>> free_devmap_managed_page(), correct? >>> >>> >>> 3) FOLL_PIN >>> >>> While you write "no one should be allowed to pin such memory", patch #2 >>> only blocks FOLL_LONGTERM. So I assume we allow ordinary FOLL_PIN and >>> you might want to be a bit more precise? >>> >>> >>> ... I'm pretty sure we cannot FOLL_PIN DEVICE_PRIVATE pages, but can we >>> FILL_PIN DEVICE_EXCLUSIVE pages? I strongly assume so? >>> >>> >>> Thanks for any information. >>> >> (digging a bit more, I realized that device exclusive pages are not >> actually/necessarily ZONE_DEVICE pages -- so I assume DEVICE_COHERENT >> will be the actual first PageAnon() ZONE_DEVICE pages that can be >> present in a page table.) > > I think DEVICE_GENERIC pages can also be mapped in the page table. In > fact, the first version of our patches attempted to add migration > support to DEVICE_GENERIC. But we were convinced to create a new > ZONE_DEVICE page type for our use case instead. Do you know if DEVICE_GENERIC pages would end up as PageAnon()? My assumption was that they would be part of a special mapping. -- Thanks, David / dhildenb