[PATCH] mm/hmm: kconfig split HMM address space mirroring from device memory

[PATCH] mm/hmm: kconfig split HMM address space mirroring from device memory

[jglisse]

From: Jérôme Glisse <jglisse@redhat.com>

To allow building device driver that only care about address space
mirroring (like RDMA ODP) on platform that do not have all the pre-
requisite for HMM device memory (like ZONE_DEVICE on ARM) split the
HMM_MIRROR option dependency from the HMM_DEVICE dependency.

Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
Cc: Leon Romanovsky <leonro@mellanox.com>
Cc: Jason Gunthorpe <jgg@mellanox.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Ralph Campbell <rcampbell@nvidia.com>
Cc: John Hubbard <jhubbard@nvidia.com>
---
 mm/Kconfig | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/mm/Kconfig b/mm/Kconfig
index 2e6d24d783f7..00d9febbc775 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -679,12 +679,13 @@ config ZONE_DEVICE
 config ARCH_HAS_HMM
 	bool
 	default y
-	depends on (X86_64 || PPC64)
-	depends on ZONE_DEVICE
 	depends on MMU && 64BIT
-	depends on MEMORY_HOTPLUG
-	depends on MEMORY_HOTREMOVE
-	depends on SPARSEMEM_VMEMMAP
+
+config ARCH_HAS_HMM_DEVICE
+	bool
+	default y
+	depends on (X86_64 || PPC64)
+	depends on ARCH_HAS_ZONE_DEVICE
 
 config MIGRATE_VMA_HELPER
 	bool
@@ -710,7 +711,8 @@ config HMM_MIRROR
 
 config DEVICE_PRIVATE
 	bool "Unaddressable device memory (GPU memory, ...)"
-	depends on ARCH_HAS_HMM
+	depends on ARCH_HAS_HMM_DEVICE
+	depends on ZONE_DEVICE
 	select HMM
 	select DEV_PAGEMAP_OPS
 
@@ -721,7 +723,8 @@ config DEVICE_PRIVATE
 
 config DEVICE_PUBLIC
 	bool "Addressable device memory (like GPU memory)"
-	depends on ARCH_HAS_HMM
+	depends on ARCH_HAS_HMM_DEVICE
+	depends on ZONE_DEVICE
 	select HMM
 	select DEV_PAGEMAP_OPS
 
-- 
2.20.1

Re: [PATCH] mm/hmm: kconfig split HMM address space mirroring from device memory

[Leon Romanovsky]

On Thu, Apr 11, 2019 at 02:03:26PM -0400, jglisse@redhat.com wrote:
> From: Jérôme Glisse <jglisse@redhat.com>
>
> To allow building device driver that only care about address space
> mirroring (like RDMA ODP) on platform that do not have all the pre-
> requisite for HMM device memory (like ZONE_DEVICE on ARM) split the
> HMM_MIRROR option dependency from the HMM_DEVICE dependency.
>
> Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
> Cc: Leon Romanovsky <leonro@mellanox.com>
> Cc: Jason Gunthorpe <jgg@mellanox.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Ralph Campbell <rcampbell@nvidia.com>
> Cc: John Hubbard <jhubbard@nvidia.com>
> ---
>  mm/Kconfig | 17 ++++++++++-------
>  1 file changed, 10 insertions(+), 7 deletions(-)
>

Thanks,

It gave me an option simply enable HMM_MIRROR without too much hassle as
it was before and compile your v3 [1].

Tested-by: Leon Romanovsky <leonro@mellanox.com>

[1] https://patchwork.kernel.org/patch/10894281/

Re: [PATCH] mm/hmm: kconfig split HMM address space mirroring from device memory

[Guenter Roeck]

On Thu, Apr 11, 2019 at 02:03:26PM -0400, jglisse@redhat.com wrote:
> From: Jérôme Glisse <jglisse@redhat.com>
> 
> To allow building device driver that only care about address space
> mirroring (like RDMA ODP) on platform that do not have all the pre-
> requisite for HMM device memory (like ZONE_DEVICE on ARM) split the
> HMM_MIRROR option dependency from the HMM_DEVICE dependency.
> 
> Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
> Cc: Leon Romanovsky <leonro@mellanox.com>
> Cc: Jason Gunthorpe <jgg@mellanox.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Ralph Campbell <rcampbell@nvidia.com>
> Cc: John Hubbard <jhubbard@nvidia.com>
> Tested-by: Leon Romanovsky <leonro@mellanox.com>

In case it hasn't been reported already:

mm/hmm.c: In function 'hmm_vma_handle_pmd':
mm/hmm.c:537:8: error: implicit declaration of function 'pmd_pfn'; did you mean 'pte_pfn'?

and similar errors when building alpha:allmodconfig (and maybe others).

Guenter

Re: [PATCH] mm/hmm: kconfig split HMM address space mirroring from device memory

[Jerome Glisse]

On Wed, Apr 17, 2019 at 11:21:18AM -0700, Guenter Roeck wrote:
> On Thu, Apr 11, 2019 at 02:03:26PM -0400, jglisse@redhat.com wrote:
> > From: Jérôme Glisse <jglisse@redhat.com>
> > 
> > To allow building device driver that only care about address space
> > mirroring (like RDMA ODP) on platform that do not have all the pre-
> > requisite for HMM device memory (like ZONE_DEVICE on ARM) split the
> > HMM_MIRROR option dependency from the HMM_DEVICE dependency.
> > 
> > Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
> > Cc: Leon Romanovsky <leonro@mellanox.com>
> > Cc: Jason Gunthorpe <jgg@mellanox.com>
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: Ralph Campbell <rcampbell@nvidia.com>
> > Cc: John Hubbard <jhubbard@nvidia.com>
> > Tested-by: Leon Romanovsky <leonro@mellanox.com>
> 
> In case it hasn't been reported already:
> 
> mm/hmm.c: In function 'hmm_vma_handle_pmd':
> mm/hmm.c:537:8: error: implicit declaration of function 'pmd_pfn'; did you mean 'pte_pfn'?

No it is pmd_pfn

> 
> and similar errors when building alpha:allmodconfig (and maybe others).

Does HMM_MIRROR get enabled in your config ? It should not
does adding depends on (X86_64 || PPC64) to ARCH_HAS_HMM
fix it ? I should just add that there for arch i do build.

Cheers,
Jérôme

Re: [PATCH] mm/hmm: kconfig split HMM address space mirroring from device memory

[Guenter Roeck]

On Wed, Apr 17, 2019 at 02:26:18PM -0400, Jerome Glisse wrote:
> On Wed, Apr 17, 2019 at 11:21:18AM -0700, Guenter Roeck wrote:
> > On Thu, Apr 11, 2019 at 02:03:26PM -0400, jglisse@redhat.com wrote:
> > > From: Jérôme Glisse <jglisse@redhat.com>
> > > 
> > > To allow building device driver that only care about address space
> > > mirroring (like RDMA ODP) on platform that do not have all the pre-
> > > requisite for HMM device memory (like ZONE_DEVICE on ARM) split the
> > > HMM_MIRROR option dependency from the HMM_DEVICE dependency.
> > > 
> > > Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
> > > Cc: Leon Romanovsky <leonro@mellanox.com>
> > > Cc: Jason Gunthorpe <jgg@mellanox.com>
> > > Cc: Andrew Morton <akpm@linux-foundation.org>
> > > Cc: Ralph Campbell <rcampbell@nvidia.com>
> > > Cc: John Hubbard <jhubbard@nvidia.com>
> > > Tested-by: Leon Romanovsky <leonro@mellanox.com>
> > 
> > In case it hasn't been reported already:
> > 
> > mm/hmm.c: In function 'hmm_vma_handle_pmd':
> > mm/hmm.c:537:8: error: implicit declaration of function 'pmd_pfn'; did you mean 'pte_pfn'?
> 
> No it is pmd_pfn
> 
FWIW, this is a compiler message.

> > 
> > and similar errors when building alpha:allmodconfig (and maybe others).
> 
> Does HMM_MIRROR get enabled in your config ? It should not
> does adding depends on (X86_64 || PPC64) to ARCH_HAS_HMM
> fix it ? I should just add that there for arch i do build.
> 

The eror is seen with is alpha:allmodconfig. "make ARCH=alpha allmodconfig".
It does set CONFIG_ARCH_HAS_HMM=y.

This patch has additional problems. For arm64:allmodconfig
and many others, when running "make ARCH=arm64 allmodconfig":

WARNING: unmet direct dependencies detected for DEVICE_PRIVATE
  Depends on [n]: ARCH_HAS_HMM_DEVICE [=n] && ZONE_DEVICE [=n]
  Selected by [m]:
  - DRM_NOUVEAU_SVM [=y] && HAS_IOMEM [=y] && ARCH_HAS_HMM [=y] && DRM_NOUVEAU [=m] && STAGING [=y]

WARNING: unmet direct dependencies detected for DEVICE_PRIVATE
  Depends on [n]: ARCH_HAS_HMM_DEVICE [=n] && ZONE_DEVICE [=n]
  Selected by [m]:
  - DRM_NOUVEAU_SVM [=y] && HAS_IOMEM [=y] && ARCH_HAS_HMM [=y] && DRM_NOUVEAU [=m] && STAGING [=y]

WARNING: unmet direct dependencies detected for DEVICE_PRIVATE
  Depends on [n]: ARCH_HAS_HMM_DEVICE [=n] && ZONE_DEVICE [=n]
  Selected by [m]:
  - DRM_NOUVEAU_SVM [=y] && HAS_IOMEM [=y] && ARCH_HAS_HMM [=y] && DRM_NOUVEAU [=m] && STAGING [=y]

This in turn results in:

arch64-linux-ld: mm/memory.o: in function `do_swap_page':
memory.c:(.text+0x798c): undefined reference to `device_private_entry_fault'

not only on arm64, but on other architectures as well.

All those problems are gone after reverting this patch.

Guenter

Re: [PATCH] mm/hmm: kconfig split HMM address space mirroring from device memory

[Jerome Glisse]

On Wed, Apr 17, 2019 at 12:33:35PM -0700, Guenter Roeck wrote:
> On Wed, Apr 17, 2019 at 02:26:18PM -0400, Jerome Glisse wrote:
> > On Wed, Apr 17, 2019 at 11:21:18AM -0700, Guenter Roeck wrote:
> > > On Thu, Apr 11, 2019 at 02:03:26PM -0400, jglisse@redhat.com wrote:
> > > > From: Jérôme Glisse <jglisse@redhat.com>
> > > > 
> > > > To allow building device driver that only care about address space
> > > > mirroring (like RDMA ODP) on platform that do not have all the pre-
> > > > requisite for HMM device memory (like ZONE_DEVICE on ARM) split the
> > > > HMM_MIRROR option dependency from the HMM_DEVICE dependency.
> > > > 
> > > > Signed-off-by: Jérôme Glisse <jglisse@redhat.com>
> > > > Cc: Leon Romanovsky <leonro@mellanox.com>
> > > > Cc: Jason Gunthorpe <jgg@mellanox.com>
> > > > Cc: Andrew Morton <akpm@linux-foundation.org>
> > > > Cc: Ralph Campbell <rcampbell@nvidia.com>
> > > > Cc: John Hubbard <jhubbard@nvidia.com>
> > > > Tested-by: Leon Romanovsky <leonro@mellanox.com>
> > > 
> > > In case it hasn't been reported already:
> > > 
> > > mm/hmm.c: In function 'hmm_vma_handle_pmd':
> > > mm/hmm.c:537:8: error: implicit declaration of function 'pmd_pfn'; did you mean 'pte_pfn'?
> > 
> > No it is pmd_pfn
> > 
> FWIW, this is a compiler message.
> 
> > > 
> > > and similar errors when building alpha:allmodconfig (and maybe others).
> > 
> > Does HMM_MIRROR get enabled in your config ? It should not
> > does adding depends on (X86_64 || PPC64) to ARCH_HAS_HMM
> > fix it ? I should just add that there for arch i do build.
> > 
> 
> The eror is seen with is alpha:allmodconfig. "make ARCH=alpha allmodconfig".
> It does set CONFIG_ARCH_HAS_HMM=y.
> 
> This patch has additional problems. For arm64:allmodconfig
> and many others, when running "make ARCH=arm64 allmodconfig":
> 
> WARNING: unmet direct dependencies detected for DEVICE_PRIVATE
>   Depends on [n]: ARCH_HAS_HMM_DEVICE [=n] && ZONE_DEVICE [=n]
>   Selected by [m]:
>   - DRM_NOUVEAU_SVM [=y] && HAS_IOMEM [=y] && ARCH_HAS_HMM [=y] && DRM_NOUVEAU [=m] && STAGING [=y]
> 
> WARNING: unmet direct dependencies detected for DEVICE_PRIVATE
>   Depends on [n]: ARCH_HAS_HMM_DEVICE [=n] && ZONE_DEVICE [=n]
>   Selected by [m]:
>   - DRM_NOUVEAU_SVM [=y] && HAS_IOMEM [=y] && ARCH_HAS_HMM [=y] && DRM_NOUVEAU [=m] && STAGING [=y]
> 
> WARNING: unmet direct dependencies detected for DEVICE_PRIVATE
>   Depends on [n]: ARCH_HAS_HMM_DEVICE [=n] && ZONE_DEVICE [=n]
>   Selected by [m]:
>   - DRM_NOUVEAU_SVM [=y] && HAS_IOMEM [=y] && ARCH_HAS_HMM [=y] && DRM_NOUVEAU [=m] && STAGING [=y]
> 
> This in turn results in:
> 
> arch64-linux-ld: mm/memory.o: in function `do_swap_page':
> memory.c:(.text+0x798c): undefined reference to `device_private_entry_fault'
> 
> not only on arm64, but on other architectures as well.
> 
> All those problems are gone after reverting this patch.
> 
> Guenter

Andrew let drop this patch i need to fix nouveau Kconfig first.

Cheers,
Jérôme