RE: Additional Huge Pages

["Albert, Des" <des.albert@hpe.com>]

Hi Matt

I have discussed this topic in more detail with the primary maintainer of this kernel code. He confirmed that the additional huge pages are currently of most value to those who develop software that supports the HPE Slingshot High Speed Network. It is therefore of considerable benefit where networks access large areas of memory with PGAS (Partitioned Global Address Space ) or Symmetrical Hierarchical Memory ( SHMEM )  Because these programming concepts are relatively specific to HPC, it seems likely that it will remain a niche topic.  
I suspect that there is an assumption that offering code to kernel.org will, in some way, reduce the HPE resources required to maintain the code. I doubt that would be possible. In the opinion of the developer, the primary benefit of contributing code to kernel.org comes from ensuring that other changes to the Linux kernel do not adversely affect the kernel code that HPE has contributed and will continue to maintain.  

At this time, the primary maintainer is dedicated to other projects and is therefore not available for more detailed discussions. My primary goal has been to establish contact with the appropriate maintainers at kernel.org and seek responses to the suggestion of offering the code.

Des 

-----Original Message-----
From: Matthew Wilcox <willy@infradead.org> 
Sent: Friday, July 22, 2022 12:39 PM
To: Albert, Des <des.albert@hpe.com>
Cc: Mike Kravetz <mike.kravetz@oracle.com>; songmuchun@bytedance.com; linux-mm@kvack.org
Subject: Re: Additional Huge Pages

On Fri, Jul 22, 2022 at 07:20:51PM +0000, Albert, Des wrote:
> This is the first time I have heard of the folio abstraction as the future for memory management. When you mention that future hugetbls work will be based on that concept, it seems unlikely that there would be interest in code that is not consistent with those developments. I also doubt that there would be a justification to 'update' the code to be consistent with future kernel developments.
> 
> I am therefore forming the impression that this idea may not be of interest to the Linux kernel community, however, I do not the detailed technical depth of the development team.
> 
> Do you have some more information about this folio abstraction plan ?

Hi Des!  I'm the lead on the folio abstraction plan, so hopefully I can be of some help.

Folios, like your Cray Hugepages, broaden the supported page sizes.
They were originally conceived for relatively small page sizes (eg
16kB-256kB) and have been implemented so far only for the XFS filesystem.
Other filesystems are in progress.

This is the first hint we've had that people are interested in folio sizes above 2MB.  I think the folio work should make supporting this Cray requirement much easier.  It's certainly good to know that this is interesting before we do too much work on converting the existing hugetlb code over to folios.  Are you able to direct any developers to help us with this?  We can definitely work together on this project; we've had a similar collaboration running for a few years now on the Transparent Huge Page side of things.