Hi Mike

This code is targeted at application usage. Developers load an appropriate environment module then compile and link ( see attached man page )

As can be seen in this document, the features were initially developed to support proprietary Cray High Speed Networks ( Gemini and Aries ) and associated PGAS or SHMEM programming models. In this respect, it can be regarded as legacy code that has not kept pace with recent developments.

It has been challenging to determine the relevance of this feature to current hardware and applications. My requests to developers and benchmark specialists for information about the benefits it provides have not revealed much specific data. There is a general impression that it would be useful for GPU applications and MPI in large HPC systems but I suspect that it would require some very advanced knowledge of memory management for a developer to know precisely how and when to apply it.

This is the first time I have heard of the folio abstraction as the future for memory management. When you mention that future hugetbls work will be based on that concept, it seems unlikely that there would be interest in code that is not consistent with those developments. I also doubt that there would be a justification to 'update' the code to be consistent with future kernel developments.

I am therefore forming the impression that this idea may not be of interest to the Linux kernel community, however, I do not the detailed technical depth of the development team.

Do you have some more information about this folio abstraction plan ?

Des

-----Original Message-----
From: Mike Kravetz <mike.kravetz@oracle.com> 
Sent: Friday, July 22, 2022 11:12 AM
To: Albert, Des <des.albert@hpe.com>
Cc: songmuchun@bytedance.com; linux-mm@kvack.org
Subject: Re: Additional Huge Pages

On 07/22/22 17:20, Albert, Des wrote:
> Hi
> 
> I am the Product Manager for the HPE Cray Operating System ( formerly 
> Cray Linux Environment )
> 
> One of the features of this product is a component known as additional huge pages. This is kernel code that enables the selection of 'non-standard' huge page sizes.
> For example, the current implementation allows for selection of huge page sizes of 2, 4, 8, 16, 32, 64, 128, 256 and 512 MB as well as 1 and 2 GB.
> 

Interesting.
Are these non-standard huge pages sizes targeted at application usage, or internal kernel APIs.  If applications, what API is used?  Is it similar/the same as hugetlb?

Within the kernel, support for 'arbitrary page sizes' is provided by the folio abstraction.  hugetlb code will be moving to that in the future.
Any new code such as this whould be based on folios.

> We are currently evaluating the concept of providing this code to kernel.org. I realize that this would require dedication of technical resources to work with maintainers.
> 
> I would like to know if there is interest in this suggestion. I realize that Transparent Huge Pages may be regarded as a more general approach to this requirement.
> 

I guess interest would depend on the use cases and potential advantages of this feature.  You should be able to speak to this based on your current usage.
--
Mike Kravetz