Re: [LSF/MM/BPF] Whither Highmem?

["Kirill A. Shutemov" <kirill@shutemov.name>]

On Mon, May 08, 2023 at 01:23:24AM +0100, Matthew Wilcox wrote:
> On Mon, May 08, 2023 at 02:43:30AM +0300, Kirill A. Shutemov wrote:
> > On Mon, May 08, 2023 at 12:20:42AM +0100, Matthew Wilcox wrote:
> > > 
> > > I see there's a couple of spots on the schedule open, so here's something
> > > fun we could talk about.
> > > 
> > > Highmem was originally introduced to support PAE36 (up to 64GB) on x86
> > > in the late 90s.  It's since been used to support a similar extension
> > > on ARM (maybe other 32-bit architectures?)
> > > 
> > > Things have changed a bit since then.  There aren't a lot of systems
> > > left which have more than 4GB of memory _and_ are incapable of running a
> > > 64-bit kernel.
> > 
> > Actual limit is lower. With 3G/1G userspace/kernel split you will have
> > somewhere about 700Mb of virtual address space for direct mapping.
> > 
> > But, I would like to get rid of highmem too. Not sure how realistic it is.
> 
> Right, I was using 4GB because on x86, we have two config options that
> enable highmem, CONFIG_HIGHMEM4G and CONFIG_HIGHMEM64G.  If we get rid
> of the latter, it could be a nice win?

Not really. CONFIG_HIGHMEM64G is basically synonym for PAE that has more
goodies beyond wider phys_addr_t, like NX bit support. PAE and HIGHMEM4G
are mutually exclusive (but NOHIGHMEM is fine).

> Also, the more highmem we have, the more kinds of things we need to put in
> highmem.  Say we have a 3:1 ratio of high to lowmem.  On my 16GB laptop,
> I have 5GB of Cached and 8.5GB of Anon.  That's 13.5GB, so assuming that
> ratio would be similar for a 4GB laptop, it's 5.4:1 and storing _just_
> anon & cached pages in highmem would be more than enough.
> 
> (fwiw, PageTables is 125MB)
> 
> Maybe there's a workload that needs, eg page tables or fs metadata to
> be stored in highmem.  Other than pathological attempts to map one
> page per 2MB, I don't think those exist.
> 
> Something I forgot to say is that I do not think we'll see highmem being
> needed on 64-bit systems.

I hope not. CPU designers must know by now to provide at least 2-bit wider
virtual address space than physical address space.

> We already have CPUs with 128-bit registers,
> and have since the Pentium 3.  128-bit ALUs are missing, but as long as
> we're very firm with CPU vendors that this is the kind of nonsense up
> with which we shall not put, I think we can get 128-bit normal registers
> at the same time that they change the page tables to support more than
> 57 bits of physical memory.

Current architectural limit on x86 PA is 52.

I think we would need 128-bit PTEs before we run out of physical address
space (or get 128-bit GPRs). We don't have bits for new features in PTE
and started to eat out PFN bits.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov