How should we RCU-free folios?

[Matthew Wilcox <willy@infradead.org>]

When folios are allocated separately from the underlying pages they
represent, they must also be freed.  See
https://kernelnewbies.org/MatthewWilcox/FolioAlloc

Since we want to do lockless lookups of folios in the page cache and
GUP, we must RCU free the folios somehow.  As I see it, we have three
options:

1. Free the folio back to the slab immediately, and mark the slab as
TYPESAFE_BY_RCU.  That means that the folio may get reallocated at
any time, but it must always remain a folio (until an RCU grace period
has passed and then the entire slab may be reallocated to a different
purpose).  Lookups will do:

a. Get a pointer to the folio
b. Tryget a refcount on the folio
c. If it succeeds, re-check the folio is still the one we want
   (If pagecache, check the xarray still points to the folio; if GUP,
   check the page still points to the folio)

2. RCU-free the folio.  The folio will not be reallocated until the
reader drops the RCU read lock.  The read side still needs to tryget
the folio refcount.  However, if it succeeds, it does not need to
re-check the pointer to the folio as the folio cannot have been
freed.  The downside is that folios will hang around in the system for
longer before being reallocated, and this may be an unacceptable
increase in memory usage.

3. RCU free the folio and RCU free the memory it controls.  Now an
RCU-protected lookup doesn't need to bump the refcount; if it found the
pointer, it knows the memory cannot be freed.  I think this is a
step too far and would 

I'm favouring option 1; it's what we currently do.  But I wanted to
give people a chance to chime in and tell me my tradeoffs are wrong.
Or propose a fourth option.