Re: sequential 1MB mmap read ends in 1 page sync read-ahead

[Bernd Schubert <bernd.schubert@fastmail.fm>]

On 4/15/23 00:02, Matthew Wilcox wrote:
> On Fri, Apr 14, 2023 at 10:47:39PM +0200, Bernd Schubert wrote:
>>> Up to bs=512K it works fine, 1M (and for what it matters
>>> already 768K) introduce the order=0 issue.
>>
>> Hmm, I replaced memcpy with dumb version, that copies byte by byte - problem
>> gone. Is it possible that the optimized memcpy causes caused kind of random
>> memory access and confuses mm / readahead?
>> And somehow your memcpy or system is not doing that?
> 
> Oh, that would make sense!  If the memcpy() works high-to-low, then
> you'd see exactly the behaviour you're reporting.  Whereas low-to-high
> results in the behaviour I'm seeing.

In my case it is not exactly high-to-low, it is more low, high, then 
from high to low. Issue goes away with a sufficiently large RA size. And 
RA behaves better when POSIX_MADV_SEQUENTIAL is not done.
For sure memcpy implementation and cpu depending (I have tested on avx 
and avx2 systems).

> 
> Hm.  I don't know what to do about that.  Maybe a "sufficiently large"
> memcpy should call posix_madvise(src, n, POSIX_MADV_WILLNEED)


What would speak against ignoring POSIX_MADV_SEQUENTIAL for RA when it 
would detect that something breaks the contract?

But then I don't know how much mmap/memcpy is used for large memcpy or 
memcmp - maybe we should just document the issue right now in the mmap 
man page? Especially that POSIX_MADV_SEQUENTIAL should be avoided?



Thanks,
Bernd