msync() seems not to clean the data cache

msync() seems not to clean the data cache

[John Reiser]

The system call write() cleans the hardware data cache (writes any dirty
values from the data cache into RAM memory) before passing to the VFS
the region to be written.  The system call msync() should do likewise.
Currently msync() does not clean the hardware data cache, as seen on
PowerPC, PowerPC64, and arm64; and probably any CPU that does not
have a Write-through cache.  (x86 and x86_64 do have write-through.)

Please fix system call msync() so that it cleans the hardware data cache
first.

Re: msync() seems not to clean the data cache

[Matthew Wilcox]

On Fri, Apr 25, 2025 at 12:17:37PM -0700, John Reiser wrote:
> The system call write() cleans the hardware data cache (writes any dirty
> values from the data cache into RAM memory) before passing to the VFS
> the region to be written.  The system call msync() should do likewise.
> Currently msync() does not clean the hardware data cache, as seen on
> PowerPC, PowerPC64, and arm64; and probably any CPU that does not
> have a Write-through cache.  (x86 and x86_64 do have write-through.)

I think you're right; we don't flush before writeback.  Does this
fix your problem?

+++ b/mm/page-writeback.c
@@ -2432,6 +2432,7 @@ static bool folio_prepare_writeback(struct address_space *mapping,
        if (!folio_clear_dirty_for_io(folio))
                return false;

+       flush_dcache_folio(folio);
        return true;
 }


By the way, x86 does not have writethrough caches.  Instead it has DMA
coherent caches, so modified cachelines in the cache will take
precedence over the contents of memory.

Re: msync() seems not to clean the data cache

[John Reiser]

On 4/25/25 12:31 PM, Matthew Wilcox wrote:
> On Fri, Apr 25, 2025 at 12:17:37PM -0700, John Reiser wrote:
>> The system call write() cleans the hardware data cache (writes any dirty
>> values from the data cache into RAM memory) before passing to the VFS
>> the region to be written.  The system call msync() should do likewise.
>> Currently msync() does not clean the hardware data cache, as seen on
>> PowerPC, PowerPC64, and arm64; and probably any CPU that does not
>> have a Write-through cache.  (x86 and x86_64 do have write-through.)
> 
> I think you're right; we don't flush before writeback.  Does this
> fix your problem?
> 
> +++ b/mm/page-writeback.c
> @@ -2432,6 +2432,7 @@ static bool folio_prepare_writeback(struct address_space *mapping,
>          if (!folio_clear_dirty_for_io(folio))
>                  return false;
> 
> +       flush_dcache_folio(folio);
>          return true;
>   }
> 
> 
It looks promising,  but I got lost tracing the logic by hand, 
especially for my seven $ARCH.  Building and testing will take a while.

The context is UPX support for Enforcing mode of SELinux, as outlined:
    fd = memfd_create("upx", MFD_EXEC);
    ftruncate(fd, length);
    maddr = mmap(0, length, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
    decompress(maddr, length, compressed_input);
    msync(maddr, length);   // fd was losing random data here
    munmap(maddr, length);  // paranoia over VMA "hangover"
    maddr2 = mmap(maddr, length, PROT_READ|PROT_EXEC,
       MAP_SHARED|MAP_FIXED, fd, 0);
See bug report at  https://github.com/upx/upx/issues/907 .
Meanwhile explicit user-mode cache cleaning (using $ARCH-dependent
system call, or hand code, or heuristics) is necessary because of
the inertia of many many installed Linux instances.