Questions on cache flushing in do_wp_page

Questions on cache flushing in do_wp_page

[Kanoj Sarcar]

I am trying to understand what the primitives flush_page_to_ram and
flush_cache_page do. I am vaguely aware of the issues of non coherent
io and virtual aliasing/virtual coherency error. Specially, I am
trying to guess at what these primitives might be doing in the
do_wp_page() routine. The only processors which I have worked with
are MIPS and Intel processors. In Intel, these two primitives are
null since the caches are fully coherent. For MIPS, I can't find
a good reason why either function would be neccesary in
do_wp_page(). I am CCing David Miller and Ralf Baechle whose
names appear in the copyright messages in arch/mips/mm/r4xx0.c.
Anyone else with m68k/ppc/sparc etc expertise please feel free to
comment.

I am looking at the piece of code in do_wp_page that reads:

        copy_cow_page(old_page,new_page);
        flush_page_to_ram(old_page);
        flush_page_to_ram(new_page);
        flush_cache_page(vma, address);
        set_pte(page_table, pte_mkwrite(pte_mkdirty(mk_pte(new_page, vma->vm_page_prot))));
        free_page(old_page);
        flush_tlb_page(vma, address);

I can see that the flush_cache_page() is needed for a processor that
has virtually indexed, virtually tagged L1, but is there any such
processor that Linux supports (sparc?)? Does this processor also
need virtual alias avoidance support from the os because the processor
can not detect such aliasing?

For MIPS R4000PC, with a virtually indexed physically tagged L1, I
think it might be enough to just have a flush_page_to_ram(new_page);
to avoid aliasing issues ...

In any case, I can't see a reason for flush_page_to_ram(old_page);
Can anyone tell me why that might be needed (on whichever processor)?

Another place that I can't explain why a flush_cache_page might be
needed is in:

        case 1:
                /* We can release the kernel lock now.. */
                unlock_kernel();

                flush_cache_page(vma, address);
                set_pte(page_table, pte_mkdirty(pte_mkwrite(pte)));
                flush_tlb_page(vma, address);
end_wp_page:
                if (new_page)
                        free_page(new_page);
                return 1;

Thanks.

Kanoj
kanoj@engr.sgi.com

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: Questions on cache flushing in do_wp_page

[David S. Miller]

   I am trying to understand what the primitives flush_page_to_ram and
   flush_cache_page do.

flush_cache_page()

	Remove all "user/tlb view" references to the page in the cache
	(any cache, L1, L2, BCACHE, etc.) for this user virtual mapping
	in address space mm.  It should be back flushed to ram if
	necessary on non-coherent copy-back style caches.

flush_page_to_ram()

	Push kernel address space references to the page in the cache
	(any cache, L1, L2, BCACHE, etc.).  It should be back flushed
	to ram if necessary on non-coherent copy-back style caches
	because we want the user to see the most up to data copy
	in his mapping.

In general the first is used when we tear down or change userspace
mappings of a page and the latter is done when we are making the page
visible to user space.

As per virtual cache aliasing issues we handle them in other ways,
and it is done on a port-by-port basis since most machines need not
handle this brain damage.

For IPC shared memory some ports enforce an alignment.  For other MMU
activities, the update_mmu_cache method can do things like remap a
page as non-cacheable in all user references if an alias has been
created which is unavoidable.  update_mmu_cache is sort of the "catch
all" area for handling stuff like this, you could use it to work
around the MIPS R4x00 "branch at end of page" hardware bug for example.

Later,
David S. Miller
davem@redhat.com
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: Questions on cache flushing in do_wp_page

[Kanoj Sarcar]

> 
> flush_cache_page()
> 
> 	Remove all "user/tlb view" references to the page in the cache
> 	(any cache, L1, L2, BCACHE, etc.) for this user virtual mapping
> 	in address space mm.  It should be back flushed to ram if
> 	necessary on non-coherent copy-back style caches.
> 
> flush_page_to_ram()
> 
> 	Push kernel address space references to the page in the cache
> 	(any cache, L1, L2, BCACHE, etc.).  It should be back flushed
> 	to ram if necessary on non-coherent copy-back style caches
> 	because we want the user to see the most up to data copy
> 	in his mapping.
> 
> In general the first is used when we tear down or change userspace
> mappings of a page and the latter is done when we are making the page
> visible to user space.
> 
> As per virtual cache aliasing issues we handle them in other ways,
> and it is done on a port-by-port basis since most machines need not
> handle this brain damage.

I am assuming then that in the above, your definition of "non-coherent"
does not include virtual coherency/aliasing issues, since the above
paragraph seems to imply that those issues are handled differently.
Applying the above formalisms to the MIPS processor in do_wp_page,
I still can't see why a cache wbinv would be done by the
flush_page_to_ram(old_page); And if I can not use the argument of
cache aliasing, I am at a complete loss to explain either of
flush_page_to_ram(new_page); and flush_cache_page(vma, address);
doing cache wbinv on the MIPS.

You do mention in the general case where the primitives need to be
invoked, except I still don't understand which processors can define
the primitives as no-ops (Intel) and which should do some real work
(like the MIPS seems to be doing). Is there some way to figure out
how a given processor/architecture needs to define these routines?

Thanks.

Kanoj

> 
> For IPC shared memory some ports enforce an alignment.  For other MMU
> activities, the update_mmu_cache method can do things like remap a
> page as non-cacheable in all user references if an alias has been
> created which is unavoidable.  update_mmu_cache is sort of the "catch
> all" area for handling stuff like this, you could use it to work
> around the MIPS R4x00 "branch at end of page" hardware bug for example.
> 
> Later,
> David S. Miller
> davem@redhat.com
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: Questions on cache flushing in do_wp_page

[David S. Miller]

   > As per virtual cache aliasing issues we handle them in other ways,
   > and it is done on a port-by-port basis since most machines need not
   > handle this brain damage.

   I am assuming then that in the above, your definition of "non-coherent"
   does not include virtual coherency/aliasing issues, since the above
   paragraph seems to imply that those issues are handled differently.

Yes, and I will say it again, broken caches like this are handled
by update_mmu_cache(), nowhere else.

It writes data back into the coherency space if necessary, that is all
these two things do, nothing more.  They do not remap pages, they do
not resolve alias conflicts, they are not meant to.

flush_cache_page(vma, page) is meant to also take care of the case
where for some reason the TLB entry must exist for the cache entry to
be valid as well.  This is the case on the HyperSparc's combined I/D
L2 cache (it has no L1 cache), you cannot flush out cache entries
which have no translation, it will make the cpu trap.  Sparc/sun4c's
mmu is like this too.

   Applying the above formalisms to the MIPS processor in do_wp_page,
   I still can't see why a cache wbinv would be done by the
   flush_page_to_ram(old_page); And if I can not use the argument of
   cache aliasing, I am at a complete loss to explain either of
   flush_page_to_ram(new_page); and flush_cache_page(vma, address);
   doing cache wbinv on the MIPS.

Point to note, the MIPS code in the kernel tree is ages out of date
and is by no means a good reference.  The Sparc code is up to date so
you can check what we do there, the 32-bit port has to deal with all
of these issues in various cache/mmu combinations.  Happily sparc64 is
several orders of magnitude easier and nops most of it out.

   You do mention in the general case where the primitives need to be
   invoked, except I still don't understand which processors can define
   the primitives as no-ops (Intel) and which should do some real work
   (like the MIPS seems to be doing). Is there some way to figure out
   how a given processor/architecture needs to define these routines?

The kernel implicitly aliases with userspace on "alias problematic"
caches, this is why flush_page_to_ram() is a seperate primitive.

View it this way, if physical page X is mapped in userspace at address
Y, and you write bytes into the page in the kernel at the kernel
mapping, and the user will not immediately see this data if he were to
just then read it at Y, you need flush_page_to_ram() to do something.

Later,
David S. Miller
davem@redhat.com
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/