Page Attribute Table (PAT) support?

Page Attribute Table (PAT) support?

[Timur Tabi]

The Page Attribute Table (PAT) is an extension to the x86 page table format
that lets you enable Write Combining on a per-page basis.  Details can be found
in chapter 9.13 of the Intel Architecture Software Developer's Manual, Volume 3
(System Programming).

I noticed that 2.4 doesn't support the Page Attribute Table, despite the fact
that it has a X86_FEATURE_PAT macro in processor.h.  Are there any plans to add
this support?  Ideally, I'd like it to be as a parameter for ioremap.


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: Page Attribute Table (PAT) support?

[Jeff Hartmann]

Timur Tabi wrote:

> The Page Attribute Table (PAT) is an extension to the x86 page table format
> that lets you enable Write Combining on a per-page basis.  Details can be found
> in chapter 9.13 of the Intel Architecture Software Developer's Manual, Volume 3
> (System Programming).
> 
> I noticed that 2.4 doesn't support the Page Attribute Table, despite the fact
> that it has a X86_FEATURE_PAT macro in processor.h.  Are there any plans to add
> this support?  Ideally, I'd like it to be as a parameter for ioremap.

I'm actually writing support for the PAT as we speak.  I already have 
working code for PAT setup.  Just having a parameter for ioremap is not 
enough, unfortunately.  According to the Intel Architecture Software 
Developer's Manual we have to remove all mappings of the page that are 
cached.  Only then can they be mapped with per page write combining.  I 
should have working code by the 2.5.x timeframe.  I can also discuss the 
planned interface if anyone is interested.

-Jeff

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: Page Attribute Table (PAT) support?

[Timur Tabi]

** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Wed, 24 Jan
2001 11:45:43 -0700


> I'm actually writing support for the PAT as we speak.  I already have 
> working code for PAT setup.  Just having a parameter for ioremap is not 
> enough, unfortunately.  According to the Intel Architecture Software 
> Developer's Manual we have to remove all mappings of the page that are 
> cached.  Only then can they be mapped with per page write combining.  I 
> should have working code by the 2.5.x timeframe.  I can also discuss the 
> planned interface if anyone is interested.

I'm interested.  Would it be possible to port this support to 2.2, or would
that be too much work?


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: Page Attribute Table (PAT) support?

[Timur Tabi]

** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Wed, 24 Jan
2001 11:45:43 -0700


> I'm actually writing support for the PAT as we speak.  I already have 
> working code for PAT setup.  Just having a parameter for ioremap is not 
> enough, unfortunately.  According to the Intel Architecture Software 
> Developer's Manual we have to remove all mappings of the page that are 
> cached.

For our specific purposes, that's not important.  We already flush the cache
before we create uncached regions (via ioremap_nocache).  I understand that as a
general Linux feature, you can't ignore cache incoherency, but I don't think
it's a hard requirement.


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: Page Attribute Table (PAT) support?

[Timur Tabi]

** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Wed, 24 Jan
2001 11:45:43 -0700


> I'm actually writing support for the PAT as we speak.  I already have 
> working code for PAT setup.  Just having a parameter for ioremap is not 
> enough, unfortunately.  According to the Intel Architecture Software 
> Developer's Manual we have to remove all mappings of the page that are 
> cached.

For our specific purposes, that's not important.  We already flush the cache
before we create uncached regions (via ioremap_nocache).  I understand that as a
general Linux feature, you can't ignore cache incoherency, but I don't think
it's a hard requirement.


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: Page Attribute Table (PAT) support?

[Jeff Hartmann]

Timur Tabi wrote:

> ** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Wed, 24 Jan
> 2001 11:45:43 -0700
> 
> 
> 
>> I'm actually writing support for the PAT as we speak.  I already have 
>> working code for PAT setup.  Just having a parameter for ioremap is not 
>> enough, unfortunately.  According to the Intel Architecture Software 
>> Developer's Manual we have to remove all mappings of the page that are 
>> cached.
> 
> 
> For our specific purposes, that's not important.  We already flush the cache
> before we create uncached regions (via ioremap_nocache).  I understand that as a
> general Linux feature, you can't ignore cache incoherency, but I don't think
> it's a hard requirement.

Actually you can't ignore it or the processor will have a heart attack 
if the cached page mapping is used even speculatively.  I've done some 
experimenting, if the page is mapped cached in one place, and UCWC in 
another, things will not work.  Its extremely likely the processor will 
cease to function.  Its not like having cached and uncached mappings of 
a page (which does work on the Intel processors, we use that feature in 
the agpgart and the DRM in fact.)  When you mark a page UCWC, you better 
have removed all cached mappings or your asking for REAL trouble.

-Jeff

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: Page Attribute Table (PAT) support?

[Timur Tabi]

** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Wed, 24 Jan
2001 13:41:41 -0700

> When you mark a page UCWC, you better 
> have removed all cached mappings or your asking for REAL trouble.

What exactly do you mean by "removed all cached mappings"?  Does that mean that
if one virtual address is a UCWC mapping of a physical page, then ALL virtual
addresses mapped to that page must also be UCWC?

In my driver, I use ioremap_nocache() on physical memory (real RAM) to create
an uncached "alias" (a virtual address) to a physical page of RAM.  When I
access the memory via this virtual address, the memory access is not cached.
What I reall need is to be able to also have that virtual address as Write
Combined.

Since all physical RAM is mapped as cached via the kernel (on a 1-to-1 basis),
and since there can be several other virtual addresses that point to that memory
(e.g. user virtual address), I can't see how these virtual addresses can be
removed.


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: Page Attribute Table (PAT) support?

[Jeff Hartmann]

Timur Tabi wrote:

> ** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Wed, 24 Jan
> 2001 13:41:41 -0700
> 
> 
>> When you mark a page UCWC, you better 
>> have removed all cached mappings or your asking for REAL trouble.
> 
> 
> What exactly do you mean by "removed all cached mappings"?  Does that mean that
> if one virtual address is a UCWC mapping of a physical page, then ALL virtual
> addresses mapped to that page must also be UCWC?
> 
> In my driver, I use ioremap_nocache() on physical memory (real RAM) to create
> an uncached "alias" (a virtual address) to a physical page of RAM.  When I
> access the memory via this virtual address, the memory access is not cached.
> What I reall need is to be able to also have that virtual address as Write
> Combined.

	Pages with multiple mappings aren't really supported by the Intel ia32 
architecture.  The trick you do above works, but is strongly discouraged 
by the Intel documentation.  The documentation says that if you do this 
with UCWC memory, it won't work (and will result in undefined processor 
behavior.)  My experiments with the PAT seem to agree with the 
documentation.

> 
> Since all physical RAM is mapped as cached via the kernel (on a 1-to-1 basis),
> and since there can be several other virtual addresses that point to that memory
> (e.g. user virtual address), I can't see how these virtual addresses can be
> removed.

	We have to remove the kernel page table mappings.  (Convert the 4MB pages 
to individual pte's, then change the individual pte so its pgprot value 
is correct.)

	When you allocate memory in the kernel, you OWN it.  Its not mapped into 
a user space process unless you put it there.  I have to point out that 
this interface requires the user to insure these pages are never ever 
swapped, or mapped cached.  This isn't a huge restriction, since we 
aren't providing a user land interface.  If we try to handle all/some of 
these cases in the kernel, it becomes a very large problem.  We would 
have to add arch specific bits for special cache handling, Do smp cache 
flushes all over the place, etc.  Its really not worth it.

-Jeff

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: Page Attribute Table (PAT) support?

[Timur Tabi]

** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Wed, 24 Jan
2001 14:15:32 -0700


> Pages with multiple mappings aren't really supported by the Intel ia32 
> architecture.  The trick you do above works, but is strongly discouraged 
> by the Intel documentation.  The documentation says that if you do this 
> with UCWC memory, it won't work (and will result in undefined processor 
> behavior.)  My experiments with the PAT seem to agree with the 
> documentation.

Is it still a problem even we never access the memory through the non-UCWC
memory? 

> 	We have to remove the kernel page table mappings.  (Convert the 4MB pages 
> to individual pte's, then change the individual pte so its pgprot value 
> is correct.)

That sounds like an incredible amount of work!  Are you going to support this
in your code?  I wouldn't even know how to begin implementing that.

> 	When you allocate memory in the kernel, you OWN it.  Its not mapped into 
> a user space process unless you put it there.  I have to point out that 
> this interface requires the user to insure these pages are never ever 
> swapped, or mapped cached.  This isn't a huge restriction, since we 
> aren't providing a user land interface.  If we try to handle all/some of 
> these cases in the kernel, it becomes a very large problem.  We would 
> have to add arch specific bits for special cache handling, Do smp cache 
> flushes all over the place, etc.  Its really not worth it.

Ok, so what are you going to provide?  All I need is a method to access a
particular physical page, whether it's part of real RAM or not, in an UCWC
manner.  Can you give me a hint as to how that cability can be provided?


-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please direct the reply to the mailing list only.  Don't send another copy to me.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: Page Attribute Table (PAT) support?

[Jeff Hartmann]

Timur Tabi wrote:

> ** Reply to message from Jeff Hartmann <jhartmann@valinux.com> on Wed, 24 Jan
> 2001 11:45:43 -0700
> 
> 
> 
>> I'm actually writing support for the PAT as we speak.  I already have 
>> working code for PAT setup.  Just having a parameter for ioremap is not 
>> enough, unfortunately.  According to the Intel Architecture Software 
>> Developer's Manual we have to remove all mappings of the page that are 
>> cached.  Only then can they be mapped with per page write combining.  I 
>> should have working code by the 2.5.x timeframe.  I can also discuss the 
>> planned interface if anyone is interested.
> 
> 
> I'm interested.  Would it be possible to port this support to 2.2, or would
> that be too much work?

Its most likely that it will be 2.5.x only for awhile.  I'm not sure at 
this point if it will make it into 2.4.x, much less 2.2.x.

Basically the high level interface will be a several allocation routines 
which handle the cache/remapping issues for you.  They will allocate 
groups of pages at once (to minimize smp cache flush issues.)

Let me give fair warning this is rough writeup of provided interfaces. 
I'm still not completely done with the design, and it might go through 
several iterations before I'm done with it.  There is no plan to provide 
a generic user land interface for these functions.

Here is some function prototypes:

extern int pat_allow_page_write_combine(void);

	This function is called by someone wanting to use the write combine 
allocation routines.  If it returns 1, per page write combining is 
available.


extern struct page_list *pat_wrtcomb_alloc_page_list(int gfp_mask,
	unsigned long order,
	int num_pages);

	An allocation routine which allocates a group of write combined pages.

extern struct virtual_page_list *pat_wrtcomb_vmalloc_page_list(
	int gfp_mask,
	int num_pages);

	An allocation routine which allocates a group of write combined pages and 
maps them contiguously in kernel virtual memory.

extern void pat_wrtcomb_free_page_list(struct page_list *list);
extern void pat_wrtcomb_vfree_page_list(struct virtual_page_list *list);
	The corresponding free routines.


The following functions would be provided for changing a 4mb mapping to 
the individual pte's, and vice versa.


extern int convert_4mb_page_to_individual(unsigned long address);
	This function would test to see if the page address given is mapped 
through a 4mb page in the kernel page tables.  If it is, ever address 
described by this 4mb page will get converted to individual pte entries.

extern int convert_individual_to_4mb_page(unsigned long address);
	This does the opposite of the above function when all pages in a range 
have the same page protection.

These functions are in turn supported by the following page list 
routines, this is the only part which would be arch-independant:
struct page_list {
         unsigned long   *pages;
         int             num_pages;
         unsigned long   order;
};

struct virtual_page_list {
         struct page_list        *list;
         void                    *addr;
};

extern struct page_list *allocate_page_list(int gfp_mask,
                                             unsigned long order,
                                             int num_pages);

extern struct virtual_page_list *vmalloc_page_list(unsigned long size,
                                                    int gfp_mask,
                                                    pgprot_t prot);

	These functions allocate a page_list or a page_list mapped into kernel 
virtual memory.

extern void free_page_list(struct page_list *list);
extern void vfree_page_list(struct virtual_page_list *list);
	The corresponding free routines.



-Jeff

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/