pgd/pmd/pte and x86 kernel virtual addresses

pgd/pmd/pte and x86 kernel virtual addresses

[Timur Tabi]

On x86, when running the kernel, all memory is mapped with a simple offset. 
The virtual address is merely an offset from the physical address.

Does that mean that the pgd/pmd/pte tables are still used?  Basically, what I'm
trying to do is find the pte for a given physical page.  That is, I'm looking
for a function that looks like this:

pte_t *find_pte(mem_map_t *mm);

Given a pointer to a mem_map_t, it returns the pointer to the pte_t for that
physical page.  Is there such a function?  I've found things like this:

#define pte_offset(dir, address) ((pte_t *) pmd_page(*(dir)) + \
			__pte_offset(address))

but what value do I use for "dir"?




--
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please don't cc: me, because then I'll just get two copies of the same message.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: pgd/pmd/pte and x86 kernel virtual addresses

[Benjamin C.R. LaHaise]

On Thu, 24 Aug 2000, Timur Tabi wrote:

> On x86, when running the kernel, all memory is mapped with a simple offset. 
> The virtual address is merely an offset from the physical address.

> Does that mean that the pgd/pmd/pte tables are still used?

x86 only uses two level page tables, so effectively only the pgd and pte
are used.  Unlike some CPUs like sparc, all virtual mappings do indeed
have entries in the pgd (but not always in the ptes).

>  Basically, what I'm
> trying to do is find the pte for a given physical page.  That is, I'm looking
> for a function that looks like this:
> 
> pte_t *find_pte(mem_map_t *mm);
> 
> Given a pointer to a mem_map_t, it returns the pointer to the pte_t for that
> physical page.  Is there such a function?  I've found things like this:

There is no such function, and there cannot be for kernel addresses
since on most x86s, the kernel makes use of 4MB pages to map chunks of
memory.  If you're looking for the user addresses associated with a
physical page, there are several ways of doing so, but none of them are
implemented in the current kernel.

Why do you need this/what are you trying to do?

		-ben

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: pgd/pmd/pte and x86 kernel virtual addresses

[Timur Tabi]

** Reply to message from "Benjamin C.R. LaHaise" <blah@kvack.org> on Thu, 24
Aug 2000 19:43:41 -0400 (EDT)


> >  Basically, what I'm
> > trying to do is find the pte for a given physical page.  That is, I'm looking
> > for a function that looks like this:
> > 
> > pte_t *find_pte(mem_map_t *mm);
> > 
> > Given a pointer to a mem_map_t, it returns the pointer to the pte_t for that
> > physical page.  Is there such a function?  I've found things like this:
> 
> There is no such function, and there cannot be for kernel addresses
> since on most x86s, the kernel makes use of 4MB pages to map chunks of
> memory.  If you're looking for the user addresses associated with a
> physical page, there are several ways of doing so, but none of them are
> implemented in the current kernel.

I thought that memory that's not allocated by a user process (i.e. allocated by
a driver that calls get_free_pages) doesn't have a user address.  Is that wrong?

> 
> Why do you need this/what are you trying to do?

What I'm trying to do is allocate some memory via get_free_pages, and then mark
that memory as uncacheable.


--
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please don't cc: me, because then I'll just get two copies of the same message.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: pgd/pmd/pte and x86 kernel virtual addresses

[Benjamin C.R. LaHaise]

On Fri, 25 Aug 2000, Timur Tabi wrote:

> What I'm trying to do is allocate some memory via get_free_pages, and then mark
> that memory as uncacheable.

ioremap_nocache should be able to do what you want.

		-ben

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: pgd/pmd/pte and x86 kernel virtual addresses

[Timur Tabi]

** Reply to message from "Benjamin C.R. LaHaise" <blah@kvack.org> on Fri, 25
Aug 2000 12:45:18 -0400 (EDT)


> > What I'm trying to do is allocate some memory via get_free_pages, and then mark
> > that memory as uncacheable.
> 
> ioremap_nocache should be able to do what you want.

Well, that's what I tried to explain in my previous email which people seem to
be ignoring.

I tried ioremap_nocache, and it doesn't appear to be working.  There are a
number of problems:

1) I'm trying to mark regular RAM as uncacheable, and ioremap_nocache()
requires me to munge the PG_Reservered bit for each page before I can do that. 
Ugly.

2) ioremap_nocache() allocates virtual RAM.  I already have a virtual address,
I don't need another one.

3) The unmap function for ioremap_nocache() is a no-op.  So after I remap and
mark the page as uncacheable, there's no way to restore it after I'm done!

4) Even with all this, it appears that the function isn't working.  I've
attached a logical analyzer to the memory bus, and writes are not being sent
out, leading me to believe the memory is still being cached.



--
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please don't cc: me, because then I'll just get two copies of the same message.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: pgd/pmd/pte and x86 kernel virtual addresses

[Benjamin C.R. LaHaise]

On Fri, 25 Aug 2000, Timur Tabi wrote:

> ** Reply to message from "Benjamin C.R. LaHaise" <blah@kvack.org> on Fri, 25
> Aug 2000 12:45:18 -0400 (EDT)
> 
> 
> > > What I'm trying to do is allocate some memory via get_free_pages, and then mark
> > > that memory as uncacheable.
> > 
> > ioremap_nocache should be able to do what you want.
> 
> Well, that's what I tried to explain in my previous email which people seem to
> be ignoring.
> 
> I tried ioremap_nocache, and it doesn't appear to be working.  There are a
> number of problems:
> 
> 1) I'm trying to mark regular RAM as uncacheable, and ioremap_nocache()
> requires me to munge the PG_Reservered bit for each page before I can do that. 
> Ugly.

Yeap, that's right.

> 2) ioremap_nocache() allocates virtual RAM.  I already have a virtual address,
> I don't need another one.

That's because there are no ptes for most RAM in the kernel.
ioremap_nocache does not allocate RAM, only a mapping for the address
space.  Actually, passing in physical RAM to ioremap_nocache may not work
on all platforms.

> 3) The unmap function for ioremap_nocache() is a no-op.  So after I remap and
> mark the page as uncacheable, there's no way to restore it after I'm done!

Bummer.

> 4) Even with all this, it appears that the function isn't working.  I've
> attached a logical analyzer to the memory bus, and writes are not being sent
> out, leading me to believe the memory is still being cached.

On x86, you're better off setting the MTRRs to get non-cached behaviour,
but that's still the wrong thing to do when you're talking about main
memory.  Better still is to not rely on uncachable mappings at all.  x86
is a cache coherent architechure -- why do you need uncachable mappings of
main memory???. 

		-ben

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: pgd/pmd/pte and x86 kernel virtual addresses

[Timur Tabi]

** Reply to message from "Benjamin C.R. LaHaise" <blah@kvack.org> on Fri, 25
Aug 2000 12:59:19 -0400 (EDT)


> > 2) ioremap_nocache() allocates virtual RAM.  I already have a virtual address,
> > I don't need another one.
> 
> That's because there are no ptes for most RAM in the kernel.
> ioremap_nocache does not allocate RAM, only a mapping for the address
> space.  Actually, passing in physical RAM to ioremap_nocache may not work
> on all platforms.

It only needs to work on one: x86

If I use ioremap_nocache(), I effectively have two virtual pointers to the same
physical pointer.  The first is the normal virtual pointer for kernel memory,
and the second is the one returned by ioremap_nocache().  I was under the
understanding that caching is enabled on physical pages only, so it shouldn't
matter which virtual address I use.  Is that correct?

> > 4) Even with all this, it appears that the function isn't working.  I've
> > attached a logical analyzer to the memory bus, and writes are not being sent
> > out, leading me to believe the memory is still being cached.
> 
> On x86, you're better off setting the MTRRs to get non-cached behaviour,
> but that's still the wrong thing to do when you're talking about main
> memory.  Better still is to not rely on uncachable mappings at all.  x86
> is a cache coherent architechure -- why do you need uncachable mappings of
> main memory???.

MTRR's are not an option, because chances are we won't have any free MTRR's to
work with.  Besides, I can do what I want on Windows 2000 without MTRR's.  My
driver is for a device which sits on the memory bus itself and responds to
memory reads/writes.  If I can't disable caching, I can't talk to the device.

The odd thing is that ioremap_nocache() did work at one point, but not any
more, and I can't figure out why.



--
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please don't cc: me, because then I'll just get two copies of the same message.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: pgd/pmd/pte and x86 kernel virtual addresses

[Benjamin C.R. LaHaise]

On Fri, 25 Aug 2000, Timur Tabi wrote:

> If I use ioremap_nocache(), I effectively have two virtual pointers to the same
> physical pointer.  The first is the normal virtual pointer for kernel memory,
> and the second is the one returned by ioremap_nocache().  I was under the
> understanding that caching is enabled on physical pages only, so it shouldn't
> matter which virtual address I use.  Is that correct?

No.  Depending on which virtual address you use, you will get different
behaviour (cached vs not).

> MTRR's are not an option, because chances are we won't have any free MTRR's to
> work with.  Besides, I can do what I want on Windows 2000 without MTRR's.  My
> driver is for a device which sits on the memory bus itself and responds to
> memory reads/writes.  If I can't disable caching, I can't talk to the device.

Ummm, then why is it in the range of normally cachable memory?  On Pentium
class machines there is a signal which indicates if a given memory access
is cachable/not.  On P6/later K6s/K7s you must use the MTRRs.

> The odd thing is that ioremap_nocache() did work at one point, but not any
> more, and I can't figure out why.

Technically ioremap should only be used on io addresses.  What in
particular is not working -- is the mapping incorrect, or is the mapping
being cached?  If the mapping is still being cached from previous
accesses, you will need to flush the CPU's cache of any stale cache lines.

		-ben

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: pgd/pmd/pte and x86 kernel virtual addresses

[Stephen C. Tweedie]

Hi,

On Fri, Aug 25, 2000 at 01:46:33PM -0500, Timur Tabi wrote:

> physical pointer.  The first is the normal virtual pointer for kernel memory,
> and the second is the one returned by ioremap_nocache().  I was under the
> understanding that caching is enabled on physical pages only, so it shouldn't
> matter which virtual address I use.  Is that correct?

No.  The no-cache bit is set in the page table entry, so depends on
the virtual address.  There is a *different* form of memory access
control which can be used to make memory non-cachable, and that is the
"mtrr" (Memory Type Range Register), which exists in different forms
on all recent Intel and AMD cpus.  Mtrrs work on physical addresses,
but that is not what ioremap_nocache() uses.

--Stephen

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: pgd/pmd/pte and x86 kernel virtual addresses

[Timur Tabi]

** Reply to message from "Stephen C. Tweedie" <sct@redhat.com> on Tue, 29 Aug
2000 00:09:35 +0100


> > physical pointer.  The first is the normal virtual pointer for kernel memory,
> > and the second is the one returned by ioremap_nocache().  I was under the
> > understanding that caching is enabled on physical pages only, so it shouldn't
> > matter which virtual address I use.  Is that correct?
> 
> No.  The no-cache bit is set in the page table entry, so depends on
> the virtual address.  There is a *different* form of memory access
> control which can be used to make memory non-cachable, and that is the
> "mtrr" (Memory Type Range Register), which exists in different forms
> on all recent Intel and AMD cpus.  Mtrrs work on physical addresses,
> but that is not what ioremap_nocache() uses.

Ok, thanks!  That explains a whole lot.

Another question: is there a way I can enable Write Combining on a page,
without using MTRR's?



-- 
Timur Tabi - ttabi@interactivesi.com
Interactive Silicon - http://www.interactivesi.com

When replying to a mailing-list message, please don't cc: me, because then I'll just get two copies of the same message.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/