About reading /proc/*/mem

About reading /proc/*/mem

[Richard F Weber]

Ok, so as a rehash, the ptrace & open(),lseek() on /proc/*/mem should 
both work about the same.  After a lot of struggling, I've gotten the 
ptrace to work right & spit out the data I want/need.  However there is 
one small problem, SIGSTOP.

ptrace() appears to set up the child process to do a SIGSTOP whenever 
any interrupt is received.  Which is kind of a bad thing for what I'm 
looking to do.  I guess I'm trying to write a non-intrusive debugger 
that can be used to view static variables stored in the heap of an 
application.

On other OS's, this can be done just by popping open /proc/*/mem, and 
reading the data as needed, allowing the child process to continue 
processing away as if nothing is going on.  I'm looking to do the same 
sort of task under Linux. 

Unfortunately, ptrace() probobally isn't going to allow me to do that.  
So my next question is does opening /proc/*/mem force the child process 
to stop on every interrupt (just like ptrace?)

Second, I would imagine opening /dev/mem (or /proc/kcore) would get me 
into the physical memory of the system itself.  How would I know what 
the starting physical memory addresses of a processes data is to start at:

What I do see is under /proc/*/maps is the entries:
08048000-08049000 r-xp 00000000 03:01 718368     /devel/mem_probe/child
08049000-0804a000 rw-p 00000000 03:01 718368     /devel/mem_probe/child
40000000-40015000 r-xp 00000000 03:01 310089     /lib/ld-2.2.so
40015000-40016000 rw-p 00014000 03:01 310089     /lib/ld-2.2.so
40016000-40017000 rwxp 00000000 00:00 0
40017000-40018000 rw-p 00000000 00:00 0
40027000-4013f000 r-xp 00000000 03:01 310096     /lib/libc-2.2.so
4013f000-40144000 rw-p 00117000 03:01 310096     /lib/libc-2.2.so
40144000-40148000 rw-p 00000000 00:00 0
bfffe000-c0000000 rwxp fffff000 00:00 0


I would assume that this tells me that memory addresses 
0x08049000-0x804a000 are mapped to the physical address of 0x718368.  
However Going to this address, and then doing an lseek(SEEK_CUR)out to 
my expected variable offset doesn't get me the result I'm expecting.  Is 
the 0x718368 the right location to be looking at, or is there some 
translation that needs to get done (* by page size, translate into 
hex/from hex, etc.) I can't find any documentation indicating what each 
column represents so it's just a stab on my part.

Thanks for the good information so far.

--Rich

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: About reading /proc/*/mem

[Eric W. Biederman]

"Richard F Weber" <rfweber@link.com> writes:

> Ok, so as a rehash, the ptrace & open(),lseek() on /proc/*/mem should 
> both work about the same.  After a lot of struggling, I've gotten the 
> ptrace to work right & spit out the data I want/need.  However there is 
> one small problem, SIGSTOP.
> 
> ptrace() appears to set up the child process to do a SIGSTOP whenever 
> any interrupt is received.  Which is kind of a bad thing for what I'm 
> looking to do.  I guess I'm trying to write a non-intrusive debugger 
> that can be used to view static variables stored in the heap of an 
> application.
> 
> On other OS's, this can be done just by popping open /proc/*/mem, and 
> reading the data as needed, allowing the child process to continue 
> processing away as if nothing is going on.  I'm looking to do the same 
> sort of task under Linux. 
> 
> Unfortunately, ptrace() probobally isn't going to allow me to do that.  
> So my next question is does opening /proc/*/mem force the child process 
> to stop on every interrupt (just like ptrace?)


The not stopping the child should be the major difference between
/proc/*/mem and ptrace.


> Second, I would imagine opening /dev/mem (or /proc/kcore) would get me 
> into the physical memory of the system itself.  How would I know what 
> the starting physical memory addresses of a processes data is to start at:

You don't even want to go there.  You've got the wrong model in your head.

> 0x08049000-0x804a000 are mapped to the physical address of 0x718368.  
Nope 0x718368 is the inode of an on-disk file.

> However Going to this address, and then doing an lseek(SEEK_CUR)out to 
> my expected variable offset doesn't get me the result I'm expecting.  Is 
> the 0x718368 the right location to be looking at, or is there some 
> translation that needs to get done (* by page size, translate into 
> hex/from hex, etc.) I can't find any documentation indicating what each 
> column represents so it's just a stab on my part.

man proc or reading the source works.

> Thanks for the good information so far.
> 
> --Rich
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux.eu.org/Linux-MM/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: About reading /proc/*/mem

[Alexander Viro]

On 1 May 2001, Eric W. Biederman wrote:

> > Unfortunately, ptrace() probobally isn't going to allow me to do that.  
> > So my next question is does opening /proc/*/mem force the child process 
> > to stop on every interrupt (just like ptrace?)
> 
> 
> The not stopping the child should be the major difference between
> /proc/*/mem and ptrace.

Could somebody tell me what would one do with data read from memory
of process that is currently running?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: About reading /proc/*/mem

[Richard F Weber]

[-- Attachment #1: Type: text/plain, Size: 1933 bytes --]

The main thing I'm looking to do is examine data that's part of a 
real-time process.  The process's execution can't be interrupted, 
otherwise it makes debugging it inaccurate.  The applications is 
certainly not looking to see every line of code get executed, but rather 
have a real-time monitor of a symbol as it gets modified.  Now 
viewing/selecting the symbol is done through a combination of nm's & a 
console based util (hopefully GTK Based in the future).  Other 
applications include recording this data to disk for later playback & 
analysis.

Now the next logical step would be to create a debug module in the RT 
system itself that dumps out the values we care about.  The problem with 
this is we are looking at a lot of legacy code (done in fortran, C & 
Ada) as well as tons of variables.  By peeking at the memory on the fly 
we can dynamically decide which values are important for this run, 
without having to record all possible data to the disk (which in itself 
would be quite painful since disk accesses would make debugging again 
difficult).

Granted, it's probobally not a very popular application, but it's still 
one which is present in many of the Big Unixes, and so far has me 
stumped as to how to get it working correctly under Linux.

--Rich

Alexander Viro wrote:

>On 1 May 2001, Eric W. Biederman wrote:
>
>>>Unfortunately, ptrace() probobally isn't going to allow me to do that.  
>>>So my next question is does opening /proc/*/mem force the child process 
>>>to stop on every interrupt (just like ptrace?)
>>>
>>
>>The not stopping the child should be the major difference between
>>/proc/*/mem and ptrace.
>>
>
>Could somebody tell me what would one do with data read from memory
>of process that is currently running?
>
>--
>To unsubscribe, send a message with 'unsubscribe linux-mm' in
>the body to majordomo@kvack.org.  For more info on Linux MM,
>see: http://www.linux.eu.org/Linux-MM/
>


[-- Attachment #2: Type: text/html, Size: 2489 bytes --]

Re: About reading /proc/*/mem

[Alexander Viro]

On Tue, 1 May 2001, Richard F Weber wrote:

> The main thing I'm looking to do is examine data that's part of a 
> real-time process.  The process's execution can't be interrupted, 
> otherwise it makes debugging it inaccurate.  The applications is 
> certainly not looking to see every line of code get executed, but rather 
> have a real-time monitor of a symbol as it gets modified.  Now 
> viewing/selecting the symbol is done through a combination of nm's & a 
> console based util (hopefully GTK Based in the future).  Other 
> applications include recording this data to disk for later playback & 
> analysis.
> 
> Now the next logical step would be to create a debug module in the RT 
> system itself that dumps out the values we care about.  The problem with 
> this is we are looking at a lot of legacy code (done in fortran, C & 
> Ada) as well as tons of variables.  By peeking at the memory on the fly 
> we can dynamically decide which values are important for this run, 
> without having to record all possible data to the disk (which in itself 
> would be quite painful since disk accesses would make debugging again 
> difficult).

You want the data in each frame be taken at the same moment. Otherwise
you are going to see inconsistent data. I.e. tons of false warnings saying
that data corruption is going on when there's none.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: About reading /proc/*/mem

[Stephen C. Tweedie]

Hi,

On Tue, May 01, 2001 at 12:35:29PM -0400, Alexander Viro wrote:
> 
> On 1 May 2001, Eric W. Biederman wrote:
> 
> > > Unfortunately, ptrace() probobally isn't going to allow me to do that.  
> > > So my next question is does opening /proc/*/mem force the child process 
> > > to stop on every interrupt (just like ptrace?)
> > 
> > 
> > The not stopping the child should be the major difference between
> > /proc/*/mem and ptrace.
> 
> Could somebody tell me what would one do with data read from memory
> of process that is currently running?

As long as we have the appropriate page table lock while doing the
physical page lookup, and grab a refcount on the page with the lock
held, we'll get a valid physical memory location to read to the user.
We don't need any stronger guarantee than that --- if the target
process is playing mmap games or modifying the memory while the read
happens, the result is unpredictable but safe.

Cheers,
 Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: About reading /proc/*/mem

[Richard F Weber]

[-- Attachment #1: Type: text/plain, Size: 2042 bytes --]



Alexander Viro wrote:

>
>On 1 May 2001, Eric W. Biederman wrote:
>
>>>Unfortunately, ptrace() probobally isn't going to allow me to do that.  
>>>So my next question is does opening /proc/*/mem force the child process 
>>>to stop on every interrupt (just like ptrace?)
>>>
>>
>>The not stopping the child should be the major difference between
>>/proc/*/mem and ptrace.
>>
>
>Could somebody tell me what would one do with data read from memory
>of process that is currently running?
>
After doing some digging around, I found this URL:   (Sorry, the 
original page at nsa seems to have disappeared).

http://www.google.com/search?q=cache:nsa.gov/selinux/slinux-200101020953/node57.html+access+physical+memory+/proc+/mem&hl=en

In any case, it indicates the the /proc/*/mem file can only be read by 
the process itself on the fly, or by a parent process that has stopped 
execution of the child via SIGSTOP.  So it seems so far that the 
behavior of /proc/*/mem is exactly the same as the behavior of ptrace in 
that it forces stoppage of execution in order to read memory.  
Bummer.(Or has this changed from v 2.2.x to v2.4.x?)

So how else can I access the process memory?  I'm wondering if it'd be 
feasible to hack the kernel to add an extra ptrace_nostop() fn that 
would allow ptracing without forcing a stop in the process.  I'd really 
rather not have to hack the kernel unless I really have to though.

My second thought is again going back to finding the translation of 
where the process physically lives in the virtual memory itself.  So 
this way I can just go and directly look at the memory of the process.  
But then this runs into problems as to finding where that process lives, 
and being told my thoughts on this are totally wrong.

I suppose the question is I know the process exists, I know the process 
is pinned into memory so it can't get swapped out, I just need to know 
how to translate the process's virtual address of 0x080495998 to some 
chunk of physical memory, and then take a look at it.

Thanks.

--Rich

[-- Attachment #2: Type: text/html, Size: 2679 bytes --]

Re: About reading /proc/*/mem

[Richard F Weber]

Well, just wanted to send an e-mail to let the group know what I've 
found & what my current workaround is, as well as make sure that it's 
recorded somewhere that will get searchable to save future hackers the 
trouble.

So far the story is I'm trying to get access to another processes memory 
structure for non-intrusive debugging (mainly to just track statically 
allocated variables to see how they change).  The technique used on 
other Operating systems was to simply open /proc/pid/mem, lseek to the 
proper location, and read the value directly from the processes mapped 
memory.

Unfortunately, on Linux this is not the case.  It appears that 
/proc/pid/mem is not available to another process, unless there is a 
ptrace(PTRACE_ATTACH) to bind the two processes together.  However, the 
problem with this method is that ptrace forces the child process to do a 
SIGSTOP whenever any signal is received.  In a real-time system, this 
isn't a good method to debug a process when it's important for the 
process being debugged to run correctly.

This brings up an interesting feature though, if you access 
/proc/self/mem, you can access anywhere in that process.  A few people 
suggested using a debugging process, or debugging routines which would 
again be a bit intrusive for use.  However, if you make a child thread, 
the child thread will get executed independently, but still have full 
access to the native application's memory.  This will basically provide 
the functionality I'm looking for.

So in conclusion, I have a call to 
pthread_create(thread_ptr,NULL,debug_fn,NULL) which is used to create a 
new thread that is designed to run the debug_fn function, which is the 
main routine from the original debug application.

Thanks to everyone for answering my questions.  I think a few people 
suggested I look at threads, but I don't remember who you were.  Thanks 
again for your help.


BTW:  As a future feature suggestion, what about having a 
/proc/pid/nbmem that would provide non-blocking access to a processes 
memory.  I would think it'd be kind of a redundant hack in the kernel to 
allow this, and since it will be a security concern make it an option 
you have to turn on in the kernel for recompilation (of course if it was 
a module that'd be sweeter still).

--Rich


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: About reading /proc/*/mem

[Richard F Weber]

[-- Attachment #1: Type: text/plain, Size: 1919 bytes --]


Eric W. Biederman wrote:

>"Richard F Weber" <rfweber@link.com> writes:
>
>>Ok, so as a rehash, the ptrace & open(),lseek() on /proc/*/mem should 
>>both work about the same.  After a lot of struggling, I've gotten the 
>>ptrace to work right & spit out the data I want/need.  However there is 
>>one small problem, SIGSTOP.
>>
>>ptrace() appears to set up the child process to do a SIGSTOP whenever 
>>any interrupt is received.  Which is kind of a bad thing for what I'm 
>>looking to do.  I guess I'm trying to write a non-intrusive debugger 
>>that can be used to view static variables stored in the heap of an 
>>application.
>>
>>On other OS's, this can be done just by popping open /proc/*/mem, and 
>>reading the data as needed, allowing the child process to continue 
>>processing away as if nothing is going on.  I'm looking to do the same 
>>sort of task under Linux. 
>>
>>Unfortunately, ptrace() probobally isn't going to allow me to do that.  
>>So my next question is does opening /proc/*/mem force the child process 
>>to stop on every interrupt (just like ptrace?)
>>
>
>
>The not stopping the child should be the major difference between
>/proc/*/mem and ptrace.
>
See this is where I start seeming to have problems.  I can open 
/proc/*/mem & lseek, but reads come back as "No such process".  However, 
if I first do a ptrace(PTRACE_ATTACH), then I can read the data, but the 
process stops.  I've kind of dug through the sys_ptrace() code under 
/usr/src/linux/arch/i386/kernel/ptrace.c, and can see and understand 
generally what it's doing, but that's getting into serious kernel-land 
stuff.  I wouldn't expect it to be this difficult to just open up 
another processes /proc/*/mem file to read data from.

Is there something obvious I'm missing?  It seems to keep pointing back 
to ptrace & /proc/*/mem are very closely related (ie: the same) 
including stopping of the child.

Thanks.

--Rich

[-- Attachment #2: Type: text/html, Size: 2339 bytes --]

Re: About reading /proc/*/mem

[Alexander Viro]

On Tue, 1 May 2001, Richard F Weber wrote:

> See this is where I start seeming to have problems.  I can open 
> /proc/*/mem & lseek, but reads come back as "No such process".  However, 
> if I first do a ptrace(PTRACE_ATTACH), then I can read the data, but the 
> process stops.  I've kind of dug through the sys_ptrace() code under 
> /usr/src/linux/arch/i386/kernel/ptrace.c, and can see and understand 
> generally what it's doing, but that's getting into serious kernel-land 
> stuff.  I wouldn't expect it to be this difficult to just open up 
> another processes /proc/*/mem file to read data from.
 
> Is there something obvious I'm missing?  It seems to keep pointing back 
> to ptrace & /proc/*/mem are very closely related (ie: the same) 
> including stopping of the child.

OK, here's something I really don't understand. Suppose that I tell your
debugger to tell me when in the executed program foo becomes greater than
bar[0] + 14. Or when cyclic list foo becomes longer than 1 element
(i.e. foo.next != foo.prev).

How do you do that if program is running? If you don't guarantee that
it doesn't run during the access to its memory (moreover, between sever
such accesses) - the data you get is worthless.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: About reading /proc/*/mem

[Richard F Weber]

[-- Attachment #1: Type: text/plain, Size: 3006 bytes --]

Actually what we are looking to do is a lot more simplistic than that 
what you are suggesting.  The RT applications is the Host portion of a 
flight Sim.  So for example, we want to see what the system thinks it's 
current altitude is at, or what position the throttle is at, etc.  So 
there isn't a lot of dynamic list generation, or pointer manipulation 
going on.  Having dynamic data generation and/or pointers is another 
nasty hole that gets opened up that nobody here is quite ready to step into.

So the majority of the data is static, as well as small enough to be 
atomic.  Mostly 32-bit floats & ints with occaisional calls to 64-bit 
floats & ints. 

Don't forget too, that since this is running in a real-time environment, 
that alot of things are expected to be guaranteed.  We know a certain 
process is going to be running at 50 Hz, or 200Hz.  So when it's 
timeslice is complete it really better be done or else we have 
architecture problems.  The child will be getting interrupts & starting, 
running, & finishing 50 times a second, meanwhile, the debug process may 
be running at 1 Hzand updating relatively slowly.  Assuming this is a 
uniprocessor, the child process should be totally done running when the 
debug process goes in to get updates.  On an SMP there will probobally 
be some synchronization problems, but we'll worry about that when we get 
there.

What I'm trying to do is port an application that was working on Suns, 
SGI's & Concurrent PowerHawks over to Linux.  All three used the /proc 
to access the memory, and Linux doesn't seem to offer the same kind of 
support.  Which brings me back to my original question of how to tie 
into the process's memory without interrupting it's execution.

--Rich

Alexander Viro wrote:

>
>On Tue, 1 May 2001, Richard F Weber wrote:
>
>>See this is where I start seeming to have problems.  I can open 
>>/proc/*/mem & lseek, but reads come back as "No such process".  However, 
>>if I first do a ptrace(PTRACE_ATTACH), then I can read the data, but the 
>>process stops.  I've kind of dug through the sys_ptrace() code under 
>>/usr/src/linux/arch/i386/kernel/ptrace.c, and can see and understand 
>>generally what it's doing, but that's getting into serious kernel-land 
>>stuff.  I wouldn't expect it to be this difficult to just open up 
>>another processes /proc/*/mem file to read data from.
>>
>>Is there something obvious I'm missing?  It seems to keep pointing back 
>>to ptrace & /proc/*/mem are very closely related (ie: the same) 
>>including stopping of the child.
>>
>
>OK, here's something I really don't understand. Suppose that I tell your
>debugger to tell me when in the executed program foo becomes greater than
>bar[0] + 14. Or when cyclic list foo becomes longer than 1 element
>(i.e. foo.next != foo.prev).
>
>How do you do that if program is running? If you don't guarantee that
>it doesn't run during the access to its memory (moreover, between sever
>such accesses) - the data you get is worthless.
>


[-- Attachment #2: Type: text/html, Size: 3488 bytes --]