Linux VM Documentation - Draft 1

Linux VM Documentation - Draft 1

[Mel Gorman]

Well, despite numerous setbacks, disasters and various panic-attacks, I've
finally got a first draft together for documentation of the Linux VM. This
is still incomplete but will hopefully still be a valuable resource to
those wishing to understand the VM.

It is based on 2.4.20 as the 2.5.x one still changes too much too
regularly to make documenting it feasible. I do believe though that having
a good understanding of the 2.4.20 VM is 80% of the work to understanding
the 2.5.x one at least. There is a few notable areas not covered yet but
will be over the next month or two but I am releasing this early so I can
start getting feedback and correcting any errors or poor assumptions now
rather than later. The areas are;

o Swap area management   (swap.c, swapfile.c etc)
o High memory management (highmem.c)
o Memory locking         (mlock.c)
o Mem init (May not cover as it's very arch specific and there is docs out
            there on the subject already)
o Shared memory (May not cover this at all as it is really an IPC field)
o Buffer management (Same, except it's of more importance to IO)

The documentation comes in two parts. The first is "Understanding the
Linux Virtual Memory Manager" and it does pretty much as described. It is
available in three formats, PDF, HTML and plain text.

Understanding the Linux Virtual Memory Manager
PDF:  http://www.csn.ul.ie/~mel/projects/vm/guide/pdf/understand.pdf
HTML: http://www.csn.ul.ie/~mel/projects/vm/guide/html/understand/
Text: http://www.csn.ul.ie/~mel/projects/vm/guide/text/understand.txt

The second part is a code commentary which is literally a guided tour
through the code. It is intended to help decipher the more cryptic
sections as well as identify the code patterns that are prevalent through
the code. I decided to have the code separate from the first document as
maintaining the code in the document would be too painful

Code Commentary on the Linux Virtual Memory Manager
PDF:  http://www.csn.ul.ie/~mel/projects/vm/guide/pdf/code.pdf
HTML: http://www.csn.ul.ie/~mel/projects/vm/guide/html/code
Text: http://www.csn.ul.ie/~mel/projects/vm/guide/text/code.txt

Any feedback, comments or suggestions are welcome from anyone with a VM
interest but I would appreciate if people already familiar with the VM
would even give a brief read to check for technical accuracy. There was
rarely an authoritative source to check to make sure I was right and I
didn't want to be asking questions every 5 minutes on IRC or mailing
lists :-)

Enjoy...

-- 
Mel Gorman
MSc Student, University of Limerick
http://www.csn.ul.ie/~mel



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Linux VM Documentation - Draft 1

[Willy Tarreau]

On Sun, Jan 12, 2003 at 02:11:37AM +0000, Mel Gorman wrote:
 
> Any feedback, comments or suggestions are welcome from anyone with a VM

one feedback : Thanks a lot !!! This is invaluable work. I don't have the
skills to tell you if/where you let mistakes, but your documents will help me
(and probably many people) understanding this important kernel part.

Congratulations, really !
Willy

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Linux VM Documentation - Draft 1

[Marcus Alanen]

>Well, despite numerous setbacks, disasters and various panic-attacks, I've
>finally got a first draft together for documentation of the Linux VM. This
>is still incomplete but will hopefully still be a valuable resource to
>those wishing to understand the VM.

Excellent! A lot of people will learn from this, thank you!

>Understanding the Linux Virtual Memory Manager
>PDF:  http://www.csn.ul.ie/~mel/projects/vm/guide/pdf/understand.pdf
>HTML: http://www.csn.ul.ie/~mel/projects/vm/guide/html/understand/
>Text: http://www.csn.ul.ie/~mel/projects/vm/guide/text/understand.txt

Do you take patches? I couldn't find the .tex file.

Regards,
Marcus

-- 
Marcus Alanen
maalanen@abo.fi
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Linux VM Documentation - Draft 1

[Mel Gorman]

On Sun, 12 Jan 2003, Marcus Alanen wrote:

> Excellent! A lot of people will learn from this, thank you!
>

Your welcome!

> Do you take patches? I couldn't find the .tex file.
>

I wasn't sure how suitable patches would be for documentation but I'll try
anything once. A tar ball of the current tex source is at
http://www.csn.ul.ie/~mel/projects/vm/guide/vm_book.tar.gz . There is a
CVS tree but it's on a computer thats already heavily loaded so I don't
want to have it hammered.

The tex sources are in tex/understand and tex/code . To create a DVI,
simply ./make dvi . If you add "understand" or "code", it'll just generate
that book.

-- 
Mel Gorman
MSc Student, University of Limerick
http://www.csn.ul.ie/~mel

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Linux VM Documentation - Draft 1

[Rob Wilkens]

On Sun, 2003-01-12 at 10:36, Mel Gorman wrote:
> I wasn't sure how suitable patches would be for documentation but I'll try
> anything once. A tar ball of the current tex source is at
> http://www.csn.ul.ie/~mel/projects/vm/guide/vm_book.tar.gz . There is a
> CVS tree but it's on a computer thats already heavily loaded so I don't
> want to have it hammered.
> 
> The tex sources are in tex/understand and tex/code . To create a DVI,
> simply ./make dvi . If you add "understand" or "code", it'll just generate
> that book.

Not worthy of re-releasing but one small "patch" is page 16 of the pdf
(labelled as page 9 on the top) which reads:
	"the init code is quiet architecture dependent"
but should probably read
	"the init code is quite architecture dependent"
It's a small thing, but whenever I read a technical document and see
small errors like that, it distracts me from the main point of the
article and makes concentrating on the original document harder.

I don't know how to code in tex (maybe one of these years I should
learn), otherwise I'd offer a tex patch for you to apply.  

I'm sure I'm not the first to catch that error, given the number of
people who are probably interested in the article you wrote, it's quite
a document.  It was also very nice of you to share it with the world
gratis.

-Rob

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Linux VM Documentation - Draft 1

[Mel Gorman]

On Sun, 12 Jan 2003, Rob Wilkens wrote:

> (labelled as page 9 on the top) which reads:
> 	"the init code is quiet architecture dependent"
> but should probably read
> 	"the init code is quite architecture dependent"
> It's a small thing, but whenever I read a technical document and see
> small errors like that

A common complaint. I regret to say that even though English is my first
language, my grammer and spelling leaves a lot to be desired at times so
corrections are welcome. I've searched through both documents and have
cleaned up all bad usage of quiet but I won't rerelease yet until I have a
few significant changes together.

Thanks

-- 
Mel Gorman
MSc Student, University of Limerick
http://www.csn.ul.ie/~mel


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Linux VM Documentation - Draft 1

[andrea.glorioso]

>>>>> "mg" == Mel Gorman <mel@csn.ul.ie> writes:

    mg> I wasn't sure how  suitable patches would be for documentation
    mg> but  I'll  try anything once.  A tar  ball of  the current tex
    mg> source                               is                     at
    mg> http://www.csn.ul.ie/~mel/projects/vm/guide/vm_book.tar.gz
    mg> .  There is a  CVS tree but it's  on  a computer thats already
    mg> heavily loaded so I don't want to have it hammered.

If  the  license of  your work  allows,  you could   open a project on
Sourcerforge,  Berlios  or Savannah  (under non-GNU  projects ;) ) and
import your CVS tree there.  I still have to  read your documents, but
I think this  is a wonderful  project and it's very worth contributing
to it - so any infrastructural help in this sense would make sense.

Thank you for your work, I'm looking forward to having a look to it.

bye,

andrea
--
Andrea Glorioso                   andrea.glorioso@binary-only.com
Binary Only                           http://www.binary-only.com/
Via A. Zanolini, 7/b                  Tel:     +39-348.921.43.79
40126 Bologna                         Fax:     +39-051-930.31.133
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Linux VM Documentation - Draft 1

[Mel Gorman]

On Sun, 12 Jan 2003 andrea.glorioso@binary-only.com wrote:

> If  the  license of  your work  allows,  you could   open a project on
> Sourcerforge,  Berlios  or Savannah  (under non-GNU  projects ;) ) and
> import your CVS tree there.

There is a savannagh project called the Linux Kernel Documentation Project
(LKDP) (http://savannah.nongnu.org/projects/lkdp) set up by Abhishek
Nayani but it has been inactive for some time. I will eventually merge
with it (I have made contributions to it in the past) but am waiting to
get the last chapters finished first. It might be me being awkward but
it's difficult to have a number of people working on one document and
keeping the writing style consistent.

-- 
Mel Gorman
MSc Student, University of Limerick
http://www.csn.ul.ie/~mel


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Linux VM Documentation - Draft 2

[Mel Gorman]

Here goes draft 2. It is mainly spelling, grammar (spelled it right this
time) and formatting corrections largely thanks to Brian O'Connor and
David Woodhouse among others who went through it with a fine tooth comb
and sent me corrections.

The main technical error was pointed out by Ingo Oeser where I didn't go
through how data is copied from userspace properly. I've written a small
paragraph for the moment but have added a section on Copying To/From
Userspace to the TODO list which currently looks something like

o Swap area management
o High memory management
o Memory locking
o Copying To/From Userspace
o Arch independent initialisation (not covering arch dependent)
o Locking

It'll be a few weeks at the very least before I get them done though so
don't hold your breath just yet (Documentation is remarkably slow work).
If anyone sees other parts missing, has suggestions or sees more mistakes,
forward them on as any feedback is welcome.

The updated docs are at the same place, the links are

PDF:  http://www.csn.ul.ie/~mel/projects/vm/guide/pdf/understand.pdf
      http://www.csn.ul.ie/~mel/projects/vm/guide/pdf/code.pdf
HTML: http://www.csn.ul.ie/~mel/projects/vm/guide/html/understand/
      http://www.csn.ul.ie/~mel/projects/vm/guide/html/code/
Text: http://www.csn.ul.ie/~mel/projects/vm/guide/text/understand.txt
      http://www.csn.ul.ie/~mel/projects/vm/guide/text/code.txt

Enjoy....

--
Mel Gorman
University of Limerick




--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Linux VM Documentation - Draft 2

[Mel Gorman]

At the risk of looking bad replying to myself, I re-generated the HTML
output of the documents. A lot of the figures, especially call graphs did
not come out because latex2html had trouble with macros and the navigation
icons were totally broken. I didn't notice because it is only the PDF I
read myself.

The HTML output is still not as good as it should be so if you see any
inconsistencies, the PDF should be consulted. I'm hoping there is no major
discrepancies left though.

-- 
Mel Gorman
MSc Student, University of Limerick
http://www.csn.ul.ie/~mel

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/