From mboxrd@z Thu Jan 1 00:00:00 1970 Date: Tue, 9 May 2000 11:36:03 -0400 (EDT) From: "Benjamin C.R. LaHaise" Subject: Re: How much to malloc(), without running into swap...? In-Reply-To: <3913F0C4.D546D155@gnu.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-linux-mm@kvack.org Return-Path: To: Andrew Clausen Cc: linux-mm@kvack.org, parted@gnu.org List-ID: On Sat, 6 May 2000, Andrew Clausen wrote: > Hi all, (please cc me) > > I'm hacking GNU Parted, which can (amongst other things) resize > file systems. It's performance is GREATLY improved if large > disk buffers are used, provided it doesn't need to swap to access > the buffers ;-) mlock() is your friend. > So, how can I maximize the buffer sizes, without running into > swap? Note: I don't want to disable swap, because a certain > (large) minimum is required for storing metadata, etc., so > low-memory machines might want to use a swap (despite it being > slow). > > So, I want to know: > (a) how much I can malloc() without swapping > (b) how much I can malloc() with swapping > > Also, I presume all IO is going to have to go through the buffer > cache, etc., so having a larger buffers means more consumption > on the kernel side of things. OTOH, it can probably kick out > old cached data fairly quickly (i.e. data used by other programs) > Any ideas on how to do the calc's? > > BTW, is /proc/meminfo a good idea? That's probably your best bet if your app is the only thing running in the system. Note that you'll get some feedback about things at mlock time -- if there just isn't enough memory free, mlock() will fail, so you can use that to scale back your memory requirements if other parts of the system are eating memory. With swapping, things get trickier -- you'll probably want to minimise the mlock()'d data. It would probably be better to avoid swapping altogether for your app since it would cause a ridiculous growth in the running time (and probably can be avoided). You might want to look into using the raw devices when possible since you can probably make a better guess at the best way to order your io than the buffer cache currently will. Doing that will also get the added benefit of avoiding any excess swapping that might be generated by competition for memory by the buffer cache. Hope this helps, -ben -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux.eu.org/Linux-MM/