From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx150.postini.com [74.125.245.150]) by kanga.kvack.org (Postfix) with SMTP id 300606B0074 for ; Sun, 13 Jan 2013 10:32:59 -0500 (EST) References: <1334483226.20721.YahooMailNeo@web162003.mail.bf1.yahoo.com> <1334490429.67558.YahooMailNeo@web162006.mail.bf1.yahoo.com> <20120418211032.47b243da@pyramind.ukuu.org.uk> <1334842941.92324.YahooMailNeo@web162006.mail.bf1.yahoo.com> Message-ID: <1358091177.96940.YahooMailNeo@web160103.mail.bf1.yahoo.com> Date: Sun, 13 Jan 2013 07:32:57 -0800 (PST) From: PINTU KUMAR Reply-To: PINTU KUMAR Subject: Introducing Aggressive Low Memory Booster [1] In-Reply-To: <1334842941.92324.YahooMailNeo@web162006.mail.bf1.yahoo.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="1520606428-2004289184-1358091177=:96940" Sender: owner-linux-mm@kvack.org List-ID: To: "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" Cc: "linux-arm-kernel@lists.infradead.org" , "pintu.k@samsung.com" , Anton Vorontsov , Alan Cox , richard -rw- weinberger , "patches@linaro.org" , Mel Gorman , Wanpeng Li --1520606428-2004289184-1358091177=:96940 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Hi,=0A=0AHere I am trying to introduce a new feature in kernel called "Aggr= essive Low Memory Booster".=0AThe main advantage of this will be to boost t= he available free memory of the system to "certain level" during extremely = low memory condition.=0A=0APlease provide your comments to improve further.= =0ACan it be used along with vmpressure_fd ???=0A=0A=0AIt can be invoked as= follows:=0A=A0=A0=A0 a) Automatically by kernel memory management when the= memory threshold falls below 10MB.=0A=A0=A0=A0 b) From user space program/= scripts by passing the "required amount of memory to be reclaimed".=0A=A0= =A0=A0 Example: echo 100 > /dev/shrinkmem=0A=A0=A0=A0 c) using sys interfac= e - /sys/kernel/debug/shrinkallmem=0A=A0=A0=A0 d) using an ioctl call and r= eturning number of pages reclaimed.=0A=A0=A0=A0 e) using a new system call = - shrinkallmem(&nrpages);=0A=A0=A0=A0 f) During CMA to reclaim and shrink a= specific CMA regions.=0A=0A=0AI have developed a kernel module to verify t= he (b) part.=0A=0AHere is the snapshot of the write call:=0A+static ssize_t= shrinkmem_write(struct file *file, const char *buff,=0A+=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= size_t length, loff_t *pos)=0A+{=0A+=A0=A0=A0=A0=A0=A0=A0 int ret =3D -1;= =0A+=A0=A0=A0=A0=A0=A0=A0 unsigned long memsize =3D 0;=0A+=A0=A0=A0=A0=A0= =A0=A0 unsigned long nr_reclaim =3D 0;=0A+=A0=A0=A0=A0=A0=A0=A0 unsigned lo= ng pages =3D 0;=0A+=A0=A0=A0=A0=A0=A0=A0 ret =3D kstrtoul_from_user(buff, l= ength, 0, &memsize);=0A+=A0=A0=A0=A0=A0=A0=A0 if (ret < 0) {=0A+=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 printk(KERN_ERR "[SHRINKMEM]: kstrtoul= _from_user: Failed !\n");=0A+=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 = return -1;=0A+=A0=A0=A0=A0=A0=A0=A0 }=0A+=A0=A0=A0=A0=A0=A0=A0 printk(KERN_= INFO "[SHRINKMEM]: memsize(in MB) =3D %ld\n",=0A+=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 (unsi= gned long)memsize);=0A+=A0=A0=A0=A0=A0=A0=A0 memsize =3D memsize*(1024UL*10= 24UL);=0A+=A0=A0=A0=A0=A0=A0=A0 nr_reclaim =3D memsize / PAGE_SIZE;=0A+=A0= =A0=A0=A0=A0=A0=A0 pages =3D shrink_all_memory(nr_reclaim);=0A+=A0=A0=A0=A0= =A0=A0=A0 printk(KERN_INFO ": Number of Pages Freed: %lu\n", pag= es);=0A+=A0=A0=A0=A0=A0=A0=A0 return pages;=0A+}=0APlease note: This requir= es CONFIG_HIBERNATION to be permanently enabled in the kernel.=0A=0A=0ASeve= ral experiments have been performed on Ubuntu(kernel 3.3) to verify it unde= r low memory conditions.=0A=0AFollowing are some results obtained:=0A------= -------------------------------=0A=0ANode 0, zone=A0=A0=A0=A0=A0 DMA=A0=A0= =A0 290=A0=A0=A0 115=A0=A0=A0=A0=A0 0=A0=A0=A0=A0=A0 0=A0=A0=A0=A0=A0 0=A0= =A0=A0=A0=A0 0=A0=A0=A0=A0=A0 0=A0=A0=A0=A0=A0 0=A0=A0=A0=A0=A0 0=A0=A0=A0= =A0=A0 0=A0=A0=A0=A0=A0 0=0ANode 0, zone=A0=A0 Normal=A0=A0=A0 304=A0=A0=A0= 540=A0=A0=A0 116=A0=A0=A0=A0 13=A0=A0=A0=A0=A0 2=A0=A0=A0=A0=A0 2=A0=A0=A0= =A0=A0 0=A0=A0=A0=A0=A0 0=A0=A0=A0=A0=A0 0=A0=A0=A0=A0=A0 0=A0=A0=A0=A0=A0 = 0=0A=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=0A=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 total=A0=A0=A0=A0=A0=A0 used= =A0=A0=A0=A0=A0=A0 free=A0=A0=A0=A0 shared=A0=A0=A0 buffers=A0=A0=A0=A0 cac= hed=0AMem:=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 497=A0=A0=A0=A0=A0=A0=A0 487=A0=A0= =A0=A0=A0=A0=A0=A0 10=A0=A0=A0=A0=A0=A0=A0=A0=A0 0=A0=A0=A0=A0=A0=A0=A0=A0 = 63=A0=A0=A0=A0=A0=A0=A0 303=0A-/+ buffers/cache:=A0=A0=A0=A0=A0=A0=A0 120= =A0=A0=A0=A0=A0=A0=A0 376=0ASwap:=A0=A0=A0=A0=A0=A0=A0=A0 1458=A0=A0=A0=A0= =A0=A0=A0=A0 34=A0=A0=A0=A0=A0=A0 1424=0ATotal:=A0=A0=A0=A0=A0=A0=A0 1956= =A0=A0=A0=A0=A0=A0=A0 522=A0=A0=A0=A0=A0=A0 1434=0A=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0ATotal Memory Freed: 3= 42 MB=0ATotal Memory Freed: 53 MB=0ATotal Memory Freed: 23 MB=0ATotal Memor= y Freed: 10 MB=0ATotal Memory Freed: 15 MB=0ATotal Memory Freed: -1 MB=0ANo= de 0, zone=A0=A0=A0=A0=A0 DMA=A0=A0=A0=A0=A0 6=A0=A0=A0=A0=A0 6=A0=A0=A0=A0= =A0 7=A0=A0=A0=A0=A0 8=A0=A0=A0=A0 10=A0=A0=A0=A0=A0 9=A0=A0=A0=A0=A0 7=A0= =A0=A0=A0=A0 4=A0=A0=A0=A0=A0 1=A0=A0=A0=A0=A0 0=A0=A0=A0=A0=A0 0=0ANode 0,= zone=A0=A0 Normal=A0=A0 2129=A0=A0 2612=A0=A0 2166=A0=A0 1723=A0=A0 1260= =A0=A0=A0 759=A0=A0=A0 359=A0=A0=A0 108=A0=A0=A0=A0 10=A0=A0=A0=A0=A0 0=A0= =A0=A0=A0=A0 0=0A=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=0A=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 total=A0=A0=A0=A0= =A0=A0 used=A0=A0=A0=A0=A0=A0 free=A0=A0=A0=A0 shared=A0=A0=A0 buffers=A0= =A0=A0=A0 cached=0AMem:=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 497=A0=A0=A0=A0=A0=A0= =A0=A0 47=A0=A0=A0=A0=A0=A0=A0 449=A0=A0=A0=A0=A0=A0=A0=A0=A0 0=A0=A0=A0=A0= =A0=A0=A0=A0=A0 0=A0=A0=A0=A0=A0=A0=A0=A0=A0 5=0A-/+ buffers/cache:=A0=A0= =A0=A0=A0=A0=A0=A0 41=A0=A0=A0=A0=A0=A0=A0 455=0ASwap:=A0=A0=A0=A0=A0=A0=A0= =A0 1458=A0=A0=A0=A0=A0=A0=A0=A0 97=A0=A0=A0=A0=A0=A0 1361=0ATotal:=A0=A0= =A0=A0=A0=A0=A0 1956=A0=A0=A0=A0=A0=A0=A0 145=A0=A0=A0=A0=A0=A0 1811=0A=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= =0AIt was verified using a sample shell script "reclaim_memory.sh" which ke= eps recovering memory by doing "echo 500 > /dev/shrinkmem" until no further= reclaim is possible.=0A=0AThe experiments were performed with various scen= arios as follows:=0Aa) Just after the boot up - (could recover around 150MB= with 512MB RAM)=0Ab) After running many applications include youtube video= s, large tar files download - =0A=0A=A0=A0 [until free mem becomes < 10MB]= =0A=A0=A0 [Could recover around 300MB in one shot]=0Ac) Run reclaim, while = download is in progress and video still playing - (Not applications killed)= =0A=0Ad) revoke all background applications again, after running reclaim - = (No impact, normal behavior)=0A=A0=A0 [Just it took little extra time to la= unch, as if it was launched for first time]=0A=0A=0APlease see more discuss= ions on this in the last year mailing list:=0A=0Ahttps://lkml.org/lkml/2012= /4/15/35 =0A=0A=0AThank You!=0AWith regards,=0APintu Kumar=0ASamsung - Indi= a --1520606428-2004289184-1358091177=:96940 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
Hi,

Here I am trying to introduce a new feature in kernel called "Agg= ressive Low Memory Booster".
The main advantage of this will be= to boost the available free memory of the system to "certain level" during= extremely low memory condition.

Please provide y= our comments to improve further.
Can it be used along with vmpressure_fd ???=

It can be invoked as follows:
    a) Automatically by kernel memory managem= ent when the memory threshold falls below 10MB.
&= nbsp;   b) From user space program/scr= ipts by passing the "required amount of memory to be reclaimed".
    Example: echo 100 > /dev/shrink= mem
    c) using sys interface - /sys/kernel/debug/shrinkallmem
 = ;   d) using an ioctl call and returni= ng number of pages reclaimed.
    = e) using a new system call - shrinkallmem(&n= rpages);
    f) During CMA to reclaim and shrink a specific CMA regions.

Here is the = snapshot of the write call:
+static ssize_t shrinkmem_write(struct file *fil= e, const char *buff,
+          &nb= sp;            =          size_t length, loff_t *pos= )
+{
+        int ret =3D -1;
+=         unsigned long memsize =3D 0;
= +        unsigned long nr_reclaim =3D 0;=
+        unsigned long pages =3D 0;<= br>+        ret =3D kstrtoul_from_user(b= uff, length, 0, &memsize);
+      &nbs= p; if (ret < 0) {
+        &n= bsp;       printk(KERN_ERR "[SHRINKMEM]: kstr= toul_from_user: Failed !\n");
+       = ;         return -1;
+        }
+   = ;     printk(KERN_INFO "[SHRINKMEM]: memsize(in MB) =3D= %ld\n",
+          &n= bsp;            = ;         (unsigned long)memsize);<= br>+        memsize =3D memsize*(1024UL*= 1024UL);
+        nr_reclaim =3D mems= ize / PAGE_SIZE;
+        pages =3D s= hrink_all_memory(nr_reclaim);
+       = ; printk(KERN_INFO "<SHRINKMEM>: Number of Pages Freed: %lu\n", pages= );
+        return pages;
+}
=
Please note: This requires CONFIG_HIBERNATION to be permanently enabled in the ke= rnel.

Several experiments have been performed on Ubuntu(kernel 3.3) = to verify it under low memory conditions.

Following are some results obt= ained:
-------------------------------------
Node 0, zone &nbs= p;    DMA    290    115 &= nbsp;    0      0   =    0      0     = ; 0      0      0 &n= bsp;    0      0
Node 0, zone&nb= sp;  Normal    304    540  &nb= sp; 116     13      2 &nb= sp;    2      0   &n= bsp;  0      0      = 0      0
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
     &n= bsp;       total     = ;  used       free     shar= ed    buffers     cached
Mem: &n= bsp;         497   &= nbsp;    487        = 10          0  &nbs= p;      63      &nbs= p; 303
-/+ buffers/cache:        120&= nbsp;       376
Swap:   &nb= sp;     1458       &= nbsp; 34       1424
Total:  &nbs= p;     1956        5= 22       1434
=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Total Memory Freed: 342= MB
Total Memory Freed: 53 MB
Total Memory Freed: 23 MB
Total Memory Freed: 10 MB
Total Memory Freed: 15= MB
Total Memory Freed: -1 MB
Node 0, zone    &nb= sp; DMA      6      6&nbs= p;     7      8  &nb= sp;  10      9      = 7      4      1 &nbs= p;    0      0
Node 0, zone = ;  Normal   2129   2612   2166 &nbs= p; 1723   1260    759    359 &= nbsp;  108     10      0&= nbsp;     0
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
      = ;       total     &n= bsp; used       free     shar= ed    buffers     cached
Mem: &n= bsp;         497   &= nbsp;     47        = 449          0  &nbs= p;       0      = ;    5
-/+ buffers/cache:     &n= bsp;   41        455
Swap:&= nbsp;        1458    = ;     97       1361
To= tal:        1956    =     145       1811
=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
=

It was verified using a sample shell script "re= claim_memory.sh" which keeps recovering memory by doing "echo 500 > /dev= /shrinkmem" until no further reclaim is possible.

The experiments were p= erformed with various scenarios as follows:
a) Just after the boot up - (could recover around 150MB with 512MB RAM)
b) After running many applicati= ons include youtube videos, large tar files download -
   [until free mem becomes < 10MB]
   [Could recover around 300MB in one shot]
c) Run reclaim, while download is in progress and video still play= ing - (Not applications killed)
d) revoke all back= ground applications again, after running reclaim - (No impact, normal behav= ior)
   [Just it took little extra time= to launch, as if it was launched for first time]


Please see more discussions on this in t= he last year mailing list:
https://lkml.org/lkml/2012/4/15/35
=

Thank You!
With regards,
Pintu Kumar
Samsung - India


<= /body> --1520606428-2004289184-1358091177=:96940-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org