* [PATCH selftests 5/6] selftests: vm: Try harder to allocate huge pages [not found] <1446334510.2595.13.camel@decadent.org.uk> @ 2015-10-31 23:39 ` Ben Hutchings 2015-11-10 20:01 ` David Rientjes 0 siblings, 1 reply; 5+ messages in thread From: Ben Hutchings @ 2015-10-31 23:39 UTC (permalink / raw) To: Shuah Khan; +Cc: linux-api, linux-mm [-- Attachment #1: Type: text/plain, Size: 1608 bytes --] If we need to increase the number of huge pages, drop caches first to reduce fragmentation and then check that we actually allocated as many as we wanted. Retry once if that doesn't work. Signed-off-by: Ben Hutchings <ben@decadent.org.uk> --- The test always fails for me in a 1 GB VM without this. Ben. tools/testing/selftests/vm/run_vmtests | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/vm/run_vmtests b/tools/testing/selftests/vm/run_vmtests index 9179ce8..97ed1b2 100755 --- a/tools/testing/selftests/vm/run_vmtests +++ b/tools/testing/selftests/vm/run_vmtests @@ -20,13 +20,26 @@ done < /proc/meminfo if [ -n "$freepgs" ] && [ -n "$pgsize" ]; then nr_hugepgs=`cat /proc/sys/vm/nr_hugepages` needpgs=`expr $needmem / $pgsize` - if [ $freepgs -lt $needpgs ]; then + tries=2 + while [ $tries -gt 0 ] && [ $freepgs -lt $needpgs ]; do lackpgs=$(( $needpgs - $freepgs )) + echo 3 > /proc/sys/vm/drop_caches echo $(( $lackpgs + $nr_hugepgs )) > /proc/sys/vm/nr_hugepages if [ $? -ne 0 ]; then echo "Please run this test as root" exit 1 fi + while read name size unit; do + if [ "$name" = "HugePages_Free:" ]; then + freepgs=$size + fi + done < /proc/meminfo + tries=$((tries - 1)) + done + if [ $freepgs -lt $needpgs ]; then + printf "Not enough huge pages available (%d < %d)\n" \ + $freepgs $needpgs + exit 1 fi else echo "no hugetlbfs support in kernel?" -- Ben Hutchings All extremists should be taken out and shot. [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 811 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH selftests 5/6] selftests: vm: Try harder to allocate huge pages 2015-10-31 23:39 ` [PATCH selftests 5/6] selftests: vm: Try harder to allocate huge pages Ben Hutchings @ 2015-11-10 20:01 ` David Rientjes 2015-11-10 20:05 ` Shuah Khan 2015-11-10 21:48 ` Ben Hutchings 0 siblings, 2 replies; 5+ messages in thread From: David Rientjes @ 2015-11-10 20:01 UTC (permalink / raw) To: Ben Hutchings; +Cc: Shuah Khan, linux-api, linux-mm [-- Attachment #1: Type: TEXT/PLAIN, Size: 1902 bytes --] On Sat, 31 Oct 2015, Ben Hutchings wrote: > If we need to increase the number of huge pages, drop caches first > to reduce fragmentation and then check that we actually allocated > as many as we wanted.A A Retry once if that doesn't work. > > Signed-off-by: Ben Hutchings <ben@decadent.org.uk> > --- > The test always fails for me in a 1 GB VM without this. > > Ben. > > A tools/testing/selftests/vm/run_vmtests | 15 ++++++++++++++- > A 1 file changed, 14 insertions(+), 1 deletion(-) > > diff --git a/tools/testing/selftests/vm/run_vmtests b/tools/testing/selftests/vm/run_vmtests > index 9179ce8..97ed1b2 100755 > --- a/tools/testing/selftests/vm/run_vmtests > +++ b/tools/testing/selftests/vm/run_vmtests > @@ -20,13 +20,26 @@ done < /proc/meminfo > A if [ -n "$freepgs" ] && [ -n "$pgsize" ]; then > A nr_hugepgs=`cat /proc/sys/vm/nr_hugepages` > A needpgs=`expr $needmem / $pgsize` > - if [ $freepgs -lt $needpgs ]; then > + tries=2 > + while [ $tries -gt 0 ] && [ $freepgs -lt $needpgs ]; do > A lackpgs=$(( $needpgs - $freepgs )) > + echo 3 > /proc/sys/vm/drop_caches > A echo $(( $lackpgs + $nr_hugepgs )) > /proc/sys/vm/nr_hugepages > A if [ $? -ne 0 ]; then > A echo "Please run this test as root" > A exit 1 > A fi > + while read name size unit; do > + if [ "$name" = "HugePages_Free:" ]; then > + freepgs=$size > + fi > + done < /proc/meminfo > + tries=$((tries - 1)) > + done > + if [ $freepgs -lt $needpgs ]; then > + printf "Not enough huge pages available (%d < %d)\n" \ > + A A A A A A A $freepgs $needpgs > + exit 1 > A fi > A else > A echo "no hugetlbfs support in kernel?" > I know this patch is in -mm and hasn't been merged by Linus yet, but I'm wondering why the multiple /proc/sys/vm/drop_caches is helping? Would it simply suffice to put a sleep in there instead or is drop_caches actually doing something useful a second time around? ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH selftests 5/6] selftests: vm: Try harder to allocate huge pages 2015-11-10 20:01 ` David Rientjes @ 2015-11-10 20:05 ` Shuah Khan 2015-11-10 21:48 ` Ben Hutchings 1 sibling, 0 replies; 5+ messages in thread From: Shuah Khan @ 2015-11-10 20:05 UTC (permalink / raw) To: David Rientjes, Ben Hutchings; +Cc: linux-api, linux-mm, Shuah Khan On 11/10/2015 01:01 PM, David Rientjes wrote: > On Sat, 31 Oct 2015, Ben Hutchings wrote: > >> If we need to increase the number of huge pages, drop caches first >> to reduce fragmentation and then check that we actually allocated >> as many as we wanted. Retry once if that doesn't work. >> >> Signed-off-by: Ben Hutchings <ben@decadent.org.uk> >> --- >> The test always fails for me in a 1 GB VM without this. >> >> Ben. >> >> tools/testing/selftests/vm/run_vmtests | 15 ++++++++++++++- >> 1 file changed, 14 insertions(+), 1 deletion(-) >> >> diff --git a/tools/testing/selftests/vm/run_vmtests b/tools/testing/selftests/vm/run_vmtests >> index 9179ce8..97ed1b2 100755 >> --- a/tools/testing/selftests/vm/run_vmtests >> +++ b/tools/testing/selftests/vm/run_vmtests >> @@ -20,13 +20,26 @@ done < /proc/meminfo >> if [ -n "$freepgs" ] && [ -n "$pgsize" ]; then >> nr_hugepgs=`cat /proc/sys/vm/nr_hugepages` >> needpgs=`expr $needmem / $pgsize` >> - if [ $freepgs -lt $needpgs ]; then >> + tries=2 >> + while [ $tries -gt 0 ] && [ $freepgs -lt $needpgs ]; do >> lackpgs=$(( $needpgs - $freepgs )) >> + echo 3 > /proc/sys/vm/drop_caches >> echo $(( $lackpgs + $nr_hugepgs )) > /proc/sys/vm/nr_hugepages >> if [ $? -ne 0 ]; then >> echo "Please run this test as root" >> exit 1 >> fi >> + while read name size unit; do >> + if [ "$name" = "HugePages_Free:" ]; then >> + freepgs=$size >> + fi >> + done < /proc/meminfo >> + tries=$((tries - 1)) >> + done >> + if [ $freepgs -lt $needpgs ]; then >> + printf "Not enough huge pages available (%d < %d)\n" \ >> + $freepgs $needpgs >> + exit 1 >> fi >> else >> echo "no hugetlbfs support in kernel?" >> > > I know this patch is in -mm and hasn't been merged by Linus yet, but I'm > wondering why the multiple /proc/sys/vm/drop_caches is helping? Would it > simply suffice to put a sleep in there instead or is drop_caches actually > doing something useful a second time around? > I sent this up for merge in my pull request. Adding sleep would increase test run-time. Something to keep in mind. thanks, -- Shuah -- Shuah Khan Sr. Linux Kernel Developer Open Source Innovation Group Samsung Research America (Silicon Valley) shuahkh@osg.samsung.com | (970) 217-8978 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH selftests 5/6] selftests: vm: Try harder to allocate huge pages 2015-11-10 20:01 ` David Rientjes 2015-11-10 20:05 ` Shuah Khan @ 2015-11-10 21:48 ` Ben Hutchings 2015-11-11 1:11 ` David Rientjes 1 sibling, 1 reply; 5+ messages in thread From: Ben Hutchings @ 2015-11-10 21:48 UTC (permalink / raw) To: David Rientjes; +Cc: Shuah Khan, linux-api, linux-mm [-- Attachment #1: Type: text/plain, Size: 2978 bytes --] On Tue, 2015-11-10 at 12:01 -0800, David Rientjes wrote: > On Sat, 31 Oct 2015, Ben Hutchings wrote: > > > If we need to increase the number of huge pages, drop caches first > > to reduce fragmentation and then check that we actually allocated > > as many as we wanted. Retry once if that doesn't work. > > > > Signed-off-by: Ben Hutchings <ben@decadent.org.uk> > > --- > > The test always fails for me in a 1 GB VM without this. > > > > Ben. > > > > tools/testing/selftests/vm/run_vmtests | 15 ++++++++++++++- > > 1 file changed, 14 insertions(+), 1 deletion(-) > > > > diff --git a/tools/testing/selftests/vm/run_vmtests b/tools/testing/selftests/vm/run_vmtests > > index 9179ce8..97ed1b2 100755 > > --- a/tools/testing/selftests/vm/run_vmtests > > +++ b/tools/testing/selftests/vm/run_vmtests > > @@ -20,13 +20,26 @@ done < /proc/meminfo > > if [ -n "$freepgs" ] && [ -n "$pgsize" ]; then > > nr_hugepgs=`cat /proc/sys/vm/nr_hugepages` > > needpgs=`expr $needmem / $pgsize` > > - if [ $freepgs -lt $needpgs ]; then > > + tries=2 > > + while [ $tries -gt 0 ] && [ $freepgs -lt $needpgs ]; do > > lackpgs=$(( $needpgs - $freepgs )) > > + echo 3 > /proc/sys/vm/drop_caches > > echo $(( $lackpgs + $nr_hugepgs )) > /proc/sys/vm/nr_hugepages > > if [ $? -ne 0 ]; then > > echo "Please run this test as root" > > exit 1 > > fi > > + while read name size unit; do > > + if [ "$name" = "HugePages_Free:" ]; then > > + freepgs=$size > > + fi > > + done < /proc/meminfo > > + tries=$((tries - 1)) > > + done > > + if [ $freepgs -lt $needpgs ]; then > > + printf "Not enough huge pages available (%d < %d)\n" \ > > + $freepgs $needpgs > > + exit 1 > > fi > > else > > echo "no hugetlbfs support in kernel?" > > > > I know this patch is in -mm and hasn't been merged by Linus yet, but I'm > wondering why the multiple /proc/sys/vm/drop_caches is helping? Would it > simply suffice to put a sleep in there instead or is drop_caches actually > doing something useful a second time around? Initially I just retried setting nr_hugepages up to 10 times, which wasn't sufficient. Then I added the drop_caches, and after that setting nr_hugepages tended to worked first time so I reduced the retry count. It might not be necessary to retry at all. Ben. -- Ben Hutchings All the simple programs have been written, and all the good names taken. [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 811 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH selftests 5/6] selftests: vm: Try harder to allocate huge pages 2015-11-10 21:48 ` Ben Hutchings @ 2015-11-11 1:11 ` David Rientjes 0 siblings, 0 replies; 5+ messages in thread From: David Rientjes @ 2015-11-11 1:11 UTC (permalink / raw) To: Ben Hutchings; +Cc: Shuah Khan, linux-api, linux-mm [-- Attachment #1: Type: TEXT/PLAIN, Size: 768 bytes --] On Tue, 10 Nov 2015, Ben Hutchings wrote: > > I know this patch is in -mm and hasn't been merged by Linus yet, but I'm > > wondering why the multiple /proc/sys/vm/drop_caches is helping?A Would it > > simply suffice to put a sleep in there instead or is drop_caches actually > > doing something useful a second time around? > > Initially I just retried setting nr_hugepages up to 10 times, which > wasn't sufficient. A Then I added the drop_caches, and after that > setting nr_hugepages tended to worked first time so I reduced the retry > count. A It might not be necessary to retry at all. > Ok, thanks. I was just trying to make sure that the additional drop_caches wasn't actually required for the test, in which case we would have to fix drop_caches :) ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2015-11-11 1:11 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <1446334510.2595.13.camel@decadent.org.uk>
2015-10-31 23:39 ` [PATCH selftests 5/6] selftests: vm: Try harder to allocate huge pages Ben Hutchings
2015-11-10 20:01 ` David Rientjes
2015-11-10 20:05 ` Shuah Khan
2015-11-10 21:48 ` Ben Hutchings
2015-11-11 1:11 ` David Rientjes
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox