[PATCH] mm/mmap.c: make brk() check RLIMIT_AS before page-aligning requested amount

[Kartavya Vashishtha <sendtokartavya@gmail.com>]

Currently, brk() only checks against RLIMIT_DATA when validating whether
the requested amount of memory is valid with respect to rlimit.

RLIMIT_AS is checked later in the `may_expand_vm` call in `do_brk_flags`,
but that call occurs after aligning the new brk to a page boundary, making
the following possible:

1. Allocate a non-page-sized amount of memory with brk()

2. brk() will internally page-align the requested amount, and allocate
the necessary amount of pages.

3. Set RLIMIT_AS to 1 byte using setrlimit.

4. Calling brk() again with a small increment (such that it does not
overflow to the next page) will succeed.

This violates setrlimit RLIMIT_AS, since the call succeeds despite a
1 byte limit.

The following code snippet reproduces this behavior:

```

int main() {
        void * mem = malloc(4096);
        sbrk(32);

	// set RLIMIT_AS for the processe's address space to 1 byte
        // This causes all future calls to sbrk to fail
        struct rlimit lim;
        getrlimit(RLIMIT_AS, &lim);
        lim.rlim_cur = 1;
        printf("lim.rlim_max: %ld\n", lim.rlim_max);
        setrlimit(RLIMIT_AS, &lim);

        printf("Mallocing an additional 8 bytes, which requires more
	       "memory from sbrk, but sbrk SHOULD fail\n");
        void * ptr = sbrk(8);
        printf("sbrk result: %p\n", ptr);
        if (ptr != -1) {
                printf("sbrk unexpectedly passed\n");
        } else {
                printf("sbrk expectedly failed\n");
        }
        free(mem);
}
```

Signed-off-by: Kartavya Vashishtha <sendtokartavya@gmail.com>
---
 mm/mmap.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/mm/mmap.c b/mm/mmap.c
index d0dfc85b209b..5f7fc6591323 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -253,8 +253,8 @@ SYSCALL_DEFINE1(brk, unsigned long, brk)
 	 * segment grow beyond its set limit the in case where the limit is
 	 * not page aligned -Ram Gupta
 	 */
-	if (check_data_rlimit(rlimit(RLIMIT_DATA), brk, mm->start_brk,
-			      mm->end_data, mm->start_data))
+	if (check_data_rlimit(min(rlimit(RLIMIT_AS), rlimit(RLIMIT_DATA)),
+			      brk, mm->start_brk, mm->end_data, mm->start_data))
 		goto out;
 
 	newbrk = PAGE_ALIGN(brk);
-- 
2.34.1