From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3382D0C5E3 for ; Fri, 25 Oct 2024 08:35:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 42FB46B0085; Fri, 25 Oct 2024 04:35:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3DFED6B0088; Fri, 25 Oct 2024 04:35:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 259546B008A; Fri, 25 Oct 2024 04:35:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 026886B0085 for ; Fri, 25 Oct 2024 04:35:26 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 84B34A0704 for ; Fri, 25 Oct 2024 08:34:51 +0000 (UTC) X-FDA: 82711465116.11.E97B40D Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf06.hostedemail.com (Postfix) with ESMTP id 47C84180011 for ; Fri, 25 Oct 2024 08:35:10 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=miQC9x2C; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=JADSNbIZ; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=qSKqa8Hm; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="AsERz/0Q"; spf=pass (imf06.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729845169; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4cSs+iIaLe9LqrnxJ0hkoB/7+YWeu/DvRnAfwYtueNc=; b=W6IfTyD/xEx/Jlv30C0JdOFwzJE3OlfConN07wem+x8+vB1Flzb+Js3Pp/j7iBGttCXzpo 1eFnfI9f10kaHkCv0W2GGNtoV/slC0Oy+gymvXz5PShPz6Z+JZhL/mcyfA+eGCCAkNFfHh Xh9JVAW8My2SrL6IspfgpBj9bmmHwuU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729845169; a=rsa-sha256; cv=none; b=zluckyKWmWzpHBE6GHCvA5rcFQ2/L0+uHceIMpOUC/ood4kp19dRM0zP0f6eVv46G+3k4Y jspGbj4hKDIwgotFRk88Pgb8X2ujv5T8KJ8q3f/11xc9oJeGPCNrbvfvkV7Bs5Fec6qwDX SxnlqvJdU6qCTpqklGLxKIlJEbLHOD4= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=miQC9x2C; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=JADSNbIZ; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=qSKqa8Hm; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="AsERz/0Q"; spf=pass (imf06.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 0EBD61FB5D; Fri, 25 Oct 2024 08:35:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1729845322; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=4cSs+iIaLe9LqrnxJ0hkoB/7+YWeu/DvRnAfwYtueNc=; b=miQC9x2C7jlYN5gsjGzwsPZ0gqujYtKKO/c6snUGS5l6GCnPTXrUfwIOAzA5PF9MWE/w3l P0G+uR1jxX1Vi6MlNg4iWzSUqEmN6McxOiLoXVBui1TyjZvWMI+8wTVMEbTpHCDXsMkN3Y VVzTY0ErL4uYZfJuVDBIf2c60Bb85u8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1729845322; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=4cSs+iIaLe9LqrnxJ0hkoB/7+YWeu/DvRnAfwYtueNc=; b=JADSNbIZ+SdypF+9jdNXjJ2Qvl9wNr46Q2X3hblVNN8GxXHKa8Gye2qXtLeHXJo6KT4bWh FxRzzwJ82MPG/2Cw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1729845321; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=4cSs+iIaLe9LqrnxJ0hkoB/7+YWeu/DvRnAfwYtueNc=; b=qSKqa8Hm0ruLCUwB8Dp0+R7VRbvAa8tZVuR2kKaTpMFAoVcmulbgkTDZDmO8y8dDpexfis frwx8FLcHozcoHo16jL7timAQZ7/uu9Ciy0noiMA7lGSIbYgS8gVDxoLzHdOVrRPBOQexs +4G3e5HWy9C/lTWeEv9yiIYjKEPyAqs= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1729845321; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=4cSs+iIaLe9LqrnxJ0hkoB/7+YWeu/DvRnAfwYtueNc=; b=AsERz/0QKLHkUcozPsN3b5o5r/8x6TV0BxGAKMLUh4c0XDbkxW126UgayF4gAk+eX0kYdb Gt/LL3B91FmG09CA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id EDD58136F5; Fri, 25 Oct 2024 08:35:20 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id ZAq5OUhYG2fyZAAAD6G6ig (envelope-from ); Fri, 25 Oct 2024 08:35:20 +0000 Message-ID: Date: Fri, 25 Oct 2024 10:35:20 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 7/8] mm: refactor __mmap_region() Content-Language: en-US To: Lorenzo Stoakes , Andrew Morton Cc: "Liam R . Howlett" , Jann Horn , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linus Torvalds , Peter Xu References: <0b1da31b49d47ccb930d36f509d50d04c0422b73.1729715266.git.lorenzo.stoakes@oracle.com> From: Vlastimil Babka Autocrypt: addr=vbabka@suse.cz; keydata= xsFNBFZdmxYBEADsw/SiUSjB0dM+vSh95UkgcHjzEVBlby/Fg+g42O7LAEkCYXi/vvq31JTB KxRWDHX0R2tgpFDXHnzZcQywawu8eSq0LxzxFNYMvtB7sV1pxYwej2qx9B75qW2plBs+7+YB 87tMFA+u+L4Z5xAzIimfLD5EKC56kJ1CsXlM8S/LHcmdD9Ctkn3trYDNnat0eoAcfPIP2OZ+ 9oe9IF/R28zmh0ifLXyJQQz5ofdj4bPf8ecEW0rhcqHfTD8k4yK0xxt3xW+6Exqp9n9bydiy tcSAw/TahjW6yrA+6JhSBv1v2tIm+itQc073zjSX8OFL51qQVzRFr7H2UQG33lw2QrvHRXqD Ot7ViKam7v0Ho9wEWiQOOZlHItOOXFphWb2yq3nzrKe45oWoSgkxKb97MVsQ+q2SYjJRBBH4 8qKhphADYxkIP6yut/eaj9ImvRUZZRi0DTc8xfnvHGTjKbJzC2xpFcY0DQbZzuwsIZ8OPJCc LM4S7mT25NE5kUTG/TKQCk922vRdGVMoLA7dIQrgXnRXtyT61sg8PG4wcfOnuWf8577aXP1x 6mzw3/jh3F+oSBHb/GcLC7mvWreJifUL2gEdssGfXhGWBo6zLS3qhgtwjay0Jl+kza1lo+Cv BB2T79D4WGdDuVa4eOrQ02TxqGN7G0Biz5ZLRSFzQSQwLn8fbwARAQABzSBWbGFzdGltaWwg QmFia2EgPHZiYWJrYUBzdXNlLmN6PsLBlAQTAQoAPgIbAwULCQgHAwUVCgkICwUWAgMBAAIe AQIXgBYhBKlA1DSZLC6OmRA9UCJPp+fMgqZkBQJkBREIBQkRadznAAoJECJPp+fMgqZkNxIQ ALZRqwdUGzqL2aeSavbum/VF/+td+nZfuH0xeWiO2w8mG0+nPd5j9ujYeHcUP1edE7uQrjOC Gs9sm8+W1xYnbClMJTsXiAV88D2btFUdU1mCXURAL9wWZ8Jsmz5ZH2V6AUszvNezsS/VIT87 AmTtj31TLDGwdxaZTSYLwAOOOtyqafOEq+gJB30RxTRE3h3G1zpO7OM9K6ysLdAlwAGYWgJJ V4JqGsQ/lyEtxxFpUCjb5Pztp7cQxhlkil0oBYHkudiG8j1U3DG8iC6rnB4yJaLphKx57NuQ PIY0Bccg+r9gIQ4XeSK2PQhdXdy3UWBr913ZQ9AI2usid3s5vabo4iBvpJNFLgUmxFnr73SJ KsRh/2OBsg1XXF/wRQGBO9vRuJUAbnaIVcmGOUogdBVS9Sun/Sy4GNA++KtFZK95U7J417/J Hub2xV6Ehc7UGW6fIvIQmzJ3zaTEfuriU1P8ayfddrAgZb25JnOW7L1zdYL8rXiezOyYZ8Fm ZyXjzWdO0RpxcUEp6GsJr11Bc4F3aae9OZtwtLL/jxc7y6pUugB00PodgnQ6CMcfR/HjXlae h2VS3zl9+tQWHu6s1R58t5BuMS2FNA58wU/IazImc/ZQA+slDBfhRDGYlExjg19UXWe/gMcl De3P1kxYPgZdGE2eZpRLIbt+rYnqQKy8UxlszsBNBFsZNTUBCACfQfpSsWJZyi+SHoRdVyX5 J6rI7okc4+b571a7RXD5UhS9dlVRVVAtrU9ANSLqPTQKGVxHrqD39XSw8hxK61pw8p90pg4G /N3iuWEvyt+t0SxDDkClnGsDyRhlUyEWYFEoBrrCizbmahOUwqkJbNMfzj5Y7n7OIJOxNRkB IBOjPdF26dMP69BwePQao1M8Acrrex9sAHYjQGyVmReRjVEtv9iG4DoTsnIR3amKVk6si4Ea X/mrapJqSCcBUVYUFH8M7bsm4CSxier5ofy8jTEa/CfvkqpKThTMCQPNZKY7hke5qEq1CBk2 wxhX48ZrJEFf1v3NuV3OimgsF2odzieNABEBAAHCwXwEGAEKACYCGwwWIQSpQNQ0mSwujpkQ PVAiT6fnzIKmZAUCZAUSmwUJDK5EZgAKCRAiT6fnzIKmZOJGEACOKABgo9wJXsbWhGWYO7mD 8R8mUyJHqbvaz+yTLnvRwfe/VwafFfDMx5GYVYzMY9TWpA8psFTKTUIIQmx2scYsRBUwm5VI EurRWKqENcDRjyo+ol59j0FViYysjQQeobXBDDE31t5SBg++veI6tXfpco/UiKEsDswL1WAr tEAZaruo7254TyH+gydURl2wJuzo/aZ7Y7PpqaODbYv727Dvm5eX64HCyyAH0s6sOCyGF5/p eIhrOn24oBf67KtdAN3H9JoFNUVTYJc1VJU3R1JtVdgwEdr+NEciEfYl0O19VpLE/PZxP4wX PWnhf5WjdoNI1Xec+RcJ5p/pSel0jnvBX8L2cmniYnmI883NhtGZsEWj++wyKiS4NranDFlA HdDM3b4lUth1pTtABKQ1YuTvehj7EfoWD3bv9kuGZGPrAeFNiHPdOT7DaXKeHpW9homgtBxj 8aX/UkSvEGJKUEbFL9cVa5tzyialGkSiZJNkWgeHe+jEcfRT6pJZOJidSCdzvJpbdJmm+eED w9XOLH1IIWh7RURU7G1iOfEfmImFeC3cbbS73LQEFGe1urxvIH5K/7vX+FkNcr9ujwWuPE9b 1C2o4i/yZPLXIVy387EjA6GZMqvQUFuSTs/GeBcv0NjIQi8867H3uLjz+mQy63fAitsDwLmR EP+ylKVEKb0Q2A== In-Reply-To: <0b1da31b49d47ccb930d36f509d50d04c0422b73.1729715266.git.lorenzo.stoakes@oracle.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Action: no action X-Stat-Signature: zp8187rtwqqiubunoby4eq6yt5cwxzj7 X-Rspamd-Queue-Id: 47C84180011 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1729845310-740795 X-HE-Meta: U2FsdGVkX18lqZ00BR83+TVO7zSx7Mv/Xqr1VF0FJhtHVOHO1UQe1yOQFjX85ODjsv27GtH8YE6gotS3ScmM6KgrXLEXV8tbDToO4UFoEgODV5lpNPDqsK390wgQTeLukVXd0P5lKSJ86a+Wkh05f4mgChfly37K2r/Mg6TyXHI8wETtremqwKjk+bL9pK+G9B0z7Vb2EjHrXKjsVFUMffg3UJ8+UG8S6N9+YWcUuVUJGlwjX+ns+wGhyW+Z99j2U0RSxFe7vO0hnzHryLGeraK5x23bQAAmePmlSNVVrJ3SA1ssLlbGZOBQxslfebXrQMcnfCQa4I9pSbmS97GZ0sVKz6Seqre3i7GRWYwX+uOYvOT7VITXV5k0fLJWPzJw++PLjQJieE1mAZQC6FCFV+1ajXPtpc+HPgI1vi5VMhDaX09EYUHe6EZQ1zb4OsngBOEvefdS16SAYppEt66lfiVh+eObrCxBg+hzAv55nihoqMIooSGTiXnK3SwJXu0lDN4OA+2jcvT2WjE4QvHy3DA3DLjL2tPNaEpYhWk8PU+8jLYkmpaza00evTbjBse7BhnLDziIc3DzchromBrr9G/hKweqQ9OY29sljVrGRahllNLbDM8khp00VX2xr0KMaiTWME+XLjrRA2FUtjfVa1WFxHRmIn3QD3pktKtbvhxELVuxuu8kCJbecatYSz8ybM7zeLHEp7zVRFP0sb1iG12fxI0Y2KwT/7TZf0+EYWrhPF/+o5cs4E+kXWj5n6I+b8zHnE84SmxqsK7elrr+y54FPC8H2hTYPX9PVgME1CJ3hD5j5c/I7h3GgBHoXJFnRyGoXTQi3gZx0E9D/LL9KLXw8YSPF7XVbT4YPEc+svEC1r9BNywP20/tPffQqlwITb8mtRh/mHRJzterd5I/y1iJUofpMDt3yKx6iVakt3uhp0d8+IXYbO/uPr3zn4UmFb43JKZl1GQGXh4s0Pw cssffeJ3 fNJ9IX1BimcDaQkxOfOJd5Baf8Brhmf32IBSq3dgQ69UWVtJJ3Rgv6GLCj2e568tdwjejrtDvgFLYeuD1vyTz1Wz2a3fDbGwbilpZTFZkkWeTy6NjwfmsGIxpbCLqIvbNizeGgoUNRBzh4/bhJp6T0hI4kn7DFlY7GfpsjE11ZQWxglgWoZsTqT3s2cJekiplyzD4ZsFjkb+dP5dX/WwKkQJoBVNh3VKm6aVdNbWy4JsANtFvPoNyfxx5JDaOqAzsLMpPB5+t98vL5supnCgS8ogNU6siVCjAXeX7Z+uzgJhyU0/kwKnKGYGadllHNWscN81GFFbOnEptV6nFANoEQY9Dk6PpdXkRjgBDJaSvC060eJY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 10/23/24 22:38, Lorenzo Stoakes wrote: > We have seen bugs and resource leaks arise from the complexity of the > __mmap_region() function. This, and the generally deeply fragile error > handling logic and complexity which makes understanding the function > difficult make it highly desirable to refactor it into something readable. > > Achieve this by separating the function into smaller logical parts which > are easier to understand and follow, and which importantly very > significantly simplify the error handling. > > Note that we now call vms_abort_munmap_vmas() in more error paths than we > used to, however in cases where no abort need occur, vms->nr_pages will be > equal to zero and we simply exit this function without doing more than we > would have done previously. > > Importantly, the invocation of the driver mmap hook via mmap_file() now has > very simple and obvious handling (this was previously the most problematic > part of the mmap() operation). > > Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka Some nits: > +static int __mmap_prepare(struct mmap_state *map, struct vma_merge_struct *vmg, > + struct list_head *uf) > { > - struct mm_struct *mm = current->mm; > - struct vm_area_struct *vma = NULL; > - pgoff_t pglen = PHYS_PFN(len); > - unsigned long charged = 0; > - struct vma_munmap_struct vms; > - struct ma_state mas_detach; > - struct maple_tree mt_detach; > - unsigned long end = addr + len; > int error; > - VMA_ITERATOR(vmi, mm, addr); > - VMG_STATE(vmg, mm, &vmi, addr, end, vm_flags, pgoff); > - > - vmg.file = file; > - /* Find the first overlapping VMA */ > - vma = vma_find(&vmi, end); > - init_vma_munmap(&vms, &vmi, vma, addr, end, uf, /* unlock = */ false); > - if (vma) { > - mt_init_flags(&mt_detach, vmi.mas.tree->ma_flags & MT_FLAGS_LOCK_MASK); > - mt_on_stack(mt_detach); > - mas_init(&mas_detach, &mt_detach, /* addr = */ 0); > + struct vma_iterator *vmi = map->vmi; > + struct vma_munmap_struct *vms = &map->vms; > + > + /* Find the first overlapping VMA and initialise unmap state. */ > + vms->vma = vma_find(vmi, map->end); > + init_vma_munmap(vms, vmi, vms->vma, map->addr, map->end, uf, > + /* unlock = */ false); > + > + /* OK, we have overlapping VMAs - prepare to unmap them. */ > + if (vms->vma) { > + mt_init_flags(&map->mt_detach, > + vmi->mas.tree->ma_flags & MT_FLAGS_LOCK_MASK); > + mt_on_stack(map->mt_detach); > + mas_init(&map->mas_detach, &map->mt_detach, /* addr = */ 0); > /* Prepare to unmap any existing mapping in the area */ > - error = vms_gather_munmap_vmas(&vms, &mas_detach); > - if (error) > - goto gather_failed; > + error = vms_gather_munmap_vmas(vms, &map->mas_detach); > + if (error) { > + /* On error VMAs will already have been reattached. */ > + vms->nr_pages = 0; > + return error; > + } > > - vmg.next = vms.next; > - vmg.prev = vms.prev; > - vma = NULL; > + map->next = vms->next; > + map->prev = vms->prev; > } else { > - vmg.next = vma_iter_next_rewind(&vmi, &vmg.prev); > + map->next = vma_iter_next_rewind(vmi, &map->prev); > } > > + /* Set up vmg for merge attempt. */ > + vmg->file = map->file; > + vmg->prev = map->prev; > + vmg->next = map->next; It seems arbitrary to do this here. vmg->flags are set in __mmap_region(). Why not all of this? We wouldn't need to pass vmg to __mmap_prepare() at all? Do any of the values chenge between here and returning and vmg needs to capture them as they are now? AFAICS not. > + > /* Check against address space limit. */ > - if (!may_expand_vm(mm, vm_flags, pglen - vms.nr_pages)) { > - error = -ENOMEM; > - goto abort_munmap; > - } > + if (!may_expand_vm(map->mm, map->flags, map->pglen - vms->nr_pages)) > + return -ENOMEM; > > - /* > - * Private writable mapping: check memory availability > - */ > - if (accountable_mapping(file, vm_flags)) { > - charged = pglen; > - charged -= vms.nr_accounted; > - if (charged) { > - error = security_vm_enough_memory_mm(mm, charged); > + /* Private writable mapping: check memory availability. */ > + if (accountable_mapping(map->file, map->flags)) { > + map->charged = map->pglen; > + map->charged -= vms->nr_accounted; > + if (map->charged) { > + error = security_vm_enough_memory_mm(map->mm, map->charged); > if (error) > - goto abort_munmap; > + return error; > } > > - vms.nr_accounted = 0; > - vm_flags |= VM_ACCOUNT; > - vmg.flags = vm_flags; > + vms->nr_accounted = 0; > + map->flags |= VM_ACCOUNT; > } > > /* > - * clear PTEs while the vma is still in the tree so that rmap > + * Clear PTEs while the vma is still in the tree so that rmap > * cannot race with the freeing later in the truncate scenario. > * This is also needed for mmap_file(), which is why vm_ops > * close function is called. > */ > - vms_clean_up_area(&vms, &mas_detach); > - vma = vma_merge_new_range(&vmg); > - if (vma) > - goto expanded; > + vms_clean_up_area(vms, &map->mas_detach); > + > + return 0; > +} > + > +static int __mmap_new_vma(struct mmap_state *map, struct vma_merge_struct *vmg, > + struct vm_area_struct **vmap) > +{ > + struct vma_iterator *vmi = map->vmi; > + int error = 0; > + bool merged = false; > + struct vm_area_struct *vma; > + > /* > * Determine the object being mapped and call the appropriate > * specific mapper. the address has already been validated, but > * not unmapped, but the maps are removed from the list. > */ > - vma = vm_area_alloc(mm); > - if (!vma) { > - error = -ENOMEM; > - goto unacct_error; > - } > + vma = vm_area_alloc(map->mm); > + if (!vma) > + return -ENOMEM; > > - vma_iter_config(&vmi, addr, end); > - vma_set_range(vma, addr, end, pgoff); > - vm_flags_init(vma, vm_flags); > - vma->vm_page_prot = vm_get_page_prot(vm_flags); > + vma_iter_config(vmi, map->addr, map->end); > + vma_set_range(vma, map->addr, map->end, map->pgoff); > + vm_flags_init(vma, map->flags); > + vma->vm_page_prot = vm_get_page_prot(map->flags); > > - if (vma_iter_prealloc(&vmi, vma)) { > + if (vma_iter_prealloc(vmi, vma)) { > error = -ENOMEM; > goto free_vma; > } > > - if (file) { > - vma->vm_file = get_file(file); > - error = mmap_file(file, vma); > - if (error) > - goto unmap_and_free_file_vma; > - > - /* Drivers cannot alter the address of the VMA. */ > - WARN_ON_ONCE(addr != vma->vm_start); > - /* > - * Drivers should not permit writability when previously it was > - * disallowed. > - */ > - VM_WARN_ON_ONCE(vm_flags != vma->vm_flags && > - !(vm_flags & VM_MAYWRITE) && > - (vma->vm_flags & VM_MAYWRITE)); > - > - vma_iter_config(&vmi, addr, end); > - /* > - * If vm_flags changed after mmap_file(), we should try merge > - * vma again as we may succeed this time. > - */ > - if (unlikely(vm_flags != vma->vm_flags && vmg.prev)) { > - struct vm_area_struct *merge; > - > - vmg.flags = vma->vm_flags; > - /* If this fails, state is reset ready for a reattempt. */ > - merge = vma_merge_new_range(&vmg); > - > - if (merge) { > - /* > - * ->mmap() can change vma->vm_file and fput > - * the original file. So fput the vma->vm_file > - * here or we would add an extra fput for file > - * and cause general protection fault > - * ultimately. > - */ > - fput(vma->vm_file); > - vm_area_free(vma); > - vma = merge; > - /* Update vm_flags to pick up the change. */ > - vm_flags = vma->vm_flags; > - goto file_expanded; > - } > - vma_iter_config(&vmi, addr, end); > - } > - > - vm_flags = vma->vm_flags; > - } else if (vm_flags & VM_SHARED) { > + if (map->file) > + error = __mmap_new_file_vma(map, vmg, &vma, &merged); > + else if (map->flags & VM_SHARED) > error = shmem_zero_setup(vma); > - if (error) > - goto free_iter_vma; > - } else { > + else > vma_set_anonymous(vma); > - } > + > + if (error) > + goto free_iter_vma; > + > + if (merged) > + goto file_expanded; > > #ifdef CONFIG_SPARC64 > /* TODO: Fix SPARC ADI! */ > - WARN_ON_ONCE(!arch_validate_flags(vm_flags)); > + WARN_ON_ONCE(!arch_validate_flags(map->flags)); > #endif > > /* Lock the VMA since it is modified after insertion into VMA tree */ > vma_start_write(vma); > - vma_iter_store(&vmi, vma); > - mm->map_count++; > + vma_iter_store(vmi, vma); > + map->mm->map_count++; > vma_link_file(vma); > > /* > * vma_merge_new_range() calls khugepaged_enter_vma() too, the below > * call covers the non-merge case. > */ > - khugepaged_enter_vma(vma, vma->vm_flags); > + khugepaged_enter_vma(vma, map->flags); > > file_expanded: > - file = vma->vm_file; > ksm_add_vma(vma); I'm wondering if this could go to the "non file expanded" region above. If we expanded a vma, it was either in ksm and stays in ksm, or it was not eligible and that couldn't have changed by expanding? > -expanded: > + *vmap = vma; > + return 0; > + > +free_iter_vma: > + vma_iter_free(vmi); > +free_vma: > + vm_area_free(vma); > + return error; > +} > +