From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9745AC27C53 for ; Fri, 7 Jun 2024 14:38:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 073406B00A4; Fri, 7 Jun 2024 10:38:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0224C6B00A5; Fri, 7 Jun 2024 10:38:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E04E96B00A6; Fri, 7 Jun 2024 10:38:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C30B86B00A4 for ; Fri, 7 Jun 2024 10:38:17 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 812ED121906 for ; Fri, 7 Jun 2024 14:38:17 +0000 (UTC) X-FDA: 82204347834.07.E374FF0 Received: from mail-yb1-f175.google.com (mail-yb1-f175.google.com [209.85.219.175]) by imf29.hostedemail.com (Postfix) with ESMTP id B1B7512000A for ; Fri, 7 Jun 2024 14:38:15 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=D+wvOEhI; spf=pass (imf29.hostedemail.com: domain of surenb@google.com designates 209.85.219.175 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717771095; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UVOnoBzoK1zAtvBRGWuYUP6Os8Y9dBiDXv9nnFzFCIo=; b=AW/ZSc8s+trJKudD/uS6AZtls1HQZyx/6njxbNRMWk//QRAMGcYMm36guMG4MHd0YQq+nj wa6QTBqKDanN8fqgtUFLrh4RTSjWhqkepx02tys2RR/dHUwPFhlzz5VGOzUq6m6Pbfx20a l8YpJPk6am/KGfC38Ya9rN7YJL/YLvE= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=D+wvOEhI; spf=pass (imf29.hostedemail.com: domain of surenb@google.com designates 209.85.219.175 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717771095; a=rsa-sha256; cv=none; b=kGNEX+7FhF39H5ndCOA3uY32KuQCfm7dp5afZnIImuoAXomVR2v/gOVbohzu1l+qS+YrYU rLWgPlxs+KPU+1/AvbKGWD9IAskeMgKnl5IBuTG2RGj1lskZRDTSH1ri6M7qQIJizBzwB0 aqXS3NJoAVDhhq1aucvfqsKA2H1rewA= Received: by mail-yb1-f175.google.com with SMTP id 3f1490d57ef6-dfab5f7e749so2285594276.0 for ; Fri, 07 Jun 2024 07:38:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1717771095; x=1718375895; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=UVOnoBzoK1zAtvBRGWuYUP6Os8Y9dBiDXv9nnFzFCIo=; b=D+wvOEhI/eP7Wf9zs6KepnIp0uBL41BdTLR/pH4XjQq2RrL7n8gb4tnHbfU1lMCK8+ c9k5WVeg0HZUuF57z+y+bQRMOj3p6VykRFf69+9H1Ybh9/9f3DkCHOZ1bJUZAKNQ3F7+ WZvdSF6xLLkS5SgB1Ga8jTbOKksiPgIeQMR7yJlFjSd9utWat5CBwFs74W0Su/oLWAfB pSw6uS6Pzl5zIRo0VrziJIu5d1RMYs5SqaFCw42VzaCpxckPWHFzl6zRritr+K76jyTS a/bMKhrllQ2fR4//y3tMpw+2JWyNOKrCmNeAnpUrJoOIofwXH4uO443rL4W4ovyI7tIf E6fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717771095; x=1718375895; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UVOnoBzoK1zAtvBRGWuYUP6Os8Y9dBiDXv9nnFzFCIo=; b=Qk7hAvvA7QNa94Q93e+2HL/Yu3xBaoKcIhiZgcrhiZpQA1BvoiT3sG4eEn/Wlkir2d G8tdnYRPy/bEcwMwCh97Dgk4iMdC/XtyeQpMEGRiPYjLRGNQQnohlWm/xMfTNzmlPTUK W1ExcF4SZj0Jl1OWqvTvc3r+sYlrO7X6zy+ewL34d4I3DrX8gCAgOO/2guTfgph2NZF5 FfZhOPXYpBIfel50VsCKvI9BNtrF02UNImzYSpLuBjypqn8/lGzzzp9A5XBoPUMvvlXX fMl4QnFHLp0+QODwjeFNyIu0NCs/W7wONxbxoclCki7p9tjxaUxSu6MpHL/N6/Mtw8Gd Mq3Q== X-Forwarded-Encrypted: i=1; AJvYcCWB8ucINUzr87OghiGm2MDNxxKmtUJkZeMMmrNINIs011Pg8ueoLHnh559sdvH8/2du6Sl70dum91Xfv0TDGjjrb+U= X-Gm-Message-State: AOJu0YwhBj0b67w/CpZH9Oorp3BKkKwX4nhKuUjgxCo/Gq7Pu6xFooEC VXahhozSNbJO2XnVe1Crxph6PR6WeAVxOo9taWKZjPnpocNrZwNJbrgziBLEs4OxNzc/BJ2cVZO 3h7WDA3hd+IgNIrPcrejYGm6T0eu9oanSaJw/ X-Google-Smtp-Source: AGHT+IEoKThZkHMiLLjP3WX0BoB3TWzKGwUQSKBSO7THX0AH5ExYgBlUbM4kN7MluTiitOjWmHibd+i9cZHc6WhpIps= X-Received: by 2002:a25:8681:0:b0:dfa:fe79:8d9b with SMTP id 3f1490d57ef6-dfafe798ff6mr1701678276.20.1717771094238; Fri, 07 Jun 2024 07:38:14 -0700 (PDT) MIME-Version: 1.0 References: <20240531163217.1584450-1-Liam.Howlett@oracle.com> <20240531163217.1584450-5-Liam.Howlett@oracle.com> In-Reply-To: <20240531163217.1584450-5-Liam.Howlett@oracle.com> From: Suren Baghdasaryan Date: Fri, 7 Jun 2024 07:38:01 -0700 Message-ID: Subject: Re: [RFC PATCH 4/5] mm/mmap: Change munmap to use vma_munmap_struct() for accounting and surrounding vmas To: "Liam R. Howlett" Cc: Andrii Nakryiko , Vlastimil Babka , sidhartha.kumar@oracle.com, Matthew Wilcox , Lorenzo Stoakes , linux-fsdevel@vger.kernel.org, bpf@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: 8swfp6bgcbs1h7yhxsqe5ici9mbkqfuj X-Rspamd-Queue-Id: B1B7512000A X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1717771095-174731 X-HE-Meta: U2FsdGVkX1+6l3j/b/WPOWi13cfl/lW6uCVFyGXHPi/+GnpFeSbKOEKXcGcasYF7BXdLRzGLFsQo2XYVXhDD5uasyfhkW13caPvLRGRj5I3/Zbw+ev8tYSAMPIcAxUrL6fvEYK+8WWxbWOJ6Ntf8B37afro0l30URbZ+ARiB7fZ50OcS+YVp/j9pIuLQKT11dJwpce1oZYDMUcS8kiQH8Qp9nll+EpxQwYmypUwgSRrLHG2w6ylPMflCt7Sv/J8d8CZqlCOZMzH2Kb+CFII35dS1xLqPojyB7Ss/40xFxAXYcvzoc1h+jGqFZN7gD/a5hq7gNF3V1gTC+bZS2R8P4EK+BDFBSPA8Tup3AD1UBDnVHEI/DB2+wekMQjmimhc1+ow+1wrKXg27NVKpTQPT5jlyp3B2XIVAoc0UYYUaPtUM8xPwBIBYw4UmeqgxjlKK/PNqnWf9aMWjpDa+cHdReRbq34sAJpr6WP0XHHF+/EO5HLdm4ZwRJh0+RIavIk1DxTFFkCdjWZXdUOG7aZcRN62FKNKN3+yStcHebEegQHGo2ysy3cUdklHm1KFJ/o1J6lMkqyc5xoD25volxt8/F+EWuD9SpD0psXUki1qpYnKoVnSF3ISzcqjjYjuNg99I5IdvoFaIes+qI2MkgsWcxwgd9fcTAUhEfZSXd7pgAF+EFG4w/7n0wEdGkdhswZd7qq92XR6zBZpjIW+cYG01qzvPwTcg08tM4Nq6ZIX5L+MxmzeMu+Z+R4KnVuVCKyRKXHCgJxVwZFoQZevCHcQ+9tbfu38CcC+3rBMuho3bPPzXNEx1fBB17NNpq2ez82XeLQBvYxA6IEpVaQONQKm9ivUMG82NaKC7rlHsULB9ON0FYPGWdvr7N/+0pKGvn8P/fol3qT+O3iNiL/HbgPXp+TZweN2J4mWRAyAkfhGVje+s5UGpgNaa/d82e0R0MZc22/pCfVHgWHIdMOkUPOG QwA2Iw2m 1GVTWt3m8GpE4/cFljPH1eUuWMhOqN1jSkU1DUd2deG7KIky/jnjBR6DM/SZyaq4GLX2yrundDeNcE1Boirw9q7aJHiQLF2um3XsVkF8PKUSUzm7RXQ/aiYD77JkqTzS7Ti1Y6kndeKKuYkBBNKg1UqntEIZJuB3XMGUr7tLKrfyEIXieQKhP04PfLdVwowAhEUQqlhXWZBxNyeoquLtTfRUxSoh1JFCRIQE6tnlxFXgXXWMZ6645AD3I0m9Zt8EknyMRZogxYz8VaYuR5zuforj8w3yFuSAjHhbZMhG/zbWUUffdjoINVd5bcipERu6nUspsSsxQEaPV9b15aa8LZY6chHKYg/71VBUlO7Mfqqm2HeDKxVifa5srAQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, May 31, 2024 at 9:33=E2=80=AFAM Liam R. Howlett wrote: > > Clean up the code by changing the munmap operation to use a structure > for the accounting and munmap variables. > > Since remove_mt() is only called in one location and the contents will > be reduce to almost nothing. The remains of the function can be added > to vms_complete_munmap_vmas(). > > Signed-off-by: Liam R. Howlett > --- > mm/internal.h | 6 ++++ > mm/mmap.c | 85 +++++++++++++++++++++++++++------------------------ > 2 files changed, 51 insertions(+), 40 deletions(-) > > diff --git a/mm/internal.h b/mm/internal.h > index 6ebf77853d68..8c02ebf5736c 100644 > --- a/mm/internal.h > +++ b/mm/internal.h > @@ -1435,12 +1435,18 @@ struct vma_munmap_struct { > struct vma_iterator *vmi; > struct mm_struct *mm; > struct vm_area_struct *vma; /* The first vma to munmap */ > + struct vm_area_struct *next; /* vma after the munmap area */ > + struct vm_area_struct *prev; /* vma before the munmap area */ > struct list_head *uf; /* Userfaultfd list_head */ > unsigned long start; /* Aligned start addr */ > unsigned long end; /* Aligned end addr */ > int vma_count; /* Number of vmas that will be re= moved */ > unsigned long nr_pages; /* Number of pages being removed = */ > unsigned long locked_vm; /* Number of locked pages */ > + unsigned long nr_accounted; /* Number of VM_ACCOUNT pages */ > + unsigned long exec_vm; > + unsigned long stack_vm; > + unsigned long data_vm; > bool unlock; /* Unlock after the munmap */ > }; > > diff --git a/mm/mmap.c b/mm/mmap.c > index 57f2383245ea..3e0930c09213 100644 > --- a/mm/mmap.c > +++ b/mm/mmap.c > @@ -482,7 +482,8 @@ static inline void init_vma_munmap(struct vma_munmap_= struct *vms, > vms->unlock =3D unlock; > vms->uf =3D uf; > vms->vma_count =3D 0; > - vms->nr_pages =3D vms->locked_vm =3D 0; > + vms->nr_pages =3D vms->locked_vm =3D vms->nr_accounted =3D 0; > + vms->exec_vm =3D vms->stack_vm =3D vms->data_vm =3D 0; > } > > /* > @@ -604,7 +605,6 @@ static inline void vma_complete(struct vma_prepare *v= p, > } > if (vp->insert && vp->file) > uprobe_mmap(vp->insert); > - validate_mm(mm); vma_complete() is used in places other than vma_shrink(). You effectively removed validate_mm() for all those other users. Is that intentional? If so, that should be documented in the changelog. > } > > /* > @@ -733,6 +733,7 @@ int vma_shrink(struct vma_iterator *vmi, struct vm_ar= ea_struct *vma, > vma_iter_clear(vmi); > vma_set_range(vma, start, end, pgoff); > vma_complete(&vp, vmi, vma->vm_mm); > + validate_mm(vma->vm_mm); > return 0; > } > > @@ -2347,30 +2348,6 @@ struct vm_area_struct *expand_stack(struct mm_stru= ct *mm, unsigned long addr) > return vma; > } > > -/* > - * Ok - we have the memory areas we should free on a maple tree so relea= se them, > - * and do the vma updates. > - * > - * Called with the mm semaphore held. > - */ > -static inline void remove_mt(struct mm_struct *mm, struct ma_state *mas) > -{ > - unsigned long nr_accounted =3D 0; > - struct vm_area_struct *vma; > - > - /* Update high watermark before we lower total_vm */ > - update_hiwater_vm(mm); > - mas_for_each(mas, vma, ULONG_MAX) { > - long nrpages =3D vma_pages(vma); > - > - if (vma->vm_flags & VM_ACCOUNT) > - nr_accounted +=3D nrpages; > - vm_stat_account(mm, vma->vm_flags, -nrpages); > - remove_vma(vma, false); > - } > - vm_unacct_memory(nr_accounted); > -} > - > /* > * Get rid of page table information in the indicated region. > * > @@ -2625,13 +2602,14 @@ static int vms_gather_munmap_vmas(struct vma_munm= ap_struct *vms, > if (error) > goto start_split_failed; > } > + vms->prev =3D vma_prev(vms->vmi); > > /* > * Detach a range of VMAs from the mm. Using next as a temp varia= ble as > * it is always overwritten. > */ > - next =3D vms->vma; > - do { > + for_each_vma_range(*(vms->vmi), next, vms->end) { > + long nrpages; > /* Does it split the end? */ > if (next->vm_end > vms->end) { > error =3D __split_vma(vms->vmi, next, vms->end, 0= ); > @@ -2640,8 +2618,21 @@ static int vms_gather_munmap_vmas(struct vma_munma= p_struct *vms, > } > vma_start_write(next); > mas_set(mas_detach, vms->vma_count++); > + nrpages =3D vma_pages(next); > + > + vms->nr_pages +=3D nrpages; > if (next->vm_flags & VM_LOCKED) > - vms->locked_vm +=3D vma_pages(next); > + vms->locked_vm +=3D nrpages; > + > + if (next->vm_flags & VM_ACCOUNT) > + vms->nr_accounted +=3D nrpages; > + > + if (is_exec_mapping(next->vm_flags)) > + vms->exec_vm +=3D nrpages; > + else if (is_stack_mapping(next->vm_flags)) > + vms->stack_vm +=3D nrpages; > + else if (is_data_mapping(next->vm_flags)) > + vms->data_vm +=3D nrpages; > > error =3D mas_store_gfp(mas_detach, next, GFP_KERNEL); > if (error) > @@ -2667,7 +2658,9 @@ static int vms_gather_munmap_vmas(struct vma_munmap= _struct *vms, > BUG_ON(next->vm_start < vms->start); > BUG_ON(next->vm_start > vms->end); > #endif > - } for_each_vma_range(*(vms->vmi), next, vms->end); > + } > + > + vms->next =3D vma_next(vms->vmi); > > #if defined(CONFIG_DEBUG_VM_MAPLE_TREE) > /* Make sure no VMAs are about to be lost. */ > @@ -2712,10 +2705,11 @@ static int vms_gather_munmap_vmas(struct vma_munm= ap_struct *vms, > * @mas_detach: The maple state of the detached vmas > * > */ > +static inline void vms_vm_stat_account(struct vma_munmap_struct *vms); > static void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, > struct ma_state *mas_detach) > { > - struct vm_area_struct *prev, *next; > + struct vm_area_struct *vma; > struct mm_struct *mm; > > mm =3D vms->mm; > @@ -2724,21 +2718,21 @@ static void vms_complete_munmap_vmas(struct vma_m= unmap_struct *vms, > if (vms->unlock) > mmap_write_downgrade(mm); > > - prev =3D vma_iter_prev_range(vms->vmi); > - next =3D vma_next(vms->vmi); > - if (next) > - vma_iter_prev_range(vms->vmi); > - > /* > * We can free page tables without write-locking mmap_lock becaus= e VMAs > * were isolated before we downgraded mmap_lock. > */ > mas_set(mas_detach, 1); > - unmap_region(mm, mas_detach, vms->vma, prev, next, vms->start, vm= s->end, > - vms->vma_count, !vms->unlock); > - /* Statistics and freeing VMAs */ > + unmap_region(mm, mas_detach, vms->vma, vms->prev, vms->next, > + vms->start, vms->end, vms->vma_count, !vms->unlock); > + /* Update high watermark before we lower total_vm */ > + update_hiwater_vm(mm); > + vms_vm_stat_account(vms); > mas_set(mas_detach, 0); > - remove_mt(mm, mas_detach); > + mas_for_each(mas_detach, vma, ULONG_MAX) > + remove_vma(vma, false); > + > + vm_unacct_memory(vms->nr_accounted); > validate_mm(mm); > if (vms->unlock) > mmap_read_unlock(mm); > @@ -3631,6 +3625,17 @@ void vm_stat_account(struct mm_struct *mm, vm_flag= s_t flags, long npages) > mm->data_vm +=3D npages; > } > > +/* Accounting for munmap */ > +static inline void vms_vm_stat_account(struct vma_munmap_struct *vms) > +{ > + struct mm_struct *mm =3D vms->mm; > + > + WRITE_ONCE(mm->total_vm, READ_ONCE(mm->total_vm) - vms->nr_pages)= ; > + mm->exec_vm -=3D vms->exec_vm; > + mm->stack_vm -=3D vms->stack_vm; > + mm->data_vm -=3D vms->data_vm; > +} > + > static vm_fault_t special_mapping_fault(struct vm_fault *vmf); > > /* > -- > 2.43.0 >