From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB335C433EF for ; Fri, 18 Feb 2022 12:21:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 214DE6B0075; Fri, 18 Feb 2022 07:21:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1C4D96B0078; Fri, 18 Feb 2022 07:21:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E949A6B007B; Fri, 18 Feb 2022 07:21:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0037.hostedemail.com [216.40.44.37]) by kanga.kvack.org (Postfix) with ESMTP id DACF16B0075 for ; Fri, 18 Feb 2022 07:21:38 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 8D15094FC3 for ; Fri, 18 Feb 2022 12:21:38 +0000 (UTC) X-FDA: 79155811476.13.4FDE8CB Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) by imf19.hostedemail.com (Postfix) with ESMTP id 0C6CE1A0009 for ; Fri, 18 Feb 2022 12:21:37 +0000 (UTC) Received: by mail-ej1-f50.google.com with SMTP id p15so14599713ejc.7 for ; Fri, 18 Feb 2022 04:21:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+FOZfGmK9Ee9HWyxoro93sTh2E++DXV5pvaMxG0mrBk=; b=S9daUL4DkyciT4OMAz6v9FmM+BjXNqe6WzHHFwVFSUo7kCrSp5MVb1P+B8l0XItu/X SybDR9ohCIDd49fCz9uTgoVdZezshyUCpvEFEHnLskNUfgus/zPK0aYbftWi3AnBPVAc 1k3Jk0uH+ATalwUnoRGFMM9y3K0qxlHchvZv/YCdanCmJfUil1zl0nP/AQoTObBhwEs8 QG/EZwVFd36fd5fzYkCBDis5gVDT/8UNgP2VnFU1i4daSgZtp8qwCj/nGRSk183SIdEl c8DdJG3DOwXGDuTUTQNO0UF/oNamOrN5bn5Oct7cJUZdzsy/S4yx+SoxKA9zzMDceAxF LRAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+FOZfGmK9Ee9HWyxoro93sTh2E++DXV5pvaMxG0mrBk=; b=qMUag943OQO3eTob1rFUiE1p6SgiRFKNX9EgGsfKH/y5jepLtgwrp0rHtjKpUzwWw1 WH7VpysK05zYdVmBtU1+hdFdkrXDw1pt+nTj+0aLfzG6omAF7C1SB19g07vWJfFktfNC u5kTwIYiVU8ifTv76ZPW/wetcIysSZHKpH2wKWFp59m2EZ4NdsfacevsooMQy5s3G7kE 3j2vUkqjMEJz/aNsSqipXMbrzbs4THitmoCnN7JACODjEUQArzuvetOMNwSY8/fZnmY/ xPYb8ZofGdXM54NHkayJkPJL5QOmEGNRsXHOAL3QYvpGoCjgdo9orifqMKz7BZihataO h1/w== X-Gm-Message-State: AOAM5336Nt7TbY5s3k72x5Th1fQM05oAJ/f/qZM9cyi2lEF7DU57CD+e Y0uZ36bWsKvahJFKvyHcCsk= X-Google-Smtp-Source: ABdhPJxLg4gyvuJjYcnBgsWo5jR7IyKPfX4GM9CvXIhx+VV6WyxqXMoi3zzVm+rcduHStchWBqty3Q== X-Received: by 2002:a17:906:1b13:b0:6ce:58d:4b78 with SMTP id o19-20020a1709061b1300b006ce058d4b78mr6324703ejg.515.1645186896729; Fri, 18 Feb 2022 04:21:36 -0800 (PST) Received: from orion.localdomain ([93.99.228.15]) by smtp.gmail.com with ESMTPSA id l8sm2141008ejp.198.2022.02.18.04.21.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Feb 2022 04:21:35 -0800 (PST) Received: by orion.localdomain (Postfix, from userid 1003) id 46728A7FF7; Fri, 18 Feb 2022 13:20:44 +0100 (CET) From: =?UTF-8?q?Jakub=20Mat=C4=9Bna?= To: linux-mm@kvack.org Cc: patches@lists.linux.dev, linux-kernel@vger.kernel.org, vbabka@suse.cz, mhocko@kernel.org, mgorman@techsingularity.net, willy@infradead.org, liam.howlett@oracle.com, hughd@google.com, kirill@shutemov.name, riel@surriel.com, rostedt@goodmis.org, peterz@infradead.org, =?UTF-8?q?Jakub=20Mat=C4=9Bna?= Subject: [RFC PATCH 2/4] [PATCH 2/4] mm: adjust page offset in mremap Date: Fri, 18 Feb 2022 13:20:17 +0100 Message-Id: <20220218122019.130274-3-matenajakub@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220218122019.130274-1-matenajakub@gmail.com> References: <20220218122019.130274-1-matenajakub@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-Rspam-User: Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=S9daUL4D; spf=pass (imf19.hostedemail.com: domain of matenajakub@gmail.com designates 209.85.218.50 as permitted sender) smtp.mailfrom=matenajakub@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 0C6CE1A0009 X-Stat-Signature: iaqihoi86h1hu9p8nhyzs3t8yuf99bb1 X-HE-Tag: 1645186897-160104 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Adjust page offset of a VMA when it's moved to a new location by mremap. This is made possible for all VMAs that do not share their anonymous pages with other processes. Previously this was possible only for not yet faulted VMAs. When the page offset does not correspond to the virtual address of the anonymous VMA any merge attempt with another VMA will fail. Signed-off-by: Jakub Mat=C4=9Bna --- mm/mmap.c | 101 ++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 95 insertions(+), 6 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index b55e11f20571..8d253b46b349 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -3224,6 +3224,91 @@ int insert_vm_struct(struct mm_struct *mm, struct = vm_area_struct *vma) return 0; } =20 +bool rbst_no_children(struct anon_vma *av, struct rb_node *node) +{ + struct anon_vma_chain *model; + struct anon_vma_chain *avc; + + if (node =3D=3D NULL) /* leaf node */ + return true; + avc =3D container_of(node, typeof(*(model)), rb); + if (avc->vma->anon_vma !=3D av) + /* + * Inequality implies avc belongs + * to a VMA of a child process + */ + return false; + return (rbst_no_children(av, node->rb_left) && + rbst_no_children(av, node->rb_right)); +} + +/* + * Check if none of the VMAs connected to the given + * anon_vma via anon_vma_chain are in child relationship + */ +bool rbt_no_children(struct anon_vma *av) +{ + struct rb_node *root_node; + + if (av =3D=3D NULL || av->degree <=3D 1) /* Higher degree might not nec= essarily imply children */ + return true; + root_node =3D av->rb_root.rb_root.rb_node; + return rbst_no_children(av, root_node); +} + +/** + * update_faulted_pgoff() - Update faulted pages of a vma + * @vma: VMA being moved + * @addr: new virtual address + * @pgoff: pointer to pgoff which is updated + * If the vma and its pages are not shared with another process, update + * the new pgoff and also update index parameter (copy of the pgoff) in + * all faulted pages. + */ +bool update_faulted_pgoff(struct vm_area_struct *vma, unsigned long addr= , pgoff_t *pgoff) +{ + unsigned long pg_iter =3D 0; + unsigned long pg_iters =3D (vma->vm_end - vma->vm_start) >> PAGE_SHIFT; + + /* 1.] Check vma is not shared with other processes */ + if (vma->anon_vma->root !=3D vma->anon_vma || !rbt_no_children(vma->ano= n_vma)) + return false; + + /* 2.] Check all pages are not shared */ + for (; pg_iter < pg_iters; ++pg_iter) { + bool pages_not_shared =3D true; + unsigned long shift =3D pg_iter << PAGE_SHIFT; + struct page *phys_page =3D follow_page(vma, vma->vm_start + shift, FOL= L_GET); + + if (phys_page =3D=3D NULL) + continue; + + /* Check page is not shared with other processes */ + if (page_mapcount(phys_page) > 1) + pages_not_shared =3D false; + put_page(phys_page); + if (!pages_not_shared) + return false; + } + + /* 3.] Update index in all pages to this new pgoff */ + pg_iter =3D 0; + *pgoff =3D addr >> PAGE_SHIFT; + + for (; pg_iter < pg_iters; ++pg_iter) { + unsigned long shift =3D pg_iter << PAGE_SHIFT; + struct page *phys_page =3D follow_page(vma, vma->vm_start + shift, FOL= L_GET); + + if (phys_page =3D=3D NULL) + continue; + lock_page(phys_page); + phys_page->index =3D *pgoff + pg_iter; + unlock_page(phys_page); + put_page(phys_page); + } + return true; +} + /* * Copy the vma structure to a new location in the same mm, * prior to moving page table entries, to effect an mremap move. @@ -3237,15 +3322,19 @@ struct vm_area_struct *copy_vma(struct vm_area_st= ruct **vmap, struct mm_struct *mm =3D vma->vm_mm; struct vm_area_struct *new_vma, *prev; struct rb_node **rb_link, *rb_parent; - bool faulted_in_anon_vma =3D true; + bool anon_pgoff_updated =3D false; =20 /* - * If anonymous vma has not yet been faulted, update new pgoff + * Try to update new pgoff for anonymous vma * to match new location, to increase its chance of merging. */ - if (unlikely(vma_is_anonymous(vma) && !vma->anon_vma)) { - pgoff =3D addr >> PAGE_SHIFT; - faulted_in_anon_vma =3D false; + if (unlikely(vma_is_anonymous(vma))) { + if (!vma->anon_vma) { + pgoff =3D addr >> PAGE_SHIFT; + anon_pgoff_updated =3D true; + } else { + anon_pgoff_updated =3D update_faulted_pgoff(vma, addr, &pgoff); + } } =20 if (find_vma_links(mm, addr, addr + len, &prev, &rb_link, &rb_parent)) @@ -3271,7 +3360,7 @@ struct vm_area_struct *copy_vma(struct vm_area_stru= ct **vmap, * safe. It is only safe to keep the vm_pgoff * linear if there are no pages mapped yet. */ - VM_BUG_ON_VMA(faulted_in_anon_vma, new_vma); + VM_BUG_ON_VMA(!anon_pgoff_updated, new_vma); *vmap =3D vma =3D new_vma; } *need_rmap_locks =3D (new_vma->vm_pgoff <=3D vma->vm_pgoff); --=20 2.34.1