From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 979A0C433FE for ; Fri, 18 Feb 2022 12:21:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 853986B007D; Fri, 18 Feb 2022 07:21:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 804126B007E; Fri, 18 Feb 2022 07:21:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6A4586B0080; Fri, 18 Feb 2022 07:21:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0185.hostedemail.com [216.40.44.185]) by kanga.kvack.org (Postfix) with ESMTP id 4ACBB6B007D for ; Fri, 18 Feb 2022 07:21:41 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 0914A59AF for ; Fri, 18 Feb 2022 12:21:41 +0000 (UTC) X-FDA: 79155811602.06.8929B77 Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) by imf10.hostedemail.com (Postfix) with ESMTP id 8AC12C0004 for ; Fri, 18 Feb 2022 12:21:40 +0000 (UTC) Received: by mail-ej1-f48.google.com with SMTP id k25so14608484ejp.5 for ; Fri, 18 Feb 2022 04:21:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=K9wD2Tr0KWFRwLLWFyz8BogF6D34vQqJ2tLSqwNrziY=; b=f894Oja0ePJSA3RxY1LN0hdK2YQrjn1gxhMzZU+xzq6qi0xHMLoxnxvT4ddpsm2RJY HLIOBTRmGR2mUDCeXMXugR5d4VGHIOlXe7xqB0FiOY6buAJUUw7AK1Tx7CbkIQ4P6TG5 0MsFuge5LxVPFGg9z5QE6rFGWa152KtVJxJjn6ILg3xeIC/WoPZ2gUnKeHv46AT1lbtv MYSFkdPz9/sQxPcCPMXZevKjjg+hl17dURo+b/gGozQ21wgsSc1n23KNT5dmAp8UWXqv R+C5f06QDVXDAGf5MEtsYiCLrLyjJOFSO41PSKJ/OxMJeWcmVQJhcs7qkmQBVGl1vIpI 36rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=K9wD2Tr0KWFRwLLWFyz8BogF6D34vQqJ2tLSqwNrziY=; b=JrhHcPeVpIjZ6W5t3eJlFY8VfqYlg1dEwIOPXt/RfKMxzr5xj1QnL0SpHja75gXMIN SNUKWnUv69OBlRs1bI857gbrMoTDB8KpfOzbJvp5m66f/UiBJXA66YYcZe52NLwonP6q pmBhR6lOCa/ffzCoqDLnilszym3lSowJTVmSifSC8hweYDBFJGsgnx2J1dVaLcOCQWYN oflNcbDXQYkNVhT2mWGb7v+xqRLSR7i3p+d9MuOFf/4B+nvm2XN9FX8PW5Nogm4xoA2i Ux4sk3XPkAN+6wRojTF3KiZLr2cYVk04K0oUYW2TllW1MkxjQrMukQBLwHfdpL6CRram EM+g== X-Gm-Message-State: AOAM532VF7E2caH436b3rNM9VttadjgtLX5FN5syx8ZufotwE8upL28Q 6qYpi12AyrFKJdU1v2cGgOE= X-Google-Smtp-Source: ABdhPJxcgIZ+10B44aMtj9wOPRGY7/4HPeI8NqgjmXec4wFs3wYKpL0ExtoiBsWf7RxBrDdDrvp5Fw== X-Received: by 2002:a17:906:3708:b0:6cf:bb34:9d2e with SMTP id d8-20020a170906370800b006cfbb349d2emr6083708ejc.665.1645186896545; Fri, 18 Feb 2022 04:21:36 -0800 (PST) Received: from orion.localdomain ([93.99.228.15]) by smtp.gmail.com with ESMTPSA id e18sm4736506edj.85.2022.02.18.04.21.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Feb 2022 04:21:35 -0800 (PST) Received: by orion.localdomain (Postfix, from userid 1003) id 49536A7FFD; Fri, 18 Feb 2022 13:20:44 +0100 (CET) From: =?UTF-8?q?Jakub=20Mat=C4=9Bna?= To: linux-mm@kvack.org Cc: patches@lists.linux.dev, linux-kernel@vger.kernel.org, vbabka@suse.cz, mhocko@kernel.org, mgorman@techsingularity.net, willy@infradead.org, liam.howlett@oracle.com, hughd@google.com, kirill@shutemov.name, riel@surriel.com, rostedt@goodmis.org, peterz@infradead.org, =?UTF-8?q?Jakub=20Mat=C4=9Bna?= Subject: [RFC PATCH 3/4] [PATCH 3/4] mm: enable merging of VMAs with different anon_vmas Date: Fri, 18 Feb 2022 13:20:18 +0100 Message-Id: <20220218122019.130274-4-matenajakub@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220218122019.130274-1-matenajakub@gmail.com> References: <20220218122019.130274-1-matenajakub@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 8AC12C0004 X-Stat-Signature: af98gukwh3d3kb61713uijyojmbdzk7u Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=f894Oja0; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of matenajakub@gmail.com designates 209.85.218.48 as permitted sender) smtp.mailfrom=matenajakub@gmail.com X-HE-Tag: 1645186900-252349 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Enable merging of a VMA even when it is linked to different anon_vma than the one it is being merged to, but only if the VMA in question does not share any page with a parent or child process. Every anonymous page stores a pointer to its anon_vma in the parameter mapping, which is now updated as part of the merge process. Signed-off-by: Jakub Mat=C4=9Bna --- include/linux/rmap.h | 17 ++++++++++++++++- mm/mmap.c | 15 ++++++++++++++- mm/rmap.c | 40 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 70 insertions(+), 2 deletions(-) diff --git a/include/linux/rmap.h b/include/linux/rmap.h index e704b1a4c06c..c8508a4ebc46 100644 --- a/include/linux/rmap.h +++ b/include/linux/rmap.h @@ -137,10 +137,13 @@ static inline void anon_vma_unlock_read(struct anon= _vma *anon_vma) */ void anon_vma_init(void); /* create anon_vma_cachep */ int __anon_vma_prepare(struct vm_area_struct *); +void reconnect_pages(struct vm_area_struct *vma, struct vm_area_struct *= next); void unlink_anon_vmas(struct vm_area_struct *); int anon_vma_clone(struct vm_area_struct *, struct vm_area_struct *); int anon_vma_fork(struct vm_area_struct *, struct vm_area_struct *); =20 +bool rbt_no_children(struct anon_vma *av); + static inline int anon_vma_prepare(struct vm_area_struct *vma) { if (likely(vma->anon_vma)) @@ -149,10 +152,22 @@ static inline int anon_vma_prepare(struct vm_area_s= truct *vma) return __anon_vma_prepare(vma); } =20 +/** + * anon_vma_merge() - Merge anon_vmas of the given VMAs + * @vma: VMA being merged to + * @next: VMA being merged + */ static inline void anon_vma_merge(struct vm_area_struct *vma, struct vm_area_struct *next) { - VM_BUG_ON_VMA(vma->anon_vma !=3D next->anon_vma, vma); + struct anon_vma *anon_vma1 =3D vma->anon_vma; + struct anon_vma *anon_vma2 =3D next->anon_vma; + + VM_BUG_ON_VMA(anon_vma1 && anon_vma2 && anon_vma1 !=3D anon_vma2 && + ((anon_vma2 !=3D anon_vma2->root) + || !rbt_no_children(anon_vma2)), vma); + + reconnect_pages(vma, next); unlink_anon_vmas(next); } =20 diff --git a/mm/mmap.c b/mm/mmap.c index 8d253b46b349..ed91d0cd2111 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1065,7 +1065,20 @@ static inline int is_mergeable_anon_vma(struct ano= n_vma *anon_vma1, if ((!anon_vma1 || !anon_vma2) && (!vma || list_is_singular(&vma->anon_vma_chain))) return 1; - return anon_vma1 =3D=3D anon_vma2; + if (anon_vma1 =3D=3D anon_vma2) + return 1; + /* + * Different anon_vma but not shared by several processes + */ + else if ((anon_vma1 && anon_vma2) && + (anon_vma1 =3D=3D anon_vma1->root) + && (rbt_no_children(anon_vma1))) + return 1; + /* + * Different anon_vma and shared -> unmergeable + */ + else + return 0; } =20 /* diff --git a/mm/rmap.c b/mm/rmap.c index 6a1e8c7f6213..1093b518b0be 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -387,6 +387,46 @@ int anon_vma_fork(struct vm_area_struct *vma, struct= vm_area_struct *pvma) return -ENOMEM; } =20 +/** + * reconnect_pages() - Reconnect physical pages from old to vma + * @vma: VMA to newly contain all physical pages of old + * @old: old VMA being merged to vma + */ +void reconnect_pages(struct vm_area_struct *vma, struct vm_area_struct *= old) +{ + struct anon_vma *anon_vma1 =3D vma->anon_vma; + struct anon_vma *anon_vma2 =3D old->anon_vma; + unsigned long pg_iter; + int pg_iters; + + if (anon_vma1 =3D=3D anon_vma2 || anon_vma1 =3D=3D NULL || anon_vma2 =3D= =3D NULL) + return; /* Nothing to do */ + + /* Modify page->mapping for all pages in old */ + pg_iter =3D 0; + pg_iters =3D (old->vm_end - old->vm_start) >> PAGE_SHIFT; + + for (; pg_iter < pg_iters; ++pg_iter) { + /* Get the physical page */ + unsigned long shift =3D pg_iter << PAGE_SHIFT; + struct page *phys_page =3D follow_page(old, old->vm_start + shift, FOL= L_GET); + struct anon_vma *page_anon_vma; + + /* Do some checks and lock the page */ + if (phys_page =3D=3D NULL) + continue; /* Virtual memory page is not mapped */ + lock_page(phys_page); + page_anon_vma =3D page_get_anon_vma(phys_page); + if (page_anon_vma !=3D NULL) { /* NULL in case of ZERO_PAGE */ + VM_BUG_ON_VMA(page_anon_vma !=3D old->anon_vma, old); + /* Update physical page's mapping */ + page_move_anon_rmap(phys_page, vma); + } + unlock_page(phys_page); + put_page(phys_page); + } +} + void unlink_anon_vmas(struct vm_area_struct *vma) { struct anon_vma_chain *avc, *next; --=20 2.34.1