From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D860C433FE for ; Fri, 11 Mar 2022 17:46:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6BD028D0005; Fri, 11 Mar 2022 12:46:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 698A28D0002; Fri, 11 Mar 2022 12:46:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2C2F18D0001; Fri, 11 Mar 2022 12:46:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0037.hostedemail.com [216.40.44.37]) by kanga.kvack.org (Postfix) with ESMTP id 1163E8D0002 for ; Fri, 11 Mar 2022 12:46:04 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id B9F36180FF4BE for ; Fri, 11 Mar 2022 17:46:03 +0000 (UTC) X-FDA: 79232833806.26.F15E66B Received: from mail-ej1-f42.google.com (mail-ej1-f42.google.com [209.85.218.42]) by imf07.hostedemail.com (Postfix) with ESMTP id 43CD54001B for ; Fri, 11 Mar 2022 17:46:03 +0000 (UTC) Received: by mail-ej1-f42.google.com with SMTP id gb39so20543930ejc.1 for ; Fri, 11 Mar 2022 09:46:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=16FTiziTuHY7luYyqhUgZZQe9XBDNooPz6CdRNgABNY=; b=oxr+bbXplN7JLlsYtHsTMq87tB9qjjwJovy4Qmphj7sIBHnv5iE72yNg7suTzlkG7R oINry5tchEi9fK7FZ7LUmaPJ+jCwkyU4D/uYP1hDIDuAHb5l5rVXjHcij7yoXtKuJ2Zp enoa7/O0Z/su2Af9pCcQsKXgeplEeEeWn/P99XC2ffJEz02lS85YB1M8YGyi/hpMUmf1 ChQ9zzI46OgPcsPId44t45Xpz1NWniqZIQo00/15hexDGgVRQkUiwrC0I30gCFe2bue3 CtMBKIRZHu7FIrQ/cg6ZOxrbEKmIPRnJXzW/DvCQWv2G24+B32J7iO8vAxujZM6w1EVv xiyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=16FTiziTuHY7luYyqhUgZZQe9XBDNooPz6CdRNgABNY=; b=2GxS5GfFuzgvO+GfWBIywG6Oe72LLeE/VCRh6x4A4kIe2dl3WkuiEWR7d13EBQQZta hVJKxMpsGGjwb145ktVoz3leuhWY3Y+KXg4uMP1dU/TJBJrc15C/DrnrkMomixkVvIR5 lvuOCpfpWhJJJ8hjknQ1umcY5ZYvYbkcWQbMsts444K1Y9+5kCH4dTOG/G51ZKZ8KwF+ apNuhjxC1WLlhvlujED4FBugXyZEYLXrJgLF0u2c/A6fenJtJz3u3drLgORc5NJP1ZVv +uZuQpqWZrsI7aeexrZ3h9bzIlKW5MsW1uocHwCWVqj1vrmXs5K02UcyEJbVgff8HUcj 2NFA== X-Gm-Message-State: AOAM533HlmheCd8XNcAxnWirJl/0dCjmCEwNVUWfECPxyJndFS7EDxVI 4ejQ8J/MZ3TQ6JL8KhaUydI= X-Google-Smtp-Source: ABdhPJwrEvlJ1jGKDiP915MpwaiTas/VtSW176/53uKYGRh7P+2bKjGc5/e1w+Yl1fYaK6RmZ7HAyQ== X-Received: by 2002:a17:907:8a0a:b0:6d8:85a6:4d42 with SMTP id sc10-20020a1709078a0a00b006d885a64d42mr9993383ejc.138.1647020761822; Fri, 11 Mar 2022 09:46:01 -0800 (PST) Received: from orion.localdomain ([93.99.228.15]) by smtp.gmail.com with ESMTPSA id k23-20020a1709062a5700b006ccd8fdc300sm3166909eje.180.2022.03.11.09.46.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 11 Mar 2022 09:46:01 -0800 (PST) Received: by orion.localdomain (Postfix, from userid 1003) id 8C736A0482; Fri, 11 Mar 2022 18:46:03 +0100 (CET) From: =?UTF-8?q?Jakub=20Mat=C4=9Bna?= To: linux-mm@kvack.org Cc: patches@lists.linux.dev, linux-kernel@vger.kernel.org, vbabka@suse.cz, mhocko@kernel.org, mgorman@techsingularity.net, willy@infradead.org, liam.howlett@oracle.com, hughd@google.com, kirill@shutemov.name, riel@surriel.com, rostedt@goodmis.org, peterz@infradead.org, =?UTF-8?q?Jakub=20Mat=C4=9Bna?= Subject: [RFC PATCH v2 4/4] [PATCH 4/4] mm: add tracing for VMA merges Date: Fri, 11 Mar 2022 18:46:02 +0100 Message-Id: <20220311174602.288010-5-matenajakub@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220311174602.288010-1-matenajakub@gmail.com> References: <20220311174602.288010-1-matenajakub@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-Rspamd-Queue-Id: 43CD54001B X-Stat-Signature: kamherhidonpswhbhnsyw5n5iq1u46kq Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=oxr+bbXp; spf=pass (imf07.hostedemail.com: domain of matenajakub@gmail.com designates 209.85.218.42 as permitted sender) smtp.mailfrom=matenajakub@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1647020763-269542 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Adds trace support for vma_merge to measure successful and unsuccessful merges of two VMAs with distinct anon_vmas and also trace support for merges made possible by update of page offset made possible by a previous patch in this series. Signed-off-by: Jakub Mat=C4=9Bna --- include/trace/events/mmap.h | 83 +++++++++++++++++++++++++++++++++++++ mm/internal.h | 12 ++++++ mm/mmap.c | 69 ++++++++++++++++-------------- 3 files changed, 133 insertions(+), 31 deletions(-) diff --git a/include/trace/events/mmap.h b/include/trace/events/mmap.h index 4661f7ba07c0..bad7abe4899c 100644 --- a/include/trace/events/mmap.h +++ b/include/trace/events/mmap.h @@ -6,6 +6,27 @@ #define _TRACE_MMAP_H =20 #include +#include <../mm/internal.h> + +#define AV_MERGE_TYPES \ + EM(MERGE_FAILED) \ + EM(AV_MERGE_FAILED) \ + EM(AV_MERGE_NULL) \ + EM(AV_MERGE_SAME) \ + EMe(AV_MERGE_DIFFERENT) + +#undef EM +#undef EMe +#define EM(a) TRACE_DEFINE_ENUM(a); +#define EMe(a) TRACE_DEFINE_ENUM(a); + +AV_MERGE_TYPES + +#undef EM +#undef EMe + +#define EM(a) { a, #a }, +#define EMe(a) { a, #a } =20 TRACE_EVENT(vm_unmapped_area, =20 @@ -42,6 +63,68 @@ TRACE_EVENT(vm_unmapped_area, __entry->low_limit, __entry->high_limit, __entry->align_mask, __entry->align_offset) ); + +TRACE_EVENT(vm_av_merge, + + TP_PROTO(int merged, enum vma_merge_res merge_prev, + enum vma_merge_res merge_next, enum vma_merge_res merge_both), + + TP_ARGS(merged, merge_prev, merge_next, merge_both), + + TP_STRUCT__entry( + __field(int, merged) + __field(enum vma_merge_res, predecessor_different_av) + __field(enum vma_merge_res, successor_different_av) + __field(enum vma_merge_res, predecessor_with_successor_different_av) + __field(int, same_count) + __field(int, diff_count) + __field(int, failed_count) + ), + + TP_fast_assign( + __entry->merged =3D merged =3D=3D 0; + __entry->predecessor_different_av =3D merge_prev; + __entry->successor_different_av =3D merge_next; + __entry->predecessor_with_successor_different_av =3D merge_both; + __entry->same_count =3D (merge_prev =3D=3D AV_MERGE_SAME) + + (merge_next =3D=3D AV_MERGE_SAME) + + (merge_both =3D=3D AV_MERGE_SAME); + __entry->diff_count =3D (merge_prev =3D=3D AV_MERGE_DIFFERENT) + + (merge_next =3D=3D AV_MERGE_DIFFERENT) + + (merge_both =3D=3D AV_MERGE_DIFFERENT); + __entry->failed_count =3D (merge_prev =3D=3D AV_MERGE_FAILED) + + (merge_next =3D=3D AV_MERGE_FAILED) + + (merge_both =3D=3D AV_MERGE_FAILED); + ), + + TP_printk("merged=3D%d predecessor=3D%s successor=3D%s predecessor_with= _successor=3D%s same_count=3D%d diff_count=3D%d failed_count=3D%d", + __entry->merged, + __print_symbolic(__entry->predecessor_different_av, AV_MERGE_TYPES), + __print_symbolic(__entry->successor_different_av, AV_MERGE_TYPES), + __print_symbolic(__entry->predecessor_with_successor_different_av, AV_= MERGE_TYPES), + __entry->same_count, __entry->diff_count, __entry->failed_count) + +); + +TRACE_EVENT(vm_pgoff_merge, + + TP_PROTO(struct vm_area_struct *vma, bool anon_pgoff_updated), + + TP_ARGS(vma, anon_pgoff_updated), + + TP_STRUCT__entry( + __field(bool, faulted) + __field(bool, updated) + ), + + TP_fast_assign( + __entry->faulted =3D vma->anon_vma; + __entry->updated =3D anon_pgoff_updated; + ), + + TP_printk("faulted=3D%d updated=3D%d\n", + __entry->faulted, __entry->updated) +); #endif =20 /* This part must be outside protection */ diff --git a/mm/internal.h b/mm/internal.h index d80300392a19..860169612192 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -34,6 +34,18 @@ struct folio_batch; /* Do not use these with a slab allocator */ #define GFP_SLAB_BUG_MASK (__GFP_DMA32|__GFP_HIGHMEM|~__GFP_BITS_MASK) =20 +/* + * Following values indicate reason for merge success or failure. + */ +enum vma_merge_res { + MERGE_FAILED, + AV_MERGE_FAILED, + AV_MERGE_NULL, + MERGE_OK =3D AV_MERGE_NULL, + AV_MERGE_SAME, + AV_MERGE_DIFFERENT, +}; + void page_writeback_init(void); =20 static inline void *folio_raw_mapping(struct folio *folio) diff --git a/mm/mmap.c b/mm/mmap.c index ccb24862e670..663f8ec46f2c 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1064,21 +1064,21 @@ static inline int is_mergeable_anon_vma(struct an= on_vma *anon_vma1, */ if ((!anon_vma1 || !anon_vma2) && (!vma || list_is_singular(&vma->anon_vma_chain))) - return 1; + return AV_MERGE_NULL; if (anon_vma1 =3D=3D anon_vma2) - return 1; + return AV_MERGE_SAME; /* * Different anon_vma but not shared by several processes */ else if ((anon_vma1 && anon_vma2) && (anon_vma1 =3D=3D anon_vma1->root) && (rbt_no_children(anon_vma1))) - return 1; + return AV_MERGE_DIFFERENT; /* * Different anon_vma and shared -> unmergeable */ else - return 0; + return AV_MERGE_FAILED; } =20 /* @@ -1099,12 +1099,10 @@ can_vma_merge_before(struct vm_area_struct *vma, = unsigned long vm_flags, struct vm_userfaultfd_ctx vm_userfaultfd_ctx, const char *anon_name) { - if (is_mergeable_vma(vma, file, vm_flags, vm_userfaultfd_ctx, anon_name= ) && - is_mergeable_anon_vma(anon_vma, vma->anon_vma, vma)) { + if (is_mergeable_vma(vma, file, vm_flags, vm_userfaultfd_ctx, anon_name= )) if (vma->vm_pgoff =3D=3D vm_pgoff) - return 1; - } - return 0; + return is_mergeable_anon_vma(anon_vma, vma->anon_vma, vma); + return MERGE_FAILED; } =20 /* @@ -1121,14 +1119,13 @@ can_vma_merge_after(struct vm_area_struct *vma, u= nsigned long vm_flags, struct vm_userfaultfd_ctx vm_userfaultfd_ctx, const char *anon_name) { - if (is_mergeable_vma(vma, file, vm_flags, vm_userfaultfd_ctx, anon_name= ) && - is_mergeable_anon_vma(anon_vma, vma->anon_vma, vma)) { + if (is_mergeable_vma(vma, file, vm_flags, vm_userfaultfd_ctx, anon_name= )) { pgoff_t vm_pglen; vm_pglen =3D vma_pages(vma); if (vma->vm_pgoff + vm_pglen =3D=3D vm_pgoff) - return 1; + return is_mergeable_anon_vma(anon_vma, vma->anon_vma, vma); } - return 0; + return MERGE_FAILED; } =20 /* @@ -1185,8 +1182,14 @@ struct vm_area_struct *vma_merge(struct mm_struct = *mm, pgoff_t pglen =3D (end - addr) >> PAGE_SHIFT; struct vm_area_struct *area, *next; int err =3D -1; - bool merge_prev =3D false; - bool merge_next =3D false; + /* + * Following three variables are used to store values + * indicating wheather this VMA and its anon_vma can + * be merged and also the type of failure or success. + */ + enum vma_merge_res merge_prev =3D MERGE_FAILED; + enum vma_merge_res merge_both =3D MERGE_FAILED; + enum vma_merge_res merge_next =3D MERGE_FAILED; =20 /* * We later require that vma->vm_flags =3D=3D vm_flags, @@ -1207,32 +1210,34 @@ struct vm_area_struct *vma_merge(struct mm_struct= *mm, =20 /* Can we merge the predecessor? */ if (prev && prev->vm_end =3D=3D addr && - mpol_equal(vma_policy(prev), policy) && - can_vma_merge_after(prev, vm_flags, + mpol_equal(vma_policy(prev), policy)) { + merge_prev =3D can_vma_merge_after(prev, vm_flags, anon_vma, file, pgoff, - vm_userfaultfd_ctx, anon_name)) { - merge_prev =3D true; - area =3D prev; + vm_userfaultfd_ctx, anon_name); } + /* Can we merge the successor? */ if (next && end =3D=3D next->vm_start && - mpol_equal(policy, vma_policy(next)) && - can_vma_merge_before(next, vm_flags, - anon_vma, file, pgoff+pglen, - vm_userfaultfd_ctx, anon_name)) { - merge_next =3D true; + mpol_equal(policy, vma_policy(next))) { + merge_next =3D can_vma_merge_before(next, vm_flags, + anon_vma, file, pgoff+pglen, + vm_userfaultfd_ctx, anon_name); } + /* Can we merge both the predecessor and the successor? */ - if (merge_prev && merge_next && - is_mergeable_anon_vma(prev->anon_vma, - next->anon_vma, NULL)) { /* cases 1, 6 */ + if (merge_prev >=3D MERGE_OK && merge_next >=3D MERGE_OK) + merge_both =3D is_mergeable_anon_vma(prev->anon_vma, next->anon_vma, N= ULL); + + if (merge_both >=3D MERGE_OK) { /* cases 1, 6 */ err =3D __vma_adjust(prev, prev->vm_start, next->vm_end, prev->vm_pgoff, NULL, prev); - } else if (merge_prev) { /* cases 2, 5, 7 */ + area =3D prev; + } else if (merge_prev >=3D MERGE_OK) { /* cases 2, 5, 7 */ err =3D __vma_adjust(prev, prev->vm_start, end, prev->vm_pgoff, NULL, prev); - } else if (merge_next) { + area =3D prev; + } else if (merge_next >=3D MERGE_OK) { if (prev && addr < prev->vm_end) /* case 4 */ err =3D __vma_adjust(prev, prev->vm_start, addr, prev->vm_pgoff, NULL, next); @@ -1246,7 +1251,7 @@ struct vm_area_struct *vma_merge(struct mm_struct *= mm, */ area =3D next; } - + trace_vm_av_merge(err, merge_prev, merge_next, merge_both); /* * Cannot merge with predecessor or successor or error in __vma_adjust? */ @@ -3321,6 +3326,8 @@ struct vm_area_struct *copy_vma(struct vm_area_stru= ct **vmap, /* * Source vma may have been merged into new_vma */ + trace_vm_pgoff_merge(vma, anon_pgoff_updated); + if (unlikely(vma_start >=3D new_vma->vm_start && vma_start < new_vma->vm_end)) { /* --=20 2.34.1