From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DD4BC433F5 for ; Tue, 21 Dec 2021 15:02:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AEE2D6B0082; Tue, 21 Dec 2021 10:02:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A9E396B0083; Tue, 21 Dec 2021 10:02:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9180D6B0085; Tue, 21 Dec 2021 10:02:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0063.hostedemail.com [216.40.44.63]) by kanga.kvack.org (Postfix) with ESMTP id 7F31B6B0082 for ; Tue, 21 Dec 2021 10:02:08 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 4203A8BEEE for ; Tue, 21 Dec 2021 15:02:08 +0000 (UTC) X-FDA: 78942116736.23.B85DEC9 Received: from mail-qv1-f44.google.com (mail-qv1-f44.google.com [209.85.219.44]) by imf28.hostedemail.com (Postfix) with ESMTP id 9030AC00A6 for ; Tue, 21 Dec 2021 15:01:52 +0000 (UTC) Received: by mail-qv1-f44.google.com with SMTP id kj16so8660187qvb.2 for ; Tue, 21 Dec 2021 07:01:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=jaGLmQY52APl8v98fQta2llNqqHuAoOAllJQBEPRHxk=; b=eBNsLd7ARojSEaIKr1fNBnwoyaIGKyZ1YHAKdIfcl9DOJfyYTkckGPZbs6qzYnm8KF UJGzZZh8frzcL5dLuDfj2s2helG8nS8rsopxSfDzz9tw/cLdXPv2mCjPvrstIh/YQPYO MbaoqWmM6lsBlKCaN1uZQkuIZ41kY/Gcg1uwL/9zkiAhhYTohOSNbbZBjIn2BSzk9Nm0 q/s7BnOn/guBwmkQKFZUpYCqMI3XtWH6SDbIOS1QqygMsL4z0JmE0xbqcgBZAaN02AU7 azLIyYmAUcTHEF5X85XZmwKKXfDKkG6YkmrdODo9qNwJSKd0aZg2I6SaJzvz7h66dyAp JdJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jaGLmQY52APl8v98fQta2llNqqHuAoOAllJQBEPRHxk=; b=OgluuxbejXyc9hUFS9uwV9k48Wo/1WkSKDCEuXm/w7dx7oj6vg1aX8B63GBlIBrGPN znTN0CNdcMHlopJnK7rTJSAxpa1RtsvMF+LUzVumXyp28hGHFv2KDLVatfIyasLwm9ed ASQwwvcgORYnws+1ksnRkRF1vAGzJwPRa2pE2kLwek+XtQP+KCbyXyn3H1Zclr2c39N6 XCIXWSl9VEBoePk7o2wSVI6dMX9LMHwFbJr3UaXHOC59GesrPYtMGNMc5UR8RWichcXf vATQDSkb6zkUlAmZ8f3KizX2N75y/80ZLHS8TnXDOj97WvL4Dc5aMYtt7Qpj6fNaIuqh L1pg== X-Gm-Message-State: AOAM532Ri6gIaeukSnJ2pJEjcLejwmpVbaMwVaYT9J9K7TuQDEH0Exov 4RjSOTJZptTxrTCnk1YXr6oOqg== X-Google-Smtp-Source: ABdhPJwMlFRiSYhQumZA31hM2R0ePuuAlQ2ZhyLXodIAC4YlYRRVox4HfZF1uSX5tXZr7Hd0UOKJ3A== X-Received: by 2002:a0c:8e08:: with SMTP id v8mr2635635qvb.93.1640098910549; Tue, 21 Dec 2021 07:01:50 -0800 (PST) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id d4sm1991371qkn.79.2021.12.21.07.01.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Dec 2021 07:01:50 -0800 (PST) From: Pasha Tatashin To: pasha.tatashin@soleen.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-m68k@lists.linux-m68k.org, anshuman.khandual@arm.com, willy@infradead.org, akpm@linux-foundation.org, william.kucharski@oracle.com, mike.kravetz@oracle.com, vbabka@suse.cz, geert@linux-m68k.org, schmitzmic@gmail.com, rostedt@goodmis.org, mingo@redhat.com, hannes@cmpxchg.org, guro@fb.com, songmuchun@bytedance.com, weixugc@google.com, gthelen@google.com, rientjes@google.com, pjt@google.com Subject: [PATCH v2 7/9] mm: simplify page_ref_* functions Date: Tue, 21 Dec 2021 15:01:38 +0000 Message-Id: <20211221150140.988298-8-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.34.1.307.g9b7440fafd-goog In-Reply-To: <20211221150140.988298-1-pasha.tatashin@soleen.com> References: <20211221150140.988298-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 9030AC00A6 X-Stat-Signature: 7s4oren8kcnc4n1sm3ndfffqpb19ex7b Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=eBNsLd7A; dmarc=none; spf=pass (imf28.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.219.44 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com X-HE-Tag: 1640098912-234296 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now, that we are using atomic_fetch* variants to add/sub/inc/dec page _refcount, it makes sense to combined page_ref_* return and non return functions. Also remove some extra trace points for non-return variants. This improves the tracability by always recording the new _refcount value after the modifications has occurred. Signed-off-by: Pasha Tatashin --- include/linux/page_ref.h | 102 +++++++++----------------------- include/trace/events/page_ref.h | 18 +----- mm/debug_page_ref.c | 14 ----- 3 files changed, 31 insertions(+), 103 deletions(-) diff --git a/include/linux/page_ref.h b/include/linux/page_ref.h index d7316881626c..243fc60ae6c8 100644 --- a/include/linux/page_ref.h +++ b/include/linux/page_ref.h @@ -8,8 +8,6 @@ #include =20 DECLARE_TRACEPOINT(page_ref_init); -DECLARE_TRACEPOINT(page_ref_mod); -DECLARE_TRACEPOINT(page_ref_mod_and_test); DECLARE_TRACEPOINT(page_ref_mod_and_return); DECLARE_TRACEPOINT(page_ref_mod_unless); DECLARE_TRACEPOINT(page_ref_freeze); @@ -27,8 +25,6 @@ DECLARE_TRACEPOINT(page_ref_unfreeze); #define page_ref_tracepoint_active(t) tracepoint_enabled(t) =20 extern void __page_ref_init(struct page *page); -extern void __page_ref_mod(struct page *page, int v); -extern void __page_ref_mod_and_test(struct page *page, int v, int ret); extern void __page_ref_mod_and_return(struct page *page, int v, int ret)= ; extern void __page_ref_mod_unless(struct page *page, int v, int u); extern void __page_ref_freeze(struct page *page, int v, int ret); @@ -41,12 +37,6 @@ extern void __page_ref_unfreeze(struct page *page, int= v); static inline void __page_ref_init(struct page *page) { } -static inline void __page_ref_mod(struct page *page, int v) -{ -} -static inline void __page_ref_mod_and_test(struct page *page, int v, int= ret) -{ -} static inline void __page_ref_mod_and_return(struct page *page, int v, i= nt ret) { } @@ -127,12 +117,7 @@ static inline int page_ref_add_return(struct page *p= age, int nr) =20 static inline void page_ref_add(struct page *page, int nr) { - int old_val =3D atomic_fetch_add(nr, &page->_refcount); - int new_val =3D old_val + nr; - - VM_BUG_ON_PAGE((unsigned int)new_val < (unsigned int)old_val, page); - if (page_ref_tracepoint_active(page_ref_mod)) - __page_ref_mod(page, nr); + page_ref_add_return(page, nr); } =20 static inline void folio_ref_add(struct folio *folio, int nr) @@ -140,30 +125,25 @@ static inline void folio_ref_add(struct folio *foli= o, int nr) page_ref_add(&folio->page, nr); } =20 -static inline void page_ref_sub(struct page *page, int nr) +static inline int page_ref_sub_return(struct page *page, int nr) { int old_val =3D atomic_fetch_sub(nr, &page->_refcount); int new_val =3D old_val - nr; =20 VM_BUG_ON_PAGE((unsigned int)new_val > (unsigned int)old_val, page); - if (page_ref_tracepoint_active(page_ref_mod)) - __page_ref_mod(page, -nr); + if (page_ref_tracepoint_active(page_ref_mod_and_return)) + __page_ref_mod_and_return(page, -nr, new_val); + return new_val; } =20 -static inline void folio_ref_sub(struct folio *folio, int nr) +static inline void page_ref_sub(struct page *page, int nr) { - page_ref_sub(&folio->page, nr); + page_ref_sub_return(page, nr); } =20 -static inline int page_ref_sub_return(struct page *page, int nr) +static inline void folio_ref_sub(struct folio *folio, int nr) { - int old_val =3D atomic_fetch_sub(nr, &page->_refcount); - int new_val =3D old_val - nr; - - VM_BUG_ON_PAGE((unsigned int)new_val > (unsigned int)old_val, page); - if (page_ref_tracepoint_active(page_ref_mod_and_return)) - __page_ref_mod_and_return(page, -nr, new_val); - return new_val; + page_ref_sub(&folio->page, nr); } =20 static inline int folio_ref_sub_return(struct folio *folio, int nr) @@ -171,14 +151,20 @@ static inline int folio_ref_sub_return(struct folio= *folio, int nr) return page_ref_sub_return(&folio->page, nr); } =20 -static inline void page_ref_inc(struct page *page) +static inline int page_ref_inc_return(struct page *page) { int old_val =3D atomic_fetch_inc(&page->_refcount); int new_val =3D old_val + 1; =20 VM_BUG_ON_PAGE((unsigned int)new_val < (unsigned int)old_val, page); - if (page_ref_tracepoint_active(page_ref_mod)) - __page_ref_mod(page, 1); + if (page_ref_tracepoint_active(page_ref_mod_and_return)) + __page_ref_mod_and_return(page, 1, new_val); + return new_val; +} + +static inline void page_ref_inc(struct page *page) +{ + page_ref_inc_return(page); } =20 static inline void folio_ref_inc(struct folio *folio) @@ -186,14 +172,20 @@ static inline void folio_ref_inc(struct folio *foli= o) page_ref_inc(&folio->page); } =20 -static inline void page_ref_dec(struct page *page) +static inline int page_ref_dec_return(struct page *page) { int old_val =3D atomic_fetch_dec(&page->_refcount); int new_val =3D old_val - 1; =20 VM_BUG_ON_PAGE((unsigned int)new_val > (unsigned int)old_val, page); - if (page_ref_tracepoint_active(page_ref_mod)) - __page_ref_mod(page, -1); + if (page_ref_tracepoint_active(page_ref_mod_and_return)) + __page_ref_mod_and_return(page, -1, new_val); + return new_val; +} + +static inline void page_ref_dec(struct page *page) +{ + page_ref_dec_return(page); } =20 static inline void folio_ref_dec(struct folio *folio) @@ -203,14 +195,7 @@ static inline void folio_ref_dec(struct folio *folio= ) =20 static inline int page_ref_sub_and_test(struct page *page, int nr) { - int old_val =3D atomic_fetch_sub(nr, &page->_refcount); - int new_val =3D old_val - nr; - int ret =3D new_val =3D=3D 0; - - VM_BUG_ON_PAGE((unsigned int)new_val > (unsigned int)old_val, page); - if (page_ref_tracepoint_active(page_ref_mod_and_test)) - __page_ref_mod_and_test(page, -nr, ret); - return ret; + return page_ref_sub_return(page, nr) =3D=3D 0; } =20 static inline int folio_ref_sub_and_test(struct folio *folio, int nr) @@ -218,17 +203,6 @@ static inline int folio_ref_sub_and_test(struct foli= o *folio, int nr) return page_ref_sub_and_test(&folio->page, nr); } =20 -static inline int page_ref_inc_return(struct page *page) -{ - int old_val =3D atomic_fetch_inc(&page->_refcount); - int new_val =3D old_val + 1; - - VM_BUG_ON_PAGE((unsigned int)new_val < (unsigned int)old_val, page); - if (page_ref_tracepoint_active(page_ref_mod_and_return)) - __page_ref_mod_and_return(page, 1, new_val); - return new_val; -} - static inline int folio_ref_inc_return(struct folio *folio) { return page_ref_inc_return(&folio->page); @@ -236,14 +210,7 @@ static inline int folio_ref_inc_return(struct folio = *folio) =20 static inline int page_ref_dec_and_test(struct page *page) { - int old_val =3D atomic_fetch_dec(&page->_refcount); - int new_val =3D old_val - 1; - int ret =3D new_val =3D=3D 0; - - VM_BUG_ON_PAGE((unsigned int)new_val > (unsigned int)old_val, page); - if (page_ref_tracepoint_active(page_ref_mod_and_test)) - __page_ref_mod_and_test(page, -1, ret); - return ret; + return page_ref_dec_return(page) =3D=3D 0; } =20 static inline int folio_ref_dec_and_test(struct folio *folio) @@ -251,17 +218,6 @@ static inline int folio_ref_dec_and_test(struct foli= o *folio) return page_ref_dec_and_test(&folio->page); } =20 -static inline int page_ref_dec_return(struct page *page) -{ - int old_val =3D atomic_fetch_dec(&page->_refcount); - int new_val =3D old_val - 1; - - VM_BUG_ON_PAGE((unsigned int)new_val > (unsigned int)old_val, page); - if (page_ref_tracepoint_active(page_ref_mod_and_return)) - __page_ref_mod_and_return(page, -1, new_val); - return new_val; -} - static inline int folio_ref_dec_return(struct folio *folio) { return page_ref_dec_return(&folio->page); diff --git a/include/trace/events/page_ref.h b/include/trace/events/page_= ref.h index 87551bb1df9e..35cd795aa7c6 100644 --- a/include/trace/events/page_ref.h +++ b/include/trace/events/page_ref.h @@ -49,7 +49,7 @@ DEFINE_EVENT(page_ref_init_template, page_ref_init, TP_ARGS(page) ); =20 -DECLARE_EVENT_CLASS(page_ref_mod_template, +DECLARE_EVENT_CLASS(page_ref_unfreeze_template, =20 TP_PROTO(struct page *page, int v), =20 @@ -83,13 +83,6 @@ DECLARE_EVENT_CLASS(page_ref_mod_template, __entry->val) ); =20 -DEFINE_EVENT(page_ref_mod_template, page_ref_mod, - - TP_PROTO(struct page *page, int v), - - TP_ARGS(page, v) -); - DECLARE_EVENT_CLASS(page_ref_mod_and_test_template, =20 TP_PROTO(struct page *page, int v, int ret), @@ -126,13 +119,6 @@ DECLARE_EVENT_CLASS(page_ref_mod_and_test_template, __entry->val, __entry->ret) ); =20 -DEFINE_EVENT(page_ref_mod_and_test_template, page_ref_mod_and_test, - - TP_PROTO(struct page *page, int v, int ret), - - TP_ARGS(page, v, ret) -); - DEFINE_EVENT(page_ref_mod_and_test_template, page_ref_mod_and_return, =20 TP_PROTO(struct page *page, int v, int ret), @@ -154,7 +140,7 @@ DEFINE_EVENT(page_ref_mod_and_test_template, page_ref= _freeze, TP_ARGS(page, v, ret) ); =20 -DEFINE_EVENT(page_ref_mod_template, page_ref_unfreeze, +DEFINE_EVENT(page_ref_unfreeze_template, page_ref_unfreeze, =20 TP_PROTO(struct page *page, int v), =20 diff --git a/mm/debug_page_ref.c b/mm/debug_page_ref.c index e32149734122..1de9d93cca25 100644 --- a/mm/debug_page_ref.c +++ b/mm/debug_page_ref.c @@ -12,20 +12,6 @@ void __page_ref_init(struct page *page) EXPORT_SYMBOL(__page_ref_init); EXPORT_TRACEPOINT_SYMBOL(page_ref_init); =20 -void __page_ref_mod(struct page *page, int v) -{ - trace_page_ref_mod(page, v); -} -EXPORT_SYMBOL(__page_ref_mod); -EXPORT_TRACEPOINT_SYMBOL(page_ref_mod); - -void __page_ref_mod_and_test(struct page *page, int v, int ret) -{ - trace_page_ref_mod_and_test(page, v, ret); -} -EXPORT_SYMBOL(__page_ref_mod_and_test); -EXPORT_TRACEPOINT_SYMBOL(page_ref_mod_and_test); - void __page_ref_mod_and_return(struct page *page, int v, int ret) { trace_page_ref_mod_and_return(page, v, ret); --=20 2.34.1.307.g9b7440fafd-goog