From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52346C433E0 for ; Sat, 27 Jun 2020 07:32:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0B9A02081A for ; Sat, 27 Jun 2020 07:32:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="t6EZGTFG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0B9A02081A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BA1426B007B; Sat, 27 Jun 2020 03:32:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B4E446B007E; Sat, 27 Jun 2020 03:32:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A163B6B0080; Sat, 27 Jun 2020 03:32:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0040.hostedemail.com [216.40.44.40]) by kanga.kvack.org (Postfix) with ESMTP id 834046B007B for ; Sat, 27 Jun 2020 03:32:37 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 755B7824556B for ; Sat, 27 Jun 2020 07:32:36 +0000 (UTC) X-FDA: 76974174312.24.limit64_040e3c326e5c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id 454821A4A0 for ; Sat, 27 Jun 2020 07:32:36 +0000 (UTC) X-HE-Tag: limit64_040e3c326e5c X-Filterd-Recvd-Size: 10358 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf24.hostedemail.com (Postfix) with ESMTP for ; Sat, 27 Jun 2020 07:32:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=nuU16mRXYOwpUOjAQFerIPbRDEcrM6bzFgPQvCmulVk=; b=t6EZGTFGBUvjVe9FE06mcfQP9/ n767z99i4jIPRAXgH9/mbSMB7z2/YITi3XlIWlAfZ5cEVNhbPatenibpnHRDe/sl6/rEwIBXSRZBh B5R6PgNKGt+bqJ46jUTvqi5oHz7KhhZcabwle7BRHv23uOHRjQ2KQu6uSFklyLr/I4Mao10HglTSW 7bvpNcxEXH1jO1UB9hjBc3/AnBPBaQ2+hLaVIO6krs64UiNsJp6N3UnZA7+qwoRU69G/KRrUCowuq embVsxiF/hDPQndhKoOtkLYTDv8CKCU0vL1mV9afQzj5X0EUgRI16ySiZNq9IhV1d2iyKgpiJqeMJ U18LHFig==; Received: from [2001:4bb8:184:76e3:595:ba65:ae56:65a6] (helo=localhost) by casper.infradead.org with esmtpsa (Exim 4.92.3 #3 (Red Hat Linux)) id 1jp5Jz-0006n0-L2; Sat, 27 Jun 2020 07:32:21 +0000 From: Christoph Hellwig To: Jens Axboe , Tejun Heo Cc: Dennis Zhou , Li Zefan , Johannes Weiner , dm-devel@redhat.com, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 06/14] block: move the bio cgroup associatation helpers to blk-cgroup.c Date: Sat, 27 Jun 2020 09:31:51 +0200 Message-Id: <20200627073159.2447325-7-hch@lst.de> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627073159.2447325-1-hch@lst.de> References: <20200627073159.2447325-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Queue-Id: 454821A4A0 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Keep the cgroup code together. Signed-off-by: Christoph Hellwig --- block/bio.c | 75 --------------------------- block/blk-cgroup.c | 103 ++++++++++++++++++++++++++++++++++++- include/linux/blk-cgroup.h | 30 ----------- 3 files changed, 101 insertions(+), 107 deletions(-) diff --git a/block/bio.c b/block/bio.c index 901d22715dd4f3..fc1299f9d86a24 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1627,81 +1627,6 @@ int bioset_init_from_src(struct bio_set *bs, struc= t bio_set *src) } EXPORT_SYMBOL(bioset_init_from_src); =20 -#ifdef CONFIG_BLK_CGROUP -/** - * bio_associate_blkg_from_css - associate a bio with a specified css - * @bio: target bio - * @css: target css - * - * Associate @bio with the blkg found by combining the css's blkg and th= e - * request_queue of the @bio. An association failure is handled by walk= ing up - * the blkg tree. Therefore, the blkg associated can be anything betwee= n @blkg - * and q->root_blkg. This situation only happens when a cgroup is dying= and - * then the remaining bios will spill to the closest alive blkg. - * - * A reference will be taken on the blkg and will be released when @bio = is - * freed. - */ -void bio_associate_blkg_from_css(struct bio *bio, - struct cgroup_subsys_state *css) -{ - struct request_queue *q =3D bio->bi_disk->queue; - struct blkcg_gq *blkg =3D q->root_blkg; - - if (bio->bi_blkg) - blkg_put(bio->bi_blkg); - - rcu_read_lock(); - if (css && css->parent) - blkg =3D blkg_lookup_create(css_to_blkcg(css), q); - bio->bi_blkg =3D blkg_tryget_closest(blkg); - rcu_read_unlock(); -} -EXPORT_SYMBOL_GPL(bio_associate_blkg_from_css); - -/** - * bio_associate_blkg - associate a bio with a blkg - * @bio: target bio - * - * Associate @bio with the blkg found from the bio's css and request_que= ue. - * If one is not found, bio_lookup_blkg() creates the blkg. If a blkg i= s - * already associated, the css is reused and association redone as the - * request_queue may have changed. - */ -void bio_associate_blkg(struct bio *bio) -{ - struct cgroup_subsys_state *css; - - rcu_read_lock(); - - if (bio->bi_blkg) - css =3D &bio_blkcg(bio)->css; - else - css =3D blkcg_css(); - - bio_associate_blkg_from_css(bio, css); - - rcu_read_unlock(); -} -EXPORT_SYMBOL_GPL(bio_associate_blkg); - -/** - * bio_clone_blkg_association - clone blkg association from src to dst b= io - * @dst: destination bio - * @src: source bio - */ -void bio_clone_blkg_association(struct bio *dst, struct bio *src) -{ - if (src->bi_blkg) { - if (dst->bi_blkg) - blkg_put(dst->bi_blkg); - blkg_get(src->bi_blkg); - dst->bi_blkg =3D src->bi_blkg; - } -} -EXPORT_SYMBOL_GPL(bio_clone_blkg_association); -#endif /* CONFIG_BLK_CGROUP */ - static void __init biovec_init_slabs(void) { int i; diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index 0ecc897b225c96..bb0607bfd771cd 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -328,7 +328,7 @@ static struct blkcg_gq *blkg_create(struct blkcg *blk= cg, * Returns the blkg or the closest blkg if blkg_create() fails as it wal= ks * down from root. */ -struct blkcg_gq *__blkg_lookup_create(struct blkcg *blkcg, +static struct blkcg_gq *__blkg_lookup_create(struct blkcg *blkcg, struct request_queue *q) { struct blkcg_gq *blkg; @@ -377,7 +377,7 @@ struct blkcg_gq *__blkg_lookup_create(struct blkcg *b= lkcg, * This looks up or creates the blkg representing the unique pair * of the blkcg and the request_queue. */ -struct blkcg_gq *blkg_lookup_create(struct blkcg *blkcg, +static struct blkcg_gq *blkg_lookup_create(struct blkcg *blkcg, struct request_queue *q) { struct blkcg_gq *blkg =3D blkg_lookup(blkcg, q); @@ -1727,6 +1727,105 @@ void blkcg_add_delay(struct blkcg_gq *blkg, u64 n= ow, u64 delta) atomic64_add(delta, &blkg->delay_nsec); } =20 +/** + * blkg_tryget_closest - try and get a blkg ref on the closet blkg + * @blkg: blkg to get + * + * This needs to be called rcu protected. As the failure mode here is t= o walk + * up the blkg tree, this ensure that the blkg->parent pointers are alwa= ys + * valid. This returns the blkg that it ended up taking a reference on = or %NULL + * if no reference was taken. + */ +static inline struct blkcg_gq *blkg_tryget_closest(struct blkcg_gq *blkg= ) +{ + struct blkcg_gq *ret_blkg =3D NULL; + + WARN_ON_ONCE(!rcu_read_lock_held()); + + while (blkg) { + if (blkg_tryget(blkg)) { + ret_blkg =3D blkg; + break; + } + blkg =3D blkg->parent; + } + + return ret_blkg; +} + +/** + * bio_associate_blkg_from_css - associate a bio with a specified css + * @bio: target bio + * @css: target css + * + * Associate @bio with the blkg found by combining the css's blkg and th= e + * request_queue of the @bio. An association failure is handled by walk= ing up + * the blkg tree. Therefore, the blkg associated can be anything betwee= n @blkg + * and q->root_blkg. This situation only happens when a cgroup is dying= and + * then the remaining bios will spill to the closest alive blkg. + * + * A reference will be taken on the blkg and will be released when @bio = is + * freed. + */ +void bio_associate_blkg_from_css(struct bio *bio, + struct cgroup_subsys_state *css) +{ + struct request_queue *q =3D bio->bi_disk->queue; + struct blkcg_gq *blkg =3D q->root_blkg; + + if (bio->bi_blkg) + blkg_put(bio->bi_blkg); + + rcu_read_lock(); + if (css && css->parent) + blkg =3D blkg_lookup_create(css_to_blkcg(css), q); + bio->bi_blkg =3D blkg_tryget_closest(blkg); + rcu_read_unlock(); +} +EXPORT_SYMBOL_GPL(bio_associate_blkg_from_css); + +/** + * bio_associate_blkg - associate a bio with a blkg + * @bio: target bio + * + * Associate @bio with the blkg found from the bio's css and request_que= ue. + * If one is not found, bio_lookup_blkg() creates the blkg. If a blkg i= s + * already associated, the css is reused and association redone as the + * request_queue may have changed. + */ +void bio_associate_blkg(struct bio *bio) +{ + struct cgroup_subsys_state *css; + + rcu_read_lock(); + + if (bio->bi_blkg) + css =3D &bio_blkcg(bio)->css; + else + css =3D blkcg_css(); + + bio_associate_blkg_from_css(bio, css); + + rcu_read_unlock(); +} +EXPORT_SYMBOL_GPL(bio_associate_blkg); + +/** + * bio_clone_blkg_association - clone blkg association from src to dst b= io + * @dst: destination bio + * @src: source bio + */ +void bio_clone_blkg_association(struct bio *dst, struct bio *src) +{ + if (src->bi_blkg) { + if (dst->bi_blkg) + blkg_put(dst->bi_blkg); + blkg_get(src->bi_blkg); + dst->bi_blkg =3D src->bi_blkg; + } +} +EXPORT_SYMBOL_GPL(bio_clone_blkg_association); + static int __init blkcg_init(void) { blkcg_punt_bio_wq =3D alloc_workqueue("blkcg_punt_bio", diff --git a/include/linux/blk-cgroup.h b/include/linux/blk-cgroup.h index a57ebe2f00abd8..60df97202314c7 100644 --- a/include/linux/blk-cgroup.h +++ b/include/linux/blk-cgroup.h @@ -183,10 +183,6 @@ extern bool blkcg_debug_stats; =20 struct blkcg_gq *blkg_lookup_slowpath(struct blkcg *blkcg, struct request_queue *q, bool update_hint); -struct blkcg_gq *__blkg_lookup_create(struct blkcg *blkcg, - struct request_queue *q); -struct blkcg_gq *blkg_lookup_create(struct blkcg *blkcg, - struct request_queue *q); int blkcg_init_queue(struct request_queue *q); void blkcg_exit_queue(struct request_queue *q); =20 @@ -480,32 +476,6 @@ static inline bool blkg_tryget(struct blkcg_gq *blkg= ) return blkg && percpu_ref_tryget(&blkg->refcnt); } =20 -/** - * blkg_tryget_closest - try and get a blkg ref on the closet blkg - * @blkg: blkg to get - * - * This needs to be called rcu protected. As the failure mode here is t= o walk - * up the blkg tree, this ensure that the blkg->parent pointers are alwa= ys - * valid. This returns the blkg that it ended up taking a reference on = or %NULL - * if no reference was taken. - */ -static inline struct blkcg_gq *blkg_tryget_closest(struct blkcg_gq *blkg= ) -{ - struct blkcg_gq *ret_blkg =3D NULL; - - WARN_ON_ONCE(!rcu_read_lock_held()); - - while (blkg) { - if (blkg_tryget(blkg)) { - ret_blkg =3D blkg; - break; - } - blkg =3D blkg->parent; - } - - return ret_blkg; -} - /** * blkg_put - put a blkg reference * @blkg: blkg to put --=20 2.26.2