From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77B71C4167D for ; Mon, 6 Nov 2023 20:58:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EC2CF8D002A; Mon, 6 Nov 2023 15:58:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E72EF8D0001; Mon, 6 Nov 2023 15:58:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3AA08D002A; Mon, 6 Nov 2023 15:58:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C246F8D0001 for ; Mon, 6 Nov 2023 15:58:23 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 94150A08A0 for ; Mon, 6 Nov 2023 20:58:23 +0000 (UTC) X-FDA: 81428742486.07.FCC60C3 Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) by imf16.hostedemail.com (Postfix) with ESMTP id AB9A3180008 for ; Mon, 6 Nov 2023 20:58:21 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=O3FcnuYi; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.54 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699304301; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZKl8aQiZoT5/Gc1O1HhE3ZzebbwHEfVwbiUJglhZ9UY=; b=3hxcxPDhPxxIMlV466Zp3Rf4QdQECfr4QlzI+04rDkK5Nsl94ZxkL3hjgAujd4FPCARuvI YiV9uGn2D6lf77JelGoFDzxCra5UW5Q7nAciR6qZdvjpmHbkJQDue34M/xIvSInuQpugnp XLD+G/iQ5T9yhvaH0E/14xWV104BABE= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=O3FcnuYi; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf16.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.54 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699304301; a=rsa-sha256; cv=none; b=H4OuQQa6ySDCGcPUDTQyv9KXxirGolDAWQL07wg10n52xau25x6hO6Ilr0pwQibdCmY1L5 uC7GjggBhfY7RATOsj8gawOWvNYAZn4PtknccVBFzlw7lBAxRK3oXdDO2Kmj6nuC0rC3Dh TwWHKF3zO+83LLoXFGKY3DcrSIDPF0U= Received: by mail-ej1-f54.google.com with SMTP id a640c23a62f3a-9c5b313b3ffso740035066b.0 for ; Mon, 06 Nov 2023 12:58:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1699304300; x=1699909100; darn=kvack.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=ZKl8aQiZoT5/Gc1O1HhE3ZzebbwHEfVwbiUJglhZ9UY=; b=O3FcnuYiPKZDEh3O9bZzdqkdmhTqUllsRa2fYSgmTO+tPG5pEA3LL4fjbGxZfjetN2 kizKhmVx2tkvlmJVyg3X9mjy0u+j1kyWQp5Ax18CyJJui/3T1eRy4HkOIFnbLIChQdCF WEsNBoWdHDoaRt526p2piVcSwLdeM5DKg5JQ3rrLCG01/gIgsynZAnnxToSvaOtXEWiT n3OAlmwoXxEM34VUdwp3MuMO/IyHlGRGi9xtirmgkWJtPWL4tuOJfQZn9FUOLaoAA4Yn MqRhpNfL3dZxrrqbGfdp6QlVWHfCcLaun3lTcAHIjDOaq9oxRyX787koojsV3PybW49K 5voQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699304300; x=1699909100; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ZKl8aQiZoT5/Gc1O1HhE3ZzebbwHEfVwbiUJglhZ9UY=; b=OLutZ59uVUlMhP62qzPxzD6w/njXsBM6+YFB7jsnxOQvewAiwF+E2npevYaP1M0SUn VTIu3HRCX9frKxUDfCdphPZbIqeG04zbp8vAudA/03eyUTMMLBaSBvgvAOXZwixfIt+o ETAMVAFQ+ezTYcSqOG2bpVa4BvmiCvnNhC/lV9aCcLmCJclbM8TdmQ26RM/De5NYTvwO tyFPPl/wVmvF2RH6nI7RZeOILdeqtbEoHNdwDW5j7kjYG0iTFj8M8NIIlezVUMZ4ffxR q9/ZbZz99qc12eF64RApQinbQW8Lq2v1rjr8Q+QShYAuT5jE4tj7VHSn6ZKBM9DhHajA M8kA== X-Gm-Message-State: AOJu0Yz1mjn2y9rMiw4e6gGuknVzGTqvwBCGX0wRrAxS3y7CvK0Jpt3e gjLyYJuSXaMPXiloEt203BQIOrmNjGXW83l+r9G27Q== X-Google-Smtp-Source: AGHT+IFrCXffedtA3mNIcSD9CL5cZrq9gg6yV7X2n90hA137Lzb005Ix2ogp4oXgiw+o+Nq7W19r1UCfCcsA5LCR+3Y= X-Received: by 2002:a17:906:da87:b0:9b7:37de:601a with SMTP id xh7-20020a170906da8700b009b737de601amr15301903ejb.49.1699304299984; Mon, 06 Nov 2023 12:58:19 -0800 (PST) MIME-Version: 1.0 References: <20231106183159.3562879-1-nphamcs@gmail.com> <20231106183159.3562879-4-nphamcs@gmail.com> In-Reply-To: From: Yosry Ahmed Date: Mon, 6 Nov 2023 12:57:44 -0800 Message-ID: Subject: Re: [PATCH v5 3/6] zswap: make shrinking memcg-aware To: Nhat Pham Cc: akpm@linux-foundation.org, hannes@cmpxchg.org, cerasuolodomenico@gmail.com, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, chrisl@kernel.org, linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, shuah@kernel.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: AB9A3180008 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 4wbfr1kzhymg6uepqdrhpdwe6mo8tigy X-HE-Tag: 1699304301-211412 X-HE-Meta: U2FsdGVkX1+zCGHD8L32gOELdkNpI1JVfD++sVJn5Azy7LuLOnw25m01PscAwqWtbS7O461hujsM2c3lvPwnwBb7pOZFnZiSLbobQf62XlOc+Iny173mrj/4knph5ENd3VLtIm3Gz7Gzt4SoYp5IPCSXe0fLklPadctt8wLEH5jp4PG/x+0e5JFjrcFmvUKiaYaHBX1qjX3soq8g49EYkr6zj9ejZiBH9djiuh8LTqHo6oIIo9KKU+kKDNwx65ymSucjkKo8BtvCpSgLDFGWOYKK1EGYnzsSjGTcD5bk7jdkvXHRc7Lon21XhQKwHTG5trx5r1CAT3LTbq0VJAqXeTwIY/8JNoY+0tSG6oLkueoRzF8tRBdwJ2dUi9wDBu4vTSSB4NwNr8DmauoEZARIKp3sJxvmNHkqcCT8Y0IatjRJ4l07qQOa0GovcRnyC5JCJ5JmpP2IlHx33F6Fj0anjbhvaWRQJ8RZuqBxcDNdr7SpMUyhs3+vejdmgYDGHgXrJsx1eIqyIvVf9hHvhC0Ji3pYCnh2u3+OhlFjqzG8rw7UB1WTrxQlckY4hK5MUNBg9L1qXYATgU3BIE76mxofZwpI2nfmFEqPcsGDpTSQZQpF6W7AL9rnhh4A/sS1S/+38iFO5LLiv11xuI4seJf9KYNb0/27b7MSC3h55vkiCYci1wxuSxNfvZW9K471vQSQNFlYVdEQN0HaTY2xLdk4ujfZ4JvUq2qM9a2VrHV9+K1KY2Eb+i3Tj7oywJU40OOiujJvk3n4niPIOScXoyriONMKhv412JRW+96Tw+RQ3EXRcZ+QQULMSSMEBDYjX0VdD3rmA8eihUb+toVZ0X9QaI55d3AxfYMa2KpX+68ORKPjxgEm5Pod9voavQfTd82ovUXpgYL9q0Iia4JmAtCXo8RvCU2ipm6RhOVqXpN3CT7oB8zmRg3lQaj+lTfcKUkCM/YPhZbOFRBTHjBLK40 F95e69w2 32hvD7z4rkoGyHakMLQyUo0td2JQ9UGtM6ytwC+BgDL9dKjLfYcx4jBuVobxmj1wjVqRiIzdsAP8k2M74dhomU5ohwtCZKDLtPHbSJIpijcms+dk11XJMlBScdd4DyjMSsPm2GknQ5zNKXO0LbrjMHvzjt5ERUx4eoL0miyVT7tObQEvwIFFFDv1Fm8F/Yi44HF153rbkt0pg0c+lOeS617J4wKkkMDQTF1/T X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: > > > > This lock is only needed to synchronize updating pool->next_shrink, > > right? Can we just use atomic operations instead? (e.g. cmpxchg()). > > I'm not entirely sure. I think in the pool destroy path, we have to also > put the next_shrink memcg, so there's that. We can use xchg() to replace it with NULL, then put the memcg ref, no? We can also just hold zswap_pools_lock while shrinking the memcg perhaps? It's not a contended lock anyway. It just feels weird to add a spinlock to protect one pointer. > > > > > > + if (pool->next_shrink == memcg) > > > + pool->next_shrink = > > > + mem_cgroup_iter(NULL, pool->next_shrink, NULL, true); > > > + spin_unlock(&pool->next_shrink_lock); > > > + } > > > + spin_unlock(&zswap_pools_lock); > > > +} > > > + > > > /********************************* > > > * zswap entry functions > > > **********************************/ > > > static struct kmem_cache *zswap_entry_cache; > > > > > > -static struct zswap_entry *zswap_entry_cache_alloc(gfp_t gfp) > > > +static struct zswap_entry *zswap_entry_cache_alloc(gfp_t gfp, int nid) > > > { > > > struct zswap_entry *entry; > > > - entry = kmem_cache_alloc(zswap_entry_cache, gfp); > > > + entry = kmem_cache_alloc_node(zswap_entry_cache, gfp, nid); > > > if (!entry) > > > return NULL; > > > entry->refcount = 1; > > [..] > > > @@ -1233,15 +1369,15 @@ bool zswap_store(struct folio *folio) > > > zswap_invalidate_entry(tree, dupentry); > > > } > > > spin_unlock(&tree->lock); > > > - > > > - /* > > > - * XXX: zswap reclaim does not work with cgroups yet. Without a > > > - * cgroup-aware entry LRU, we will push out entries system-wide based on > > > - * local cgroup limits. > > > - */ > > > objcg = get_obj_cgroup_from_folio(folio); > > > - if (objcg && !obj_cgroup_may_zswap(objcg)) > > > - goto reject; > > > + if (objcg && !obj_cgroup_may_zswap(objcg)) { > > > + memcg = get_mem_cgroup_from_objcg(objcg); > > > + if (shrink_memcg(memcg)) { > > > + mem_cgroup_put(memcg); > > > + goto reject; > > > + } > > > + mem_cgroup_put(memcg); > > > > Can we just use RCU here as well? (same around memcg_list_lru_alloc() > > call below). > > For memcg_list_lru_alloc(): there's potentially sleeping in that piece of > code I believe? I believe at the very least we'll have to use this gfp_t > flag for it to be rcu-safe: > > GFP_KERNEL | __GFP_NORETRY | __GFP_NOMEMALLOC | __GFP_NOWARN > not sure the > > Same go for this particular place IIRC - there's some sleeping done > in zswap_writeback_entry(), correct? Ah right, I missed this. My bad.