From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7EC12C6FD1D for ; Mon, 27 Mar 2023 23:23:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9F399900003; Mon, 27 Mar 2023 19:23:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9A341900002; Mon, 27 Mar 2023 19:23:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 86ADA900003; Mon, 27 Mar 2023 19:23:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 72899900002 for ; Mon, 27 Mar 2023 19:23:54 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 37F171202B9 for ; Mon, 27 Mar 2023 23:23:54 +0000 (UTC) X-FDA: 80616257988.09.2E3B55B Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) by imf06.hostedemail.com (Postfix) with ESMTP id 60718180008 for ; Mon, 27 Mar 2023 23:23:52 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=W+Tv96+p; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf06.hostedemail.com: domain of yosryahmed@google.com designates 209.85.208.43 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679959432; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2sthNEXRrhXPflNLlQaZ/R6XYUJ5sFT3zFaO3Oj8iQU=; b=NLJA0seWtRhcOmoPrHXvilFMjIjzMnK8Yp2+Z8ZeCf1OREBwXisbdNFjYSNKsb1m0RH9bi HDH/dqa9tRMQVR9yPrKbG5mApND8pthcDOp3yEy3q+kEVr7A7y6kD8n0fR+hdU9PUt5/Cn gFB8r/xfCEw8uAtD8SjN1+JAVdEAm8o= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=W+Tv96+p; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf06.hostedemail.com: domain of yosryahmed@google.com designates 209.85.208.43 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679959432; a=rsa-sha256; cv=none; b=tAyUiDgY/sceiZ9LMMEQOROpyJLlge0XvVOKz4RQoBVbczutzON6lIelh7dsWNPXm0171n unZH/dVH5h8kszPIQZZsdN/tZQ/NVzWwFfMXrjgyIVqfnTawU0KiRwguy/+yHQ75rEobQp xsblvbaYdig7mxtPK8aHBDIlaOOQb+A= Received: by mail-ed1-f43.google.com with SMTP id t10so42553035edd.12 for ; Mon, 27 Mar 2023 16:23:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1679959430; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=2sthNEXRrhXPflNLlQaZ/R6XYUJ5sFT3zFaO3Oj8iQU=; b=W+Tv96+pFUlhT5JOOrNtcy47rsqxINYX5Iydl2ZhOjD2ZICVF/srMD4pjGFArnDzTC kPELiLr2qm8nbVs2H3GBDurYFGHHHR4YppJMTiQgYYWDafqzi7oz9Du7VQA6lba1jKXq dwn2vZhC36MqALm/QJpx59PUuvFUdHNBjYQvAl4nTFdcu6Cb2GfBntbxfBbl6jFLwYkS xFSElExlWZVdm5acLMkTXu9+vioAED99Ij0oJd6h+lwLDOoAZY3FueFbhm3i8L+Th0Ha nQFF7w8Ehg6MPHtETaxtVvSE4I3f3hAepMquA1Zlg0t/TDXtMd7OTTczELyQE4Wsmo2Y 0q3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679959430; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2sthNEXRrhXPflNLlQaZ/R6XYUJ5sFT3zFaO3Oj8iQU=; b=bEXJuv4e9yF7CiZb+InozgJKoUeX7hjRnloONyu2gZMgIdQ87aDMRcoK9FBoOE/DHS Pck3PoA67MjlsXRTJbq51BOpKTrojbDaUSxcWkZ/xiBOZzJNfNExnK42bZ+urwwd9Ogt RKeqAbAzXUWiU2O8tJUWf4Brh0Y8lxSEP8YFe6PHRhRgpDR2V4GEn91UNu+jrmgnBMqk Ze6+OWfmTnsYygxXZpj0sLQWAFAjb3TSWqNBYODiRrsZqMMW9AueSmtDWM/+7tGCH9DY r9KkLpfuq2sDVfn8U6nMu/tTA18gLWYsbQiLkKnNhrQuZuUiD2zAuFTKgXSUVN23YVbb MNtw== X-Gm-Message-State: AAQBX9eQoGZpMWimliiwpd0vAAqslYO80LPaT2RnAeAAfKc9afVfGlWN y3MBiOTyBFX/qkOpK6/i4jWeoqKtVj6VkGfXsnt+ig== X-Google-Smtp-Source: AKy350axSMcFglrC9LKGtyy/jSyGyc9yKavwwCk6Ag/35Vw5JpZJlLk5zGZNXtD+EFFmpK2mYo3yPnkogMK64CtmxC8= X-Received: by 2002:a50:8e0d:0:b0:4fc:473d:3308 with SMTP id 13-20020a508e0d000000b004fc473d3308mr6749360edw.8.1679959430371; Mon, 27 Mar 2023 16:23:50 -0700 (PDT) MIME-Version: 1.0 References: <20230323040037.2389095-1-yosryahmed@google.com> <20230323040037.2389095-2-yosryahmed@google.com> In-Reply-To: From: Yosry Ahmed Date: Mon, 27 Mar 2023 16:23:13 -0700 Message-ID: Subject: Re: [RFC PATCH 1/7] cgroup: rstat: only disable interrupts for the percpu lock To: Shakeel Butt Cc: Tejun Heo , Josef Bacik , Jens Axboe , Zefan Li , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Andrew Morton , Vasily Averin , cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 60718180008 X-Stat-Signature: y1zcmrb9ar7fm7a4mb34fx4je46rk169 X-HE-Tag: 1679959432-572540 X-HE-Meta: U2FsdGVkX1+vQm+GEdvYgaldzYWuexGJByaZVcldN+5cSIRRHwozD5lmtA+gRy4ZydzIjtLOEzi1jxJyWkE8d/k2W4ufKDxrd6u/FcnxilTXfYLHqRUt68D3ciu5OE+a8h6IDtaA44ZTmrnATZ1nOiy2X32KbiOgbQsgt4cXrXu4f8uJ+/+prM1QqKYZdBK7mXDZ1niYOIbftY+aZZUO//qSUq1V5UlmJB/4vj9PYYYFCfl9ebUWiD8s6UC4Ww8H6JO2Vc7WqIS0GgfjsGSSWPyb7RlEWCoDyl0nXRwUm8xtLJ2O+z3S3z7i0u3a2AcsoC6BvAICNax4bAQcIy0s7GBFteoqoNueomrI7AGRoU42Qdp4mm4s1NX3bFvZSvGTCa7vPvi4jvY7emH7DWpKiy7b/YvCIbNHV1HIdKz7jPhPeI4pB/E6fGA/7Fb/X2dtA6e4Ue5WvgL/ZReklQ6ZkMYkZ9nJJ6KrdG3QwpuDgVXxqILM5rGlpDjBGHO6P3rKYrGZMRpiz5nBS3U7D+5mP9lO+6sa2e1/zM35+M0HzaJAc+lrzH43DFT4ZUM6lkVegpdofE4LUqTLmIanY+Z4ekmI3OPitYsWOAZIyaWy7z2SZYqVhubKngK7IUuz1cwePS+/Ie+ijEEp8f5oatysRc0fAbESxup9bCh0LNBOH0oCjkkKX4pZy/cAjVoe1Xw9I4lAVofrYHQorRztTMv8galPP20kR+7LzOaZhGFK0JkjiD+1XBdmdTcskxjzfGxXlT81aJp4cT1/cd8nFpUlhc4sbAfIKLy1qgwPNeBCoeJuXOdn0aMZ+eOr6x0bqqIz+6k0PaHNlmaIApPdQff2cYI/22CTfVXZQsshk4V7fdlz4d+eStXxgevUibqiIpCx0JJQ7IF3qIoUKE2SjKvE2t34ocbPJeGjxKpFabR1pEjakENvCGnTv9jCK/o9MjxScSArsmtw0Q6jDX7qa1m yE8A2u0o qxaAzesFV0X8eH791wm2oUYeDvEfsuVPcXAj3zhAUBUPm0s96Q7P0v4hnk2McGQOqeCraC1tTefDFB2iK1znRv50YVs+2tdv2kHere35yCvFoynVbe8Og7dMixB9ZftMA8hzLNI9aVjvOJUkSYvXdX4OvdnAGvRIsIUdQDXVvW47MxwoUi7XclNSYDDdaECIOH5qBGKADYJpVw1y839niXftdrmANtKzzVWhteUoPgNIGrzEj8RcgsHOJ7eeBeXlIYw89SPp3PfvTHLy52LQIdtNBJdnsGTyqYkQRDhPEf6Z8Pe0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Mar 24, 2023 at 9:46=E2=80=AFPM Shakeel Butt = wrote: > > On Fri, Mar 24, 2023 at 9:37=E2=80=AFPM Yosry Ahmed wrote: > > > > On Fri, Mar 24, 2023 at 9:31=E2=80=AFPM Shakeel Butt wrote: > > > > > > On Fri, Mar 24, 2023 at 7:18=E2=80=AFPM Yosry Ahmed wrote: > > > > > > > [...] > > > > Any ideas here are welcome! > > > > > > > > > > Let's move forward. It seems like we are not going to reach an > > > agreement on making cgroup_rstat_lock a non-irq lock. However there i= s > > > agreement on the memcg code of not flushing in irq context and the > > > cleanup Johannes has requested. Let's proceed with those for now. We > > > can come back to cgroup_rstat_lock later if we still see issues in > > > production. > > > > Even if we do not flush from irq context, we still flush from atomic > > contexts that will currently hold the lock with irqs disabled > > throughout the entire flush sequence. A primary purpose of this reason > > is to avoid that. > > > > We can either: > > (a) Proceed with the following approach of making cgroup_rstat_lock a > > non-irq lock. > > (b) Proceed with Tejun's suggestion of always releasing and > > reacquiring the lock at CPU boundaries, even for atomic flushes (if > > the spinlock needs a break ofc). > > (c) Something else. > > (d) keep the status quo regarding cgroup_rstat_lock > (e) decouple the discussion of cgroup_rstat_lock from the agreed > improvements. Send the patches for the agreed ones and continue > discussing cgroup_rstat_lock. Ah, I lost sight of the fact that the rest of the patch series does not strictly depend on this patch. I will respin the rest of the patch series separately. Thanks, Shakeel. Meanwhile, it would be useful to reach an agreement here to stop acquiring the cgroup_rstat_lock for a long time with irq disabled in atomic contexts. Tejun, if having the lock be non-irq is a non-starter for you, I can send a patch that instead gives up the lock and reacquires it at every CPU boundary unconditionally -- or perhaps every N CPU boundaries to avoid excessively releasing and reacquiring the lock. Something like: static void cgroup_rstat_flush_locked(struct cgroup *cgrp, bool may_sleep) { ... for_each_possible_cpu(cpu) { ... /* Always yield the at CPU boundaries to enable irqs */ spin_unlock_irq(&cgroup_rstat_lock); /* if @may_sleep, play nice and yield if necessary */ if (may_sleep) cond_resched(); spin_lock_irq(&cgroup_rstat_lock); } } If you have other ideas to avoid disabling irq's for the entire flush sequence I am also open to that. Thanks!