From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62CA7E7717F for ; Thu, 12 Dec 2024 11:09:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DDBF16B0082; Thu, 12 Dec 2024 06:09:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D64C26B0095; Thu, 12 Dec 2024 06:09:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C065C6B0096; Thu, 12 Dec 2024 06:09:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9CBCC6B0082 for ; Thu, 12 Dec 2024 06:09:23 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 4D167817E1 for ; Thu, 12 Dec 2024 11:09:23 +0000 (UTC) X-FDA: 82886035344.04.3261B18 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf25.hostedemail.com (Postfix) with ESMTP id 7F20AA000C for ; Thu, 12 Dec 2024 11:09:03 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JdU9Adqq; spf=pass (imf25.hostedemail.com: domain of gmonaco@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=gmonaco@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734001738; a=rsa-sha256; cv=none; b=LR4sbYV4UtkbhiJ9tkEqr1v6AVSgXg7TpwldTdttjBYUAYX5BTNVYyaW9+zoayVZyWi0pV /eXp2tkV9zoof0I6Kx2e0nLVzRCBAJyHDH45VxR18Ke61mYH4FB/AzHTO/D7kxoc/r0TCy tTLJ/ZDimZhNi/HhV7wfa9k6KH79xLE= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JdU9Adqq; spf=pass (imf25.hostedemail.com: domain of gmonaco@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=gmonaco@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734001738; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NVCzg2aQMKAJiIzGPvWwj8kbPMgicpJ+TDwewG2vM1M=; b=kRyTraK/90kOkzTgyaJFEpyx6UXcFMrqUx1cxKpRsL5/wY8MubkHiNvVww5byd9vpolzbI 6qOFoOhPCBvN8wodOKDXs8QzISSv/D3usfZrK+EBGiAxBj+PkkDvZIiLBaM23Ms4W3OAyr 9AewP7e0bfys9xyP+73hqEKvwh+gXRE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1734001760; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=NVCzg2aQMKAJiIzGPvWwj8kbPMgicpJ+TDwewG2vM1M=; b=JdU9Adqq9sSMB4Cy5eb9Hit6or7vOgJiUEgstYc1V2hO4cW0DyNlvxVdNg4Q14iG8YyMa2 oJsBtIXZ0VNV0oxxvO32mNzmltySh0rEvMu5cqCAmECWH8QoRyGQxvikmAWJGoPD+/LUwQ uIvhFqy1SUwvWb2Yn5LHh0xPwhuBaxg= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-623-WE-WNHIZPBaP0_wippZMGg-1; Thu, 12 Dec 2024 06:09:17 -0500 X-MC-Unique: WE-WNHIZPBaP0_wippZMGg-1 X-Mimecast-MFC-AGG-ID: WE-WNHIZPBaP0_wippZMGg Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-4361efc9d1fso4327795e9.2 for ; Thu, 12 Dec 2024 03:09:17 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734001756; x=1734606556; h=mime-version:user-agent:content-transfer-encoding:autocrypt :references:in-reply-to:date:cc:to:from:subject:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NVCzg2aQMKAJiIzGPvWwj8kbPMgicpJ+TDwewG2vM1M=; b=vsN3BHl5/YLQft00R7sLKWsBC5w7ELp39MWFn6Psd9iYCS3CymilZcJ5mnykdJngVP 8YVp8Pe/A+MER6Lc4zEIa+yYMulfcgQKqE2ybNzeOKns/berPbi3E+5QP8RADx0VasI1 0U/868bS7P5hoFecKPVpZvjp0ZPscXwBb0vo9Guh1IPCbhYx7y1Ep7jKEZZf2a7ESewc JKo4CDgck+iBwlwnClrFp4Wy88sDRnLVX/MS0px0z0ge/pqvXIi4ewybEQkzmCt0GGfY TXmrU6EH+ndrcaXnU93SHtZ03uIjEhJGthUt2/1NeMw87QZHSJ7F5LJ2NrGVZ992EJlP 9VAA== X-Forwarded-Encrypted: i=1; AJvYcCUbNuuPrUmpf2bcbn7VNu5AjR1i6yN2dupIjZdE2GISzM/q+bX/2UViXIqBx/BpqAONMMUxODqX0A==@kvack.org X-Gm-Message-State: AOJu0YxxExqYzpHpnQh890hbi/bXjO9e0JYarZtqF8OCZu6F2VTbhb7X Tr2L3DoqiDS1NETYVaC1+zsZ6cWtGTFCeHQZAlmuE+GQC1iJhB+3odiIeMcw3+TN26UkMCLjbCl 6iBhu3J5SiD4toKVE+dZMUAwNRwYQcRlRi4JZRHSUwN3ZPA5o X-Gm-Gg: ASbGncuSlfpdcm1yQ1cPetvSd8XKb7joiRnImye76/yEZ+t7zDRS0YmXP4ndhq4bhmE sxF3SMn0BRB4KnOYqfeY5zsolcHwjhwjD5E3T1prNmdjhvBXsqi9mxj2hlz42ed3AxEa0BrXig0 5Sc2/lnk+Hz30r1X3q61f6viZJ2Ey4dWmmMXW1UKz2K96wo4B72IN6Khwf9efcXlmJjUWfqWZuU YuQdcL6wbLzqYG3bWoLexEGdcoCV96PjlOZxctaJ66nEecD7I+sgey9WA6LWurO2rV2RnUYls3p axWIEO0= X-Received: by 2002:a05:600c:3b18:b0:434:fd77:5436 with SMTP id 5b1f17b1804b1-4361c38d24fmr55270145e9.15.1734001756129; Thu, 12 Dec 2024 03:09:16 -0800 (PST) X-Google-Smtp-Source: AGHT+IH8ap1IbGcgEu+lVxAQWA0eS/jI89kYifdBDCdLvLY0jc5l1y7TOiHLyavz5avNPpBRcPwp9Q== X-Received: by 2002:a05:600c:3b18:b0:434:fd77:5436 with SMTP id 5b1f17b1804b1-4361c38d24fmr55269825e9.15.1734001755714; Thu, 12 Dec 2024 03:09:15 -0800 (PST) Received: from gmonaco-thinkpadt14gen3.rmtit.csb ([185.107.56.30]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-43625706595sm13537165e9.33.2024.12.12.03.09.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 12 Dec 2024 03:09:14 -0800 (PST) Message-ID: <7e9082361b5b98f1824301c92cde929725db0db6.camel@redhat.com> Subject: Re: [PATCH] sched: Move task_mm_cid_work to mm delayed work From: Gabriele Monaco To: Mathieu Desnoyers , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Juri Lelli , Vincent Guittot , Ingo Molnar , Peter Zijlstra , Andrew Morton , Mel Gorman Date: Thu, 12 Dec 2024 12:09:11 +0100 In-Reply-To: References: <20241205083110.180134-2-gmonaco@redhat.com> <4c067b75e06aadd34eff5b60fc7c59967aa30809.camel@redhat.com> <5ba975e2-06b9-4b98-bece-d601b19a06db@efficios.com> <445b4203-940d-4817-bd45-9da757f22450@efficios.com> <481a7b7716cf4eb2d592b08558d297d343d9aa25.camel@redhat.com> <1f4a8928-8450-48e2-bf40-e75967240d79@efficios.com> <7c4d0c6800a4bd7a5cf4928e28d59fb469c944b9.camel@redhat.com> Autocrypt: addr=gmonaco@redhat.com; prefer-encrypt=mutual; keydata=mDMEZuK5YxYJKwYBBAHaRw8BAQdAmJ3dM9Sz6/Hodu33Qrf8QH2bNeNbOikqYtxWFLVm0 1a0JEdhYnJpZWxlIE1vbmFjbyA8Z21vbmFjb0ByZWRoYXQuY29tPoiZBBMWCgBBFiEEysoR+AuB3R Zwp6j270psSVh4TfIFAmbiuWMCGwMFCQWjmoAFCwkIBwICIgIGFQoJCAsCBBYCAwECHgcCF4AACgk Q70psSVh4TfJzZgD/TXjnqCyqaZH/Y2w+YVbvm93WX2eqBqiVZ6VEjTuGNs8A/iPrKbzdWC7AicnK xyhmqeUWOzFx5P43S1E1dhsrLWgP User-Agent: Evolution 3.54.2 (3.54.2-1.fc41) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: IU_mbesaYF9N90urn-iuPiWI4v5ZIj03CvEhnZw0gbE_1734001756 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 7F20AA000C X-Stat-Signature: hjd618czqqiecxe5d751pi41u3964ije X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1734001743-121409 X-HE-Meta: U2FsdGVkX18zKIeVgwxfbz5nnjQXPyDhnDe2vuaulHYi/7qLZFNfozSwXU1jWLwjYKat8t4xCxgnUnoYGP9ekPePawZPit1ndN/yp284bV/qK0rc5IOsdboTXkk/BACKJorcFqDXVZa9tlGm+WMZxqbzNlRefLxH+q7cZSCSACCIo4dNsiw26Fk+uxaKzJjwRAVl3iePCU0ZP8xMFp3wf2JjZ09nF1ITUmTPotDSZboVZ5aJdW4xBZA4WKVP/ox8wSnDgU3VSjYGGWjcBTXWGVhOP43uV4cqVNHxBdHYxTFqLatrii1zd2BY0Gj3t7/3YDlBq49cR9LW733AV1qKq/02WiprU07xRhSMOEN+Yzr0k2CbITA4gSzLZtZihfcaUH9oxK/r+Zbf4e5j1ghzrg+w1hCTw7USJx0/7f3q0DewdE8CzXVsjxuEUPT/D5HeX4JPj49wytXCHrV9TwU0vcIwyRgbswMkRAP6syk0JuOYWZG43IKgg/yP453j9mHNxW1iNgzCw8pVSBupaIqta+MIHEC35i0yq/BgSslmpcTZl5t2IQXMr5J9zdVdbV4BwD812D1QY++lBoZUWfJQyMiSK/4FoRhdLdyviUEVdq5drCr3SmMcvaGLoB7egf5LQ+d7P0eE00g4sLv0fI67YY4BZ1ac56pVi4h3VB2HY2W6s9f6wgyRYP+qMxV6DITO2nnFom7PWTNNh5GxOqm3hgH4yYT3gfevoIvLypsqFYdg96I1tfadL7UK95QZiwW/geqrWgw17tcopt6DwFE54x15gaLXMnQ6rvasoUu7x+DkMj6bnFJ3YHJsUX01L8bBu+hNEZxT/QbVdul/KI+YWpG7WyInXPffHhWJr9zAMPzHs2AD+cgRrIWprooG3TuActJo4qbJkXZxNVX2wVyPC2MXdwUwz5BwLeFhF6dQdjvSbJQhMkW7HVaKOchgdyxFaVR3npTb0OfwiOWLC+2 hu6Var4a uumeFlJDSdyw+6xhUdUHwj5tDOgHv2hF5Srw4QdqPnqhqk2zJBq/+5QlKUea108gnLKN4RZcmZvs/ZS3d+ZUZR4BFimp8W3fi8IGN/KR6regdmnPPo1lzCDCJ9TQl1PX6PbNWxkwZP+ghHqjOlm+puVyyJAatoWKCHM9Z9qLPwjBkiHHD0uOWr9M+D9xM7JyeYc/sJf8H5k4Ei9Pqv/rjpOPSPYnAVh6gwjn52dPp6GGCNwl6tPK3zK94t36C9bBm0odh0Jx7qZsSgdFGNQO07eSfPEJrI7d7GCARan4vuSec5F+6tW19b9YmkfAaJkwLX2EWfUqzIdIDR2/143xCS/wO7whsFr6nPiIBobG6vLTYtuw2dRSKrCXL1cp7L6ubyFP0CoE8Up59ZAJNu6r9PSGVpp2fGVAKP51GbTl1MYknHTuvq3bxJTqPtYHUottZ/d+3e+PVtQ94NWtfBcKnYbeQ1K1k3U/rlfEf X-Bogosity: Ham, tests=bogofilter, spamicity=0.000025, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, 2024-12-11 at 12:07 -0500, Mathieu Desnoyers wrote: > > Here's where I'm in doubt, is a compact map more desirable than > > reusing > > the same mm_cids for cache locality? >=20 > This is a tradeoff between: >=20 > A) Preserving cache locality after a transition from many threads to > few > =C2=A0=C2=A0=C2=A0 threads, or after reducing the hamming weight of the a= llowed cpu > mask. >=20 > B) Making the mm_cid guarantees wrt nr threads and allowed cpu mask > easy > =C2=A0=C2=A0=C2=A0 to document and understand. >=20 > C) Allowing applications to eventually react to mm_cid compaction > after > =C2=A0=C2=A0=C2=A0 reduction of the nr threads or allowed cpu mask, makin= g the > tracking > =C2=A0=C2=A0=C2=A0 of mm_cid compaction easier by shrinking it back towar= ds 0 or > not. >=20 > D) Making sure applications that periodically reduce and then > increase > =C2=A0=C2=A0=C2=A0 again the nr threads or allowed cpu mask still benefit= from good > =C2=A0=C2=A0=C2=A0 cache locality with mm_cid. >=20 >=20 > > If not, should we perhaps ignore the recent_cid if it's larger than > > the > > map weight? > > It seems the only way the recent_cid is unset is with migrations, > > but > > I'm not sure if forcing one would make the test vain as the cid > > could > > be dropped outside of task_mm_cid_work. > >=20 > > What do you think? >=20 > Can you try this patch ? (compile-tested only) >=20 > commit 500649e03c5c28443f431829732c580750657326 > Author: Mathieu Desnoyers > Date:=C2=A0=C2=A0 Wed Dec 11 11:53:01 2024 -0500 >=20 > =C2=A0=C2=A0=C2=A0=C2=A0 sched: shrink mm_cid allocation with nr thread/a= ffinity >=20 > diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h > index 76f5f53a645f..b92e79770a93 100644 > --- a/kernel/sched/sched.h > +++ b/kernel/sched/sched.h > @@ -3657,10 +3657,24 @@ static inline int __mm_cid_try_get(struct > task_struct *t, struct mm_struct *mm) > =C2=A0 { > =C2=A0=C2=A0 struct cpumask *cidmask =3D mm_cidmask(mm); > =C2=A0=C2=A0 struct mm_cid __percpu *pcpu_cid =3D mm->pcpu_cid; > - int cid =3D __this_cpu_read(pcpu_cid->recent_cid); > + int cid, max_nr_cid, allowed_max_nr_cid; > =C2=A0=20 > + /* > + * After shrinking the number of threads or reducing the number > + * of allowed cpus, reduce the value of max_nr_cid so expansion > + * of cid allocation will preserve cache locality if the number > + * of threads or allowed cpus increase again. > + */ > + max_nr_cid =3D atomic_read(&mm->max_nr_cid); > + while ((allowed_max_nr_cid =3D min_t(int, READ_ONCE(mm- > >nr_cpus_allowed), atomic_read(&mm->mm_users))), > + max_nr_cid > allowed_max_nr_cid) { > + if (atomic_try_cmpxchg(&mm->max_nr_cid, &max_nr_cid, > allowed_max_nr_cid)) > + break; > + } > =C2=A0=C2=A0 /* Try to re-use recent cid. This improves cache locality. *= / > - if (!mm_cid_is_unset(cid) && !cpumask_test_and_set_cpu(cid, > cidmask)) > + cid =3D __this_cpu_read(pcpu_cid->recent_cid); > + if (!mm_cid_is_unset(cid) && cid < max_nr_cid && > + =C2=A0=C2=A0=C2=A0 !cpumask_test_and_set_cpu(cid, cidmask)) > =C2=A0=C2=A0 return cid; > =C2=A0=C2=A0 /* > =C2=A0=C2=A0 * Expand cid allocation if the maximum number of concurrency > @@ -3668,12 +3682,11 @@ static inline int __mm_cid_try_get(struct > task_struct *t, struct mm_struct *mm) > =C2=A0=C2=A0 * and number of threads. Expanding cid allocation as much as > =C2=A0=C2=A0 * possible improves cache locality. > =C2=A0=C2=A0 */ > - cid =3D atomic_read(&mm->max_nr_cid); > - while (cid < READ_ONCE(mm->nr_cpus_allowed) && cid < > atomic_read(&mm->mm_users)) { > - if (!atomic_try_cmpxchg(&mm->max_nr_cid, &cid, cid + 1)) > + while (max_nr_cid < allowed_max_nr_cid) { > + if (!atomic_try_cmpxchg(&mm->max_nr_cid, &max_nr_cid, max_nr_cid + > 1)) > =C2=A0=C2=A0 continue; > - if (!cpumask_test_and_set_cpu(cid, cidmask)) > - return cid; > + if (!cpumask_test_and_set_cpu(max_nr_cid, cidmask)) > + return max_nr_cid; > =C2=A0=C2=A0 } > =C2=A0=C2=A0 /* > =C2=A0=C2=A0 * Find the first available concurrency id. Thanks for the patch, it seems much more robust than my simple condition on the weight. It passes the test (both versions) we previously discussed and doesn't seem to interfere with the general rseq functionality as checked by the other selftests. I'm not sure if I should run more tests on this one. I will come up with a V2 shortly and attach some performance evaluations. Do you want to keep your patch separate or do I submit it together with V2? Thanks, Gabriele