From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DF1EC25B75 for ; Thu, 23 May 2024 15:30:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1BA196B008A; Thu, 23 May 2024 11:30:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 168B86B0092; Thu, 23 May 2024 11:30:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 057066B0095; Thu, 23 May 2024 11:30:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id DB55F6B008A for ; Thu, 23 May 2024 11:30:52 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 6D9EFA194C for ; Thu, 23 May 2024 15:30:52 +0000 (UTC) X-FDA: 82150048344.29.D0A1416 Received: from out-177.mta1.migadu.com (out-177.mta1.migadu.com [95.215.58.177]) by imf07.hostedemail.com (Postfix) with ESMTP id C9C414001B for ; Thu, 23 May 2024 15:30:48 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=bIJ5AiLk; spf=pass (imf07.hostedemail.com: domain of roman.gushchin@linux.dev designates 95.215.58.177 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716478249; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MqJQ6S1S7dUreqk/kr9p3VEkNZipko3/hIqO3h95RDA=; b=Ql9uTdIqglHKuZvq8+LKjTzSZ92JK5mkbPKiCmKR4FtWbwz78FZOD4oqU3Q2C2pptZf0O5 LQ+CGcqNvA7ZOznXOdsTU6KRt9+WHrotsHSg8dvDM1Z9rmZLrjaMrkdY26XsKE/Gz20qPO EiS/mMqsKlkaJfXRvnz6flNqLVXYbpw= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=bIJ5AiLk; spf=pass (imf07.hostedemail.com: domain of roman.gushchin@linux.dev designates 95.215.58.177 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716478249; a=rsa-sha256; cv=none; b=VxtrM4gDF+O38uUAkxgHlXfVRvSMTqnpdiSVpzWhcSQFFybAR8t8bajtN9AprfbfA0IpWw ru2sWniRHhgJfnrsgOGnNZ9386vVc67o2uvYKBinHxPC7EhO1EodzNTkfNCll6N12xXvDS dyCGK38uHqVagYHYB6ADSQKICIMD2xg= X-Envelope-To: shakeel.butt@linux.dev DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1716478246; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MqJQ6S1S7dUreqk/kr9p3VEkNZipko3/hIqO3h95RDA=; b=bIJ5AiLkxgMWNcwwzjfqWlV8DrYGkdefLGSTiaBDdh5Sn5FZ/rHNpE3s7Q62HNrUy9+rcx Jj/c6RnqWqXsu3xbXWAcY54dvM7BusU9yaHEqN86UspFAQi2+tBYdRQeUyhFQ/WMY8SwUp ARFcKYI9wprtKL4byXQtm56KvkUBTtg= X-Envelope-To: yosryahmed@google.com X-Envelope-To: akpm@linux-foundation.org X-Envelope-To: hannes@cmpxchg.org X-Envelope-To: mhocko@kernel.org X-Envelope-To: muchun.song@linux.dev X-Envelope-To: ying.huang@intel.com X-Envelope-To: feng.tang@intel.com X-Envelope-To: fengwei.yin@intel.com X-Envelope-To: oliver.sang@intel.com X-Envelope-To: kernel-team@meta.com X-Envelope-To: linux-mm@kvack.org X-Envelope-To: linux-kernel@vger.kernel.org Date: Thu, 23 May 2024 08:30:40 -0700 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Roman Gushchin To: Shakeel Butt Cc: Yosry Ahmed , Andrew Morton , Johannes Weiner , Michal Hocko , Muchun Song , ying.huang@intel.com, feng.tang@intel.com, fengwei.yin@intel.com, oliver.sang@intel.com, kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH] memcg: rearrage fields of mem_cgroup_per_node Message-ID: References: <20240523034824.1255719-1-shakeel.butt@linux.dev> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Migadu-Flow: FLOW_OUT X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: C9C414001B X-Stat-Signature: 66xa8eoh4dzb1iabe56kuedyyic8zpeo X-Rspam-User: X-HE-Tag: 1716478248-151245 X-HE-Meta: U2FsdGVkX19yAAh9ubXlKCZVfrbkd4ew2p14SfTdYIBWbP6ywDqES9JqcU4R3uSq6lv/krGo8MNnRyF183MC9PiDqB0vzDn+IQlPtuy7Ch5GGo8RZS2x8Be15a0PqraE1DTW8VNcdTgzv3VM0U2ZolfiUSTT/wG23GClQ3/O8tPVqCM72PCLrWBIuHucTnB2sPz/cQt7osc2MAKc9oxVDNMz4y01UXSivptUlH58DKMrDM2jODIPsJ7+WaAmtyogRUJHxlUPTYOT/d3MdT6dgNmEjk+7MxSW9RiaSedr9WXDyMHZLuiqU8z1hUlbRkQH19FRnx+f4aBMUaYcU6Z/5XjkFZPeCCq156xNhLdBLkhIzZyFHeoeVrnWTATvRO+1pleZbKvo2BGNaQaj+cYRTOVhhPstFXotr7OW2v/POQeiQB8ZStB399IAKUeTOhBMVif/4eOEyTAw2tciuTk47p1sXOctEyW2R767+5vPaVBFfDMQ/P/qzCL3NkXcaFCtJnjR6geiYN9CbkPm1BqmRYIHEhnmRa1AeW8oWHmSWvYB12JCiwJOAvhUW94T3+iXVoljUMaCAkKyzTfL6TjJppAYPJT++PjFsOmlSK2hYUPpaLj10VHJjAFu/KmX6vaz3SDvbO6IU9xcdQXGclJuIUKLjotyBlQmR4b+avarmmAAEeTIomXf3z+YSo7Z/7WC9wNMPIoz9vDDdnqoK7JubpHMqzWcazl3gBhL/fwIXNLgbZ5w9m/GG5AmujyvneXzKAi85vNJrkSrLKh4mCwTh3zlEZ82vMva1/GcTBkhHqR6qFo41c6B8zNP8KhbkxCvkPOIEbopx9tBXVAApSHgr4dGqd5bwcNNow8/RNL69I0a/EoLXijHKTBmbP/9pcRxLpP+FwQQ5NTCVXDoTrpa5CaJlhgmfOg9ZX0PwtRqRprry/pJvkAyX1KYkEUol7wPYSvJt9oA5SO0EYghJRV wI/aZcMy pSO1gw0ieA6eg3zutNC9xOSGaNBhaZ5dcAFrs/GohVMkYD++pAoh3VqaK19y6HCl7QQxRh+9uBLTjXWA1ELp4m/Mq+whl0i4E/hxCuSVTHEbjUOUhdcOCJ7YIbWomW4lfOcynSExroFF81PPFSskPVJWxPGoKbitNDFX51BfReFNH/Wx65n22glq4f4zJr7EvU8sIHu1oNiaUFeHik3mroCc84g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, May 22, 2024 at 10:34:38PM -0700, Shakeel Butt wrote: > On Wed, May 22, 2024 at 09:35:57PM -0700, Yosry Ahmed wrote: > > On Wed, May 22, 2024 at 8:48 PM Shakeel Butt wrote: > [...] > > > > > > struct mem_cgroup_per_node { > > > - struct lruvec lruvec; > > > + /* Keep the read-only fields at the start */ > > > + struct mem_cgroup *memcg; /* Back pointer, we cannot */ > > > + /* use container_of */ > > > > > > struct lruvec_stats_percpu __percpu *lruvec_stats_percpu; > > > struct lruvec_stats *lruvec_stats; > > > - > > > - unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS]; > > > - > > > - struct mem_cgroup_reclaim_iter iter; > > > - > > > struct shrinker_info __rcu *shrinker_info; > > > > > > + /* memcg-v1 only stuff in middle */ > > > + > > > struct rb_node tree_node; /* RB tree node */ > > > unsigned long usage_in_excess;/* Set to the value by which */ > > > /* the soft limit is exceeded*/ > > > bool on_tree; > > > - struct mem_cgroup *memcg; /* Back pointer, we cannot */ > > > - /* use container_of */ > > > > Do we need CACHELINE_PADDING() here (or maybe make struct lruvec > > cache-aligned) to make sure the false cacheline sharing doesn't happen > > again with the fields below, or is the idea that the fields that get > > read in hot paths (memcg, lruvec_stats_percpu, lruvec_stats) are far > > at the top, and the memcg v1 elements in the middle act as a buffer? It's a good point. Once we will compile out the memcg v1 stuff, it might stop working. > > > > IOW, is sharing between the fields below and memcg v1 fields okay > > because they are not read in the hot path? If yes, I believe it's > > worth a comment. It can be easily missed if the memcg v1 soft limit is > > removed later for example. > > > > For 6.10, I wanted to keep the change simple and yes, the memcg v1 stuff > as a buffer between the pointers and lruvec/lru_zone_size fields. For > 6.11 or later kernels, I am planning to use some asserts to make sure > these fields don't share a cacheline, so later when we remove the > v1-only stuff, the asserts will make sure we keep the separate cacheline > property intact. Sounds good. Once we'll have memcg v1 stuff under a config option, we'll put those asserts in. Btw, I'm about (today/tomorrow) to post the memcg-v1 separation patchset, so it won't take a long time. Thanks!