From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54C0FC02180 for ; Wed, 15 Jan 2025 16:39:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6A94C6B007B; Wed, 15 Jan 2025 11:39:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6324F6B0082; Wed, 15 Jan 2025 11:39:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4ABBB6B0085; Wed, 15 Jan 2025 11:39:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 279986B007B for ; Wed, 15 Jan 2025 11:39:17 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id C7AA21A14DC for ; Wed, 15 Jan 2025 16:39:16 +0000 (UTC) X-FDA: 83010246312.03.226A55D Received: from mail-qt1-f172.google.com (mail-qt1-f172.google.com [209.85.160.172]) by imf18.hostedemail.com (Postfix) with ESMTP id 85D7E1C0015 for ; Wed, 15 Jan 2025 16:39:14 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=JN4HQDL5; spf=pass (imf18.hostedemail.com: domain of surenb@google.com designates 209.85.160.172 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736959154; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QaZL90gx3dqyATOSCUoKbDZ4oWlZcUee4Harbwpymrk=; b=ywqd1u3w/Fhq6M0UxxeHvX7lepOfOAzcWA8wgBpaKid/Y6JqwI9ZACxyvJi+p6bb3Uh9GI 1x8nACpxeXNn+PgaG0BHMxNynGiBHZX3ETs2a4JqbqKn5HxRlTtct0GQIJ97Lp7N2qAYIj G524NSqETTZ5K+c720UZTl1hdVgeqvI= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=JN4HQDL5; spf=pass (imf18.hostedemail.com: domain of surenb@google.com designates 209.85.160.172 as permitted sender) smtp.mailfrom=surenb@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736959154; a=rsa-sha256; cv=none; b=A0FSxK2CNgIlvvw6Qm2FygeSQfaGgDXDWksMqcwLLgS+lp7h38ilEjBrTApbglYqANwa/h uXhZsZr8lfRuMx1wHiFM3xwR8M9yFQQiL5c3XqE32FR0GJ4ZfH+kGWqnIQgBwaCb6vWUC1 jTc2wFlrbSuuVzyn9EBZqmCnnM4Cp4I= Received: by mail-qt1-f172.google.com with SMTP id d75a77b69052e-4679b5c66d0so578171cf.1 for ; Wed, 15 Jan 2025 08:39:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736959153; x=1737563953; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=QaZL90gx3dqyATOSCUoKbDZ4oWlZcUee4Harbwpymrk=; b=JN4HQDL5RlRuuSrEMCEvCnM2ZeN51b5DxLbMVni3xuM/IAzPQHmdkDhsnt23gVrS4X 0/SKBEOFsKlcQsagwI/LYW/NtZ4o+Mxr563eU640VrZZ6PH/sLUgisoMsYILz1u/1B9i rZjpzDnmdDSJnWT7ziAdYxMhI0fqiB43DtT7UywRA6yurfL6f0M1muK/kWfpU1YzXJCg mxxcxPUUGvrc63oBt/JIc4ANhPORWP/no9ZsA54wW+DrGK/SZgsgXytc7WD2RzmlGd2h InkSKnvVUF3glz8cGN/zJ3s4A0vD4clMpM9gAX/9SOgOs0eEmerCbnSGsmPu9Q8bR7/N uxlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736959153; x=1737563953; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QaZL90gx3dqyATOSCUoKbDZ4oWlZcUee4Harbwpymrk=; b=SAM9CpNZiVfW0Y+w28AsFh5E4tFDhbW3bITaGd6gAsd7oltA7t5rIRVeWW1qI1hsF6 n9/MBAS3M7rr+4pBxUJ8eqVHk73MLPKpLx+Nb0mhJ38HlSf1dFCSL8B07CNVE8SeJnWp cqVMERGaGx6vaB6Z6+z1QXvHPor59pDYjL7erj8SEvScX9GD5dgnTpS1huBqFTHGFISf 7O8n3rvcKZdt1KWJRUePw1zePLQZiYxp2TJiAWseLramKq8gJGDvjEj/NojCoN5h1acr 8wYbDaAuKf2lofy5hm5dwIkfe/jGD0HQJowNAkjXhm9j2J2Ec4mm6ofEGRPXU+dpLr00 wfzA== X-Forwarded-Encrypted: i=1; AJvYcCVYvCH9ozyKhUPnHKLduYjdSnBCah/G2kw4byBRqnhyqF/ib+/gXGc5y9z2jMuxAASjldE/01QsSA==@kvack.org X-Gm-Message-State: AOJu0Yyg7BDFMZVvqhdHa+JhHt2dnXHgjOEFiFOC/Rrpk0mkCN8tSGG9 ua5pqWQPDgzTu4eac7Qom8yqjjaB8P8+nPyJA5YZUBx11ROCyPv5EwnhOnZduv/YTo9E/36z3tC J3ngWVYx+33r7j51r5H1klDbfCTa7q89rZA3Z X-Gm-Gg: ASbGnctEGxYjGKgHlAMgFhi0idcYBtDtoGl/kj38V7h1CzHtg25E1AWYZ8hYQGmu1qJ NmBufk3yPCdtBhfG9rbuSfxZ5GpsSaUbk6W2g8z2oh3q7rewkPEPJ/8W21tCuTmjsxDE= X-Google-Smtp-Source: AGHT+IGUkYPO5ZjvmWw6xf3ICmiBy/miXu0O3/ygEQr+pM9TgveRmbvWW3Y4m127Mhfar6MytlQPEoyZvu/XoifvPQg= X-Received: by 2002:ac8:7c46:0:b0:467:462e:a51b with SMTP id d75a77b69052e-46df56c14f6mr4399061cf.14.1736959152832; Wed, 15 Jan 2025 08:39:12 -0800 (PST) MIME-Version: 1.0 References: <20250111042604.3230628-1-surenb@google.com> <20250111042604.3230628-13-surenb@google.com> <20250115105056.GY5388@noisy.programming.kicks-ass.net> In-Reply-To: <20250115105056.GY5388@noisy.programming.kicks-ass.net> From: Suren Baghdasaryan Date: Wed, 15 Jan 2025 08:39:01 -0800 X-Gm-Features: AbW1kva5KQEaW2j0QeAKNz__H-sbubypX-MpKHIs5psh00jd36ny2QIqDDCj1jY Message-ID: Subject: Re: [PATCH v9 12/17] mm: move lesser used vma_area_struct members into the last cacheline To: Peter Zijlstra Cc: akpm@linux-foundation.org, willy@infradead.org, liam.howlett@oracle.com, lorenzo.stoakes@oracle.com, david.laight.linux@gmail.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mjguzik@gmail.com, oliver.sang@intel.com, mgorman@techsingularity.net, david@redhat.com, peterx@redhat.com, oleg@redhat.com, dave@stgolabs.net, paulmck@kernel.org, brauner@kernel.org, dhowells@redhat.com, hdanton@sina.com, hughd@google.com, lokeshgidra@google.com, minchan@google.com, jannh@google.com, shakeel.butt@linux.dev, souravpanda@google.com, pasha.tatashin@soleen.com, klarasmodin@gmail.com, richard.weiyang@gmail.com, corbet@lwn.net, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@android.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam05 X-Stat-Signature: n9e5saitbg5cgk8j5c46nj14tbqoohzq X-Rspamd-Queue-Id: 85D7E1C0015 X-Rspam-User: X-HE-Tag: 1736959154-798116 X-HE-Meta: U2FsdGVkX196upEWEOdilT/2/N0x4gLJzxNbHZidwI0O+1vTLzsJLa5dn64otE4x1fF07wrDWBm6mp02OF10FgZDvYJA3qI9DHoa1v7nyOscwBkT5RmYKPz2muKWmFAjIhz9Uylp+peCtK05AYr71n9o67mXR93SJIZazzJYkxQ8qEl3/RPIeAb0KFg2nEsYXDPIQa4mSkxJr7hs1w4fTq2D9HuYG2c5YSxmw3bsOJHNe79dw6DAjZhTCfM9uCi+CjZhs4Rb17v+POvqxvnwmSvTy3Z2vRm4/4jJS9tTX+T1gobPN+i3rPGITMRjVnuUCLNF5bQuBcin/6VuoUue0p/wy1fLNQytoXSGh7IEKXIakeuZTpcnBzEBVj24PfnkMAcEQbjCBwfLHoSNHiCLm6vJQdr3I/4uuPJ6TOg2I5h5bTqjvqpFQX7HbGW30BfS5I9KIqbdaaQn9jOr61OiTtva23cVjdIkIpUIevnHTn2aiaWOjX+oR846p4FC5GDGju1/KKOro0XbEGQXnMaU53t43QQgWjkXIl1Iyw51KdBaYSlK3RLrONAcn0UWh1di+XlqL66GP8PbhJMJeBKM2+bC7zcqyClpooqGYuBNR6ig4unaku6xPSEbYxj/WEsa1ycvZefLaFKzy2kGVlu77qSVbQOU19fz8xDcmO8vY+7iiIHjZkE3S7N6j1v502ytazTcQ9U+EyqZucZPKFDJk2falpy58uoC8nm1N2NsshiXHXO5hnEcpUK4TjrbFr1ryulyfTMWtHbI6hozeneWDk7TxkhT8+8Q8XFiGNlitOElkbbZ0ogtGY1XNQcznZY84txsSYaMTPvPDSHyWElOeA26s/c2Sqm6pO/gnEsJWwf5hgJHX5DOvPSwrlarFyEHAO0fJcZ48QYUF8Kxgfz8WrE5FRS7q/SZ+dg0DzoZ4HCFw5GFdpdFrX37btH1ReklDTeUUnGQ1v9jCXAel9q Oml8IKCw IWrK2TcoJ16jlLR5O8xoPoogTUZvUlv4NhM7LTBdY1L1Bn8lUCpqYfDpJ0v1DaElA632DJf3Cdt81GJ1KFAtde05ZRr94TrynMG++vOBX6La9i/3vBhSeq3DlwCoiR/4GmyZdFASGco1RHDFp3ilKN18k7E8VttxN6AyItuB1DOS6SpDYIOpAwxsXowc+tDzKx93TLyuPHo88INGMmXE5S8CUBrbA8CiOLHKFhdMME782e9MOHyXEKP2Bbafeci5e4lbm3dDF+v5ssncINL5fR6tOVq7p5FMM4VVLfiN6gxOnwT5TWY70y0FHVeJCW79y79TxpTk3IH58sUTB0nH71L9/jPHurqPYb4vExQcO7aP0oAyB5CODWHyKuQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jan 15, 2025 at 2:51=E2=80=AFAM Peter Zijlstra wrote: > > On Fri, Jan 10, 2025 at 08:25:59PM -0800, Suren Baghdasaryan wrote: > > Move several vma_area_struct members which are rarely or never used > > during page fault handling into the last cacheline to better pack > > vm_area_struct. As a result vm_area_struct will fit into 3 as opposed > > to 4 cachelines. New typical vm_area_struct layout: > > > > struct vm_area_struct { > > union { > > struct { > > long unsigned int vm_start; /* 0 8 */ > > long unsigned int vm_end; /* 8 8 */ > > }; /* 0 16 */ > > freeptr_t vm_freeptr; /* 0 8 */ > > }; /* 0 16 */ > > struct mm_struct * vm_mm; /* 16 8 */ > > pgprot_t vm_page_prot; /* 24 8 */ > > union { > > const vm_flags_t vm_flags; /* 32 8 */ > > vm_flags_t __vm_flags; /* 32 8 */ > > }; /* 32 8 */ > > unsigned int vm_lock_seq; /* 40 4 */ > > Does it not make sense to move this seq field near the refcnt? In an earlier version, when vm_lock was not a refcount yet, I tried that and moving vm_lock_seq introduced regression in the pft test. We have that early vm_lock_seq check in the beginning of vma_start_read() and if it fails we bail out early without locking. I think that might be the reason why keeping vm_lock_seq in the first cacheling is beneficial. But I'll try moving it again now that we have vm_refcnt instead of the lock and see if pft still shows any regression. > > > /* XXX 4 bytes hole, try to pack */ > > > > struct list_head anon_vma_chain; /* 48 16 */ > > /* --- cacheline 1 boundary (64 bytes) --- */ > > struct anon_vma * anon_vma; /* 64 8 */ > > const struct vm_operations_struct * vm_ops; /* 72 8 */ > > long unsigned int vm_pgoff; /* 80 8 */ > > struct file * vm_file; /* 88 8 */ > > void * vm_private_data; /* 96 8 */ > > atomic_long_t swap_readahead_info; /* 104 8 */ > > struct mempolicy * vm_policy; /* 112 8 */ > > struct vma_numab_state * numab_state; /* 120 8 */ > > /* --- cacheline 2 boundary (128 bytes) --- */ > > refcount_t vm_refcnt (__aligned__(64)); /* 128 4 */ > > > > /* XXX 4 bytes hole, try to pack */ > > > > struct { > > struct rb_node rb (__aligned__(8)); /* 136 24 */ > > long unsigned int rb_subtree_last; /* 160 8 */ > > } __attribute__((__aligned__(8))) shared; /* 136 32 */ > > struct anon_vma_name * anon_name; /* 168 8 */ > > struct vm_userfaultfd_ctx vm_userfaultfd_ctx; /* 176 8 */ > > > > /* size: 192, cachelines: 3, members: 18 */ > > /* sum members: 176, holes: 2, sum holes: 8 */ > > /* padding: 8 */ > > /* forced alignments: 2, forced holes: 1, sum forced holes: 4 */ > > } __attribute__((__aligned__(64))); > >