From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6208FC27C4F for ; Fri, 31 May 2024 16:13:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DA4DA6B0092; Fri, 31 May 2024 12:13:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D7F896B0093; Fri, 31 May 2024 12:13:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C1CA06B0095; Fri, 31 May 2024 12:13:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A58546B0092 for ; Fri, 31 May 2024 12:13:13 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 547AA1C1343 for ; Fri, 31 May 2024 16:13:13 +0000 (UTC) X-FDA: 82179185466.24.7605229 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.17]) by imf18.hostedemail.com (Postfix) with ESMTP id 7DEE21C0002 for ; Fri, 31 May 2024 16:13:10 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=gjb+3uBb; spf=pass (imf18.hostedemail.com: domain of dave.hansen@intel.com designates 192.198.163.17 as permitted sender) smtp.mailfrom=dave.hansen@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717171991; a=rsa-sha256; cv=none; b=LqNiiFH6Tz1enFmgF39oiCPXeXm9ceAcEHAC4SYMqpdDumlTmEr16yBNzdPZUJbnaBg4fW Nw1xQ6xYAsb2ohDet3ansUkEqBr6jvilo5fdhTVgJNH9lJmAY7o4CAdPud9CyPrWg5uQZN o1uk7DCmbNIo7k/L7PVEwKAUzpf7TVU= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=gjb+3uBb; spf=pass (imf18.hostedemail.com: domain of dave.hansen@intel.com designates 192.198.163.17 as permitted sender) smtp.mailfrom=dave.hansen@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717171991; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aA+xln9Am14bbSreZVDSza35jpfNGdAL7zvzBpBt55g=; b=G8kD/f0wHqw9gO7YgOsZxdfB6dXiGMW/t66tOxoGvLbYQ1T7ZfGLpnieauzqfM5xUSTTgQ C9NZSmmXCavLQQ4K1QmZN0dp+6nYD8Pg0Sy+oUMbLOm+xgjJW3s7I4ahPakAM2yMpmzi0r eYieoWdE7GXT2+u7MaRYzY+rI9TXzzs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1717171990; x=1748707990; h=message-id:date:mime-version:subject:to:cc:references: from:in-reply-to:content-transfer-encoding; bh=7T8wP/o3cgJtL1+OvLDrTpH/pVY43kVpMfxcGleEXLE=; b=gjb+3uBb0bPHaWqkTdv9lufb/ktqDTqDS/qrb4cizATCpCBOONMVemuP uaLzy/l2wVpF7AdeSfBcHE7I7LQyy5NLNRS46Ov3me7HUuplJQA+P4aAu Z6M7rR+NUggWw+NaLdGzdMJFup2zwG8Uzmevph1iuo4iMNeDcSuF7LEom oo6cUlS1VncXReXOpHTGZmQHusOYPeEmSYu6A0uqjRzc/+p6q+h/0rpj9 G7ZYxV7qEPGE6BtCXe+n7TIPNlUw/AHAKBTL3zj+TP7rTvVQQLFu1itAr +OnT9nPItjRKquDJ4GNAgHOUTkxl6tTSw8GbKSTaTDsqTDylVTn0Uygwn Q==; X-CSE-ConnectionGUID: sAWr78/yThaAH15UoElpdg== X-CSE-MsgGUID: Hc9JtBTxTsiycLBZhVhHPA== X-IronPort-AV: E=McAfee;i="6600,9927,11089"; a="13590281" X-IronPort-AV: E=Sophos;i="6.08,204,1712646000"; d="scan'208";a="13590281" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 May 2024 09:12:43 -0700 X-CSE-ConnectionGUID: btdZbbZcSEKI3pzJF3+3JQ== X-CSE-MsgGUID: hWvexQpLSeak9c2RRXYCzA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,204,1712646000"; d="scan'208";a="59384503" Received: from uaeoff-desk1.amr.corp.intel.com (HELO [10.125.108.48]) ([10.125.108.48]) by fmviesa002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 May 2024 09:12:43 -0700 Message-ID: Date: Fri, 31 May 2024 09:12:42 -0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v11 09/12] mm: implement LUF(Lazy Unmap Flush) defering tlb flush when folios get unmapped To: Byungchul Park , linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: kernel_team@skhynix.com, akpm@linux-foundation.org, ying.huang@intel.com, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, rjgolo@gmail.com References: <20240531092001.30428-1-byungchul@sk.com> <20240531092001.30428-10-byungchul@sk.com> From: Dave Hansen Content-Language: en-US Autocrypt: addr=dave.hansen@intel.com; keydata= xsFNBE6HMP0BEADIMA3XYkQfF3dwHlj58Yjsc4E5y5G67cfbt8dvaUq2fx1lR0K9h1bOI6fC oAiUXvGAOxPDsB/P6UEOISPpLl5IuYsSwAeZGkdQ5g6m1xq7AlDJQZddhr/1DC/nMVa/2BoY 2UnKuZuSBu7lgOE193+7Uks3416N2hTkyKUSNkduyoZ9F5twiBhxPJwPtn/wnch6n5RsoXsb ygOEDxLEsSk/7eyFycjE+btUtAWZtx+HseyaGfqkZK0Z9bT1lsaHecmB203xShwCPT49Blxz VOab8668QpaEOdLGhtvrVYVK7x4skyT3nGWcgDCl5/Vp3TWA4K+IofwvXzX2ON/Mj7aQwf5W iC+3nWC7q0uxKwwsddJ0Nu+dpA/UORQWa1NiAftEoSpk5+nUUi0WE+5DRm0H+TXKBWMGNCFn c6+EKg5zQaa8KqymHcOrSXNPmzJuXvDQ8uj2J8XuzCZfK4uy1+YdIr0yyEMI7mdh4KX50LO1 pmowEqDh7dLShTOif/7UtQYrzYq9cPnjU2ZW4qd5Qz2joSGTG9eCXLz5PRe5SqHxv6ljk8mb ApNuY7bOXO/A7T2j5RwXIlcmssqIjBcxsRRoIbpCwWWGjkYjzYCjgsNFL6rt4OL11OUF37wL QcTl7fbCGv53KfKPdYD5hcbguLKi/aCccJK18ZwNjFhqr4MliQARAQABzUVEYXZpZCBDaHJp c3RvcGhlciBIYW5zZW4gKEludGVsIFdvcmsgQWRkcmVzcykgPGRhdmUuaGFuc2VuQGludGVs LmNvbT7CwXgEEwECACIFAlQ+9J0CGwMGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEGg1 lTBwyZKwLZUP/0dnbhDc229u2u6WtK1s1cSd9WsflGXGagkR6liJ4um3XCfYWDHvIdkHYC1t MNcVHFBwmQkawxsYvgO8kXT3SaFZe4ISfB4K4CL2qp4JO+nJdlFUbZI7cz/Td9z8nHjMcWYF IQuTsWOLs/LBMTs+ANumibtw6UkiGVD3dfHJAOPNApjVr+M0P/lVmTeP8w0uVcd2syiaU5jB aht9CYATn+ytFGWZnBEEQFnqcibIaOrmoBLu2b3fKJEd8Jp7NHDSIdrvrMjYynmc6sZKUqH2 I1qOevaa8jUg7wlLJAWGfIqnu85kkqrVOkbNbk4TPub7VOqA6qG5GCNEIv6ZY7HLYd/vAkVY E8Plzq/NwLAuOWxvGrOl7OPuwVeR4hBDfcrNb990MFPpjGgACzAZyjdmYoMu8j3/MAEW4P0z F5+EYJAOZ+z212y1pchNNauehORXgjrNKsZwxwKpPY9qb84E3O9KYpwfATsqOoQ6tTgr+1BR CCwP712H+E9U5HJ0iibN/CDZFVPL1bRerHziuwuQuvE0qWg0+0SChFe9oq0KAwEkVs6ZDMB2 P16MieEEQ6StQRlvy2YBv80L1TMl3T90Bo1UUn6ARXEpcbFE0/aORH/jEXcRteb+vuik5UGY 5TsyLYdPur3TXm7XDBdmmyQVJjnJKYK9AQxj95KlXLVO38lczsFNBFRjzmoBEACyAxbvUEhd GDGNg0JhDdezyTdN8C9BFsdxyTLnSH31NRiyp1QtuxvcqGZjb2trDVuCbIzRrgMZLVgo3upr MIOx1CXEgmn23Zhh0EpdVHM8IKx9Z7V0r+rrpRWFE8/wQZngKYVi49PGoZj50ZEifEJ5qn/H Nsp2+Y+bTUjDdgWMATg9DiFMyv8fvoqgNsNyrrZTnSgoLzdxr89FGHZCoSoAK8gfgFHuO54B lI8QOfPDG9WDPJ66HCodjTlBEr/Cwq6GruxS5i2Y33YVqxvFvDa1tUtl+iJ2SWKS9kCai2DR 3BwVONJEYSDQaven/EHMlY1q8Vln3lGPsS11vSUK3QcNJjmrgYxH5KsVsf6PNRj9mp8Z1kIG qjRx08+nnyStWC0gZH6NrYyS9rpqH3j+hA2WcI7De51L4Rv9pFwzp161mvtc6eC/GxaiUGuH BNAVP0PY0fqvIC68p3rLIAW3f97uv4ce2RSQ7LbsPsimOeCo/5vgS6YQsj83E+AipPr09Caj 0hloj+hFoqiticNpmsxdWKoOsV0PftcQvBCCYuhKbZV9s5hjt9qn8CE86A5g5KqDf83Fxqm/ vXKgHNFHE5zgXGZnrmaf6resQzbvJHO0Fb0CcIohzrpPaL3YepcLDoCCgElGMGQjdCcSQ+Ci FCRl0Bvyj1YZUql+ZkptgGjikQARAQABwsFfBBgBAgAJBQJUY85qAhsMAAoJEGg1lTBwyZKw l4IQAIKHs/9po4spZDFyfDjunimEhVHqlUt7ggR1Hsl/tkvTSze8pI1P6dGp2XW6AnH1iayn yRcoyT0ZJ+Zmm4xAH1zqKjWplzqdb/dO28qk0bPso8+1oPO8oDhLm1+tY+cOvufXkBTm+whm +AyNTjaCRt6aSMnA/QHVGSJ8grrTJCoACVNhnXg/R0g90g8iV8Q+IBZyDkG0tBThaDdw1B2l asInUTeb9EiVfL/Zjdg5VWiF9LL7iS+9hTeVdR09vThQ/DhVbCNxVk+DtyBHsjOKifrVsYep WpRGBIAu3bK8eXtyvrw1igWTNs2wazJ71+0z2jMzbclKAyRHKU9JdN6Hkkgr2nPb561yjcB8 sIq1pFXKyO+nKy6SZYxOvHxCcjk2fkw6UmPU6/j/nQlj2lfOAgNVKuDLothIxzi8pndB8Jju KktE5HJqUUMXePkAYIxEQ0mMc8Po7tuXdejgPMwgP7x65xtfEqI0RuzbUioFltsp1jUaRwQZ MTsCeQDdjpgHsj+P2ZDeEKCbma4m6Ez/YWs4+zDm1X8uZDkZcfQlD9NldbKDJEXLIjYWo1PH hYepSffIWPyvBMBTW2W5FRjJ4vLRrJSUoEfJuPQ3vW9Y73foyo/qFoURHO48AinGPZ7PC7TF vUaNOTjKedrqHkaOcqB185ahG2had0xnFsDPlx5y In-Reply-To: <20240531092001.30428-10-byungchul@sk.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 7DEE21C0002 X-Rspam-User: X-Rspamd-Server: rspam12 X-Stat-Signature: qwbogwaytmqyjg93mtn9n3orhzfy6y8d X-HE-Tag: 1717171990-571452 X-HE-Meta: U2FsdGVkX19o2RGggMNwawvcLgry307u5qQWf1a7JQLdu2Jvqqp12hflyNiUULrs7I+3p6b5Dnz7nNK2jcnt76UhUe1ELg//24SqzS2J03CjtUj/Dgi7W/yvmHeqyBcxRMXHaaYufPPoMrgt/irD5wRY88mhU6658r4uVU/pgIp8YyHC+ZVLY6egGJgX4vMkgm9EugN2nJePIEZKyLVRdm9PExlPudcW6IgSixsqKyLyVynBe80lKWhW8luTzEcvqpDvAu8G9XoClHDFe9Vuk80Qo4XeV14AkD55sd3Oyd+518eiPrwzZsSArLfrU5MCP/fFIzZdF1PKsrzYbQBlgn6s4FBZHzl05oKddmuO8zq9dsXESHItbQ/xbpF+IqSZN5CS48Szwk+oeWruaMc0IgD1Sy/zYFQhI9Oo27U3BKhB2mOB86Ur2KubUQk+wF45LAWEyevP+2dN4jrH4Vcu6jmQ3BnBtSW0k36ztin0uTyuZmVdm/H+nqhlnKXdEqotAc/0U2xlHGdPNyGkCFJ2AySKh8B2l5ohjd0glE4d777sHyeWVvkHFqlmk9lJAAvj1+VGap4FY5h0HBw5sEg6dQsVOWttfhRX9JNW2SUdDDtob9rRijRHFXA6ZUJimc00McdHhifHwG6bSa2YmtydvnUr1LrqqnMzrkNjCpJCy6VqQzujAzk8JDwGl1AHB/NLPcXGPKVe7kT+OrgPGA9PMxPSluEvATr6xEQdlNNf+7SDH4SiJohC255n00BDu6RvTHAXESq623aSGOPwKSK4nOINuLkTZQaxbFwpp2vETG5ql7Pk9KtwetcgAZUjoB/YYB5S7mg6ViN/yEqA4dCEOYs9Kpd50TmwIUwbG+3wGY4PDOoWAeW9396kbn8BwTNzg763DO13xOm7CcCyNI5B5auBCNTrSg6+6gbMoAVqh0VoLaIRVPAQwtIoTJ8jHk2EJvJxXUN5wqZ+Hm4LcHL iDNQERfl niA0viwWz3UH7HGcO1fubX/Np3RFY1lLrUqzBFeEse/qS+oeCZw5cpWxyZWMtSQ+JL57Tlh1ADqA7223B13o0gh7/wjmkk2MY4vT4BADkNEkpcN1en7YBI9tQAaQmBhyxEJmNaGS5I6nYW+ewK/hGChfIstbskBZ2ehvTndDHzi9yg+k+bCmuGCnlWg8rxIf7hHA5mSKaREm4/TU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 5/31/24 02:19, Byungchul Park wrote: ... > diff --git a/include/linux/fs.h b/include/linux/fs.h > index 0283cf366c2a..03683bf66031 100644 > --- a/include/linux/fs.h > +++ b/include/linux/fs.h > @@ -2872,6 +2872,12 @@ static inline void file_end_write(struct file *file) > if (!S_ISREG(file_inode(file)->i_mode)) > return; > sb_end_write(file_inode(file)->i_sb); > + > + /* > + * XXX: If needed, can be optimized by avoiding luf_flush() if > + * the address space of the file has never been involved by luf. > + */ > + luf_flush(); > } ... > +void luf_flush(void) > +{ > + unsigned long flags; > + unsigned short int ugen; > + > + /* > + * Obtain the latest ugen number. > + */ > + spin_lock_irqsave(&luf_lock, flags); > + ugen = luf_gen; > + spin_unlock_irqrestore(&luf_lock, flags); > + > + check_luf_flush(ugen); > +} Am I reading this right? There's now an unconditional global spinlock acquired in the sys_write() path? How can this possibly scale? So, yeah, I think an optimization is absolutely needed. But, on a more fundamental level, I just don't believe these patches are being tested. Even a simple microbenchmark should show a pretty nasty regression on any decently large system: > https://github.com/antonblanchard/will-it-scale/blob/master/tests/write1.c Second, I was just pointing out sys_write() as an example of how the page cache could change. Couldn't a separate, read/write mmap() of the file do the same thing and *not* go through sb_end_write()? So: fd = open("foo"); ptr1 = mmap(fd, PROT_READ); ptr2 = mmap(fd, PROT_READ|PROT_WRITE); foo = *ptr1; // populate the page cache ... page cache page is reclaimed and LUF'd *ptr2 = bar; // new page cache page is allocated and written to printk("*ptr1: %d\n", *ptr1); Doesn't the printk() see stale data? I think tglx would call all of this "tinkering". The approach to this series is to "fix" narrow, specific cases that reviewers point out, make it compile, then send it out again, hoping someone will apply it. So, for me, until the approach to this series changes: NAK, for x86. Andrew, please don't take this series. Or, if you do, please drop the patch enabling it on x86. I also have the feeling our VFS friends won't take kindly to having random luf_foo() hooks in their hot paths, optimized or not. I don't see any of them on cc.