From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F4A5E92FDE for ; Fri, 6 Oct 2023 03:48:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0E6236B0141; Thu, 5 Oct 2023 23:48:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 078546B0150; Thu, 5 Oct 2023 23:48:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E28416B0156; Thu, 5 Oct 2023 23:48:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id CC3E96B0141 for ; Thu, 5 Oct 2023 23:48:13 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 96A4716014C for ; Fri, 6 Oct 2023 03:48:13 +0000 (UTC) X-FDA: 81313653666.17.7F0C027 Received: from mail-yw1-f182.google.com (mail-yw1-f182.google.com [209.85.128.182]) by imf25.hostedemail.com (Postfix) with ESMTP id CB8E2A000F for ; Fri, 6 Oct 2023 03:48:11 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=DjSZ2VYW; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf25.hostedemail.com: domain of hughd@google.com designates 209.85.128.182 as permitted sender) smtp.mailfrom=hughd@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696564091; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8BzpawJSlmNG1RzfXiwjSDiv93YoIZJw/vsqhpYIp/Q=; b=xFnMLFwyoXm6at5EaDSS4qQ1Pm6BlIYeRYTy+nLkSnFrQM+3ogdek7mNPDOMtauZYY6F3q 8Td9Hx3yDVmo0xJOEXbGvMgkY2eY+gs3dVItsPNvztn6lC6WSCEwRqtAZlxX4dknGTPtTr OcZJ6ShMjBUkJOpq9T/AiQo1Tw2buzU= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=DjSZ2VYW; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf25.hostedemail.com: domain of hughd@google.com designates 209.85.128.182 as permitted sender) smtp.mailfrom=hughd@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696564091; a=rsa-sha256; cv=none; b=K3ypUWeJamDmDDQw5jV0vELWcwcAvMbQDq4UEKcuJwm+LKYDaKrzJevkxqTJC9AgxsxmMs zQzGvCXZOoloZ079VoPGaYEnBw+zw5sOIE1Y55PcFlvrKaGJCY+zyQXyRjoFI5o1idt4Ta fC/PPDXYFnkYM9TBBksQTVXA7dTXB48= Received: by mail-yw1-f182.google.com with SMTP id 00721157ae682-59f7f2b1036so20147707b3.3 for ; Thu, 05 Oct 2023 20:48:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1696564091; x=1697168891; darn=kvack.org; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=8BzpawJSlmNG1RzfXiwjSDiv93YoIZJw/vsqhpYIp/Q=; b=DjSZ2VYWjaTNzRLcNq9PKIt3C3Gve63AfxktCAx+dLXBijSG/zBfm2oCUnsCQqaWzU KBj4QZz2mUnX8QJ2eVjCWjSw6jJi8TsfT1d0zFx2Xq+HyWAl/6RjY3m3rMjG411Wwac7 6hNrzpQArj0prOez1+kv8mw852N0gJKVq6I6kMIPB79ugAPue1692//q5Kfei+ajeI40 jmUc/yPGByxZu2neUGR4zQAbjgXlSMo4Rjg/UXEarP0dqhGUJ1RhpeuI4lZ/asJPzejd rXEghqvujgmh8+pkV4EGACCr4NQEN7sA5vbOn1t1RVCiFkq9jp+Uh3pNV9hCImEwfCA+ FGiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696564091; x=1697168891; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=8BzpawJSlmNG1RzfXiwjSDiv93YoIZJw/vsqhpYIp/Q=; b=bWYKksAAuY3hWtFT6uOAlqALf3iXkkmidyZ7QOz9VzyhE99PLrjB5mgcGT0MYx8CJ8 QFlCli6hr3JK9TEwyLKKsjC58qxF5sI7wHF+1mMyCD7MloXc8N0yrQt3IIibBI6fZ4QP GrskHv9Lgf8eRoXeGYcL8Yf2cps+tXzBSZ7YOMhhdLwuV7nzQ7Wy7moO9IqHHAkwV5er ZRsJDJRWqqx7Y0XDSfWuH7U1wi538PXDPIG0cRp06gs4QScTKq/ihmsXTPDe8Z+Vhhp3 pDFnQ0zhy7nYd+AnThMW/x8qlwvPdTKAULjYGMnEC38BeotXPuAQo8kRmUYZfzMRHAd/ hMGg== X-Gm-Message-State: AOJu0YyWd0dcRdRt5dT3o/nvGdcoBM3/FRwQUzQPiAbYwBNNrwZjK53z WF9HkqYk31BvukmyJm4rle/Mqw== X-Google-Smtp-Source: AGHT+IFKYo13giuoU4JAbm3Yxs/mA1pEeUedUhqlOL/xzX3G8nGF4rXRIqpPGjn+oxsWrsZlrRyivg== X-Received: by 2002:a0d:d54f:0:b0:5a1:f0f8:4280 with SMTP id x76-20020a0dd54f000000b005a1f0f84280mr7981311ywd.22.1696564090019; Thu, 05 Oct 2023 20:48:10 -0700 (PDT) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id v143-20020a814895000000b0059be3d854b1sm978159ywa.109.2023.10.05.20.48.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 05 Oct 2023 20:48:08 -0700 (PDT) Date: Thu, 5 Oct 2023 20:48:00 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@ripple.attlocal.net To: Jan Kara cc: Hugh Dickins , Andrew Morton , Christian Brauner , Carlos Maiolino , Chuck Lever , Matthew Wilcox , Johannes Weiner , Axel Rasmussen , Vlastmil Babka , Sasha Levin , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH 3/8] shmem: factor shmem_falloc_wait() out of shmem_fault() In-Reply-To: <20231003131853.ramdlfw5s6ne4iqx@quack3> Message-ID: References: <6fe379a4-6176-9225-9263-fe60d2633c0@google.com> <20231003131853.ramdlfw5s6ne4iqx@quack3> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: CB8E2A000F X-Stat-Signature: zaqb9khhyq99rkz4j8mpnrzocywsfk8k X-HE-Tag: 1696564091-806318 X-HE-Meta: U2FsdGVkX18Wlsr2inq5wECvvMwuBQBkGwX/QuDCGfS9WE71YkEG31HZpMJQEHguyYGTMd7dfH9AXzIgRsuwbzLqu7jhDBgnXv/GRP+npb/icUh/WUaEOy8PREwoyxBkBtkUSlemfy6sA1HCW0/Ik2UUSXK8GBZ5yCdOr41iEpxY5KSgatbzfj6nJGPnSV3jxDsew9UdPlPmRDCzzoDvgKkVAy3ue6xfEuP9YsCibX4SlxiVlqp3GvVJ6FKClYeKSMXVrHEwO42bVxpspwLb+VH1dVLjg9Q8NQk2PVur9tvhT1ipNUoRSGFvlHb3YGb2+/VDRQzQxXMplnzSn+qIGNO0WPZhrf0aP6JgPVWRVvb0PDvRw7BexW3PdZPKLtkWBVgPt0Fn/QAuXxk0HN/a7xHyM9gRH24rAnJoiO4q8wr5qV3XRR5+OIY/eMu4kcj/yCMbmKUZJy6q45gyHQk2c7apw3KKFOnk9BtTgFr4Cqxf6xipQtAEhs5UIeyJhc3tkRLPX27AqfWUvgEUxHZ16q8y3ixkpNyNEDMmeRP614BpEwJDGRy+E0+OGrzv95QO6RZawP7NluSh3OqYKGkyHghVlNx4qPgCtG2ABODdqQghh1d9GwhEYrG1GVfiUESJ0cOPIJ0AprWpbZFifzhuryT6DvcX6NENEhGxnpR8y8aSSLCOHPijGyJGUPfhHseWLjA6sn+TsvKCIiwcydqCOAHfKDPL6wowB3+N8yzLP+vTYfaOOfcotA9RIlfQIEZlIHbukoXrb5bWbCqdiNlJpY4aOjJHR0kHAL0ryYjFFBhQDu/RL+bdWiMZTpDewzeDKC19kO5foWUuQYhwz80VsDHdnp3Q77Vxd1PqHY4nrhUD366UKVMsXQeTEyyqvrCFJuI4DRO7/4GMY+zpu8J2LzUu3s2xUbZeDKHQ0B6Hx5Dpy5gRkYtUZRF4IQf4a1V9jVVIfzgMDjugTK7YGrR VeTWPE+T ycraxYSe83zso5XAIMZ+hhw/qyRTFJbkKEjfZkz9baDLvQK8AEPlyufuvoSw2JGz0TgXIx7yw7buAHQvk8/NsjThKqiNapAKPObTu7lkE8gbLkiJ9mezxbCAlUvgUXk2cHhNFa7UU9rpJkBuACdMJDoAQSu60lD1TTWzeHPQQo+7bqn7Mc5u5p/AS4HPa59CS3jgaf7F1k4BjM8EEvjl/h8v+IjyfSokAuVjQ+MmkEE/vOaRNZgCEGySkgIGl+IsdAAdQEv0LPN2syjbcCcxEOv3oJ9H7FnFiXGj9Q6lN7LSV+JzRlVL3o1pQcu7WcU5BOMjT9SF6yIEh/EQc8o5dbDtW0qxU3I79QTWDxSmJTOuWvnGvtO/3/IHKgAK/rS7PA9GlFTHVSYBUW1ETlDzqfpSt2H/QZlDyVF1SxfMmZ4eCjVVB8xuOkWMT/PNSJ3LnKXtMTBw0aoPag4PUjdf2g4E+IvkSQRJY9xQnqknU5Mc7uvwNLBO0P/13Achxo3aEBZK3 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, 3 Oct 2023, Jan Kara wrote: > On Fri 29-09-23 20:27:53, Hugh Dickins wrote: > > That Trinity livelock shmem_falloc avoidance block is unlikely, and a > > distraction from the proper business of shmem_fault(): separate it out. > > (This used to help compilers save stack on the fault path too, but both > > gcc and clang nowadays seem to make better choices anyway.) > > > > Signed-off-by: Hugh Dickins > > Looks good. Feel free to add: > > Reviewed-by: Jan Kara Thanks a lot for all these reviews, Jan. (And I particularly enjoyed your "Autumn cleaning" remark: sweeping up the leaves, I've been glad to have "Autumn Almanac" running through my head since reading that.) > > Looking at the code I'm just wondering whether the livelock with > shmem_undo_range() couldn't be more easy to avoid by making > shmem_undo_range() always advance the index by 1 after evicting a page and > thus guaranteeing a forward progress... Because the forward progress within > find_get_entries() is guaranteed these days, it should be enough. I'm not sure that I understand your "advance the index by 1" comment. Since the "/* If all gone or hole-punch or unfalloc, we're done */" change went in, I believe shmem_undo_range() does make guaranteed forward progress; but its forward progress is not enough. I would love to delete all that shmem_falloc_wait() strangeness; and your comment excited me to look, hey, can we just delete all that stuff now, instead of shifting it aside? That would be much nicer. And if I'd replied to you yesterday, I'd have been saying yes we can. But that's because I hadn't got far enough through re-reading the various July 2014 3.16-rc mail threads. I had been excited to find myself posting a revert of the patch; before reaching Sasha's later livelock which ended up with "belt and braces" retaining the shmem_falloc wait while adding the "If all gone or hole-punch" mod. https://marc.info/?l=linux-kernel&m=140487864819409&w=2 though that thread did not resolve, and morphed into lockdep moans. So I've reverted to my usual position: that it's conceivable that something has changed meanwhile, to make that Trinity livelock no longer an issue (in particular, i_mmap_mutex changed to i_mmap_rwsem, and much later unmap_mapping_range() changed to only take it for read: but though that change gives hope, I suspect it would turn out to be ineffectual against the livelock); but that would have to be proved. If there's someone who can re-demonstrate Sasha's Trinity livelock on 3.16-with-shmem-falloc-wait-disabled, or re-demonstrate it on any later release-with-shmem-falloc-wait-disabled, but demonstrate that the livelock does not occur on 6.6-rc-with-shmem-falloc-wait-disabled (or that plus some simple adjustment of hacker's choosing): that would be great news, and we could delete all this - though probably not without bisecting to satisfy ourselves on what was the crucial change. But I never got around to running up Trinity to work on it originally, nor in the years since, nor do I expect to soon. (Vlastimil had a good simple technique for demonstrating a part of the problem, but fixing that part turned out not fix the whole issue with Trinity.) Hugh