From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44C8EC47258 for ; Mon, 29 Jan 2024 01:47:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 721EE6B0074; Sun, 28 Jan 2024 20:47:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6D2056B0075; Sun, 28 Jan 2024 20:47:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5987D6B0078; Sun, 28 Jan 2024 20:47:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 49F586B0074 for ; Sun, 28 Jan 2024 20:47:51 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id C2EF3120367 for ; Mon, 29 Jan 2024 01:47:50 +0000 (UTC) X-FDA: 81730662300.12.EA3D19F Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf19.hostedemail.com (Postfix) with ESMTP id C99701A0002 for ; Mon, 29 Jan 2024 01:47:48 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=fromorbit-com.20230601.gappssmtp.com header.s=20230601 header.b=iJXaSSfL; spf=pass (imf19.hostedemail.com: domain of david@fromorbit.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=david@fromorbit.com; dmarc=pass (policy=quarantine) header.from=fromorbit.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706492868; a=rsa-sha256; cv=none; b=z+x7W1OGzJ1GaW2vGzMAGgqaq2qxxGHaukU2ct27fJyuKNhRg8zCkTTdq4n+3znYYHDW8j b6PRzFi3AY9wHWxcqXuB1qF+xiLTyppRBke1duZaU8vU9SgfLJsMZbfP1mglu5nO39F9a0 mFLiDTMpWpNjuOa8ovOC1F5IB9EGr2w= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=fromorbit-com.20230601.gappssmtp.com header.s=20230601 header.b=iJXaSSfL; spf=pass (imf19.hostedemail.com: domain of david@fromorbit.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=david@fromorbit.com; dmarc=pass (policy=quarantine) header.from=fromorbit.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706492868; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/HXe1ZkYYO9fD1r0+7cwZjBt+if+SneUe0KqnaZMV1Y=; b=1zvl2l/kk0K7QO6sjZ7LTbK1GvLwQsohEoeQ7LO84iuLKMS+4b4N8PcJW6Pship9TE4iFz rnsRsMSgxFm8G8fRcM64tPdKVlXMIU4pn6bPPOpCnAkz9i5uYWjnVAEVUVEDhNZ5ykmfAe kyN0K+tetq6vIAoZxBbSacIEcoxTIIM= Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-1d751bc0c15so20784295ad.2 for ; Sun, 28 Jan 2024 17:47:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fromorbit-com.20230601.gappssmtp.com; s=20230601; t=1706492867; x=1707097667; darn=kvack.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=/HXe1ZkYYO9fD1r0+7cwZjBt+if+SneUe0KqnaZMV1Y=; b=iJXaSSfLPX8A6XkxnVeafCPSoaxKJZhMKjhy+rCHb3zm6Du/cLqRB8sBq1Bo3qDu7R c6JoM1Xe9XFZato1CfiValL+DeXea7aoPMNEod7BzjYP1gxQBnwnDTyWi7l/iQ6pqI9/ PfUPdAVUUeakZDN2/BxiLyOzXIV9xfWEWH++/v3vkdM1zXiadqzGK45xiL29bVsGAwi1 OPCnODs3yDDLPfjqzBooIy5ZzsD3YUBxhN8ytd8dA4tPkFNfbWZLQgBoTJGO90HeKMTz 5Ck4tnLIdAb1G8NsPTHrIKAJGLdpN2K3wtHZoMGj0q5qvXN2uXhHs6a5Yxt6/oslC6mo Klpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706492867; x=1707097667; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/HXe1ZkYYO9fD1r0+7cwZjBt+if+SneUe0KqnaZMV1Y=; b=MCad8Rju+BRDrymDQNvtF/RBNqBKdKa/UlePm40YKaQbp8CMOo7CXyj3zqXpOitPgm NceZxQPk3xiIj5tmuY2ynsgL7I5TtmA6dwtAVKJJkx/20AYmokzWpYbL59r831cTLDhz wS6SjvUoXwKY1yNKOvfC2ZTndRZKx/aqkgTkghNXt/qMPEIy4rUeHvxxQ7V9xGHA6uV5 xzB+QcotoX0/R3akiLOAHCwSQJ7gFI7drP7dn320gyPg62m4xWqHfgJNOkHxDiELbyYR p7rPOCkx+E6ixBxBw+pvjSAAEz/GwzUptm+3qKogQmZkkGE8hBxLhoxy++vmT8q440+A mRNA== X-Gm-Message-State: AOJu0Yw7Z/AVDhjL/3TUURU893qgxvka/dSC9Hmavr+V2A6KFzd4B/50 14HYHzyG4xr8pzDdQFWDhyhbk/LHQmEQg27IimNfwgyshomO3f+NABFuSduiAPk= X-Google-Smtp-Source: AGHT+IEq3c3K1ygyW/CVEYqCz0nHDHcKikCupujYmvZwln17s4zPmpinqbh5fIhJ0e17Gfa+s7b+aA== X-Received: by 2002:a17:902:d4c9:b0:1d8:c3be:8f10 with SMTP id o9-20020a170902d4c900b001d8c3be8f10mr3714366plg.46.1706492867567; Sun, 28 Jan 2024 17:47:47 -0800 (PST) Received: from dread.disaster.area (pa49-181-38-249.pa.nsw.optusnet.com.au. [49.181.38.249]) by smtp.gmail.com with ESMTPSA id iz3-20020a170902ef8300b001d8e974ed2fsm292730plb.284.2024.01.28.17.47.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 28 Jan 2024 17:47:47 -0800 (PST) Received: from dave by dread.disaster.area with local (Exim 4.96) (envelope-from ) id 1rUGkP-00GfP7-0g; Mon, 29 Jan 2024 12:47:41 +1100 Date: Mon, 29 Jan 2024 12:47:41 +1100 From: Dave Chinner To: Mike Snitzer Cc: Matthew Wilcox , Ming Lei , Andrew Morton , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Don Dutile , Raghavendra K T , Alexander Viro , Christian Brauner Subject: Re: [RFC PATCH] mm/readahead: readahead aggressively if read drops in willneed range Message-ID: References: <20240128142522.1524741-1-ming.lei@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: C99701A0002 X-Stat-Signature: 555p6qn1mpt3me6zcpky3c851i84cahx X-Rspam-User: X-HE-Tag: 1706492868-264843 X-HE-Meta: U2FsdGVkX1/9uIc24UkXAp20kGpqpQaxst2Kx4o+uYnQo0IzGjZUkPHTII4Kx9p63aXFzrD9IdYUjtRw+y/85sC0fAXp4PoibNB/hnq2QqS3ZJB222SQl2h4+x+jeffJrdSAIVRTEH9eEieweO//GzGicnbngJNx3XQnXiS8Rv/6BqokrcwC1W4QsUqyg8ZhpwUKqNfK6VAU49dByqzTleILU4eSB7/HNXvpymYGphyt0XNrZYv9hgzDTpUjx0VDntZF+8uBQk2U1RQaKKjR045PaK7DYq66ImqWbLZAJZWxDeqKwIOV0qxiiG+neU4vlPFAc6ElrnfA1iJxwHSpPnsot9vGhbyUlIVNKJaooqyp6Eq1hOD+aeQunM/KRHGucuz1+m+2sFP8ntvPqUB4SDSwZvPo/gxDJXnWj+ZiVVZDDgQ57BPDYjxwsQ58BP7+V2sd9U2kOYCMVfsJDDfjKoQqqMA0OobIbib1fG296QkTqh+yWFrtlw1rpWofIKz15UeaUgZA7cMm9ON8T1AmJ1jEb2FX0DrPtDqZxc8A0mnvZR8OimcRKmqf4IGK/RQ1sn+PFETZPLVevo+BUosb36F52URsDqyANoEBgPOd0beZ8FRHbuUJia768VQpUZn8KWHWtApGM+Tps1Oh6baRU1aa0Cgo5gdnTJ6KpDbaB+zP2zMdBe2pSMfLWKiaivBUtR2rJaSKoQXc5CM195VBWFEJtCVSeolb3YvQaHc290ErE3kik7oqOUrJV7XfiujbEQoSYEJFbqyuzrw6rMtmbw5D342Lw/BYNhYcbUDkKHwGnXm9QAv5ThGkanzYwS9s3PniKbOeRlpCbE7hi5jB3I1hDjR7l/6j94DHPqAfUKuaTw8S9x08vRMUBoyT9dZtFoISuXpriz1fjTH9qwNV5M42x18F/0dthNNhO8lBHJ2NPYD9ovf1WVPQHKFR6OVVdXvpF3nYR8jomMp0trt olkb2gaq NyvWeYCoKdHSO0fLnT4FrrLuLvCfnJdRzRA7rvrnfK0nFiRYzMAc8BCAvh4BTGJYzHzJQx6mwe/6ZV/KNphfpIY3yn+nfWuy9UW1NveSjQy6kI3iluuicu+Laq6TVvT9w3vi8R+E0cIWqkkyje1orNane5LQRjZUUtdrACiiFW+zAj3R4u+J4JK2ZDke7Bi5Wb3sweJ4gcTpD275J3mprGlw9Xkv2Wvrxrm0pvE8k9pXCLV6EoLHdPa4GlpFAdBqBhq48xv/bxAHZ7hZr+NdC2rsrTJSswMrnmOZSoqueeAZIoLv38bSqJxzDtRE3UjCzeO9RHujtCqEZ9hG6IUivGhsMKZGx4DFEgJoP3g+4nQMCOIIYPpXPjt43igdI/3hDT88DY7T452L4c/aTVwNpPngqSklHzeZn2aXOMl1tTEic4xmRgNdB4ii/Rwk0qJH2KCJjVBT8QF4EHFjjg/qlUVDAq+CEUHrYYsnJOA/8dFFrfkUOO1tEF5AQMeEUDYKl+GK0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000004, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sun, Jan 28, 2024 at 07:39:49PM -0500, Mike Snitzer wrote: > On Sun, Jan 28, 2024 at 7:22 PM Matthew Wilcox wrote: > > > > On Sun, Jan 28, 2024 at 06:12:29PM -0500, Mike Snitzer wrote: > > > On Sun, Jan 28 2024 at 5:02P -0500, > > > Matthew Wilcox wrote: > > Understood. But ... the application is asking for as much readahead as > > possible, and the sysadmin has said "Don't readahead more than 64kB at > > a time". So why will we not get a bug report in 1-15 years time saying > > "I put a limit on readahead and the kernel is ignoring it"? I think > > typically we allow the sysadmin to override application requests, > > don't we? > > The application isn't knowingly asking for readahead. It is asking to > mmap the file (and reporter wants it done as quickly as possible.. > like occurred before). ... which we do within the constraints of the given configuration. > This fix is comparable to Jens' commit 9491ae4aade6 ("mm: don't cap > request size based on read-ahead setting") -- same logic, just applied > to callchain that ends up using madvise(MADV_WILLNEED). Not really. There is a difference between performing a synchronous read IO here that we must complete, compared to optimistic asynchronous read-ahead which we can fail or toss away without the user ever seeing the data the IO returned. We want required IO to be done in as few, larger IOs as possible, and not be limited by constraints placed on background optimistic IOs. madvise(WILLNEED) is optimistic IO - there is no requirement that it complete the data reads successfully. If the data is actually required, we'll guarantee completion when the user accesses it, not when madvise() is called. IOWs, madvise is async readahead, and so really should be constrained by readahead bounds and not user IO bounds. We could change this behaviour for madvise of large ranges that we force into the page cache by ignoring device readahead bounds, but I'm not sure we want to do this in general. Perhaps fadvise/madvise(willneed) can fiddle the file f_ra.ra_pages value in this situation to override the device limit for large ranges (for some definition of large - say 10x bdi->ra_pages) and restore it once the readahead operation is done. This would make it behave less like readahead and more like a user read from an IO perspective... -Dave. -- Dave Chinner david@fromorbit.com