From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64BF4C77B70 for ; Mon, 17 Apr 2023 15:10:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E94796B0071; Mon, 17 Apr 2023 11:10:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E44F68E0002; Mon, 17 Apr 2023 11:10:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D0E2F8E0001; Mon, 17 Apr 2023 11:10:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C13C46B0071 for ; Mon, 17 Apr 2023 11:10:11 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 49AE7AC14D for ; Mon, 17 Apr 2023 15:10:11 +0000 (UTC) X-FDA: 80691218622.20.1D5CE04 Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by imf04.hostedemail.com (Postfix) with ESMTP id 8DE014000F for ; Mon, 17 Apr 2023 15:10:07 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=fastmail.fm header.s=fm3 header.b="lCW/YyIg"; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=HIVkCOtl; spf=pass (imf04.hostedemail.com: domain of bernd.schubert@fastmail.fm designates 66.111.4.25 as permitted sender) smtp.mailfrom=bernd.schubert@fastmail.fm; dmarc=pass (policy=none) header.from=fastmail.fm ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681744207; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TS9dMLl1hXAh0cv6Px1UOsJTZ3pL/MLAgWsqzx5qTU8=; b=Al/Z2lKcCRH8r/ieeHtqIoTYeDh0ZQTsEyJ4H5ZmOsTmWKwp1tCi/QXcKO0EdcZUZ6QK5M fbhYAxI5zFdavPHOr2Suwa6R0mo2v0b4cIUClE1pgAeGe7ifqLVb52vzqTgehwLlx2CLpW hQwH+NnNmrsuQiV9+uxDNGv7Okq8BJE= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=fastmail.fm header.s=fm3 header.b="lCW/YyIg"; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=HIVkCOtl; spf=pass (imf04.hostedemail.com: domain of bernd.schubert@fastmail.fm designates 66.111.4.25 as permitted sender) smtp.mailfrom=bernd.schubert@fastmail.fm; dmarc=pass (policy=none) header.from=fastmail.fm ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681744207; a=rsa-sha256; cv=none; b=L0Sqi5U8DY7ZKESxEblOSpUCIlBROmuqIFVONAs6ZQltrM1usK9dGjKkHU5xx9WOql41Zo mqto6ORy89pp+lLJeukVmzZQoFbPzNsS1ltYHzHg/NHwFA7StLjC/DO5wFgKDnYKwZb6Ap 4AhfrtytMGZSHofxHl6bKWI9VJ+03mY= Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id D98E15C0110; Mon, 17 Apr 2023 11:10:06 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Mon, 17 Apr 2023 11:10:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.fm; h= cc:cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm3; t= 1681744206; x=1681830606; bh=TS9dMLl1hXAh0cv6Px1UOsJTZ3pL/MLAgWs qzx5qTU8=; b=lCW/YyIgXLHf2QfAPXj7gabdcE+BamOdn1rtThmbgmbaec1uWc+ tAr2LJF50C7qPUFwW71jKy5nswyjufmtHV/wyfpCDgO1AYPOoUfI1RA/IIKKnx8+ 67hOTyu7SuKlG9yNvrFKGtPrGMikqR1Td5fl55zB3TO3/GZjfp5Cdr+ey57yIaaX 4yMt8IqAf6gvIAHmKjfHSZlzo6GLiQPV/GmILz/0eHG1CjXHb7+PWTFZS05soKr+ jYMrwB44jmyQE1fGWnPgygg58uczA2q2kgsOQuXagcTqoXEaQPrmvt0cmhcsykje zwx7q57UyRtZ0RI3vTXm6mLoXOXUqeSAYiA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1681744206; x=1681830606; bh=TS9dMLl1hXAh0cv6Px1UOsJTZ3pL/MLAgWs qzx5qTU8=; b=HIVkCOtlZRX+Th9ErFStcskg/WByy0+EjyvCwZZ2tp7GVQTO0w1 usZBUj+qF9W+RMX9qJ1p5UXXXLiS5vlvJm9ympTqYHOWcf6QvDRuM9nIE+AOZVNC hbPq/P6owa3nHj6HJ4MlrVIR0rrZ+BHCUd8cHtDlYttGi+2n2Dh2wlLXubfcVb5Y mPxu6+tqHelKujOtcFuK/1EenAoJJoxvnB0DrejQQEEByiNKXqR7NMmQaA9uTYwd VHl1dGAHI0yDLUtUmOPLVquWQyxAPVeTsKL4w0/gE/px9jJ07jBrnRt+HqhIG7rX PWjPsGlD/OGQl9q21vT2hD45hLbt3r3PVlw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrvdeliedgkeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeeuvghr nhguucfutghhuhgsvghrthcuoegsvghrnhgurdhstghhuhgsvghrthesfhgrshhtmhgrih hlrdhfmheqnecuggftrfgrthhtvghrnhepkeehveekleekkeejhfehgeeftdffuddujeej ieehheduueelleeghfeukeefvedunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrg hmpehmrghilhhfrhhomhepsggvrhhnugdrshgthhhusggvrhhtsehfrghsthhmrghilhdr fhhm X-ME-Proxy: Feedback-ID: id8a24192:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 17 Apr 2023 11:10:05 -0400 (EDT) Message-ID: <48eca641-e810-fac5-0ff0-8762eed3f61e@fastmail.fm> Date: Mon, 17 Apr 2023 17:10:03 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: sequential 1MB mmap read ends in 1 page sync read-ahead To: Matthew Wilcox Cc: "linux-fsdevel@vger.kernel.org" , Miklos Szeredi , Andrew Morton , linux-mm@kvack.org, linux-doc@vger.kernel.org References: <1e88b8ed-5f17-c42e-9646-6a97efd9f99c@fastmail.fm> Content-Language: en-US, de-DE From: Bernd Schubert In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Stat-Signature: t4k91yb51kiu98dppbpifsgm3by4jfgr X-Rspam-User: X-Rspamd-Queue-Id: 8DE014000F X-Rspamd-Server: rspam06 X-HE-Tag: 1681744207-706006 X-HE-Meta: U2FsdGVkX19a7v46bc8tfwzfACv7XjFFOVu21jYJ4D3vhiEtoFq1ZD5LTJEj2hXFaATnNFM6cLEBaEA1U9b6ZvxrYM6v01fI9F/d/yrxQci5ueqtpK+xueDxLv62CHfr6krgD96KzFBDEwlcAkqQNlYyYNw18oXBM3+6aKP97RkjB9t6V0knRa2tTlkoEQuVodWlI1BHzDjQxwweWVktYrxcXTKr7Hav/xk//wirSW3MxuPK+XJ6bsphIj3l997dj7BGX0aefZUDNZnOKPM/HBnBi89XqWEI6JKRsk90tT2THXABxy1mPPyo1EST7SyrEAwnLcAkRbP3foMWCvCMB2mWnE4xZj6+sn9KJsWhrmtJI2JnrLu9AVxLrAeajtleFPXHWcethMrrClm1YQNYKvQYNxhl51rVYBRc/2xyPcGongwFkY4cy0sX74G2nY6hMj/At9P5rFi4dHdZZRnvjtOIYCI8IREPHluqXZKuPrVlaqGRSAL/FkhaznvCBxzclR6NVuZHpGAnuJwQN14zvg4Gt9lvtJ4CeP0i00NQKvvj0PeRDKLWSk5X6IxjcbSZaeP+fpu6tX0S0duLkjFj3NdULLKPdJepFcXDlPjwT8rKPcJWRrWNxCvTornZ9JxE4j82D5Dwq22riMWTJC/ds0SFzeaJ3BGwS0wJsBLZt6NNkIf4l7DI4J6pmWo3/w3oP/CnZTCXLn1TcVK7Lx+rvFvlZR0O3jnWtzX9DRMfYfRTDHn8jBFvKz8Dzud0i8EJo0iBkoCpMznLBA3dsuXK8+CQcwzZCaSjRGWTKj6awR3V+3IVrxStSkMx+R5nHgDDFvmq9YY9S2L3lBcm/8sadKIaHO5H4CqICf5n8l5/6JqPCI9FJD9RxomzMOPatZp+jlF7qWgmdK8mbd7t3Nyyl7hvtLA171ZsWZXi0tjivaYqF81it8lkas7T0GORz4c3xECVNg0im1DM47D3idP 56YUjnkG yShojxeLIwGYp4QE6Pl1tnupDLMj3xDMwA7gWFWRu+Xrl96zODPuKN01sq/UqIk9XquHtFlRU4Xs9opx6InQlpUc+6j/4At8s6CnRrcYbcorruTf2WPFmVpPoXVDuO7HKQedGuB1QJruFaZl5J9n/rLWFyR0QrUm5ISg9s9rr2zTfy7BXVgg8ERW9zxQhZicq5h20EZ9fk6SwptrNRCs4i31NbMn1TLfPwG723osQooOTL7fQQb81UMvWHhdYEvgEJmTt65KTktPsHuhVxob5oPkIAejhPVIPjDdrMQA2UDGVfN9On02rD9rg9w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 4/15/23 00:02, Matthew Wilcox wrote: > On Fri, Apr 14, 2023 at 10:47:39PM +0200, Bernd Schubert wrote: >>> Up to bs=512K it works fine, 1M (and for what it matters >>> already 768K) introduce the order=0 issue. >> >> Hmm, I replaced memcpy with dumb version, that copies byte by byte - problem >> gone. Is it possible that the optimized memcpy causes caused kind of random >> memory access and confuses mm / readahead? >> And somehow your memcpy or system is not doing that? > > Oh, that would make sense! If the memcpy() works high-to-low, then > you'd see exactly the behaviour you're reporting. Whereas low-to-high > results in the behaviour I'm seeing. In my case it is not exactly high-to-low, it is more low, high, then from high to low. Issue goes away with a sufficiently large RA size. And RA behaves better when POSIX_MADV_SEQUENTIAL is not done. For sure memcpy implementation and cpu depending (I have tested on avx and avx2 systems). > > Hm. I don't know what to do about that. Maybe a "sufficiently large" > memcpy should call posix_madvise(src, n, POSIX_MADV_WILLNEED) What would speak against ignoring POSIX_MADV_SEQUENTIAL for RA when it would detect that something breaks the contract? But then I don't know how much mmap/memcpy is used for large memcpy or memcmp - maybe we should just document the issue right now in the mmap man page? Especially that POSIX_MADV_SEQUENTIAL should be avoided? Thanks, Bernd