From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5593BC48291 for ; Mon, 5 Feb 2024 09:54:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E2B8C6B0071; Mon, 5 Feb 2024 04:53:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DDADA6B0074; Mon, 5 Feb 2024 04:53:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA4076B0085; Mon, 5 Feb 2024 04:53:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B84916B0071 for ; Mon, 5 Feb 2024 04:53:59 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 94196C09A9 for ; Mon, 5 Feb 2024 09:53:59 +0000 (UTC) X-FDA: 81757288998.30.1695618 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf04.hostedemail.com (Postfix) with ESMTP id C55B74000A for ; Mon, 5 Feb 2024 09:53:57 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=VhSDuT6i; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf04.hostedemail.com: domain of ming.lei@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=ming.lei@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707126837; a=rsa-sha256; cv=none; b=7lgUcTf7yCuGULtMahLpB0kYoXkJT5MC9mHEkrHHmcp821K8qcAEoKMILsOY9NMvQr4yec ggE3z85kS9sfMBq41oRiplQPCIAMfH83mNMGTTwLRmb6qo2BQZB5yxGhH4ao8Ge4AlsnQ1 tEZiBIbinqVNKjgMus+CA336La08YgA= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=VhSDuT6i; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf04.hostedemail.com: domain of ming.lei@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=ming.lei@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707126837; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=L40x9EmC//Z/IURxbr1HiKYTbhCONWsaujp5SiZFte0=; b=Cvz9/9Ngg/3IiCiOcmcf2GWYIHemUJtWdk2f5Y/JDrtbioUnCTtlGiaqkbyIEBPx0slHAG TJgEidJo98lX+e34J0mILjZ9EnJ+yjMEl+obazn1j7idCyF6YBoTq1r5wG8Pa6NDKCTpWm 3IcwgS4e2ODPvDQs7ToQqxF+7vk/sS0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1707126837; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=L40x9EmC//Z/IURxbr1HiKYTbhCONWsaujp5SiZFte0=; b=VhSDuT6ilpmM0MDRbd/p8iSCc7n/vuYI3t/KI/fSiZaM7ychldguLH0WrisLr1WxnW9ZMw kZTCKsRINZ9neY0nIUzDKTvSzNFOH2zg3Mo/y+iwHUHN0gVb1PmpAZs4Tm9ntm2N4RKR0L bjRYnJi/46o/9lvABJfLCinCA9NISJE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-49-JhF2NxylPlyvXTa0o4_Bkg-1; Mon, 05 Feb 2024 04:53:55 -0500 X-MC-Unique: JhF2NxylPlyvXTa0o4_Bkg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0897A1065221; Mon, 5 Feb 2024 09:53:55 +0000 (UTC) Received: from fedora (unknown [10.72.116.6]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5CE813C2E; Mon, 5 Feb 2024 09:53:49 +0000 (UTC) Date: Mon, 5 Feb 2024 17:53:45 +0800 From: Ming Lei To: Dave Chinner Cc: Andrew Morton , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, David Hildenbrand , Matthew Wilcox , Alexander Viro , Christian Brauner , Don Dutile , Rafael Aquini , Mike Snitzer Subject: Re: [PATCH] mm/madvise: set ra_pages as device max request size during ADV_POPULATE_READ Message-ID: References: <20240202022029.1903629-1-ming.lei@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: C55B74000A X-Stat-Signature: iq7k1pj75wijb9eej4a5gymz5drtfwmm X-HE-Tag: 1707126837-915692 X-HE-Meta: U2FsdGVkX19i7Jn60VpLpUJYl5tCiCH96lCI7asrNbXjNM/ymQ3rOPJvr+5Pb5NQOhdxblQLppBFA+ZSFWzsVqkvKq3K+r5UeCPUb25Od4zofUskFYIDkw3ZR3GqEi/xgnLBZdDChaM1ZsMBhTWW/eoQtF7aVF1s3+GZbaGocOxTDBlOZnkwwQFJWFd/yqImhMrsgYnB/TrAuuXYANDYc7ESsHJ0PAY1hnx+Bhk+qF3Gy4C7OmW21LWkZZdgRFy2diCM5XvukgYnNyUxKBWXXhL9oM6POPRtkNS2fB0O1E6RHYorCK+x5jOhge1gFgu/cu0PwCLpA3432YHLGURsEKdWRXpv+TbYLdp0ZE2jqRiawJJMSMhE5/RbO2zrywO9QLnKJIpSnMGkNBaGBTJvciOV5vipu56kAPwelpipEQWVnXX5IuEt4WRHDJ/tsbCl4xOQyBkk40DwsJiHeQzmLQSu3FGQMVr8ORSlkFQaJ0e6Kft9IwwH1/qdnt6qR824F7kflgKspkpZfnHtZs9h4BoK+LVJT9PRwDBhw5G4QSPX6kYHjmbIb+7SjlbmSzkl651Mogw9NfZXqPXmkNJeDuPVsitWVdNgcNbMWuZmxR1JdzBJO0GnKRahiul9FrPckGeYS/2Y2EO3fCoaNb8SbSPlnL5eZhsYzda+lAcOMlsBM0n41Si+fYJAnBF+s7iuP/PAC0wVJagf39CQwkCuoybtWgTNNmEiQC1veIp34vIoKmHlHiewqLVeOcWk+wQ9InelAOeFDefi3XkjUhdIFepQnSsn5njINGfdlNyvsonjV8I/+nT8aXMrjbQbn/1CDVJ3DE07gtc3teUEfHsLPGsHPwsNjRt4LNxDPklRFrLvn80pVlULTvz1fcnv0ZOAszwNiLMESPCHrjXzy2cL+FzS2capcTlk9gLEzwRbEPULxDCmLYq9td08LVPSGtp4EOdH/+gfFGj6yxDcYqs sVGgj2i5 WsHDSReNQi1kT+zSH6DJXA5MX5NZqc7lhDAEQGyuWruWYuiAo+Xb15YQ/rK+a3dc5jEX1So6I4MAi86iiYCO2U7qOaSUh97EWckb9CsqMMP4/wa6MORqDqcaG85wjqFJOurRRJc+/HXETI2DPsK8x+IYaogUgd5gAzHTP2rtnbNCK8uYf9BkuJ/tJC8jAPP6sKiDOTMrhT8dQ1t2m+++eYwekErwQ1Kl910CM+OMNh3MK9HEHNlUpGpebNYm4XoMTqVjyymH/Lo2eOnwQ7WzhdLDqunVPVWAvJ8LmwZwcKElYrVIhr9u8u+h9K2yjFmyC6tgw8MeN+5NPqoGKB+Czdl7auW5P8WEhe2orDYVtiE8NNijHxdvBpKD9HD6GUWw7yUJAIBOR7tkODEQjJ3P9E3jD6mIGbzDJov55wPbaCGWCnn1pqP0gJRIULpSY2goSV7SON0QCZoevlXiLobpXL9E/zL4ey3m3mEoEnibsBWoIeqcz+cE2K4DeCDsxUag3oDaxCYBvSVUs0V5zoLgCdJ4/3w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Feb 05, 2024 at 10:34:47AM +1100, Dave Chinner wrote: > On Fri, Feb 02, 2024 at 10:20:29AM +0800, Ming Lei wrote: > > madvise(MADV_POPULATE_READ) tries to populate all page tables in the > > specific range, so it is usually sequential IO if VMA is backed by > > file. > > > > Set ra_pages as device max request size for the involved readahead in > > the ADV_POPULATE_READ, this way reduces latency of madvise(MADV_POPULATE_READ) > > to 1/10 when running madvise(MADV_POPULATE_READ) over one 1GB file with > > usual(default) 128KB of read_ahead_kb. > > > > Cc: David Hildenbrand > > Cc: Matthew Wilcox > > Cc: Alexander Viro > > Cc: Christian Brauner > > Cc: Don Dutile > > Cc: Rafael Aquini > > Cc: Dave Chinner > > Cc: Mike Snitzer > > Cc: Andrew Morton > > Signed-off-by: Ming Lei > > --- > > mm/madvise.c | 52 +++++++++++++++++++++++++++++++++++++++++++++++++++- > > 1 file changed, 51 insertions(+), 1 deletion(-) > > > > diff --git a/mm/madvise.c b/mm/madvise.c > > index 912155a94ed5..db5452c8abdd 100644 > > --- a/mm/madvise.c > > +++ b/mm/madvise.c > > @@ -900,6 +900,37 @@ static long madvise_dontneed_free(struct vm_area_struct *vma, > > return -EINVAL; > > } > > > > +static void madvise_restore_ra_win(struct file **file, unsigned int ra_pages) > > +{ > > + if (*file) { > > + struct file *f = *file; > > + > > + f->f_ra.ra_pages = ra_pages; > > + fput(f); > > + *file = NULL; > > + } > > +} > > + > > +static struct file *madvise_override_ra_win(struct file *f, > > + unsigned long start, unsigned long end, > > + unsigned int *old_ra_pages) > > +{ > > + unsigned int io_pages; > > + > > + if (!f || !f->f_mapping || !f->f_mapping->host) > > + return NULL; > > + > > + io_pages = inode_to_bdi(f->f_mapping->host)->io_pages; > > + if (((end - start) >> PAGE_SHIFT) < io_pages) > > + return NULL; > > + > > + f = get_file(f); > > + *old_ra_pages = f->f_ra.ra_pages; > > + f->f_ra.ra_pages = io_pages; > > + > > + return f; > > +} > > This won't do what you think if the file has been marked > FMODE_RANDOM before this populate call. Yeah. But madvise(POPULATE_READ) is actually one action, so userspace can call fadvise(POSIX_FADV_NORMAL) or fadvise(POSIX_FADV_SEQUENTIAL) before madvise(POPULATE_READ), and set RANDOM advise back after madvise(POPULATE_READ) returns, so looks not big issue in reality. > > IOWs, I don't think madvise should be digging in the struct file > readahead stuff here. It should call vfs_fadvise(FADV_SEQUENTIAL) to > do the set the readahead mode, rather that try to duplicate > FADV_SEQUENTIAL (badly). We already do this for WILLNEED to make it > do the right thing, we should be doing the same thing here. FADV_SEQUENTIAL doubles current readahead window, which is far from enough to get top performance, such as, latency of doubling (default) ra window is still 2X of setting ra windows as bdi->io_pages. If application sets small 'bdi/read_ahead_kb' just like this report, the gap can be very big. Or can we add one API/helper in fs code to set file readahead ra_pages for this use case? > > Also, AFAICT, there is no need for get_file()/fput() here - the vma > already has a reference to the struct file, and the vma should not > be going away whilst the madvise() operation is in progress. You are right, get_file() is only needed in case of dropping mm lock. Thanks, Ming