From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7393FD597A1 for ; Tue, 12 Nov 2024 20:20:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D909A6B00C1; Tue, 12 Nov 2024 15:19:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CC8F86B00C2; Tue, 12 Nov 2024 15:19:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B1DF76B00C3; Tue, 12 Nov 2024 15:19:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 916256B00C1 for ; Tue, 12 Nov 2024 15:19:59 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 45352405D6 for ; Tue, 12 Nov 2024 20:19:59 +0000 (UTC) X-FDA: 82778558856.02.616A644 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf24.hostedemail.com (Postfix) with ESMTP id D60E118000F for ; Tue, 12 Nov 2024 20:19:51 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=BcBfe+b0; spf=pass (imf24.hostedemail.com: domain of bfoster@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=bfoster@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731442709; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WXlvQkvFWX/vPGCP4GfItyIf5yJB87PKwwJ7kOXASbs=; b=RXm2tpTHwbowNennWU25qs1ywsXHjmvNvMYd52ImkcHA6aWwpc/yT4K+bszCmTJrGfofLb /4zcPVg0Px56mm345USyz5I05Y7hmOIMtXAIlSKtjfmh4uNC7dkQiJts3Z8DYyV8cc59kI lNNm3sOIoOLXCD3ex3MGBn312WNIqL0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731442709; a=rsa-sha256; cv=none; b=K6i+JA2zqenGNLegAaMz/SXLvOqDW3SP9udUl8/UiJjcYCM701S3jZVFJpSzMY91Cjrtlq DiDTS8y3vOdP0tXZPp86TLDurmipt2diAMJcW+JEmrCr3BcJFKBZyErjp9YYp6pe1wJUSg hgrOkMNoD0GG8to9EzLNzo4APr9/uuw= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=BcBfe+b0; spf=pass (imf24.hostedemail.com: domain of bfoster@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=bfoster@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1731442796; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=WXlvQkvFWX/vPGCP4GfItyIf5yJB87PKwwJ7kOXASbs=; b=BcBfe+b0VEfmckFwI4lQS5EyrUfVo11e9rzHZKxG76uW8MGhsAtJ5spgYJKClbgM+k1itN Ty/h4U2BYgzh61LtuQUs+nYFOwAXYlewZGeoicuqC/AevZLoImLdXbi4EXttA72lWr8JfL oQyRC+UhAaYZHnWqX257y3A2k8Vrjms= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-187-SSKmdJ4cMNaqMlq2w3V5xg-1; Tue, 12 Nov 2024 15:19:53 -0500 X-MC-Unique: SSKmdJ4cMNaqMlq2w3V5xg-1 X-Mimecast-MFC-AGG-ID: SSKmdJ4cMNaqMlq2w3V5xg Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 655E0195608A; Tue, 12 Nov 2024 20:19:51 +0000 (UTC) Received: from bfoster (unknown [10.22.80.120]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D65DD19560A3; Tue, 12 Nov 2024 20:19:48 +0000 (UTC) Date: Tue, 12 Nov 2024 15:21:21 -0500 From: Brian Foster To: Jens Axboe Cc: Christoph Hellwig , "Kirill A. Shutemov" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org Subject: Re: [PATCH 08/15] mm/filemap: add read support for RWF_UNCACHED Message-ID: References: <3da73668-a954-47b9-b66d-bb2e719f5590@kernel.dk> <7a4ef71f-905e-4f2a-b3d2-8fd939c5a865@kernel.dk> <3f378e51-87e7-499e-a9fb-4810ca760d2b@kernel.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Rspamd-Server: rspam10 X-Stat-Signature: ajyz1x5y3kjnobwfc1gbeu5i9e5grcw8 X-Rspamd-Queue-Id: D60E118000F X-Rspam-User: X-HE-Tag: 1731442791-920471 X-HE-Meta: U2FsdGVkX185rXxaLqqXVLJmb8JMNANbaUMegd6+isvGw6wTtaYyuf2i+felv0htONuHGZvyUcclwi4625SaLymzk/rdgdzGAZNxCrOtK0ALY2cV0ZRkkYRzldHSXwQqLBe9XqVmt6rr+KJP5sYr27EeO8N37R+k2i9IBvoiG28K5mZP5yXVc6Fu8I7WXcAjI8RHhk0OjeRcLvbtqhbtSUzhf3b+SSZpT79UnTFR5qu1Z2vHo0ovHLXEPKR9L7CzzDBPKtHHxdv2N3qKwkjxtJxi+E/oRQIFMlezHxq7S2S6jlArRW4IhshGfI3j134S7qqK2CcmAjm1rPXqBhugu2hW/erfIEMAduhlQVc0JvD11Ib3u1AhPqiQycr+QOqLOm9OodZzOY3+TycHZe0xcu6L1ZfkTyywXubvGdz3gKGMryx+vLs6jqFIebkDQAGKy7WpwMnLgOznVeMe9vldMBmVXDeQVD7Dn+NKMymYKPfxy4BRvsk56UEAqsyw46Y6TTlpmbcl0eEKraPKmwC96I/aRp9+uAN7YvfuPLoP4BrI+BiiIfDGa4nSPZlkqmaOSp8EoJywDITtu320QaLIKD3OKC+tZDVCl14lF/cVfldTOtiE/PyQs3Qm7ENX+s5fFNrhtf0oj2dEYeMfAGQ51hoVguVu285Y2SYFaieTchgpAePQbqZ0nfuniSXnXYn3ad9DyqLHmbvQRA2myWwvvWhEl/z+2fV4/v4ItaELmTAhXXz9cD4fZl91BiYWL27ieBkJtz1FdGMn6U7LhQmNJZsBT3z/nb3cutBUnPo3xvWIjIadzk1f8y9p2b92Leydgv2XGcLiFVC3sl1y/mdPNDzOQyQ+YwXUa8tWxl7pMBYr/f2LSR07BvXGIy34Z+dawwJSSShxijmYwNvdN+P4YZ1ybflFVnlAyYuCuYxJGGlBq9nbgLAXlWlUUOnhf+dze+PuTVQzLYvEkngnSfc x9D7okep 2omQqKYwj056U2/nxIvAhMBKPvGa1L1EcUBIo6A4xUC75c3TgaYZaQ0VOVFnMrBOWW27xLnNsfKah70Fy54D+rzxIauOjshMwQ1TTOWERKeUsfwj1DADL16gG4BB4FG2sDGDPXf17/XJhVomq2/AC+5VMYRPcL6b6kS9zyffHzH5UABUDYLZHyTuddWmZSGFBtQ1MpW5AG9jcBROpiN7bnxSiszpcVMHk57HpBcO5UH5sZXYcn3rV8myjPq0yG96xY/Fm4Ai8c2HWqpRI0YNGbXZjTsZzJ6BKRj+CYRRZxBz+vaMvA7/BySFABd8eKfaoaNXwTNxOjjE9xvni1/SPhXMnz4fiI4MrIdkVz8S7E934p+e2libh1qkamDCnqnTGv8ole5Tb1Uo+G6DX2JWkqlGWqg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Nov 12, 2024 at 12:45:58PM -0700, Jens Axboe wrote: > On 11/12/24 12:39 PM, Brian Foster wrote: > > On Tue, Nov 12, 2024 at 12:08:45PM -0700, Jens Axboe wrote: > >> On 11/12/24 11:44 AM, Brian Foster wrote: > >>> On Tue, Nov 12, 2024 at 10:19:02AM -0700, Jens Axboe wrote: > >>>> On 11/12/24 10:06 AM, Jens Axboe wrote: > >>>>> On 11/12/24 9:39 AM, Brian Foster wrote: > >>>>>> On Tue, Nov 12, 2024 at 08:14:28AM -0700, Jens Axboe wrote: > >>>>>>> On 11/11/24 10:13 PM, Christoph Hellwig wrote: > >>>>>>>> On Mon, Nov 11, 2024 at 04:42:25PM -0700, Jens Axboe wrote: > >>>>>>>>> Here's the slightly cleaned up version, this is the one I ran testing > >>>>>>>>> with. > >>>>>>>> > >>>>>>>> Looks reasonable to me, but you probably get better reviews on the > >>>>>>>> fstests lists. > >>>>>>> > >>>>>>> I'll send it out once this patchset is a bit closer to integration, > >>>>>>> there's the usual chicken and egg situation with it. For now, it's quite > >>>>>>> handy for my testing, found a few issues with this version. So thanks > >>>>>>> for the suggestion, sure beats writing more of your own test cases :-) > >>>>>>> > >>>>>> > >>>>>> fsx support is probably a good idea as well. It's similar in idea to > >>>>>> fsstress, but bashes the same file with mixed operations and includes > >>>>>> data integrity validation checks as well. It's pretty useful for > >>>>>> uncovering subtle corner case issues or bad interactions.. > >>>>> > >>>>> Indeed, I did that too. Re-running xfstests right now with that too. > >>>> > >>>> Here's what I'm running right now, fwiw. It adds RWF_UNCACHED support > >>>> for both the sync read/write and io_uring paths. > >>>> > >>> > >>> Nice, thanks. Looks reasonable to me at first glance. A few randomish > >>> comments inlined below. > >>> > >>> BTW, I should have also mentioned that fsx is also useful for longer > >>> soak testing. I.e., fstests will provide a decent amount of coverage as > >>> is via the various preexisting tests, but I'll occasionally run fsx > >>> directly and let it run overnight or something to get the op count at > >>> least up in the 100 millions or so to have a little more confidence > >>> there isn't some rare/subtle bug lurking. That might be helpful with > >>> something like this. JFYI. > >> > >> Good suggestion, I can leave it running overnight here as well. Since > >> I'm not super familiar with it, what would be a good set of parameters > >> to run it with? > >> > > > > Most things are on by default, so I'd probably just go with that. -p is > > useful to get occasional status output on how many operations have > > completed and you could consider increasing the max file size with -l, > > but usually I don't use more than a few MB or so if I increase it at > > all. > > When you say default, I'd run it without arguments. And then it does > nothing :-) > > Not an fs guy, I never run fsx. I run xfstests if I make changes that > may impact the page cache, writeback, or file systems. > > IOW, consider this a "I'm asking my mom to run fsx, I need to be pretty > specific" ;-) > Heh. In that case I'd just run something like this: fsx -p 100000 ... and see how long it survives. It may not necessarily be an uncached I/O problem if it fails, but depending on how reproducible a failure is, that's where a cli knob comes in handy. > > Random other thought: I also wonder if uncached I/O should be an > > exclusive mode more similar to like how O_DIRECT or AIO is implemented. > > But I dunno, maybe it doesn't matter that much (or maybe others will > > have opinions on the fstests list). > > Should probably exclude it with DIO, as it should not do anything there > anyway. Eg if you ask for DIO, it gets turned off. For some of the other > exclusions, they seem kind of wonky to me. Why can you use libaio and > io_uring at the same time, for example? > To your earlier point, if I had to guess it's probably just because it's grotty test code with sharp edges. Brian > io_uring will work just fine with both buffered and direct IO, and it'll > do the right thing with uncached as well. AIO is really a DIO only > thing, not useful for anything else. > > -- > Jens Axboe >