From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F64FCA9EBB for ; Wed, 23 Oct 2019 07:11:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0E9E32173B for ; Wed, 23 Oct 2019 07:11:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0E9E32173B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 95AE16B0003; Wed, 23 Oct 2019 03:11:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 90B026B0006; Wed, 23 Oct 2019 03:11:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 820376B0007; Wed, 23 Oct 2019 03:11:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0073.hostedemail.com [216.40.44.73]) by kanga.kvack.org (Postfix) with ESMTP id 5C0876B0003 for ; Wed, 23 Oct 2019 03:11:51 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id D878583FE for ; Wed, 23 Oct 2019 07:11:50 +0000 (UTC) X-FDA: 76074179580.16.top05_675d39075b400 X-HE-Tag: top05_675d39075b400 X-Filterd-Recvd-Size: 3802 Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by imf23.hostedemail.com (Postfix) with ESMTP for ; Wed, 23 Oct 2019 07:11:50 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 5C515B3F8; Wed, 23 Oct 2019 07:11:48 +0000 (UTC) Date: Wed, 23 Oct 2019 09:11:46 +0200 From: Michal Hocko To: Dave Chinner Cc: Mike Christie , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, martin@urbackup.org, Damien.LeMoal@wdc.com Subject: Re: [PATCH] Add prctl support for controlling PF_MEMALLOC V2 Message-ID: <20191023071146.GE754@dhcp22.suse.cz> References: <20191021214137.8172-1-mchristi@redhat.com> <20191022112446.GA8213@dhcp22.suse.cz> <5DAF2AA0.5030500@redhat.com> <20191022163310.GS9379@dhcp22.suse.cz> <20191022204344.GB2044@dread.disaster.area> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20191022204344.GB2044@dread.disaster.area> User-Agent: Mutt/1.10.1 (2018-07-13) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed 23-10-19 07:43:44, Dave Chinner wrote: > On Tue, Oct 22, 2019 at 06:33:10PM +0200, Michal Hocko wrote: Thanks for more clarifiation regarding PF_LESS_THROTTLE. [...] > > PF_IO_FLUSHER would mean that the user > > context is a part of the IO path and therefore there are certain reclaim > > recursion restrictions. > > If PF_IO_FLUSHER just maps to PF_LESS_THROTTLE|PF_MEMALLOC_NOIO, > then I'm not sure we need a new definition. Maybe that's the ptrace > flag name, but in the kernel we don't need a PF_IO_FLUSHER process > flag... Yes, the internal implementation would do something like that. I was more interested in the user space visible API at this stage. Something generic enough because exporting MEMALLOC flags is just a bad idea IMHO (especially PF_MEMALLOC). > > > >> This patch allows the userspace deamon to set the PF_MEMALLOC* flags > > > >> with prctl during their initialization so later allocations cannot > > > >> calling back into them. > > > > > > > > TBH I am not really happy to export these to the userspace. They are > > > > an internal implementation detail and the userspace shouldn't really > > > > > > They care in these cases, because block/fs drivers must be able to make > > > forward progress during writes. To meet this guarantee kernel block > > > drivers use mempools and memalloc/GFP flags. > > > > > > For these userspace components of the block/fs drivers they already do > > > things normal daemons do not to meet that guarantee like mlock their > > > memory, disable oom killer, and preallocate resources they have control > > > over. They have no control over reclaim like the kernel drivers do so > > > its easy for us to deadlock when memory gets low. > > > > OK, fair enough. How much of a control do they really need though. Is a > > single PF_IO_FLUSHER as explained above (essentially imply GPF_NOIO > > context) sufficient? > > I think some of these usrspace processes work at the filesystem > level and so really only need GFP_NOFS allocation (fuse), while > others work at the block device level (iscsi, nbd) so need GFP_NOIO > allocation. So there's definitely an argument for providing both... The main question is whether giving more APIs is really necessary. Is there any real problem to give them only PF_IO_FLUSHER and let both groups use this one? It will imply more reclaim restrictions for solely FS based ones but is this a practical problem? If yes we can always add PF_FS_$FOO later on. -- Michal Hocko SUSE Labs