From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5209C433EF for ; Wed, 19 Jan 2022 17:52:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4220F6B0072; Wed, 19 Jan 2022 12:52:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3A9A26B0073; Wed, 19 Jan 2022 12:52:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 223A76B0074; Wed, 19 Jan 2022 12:52:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0183.hostedemail.com [216.40.44.183]) by kanga.kvack.org (Postfix) with ESMTP id 0D36C6B0072 for ; Wed, 19 Jan 2022 12:52:45 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id B505B8297CD5 for ; Wed, 19 Jan 2022 17:52:44 +0000 (UTC) X-FDA: 79047781848.28.6AA3BA9 Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) by imf06.hostedemail.com (Postfix) with ESMTP id BA3A8180025 for ; Wed, 19 Jan 2022 17:52:43 +0000 (UTC) Received: by mail-wm1-f44.google.com with SMTP id v123so6723922wme.2 for ; Wed, 19 Jan 2022 09:52:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=n9+QuY0zH4n6DhdsHh/bGzBmsewJnPu1xsyY0wQOFNs=; b=Qwr4sXWlG63KPE36dssUlXfgc2ML4gn5DUWXdNefR/tdwjFsg7Y695LkndA6iRGRSt kTeZ+YBr95b8stCVFdmxVu0sQIkSPxHL5hVqZQiZmevmu74D5ALilFJ9gWC3jDvJZkHn bKdmr6EcNdM8IwETGj1/+v8U1zr2Vg+LJodTULKoDO7CzU5jTaXlF/gtnGWMITm+RWQA LnR0F810BIuRQPGwa9kw7AZzJwyju0XCYfqlG8fzIBB2Ml/04RkaM6LyvW1puu4OyK9f qRDzmmQwlVDvs38rwANgJ+GwjWXD6zOn0cIVQDXc3XkrDVbIKY5yEJIaUy2tiUsfxaGq Ec4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=n9+QuY0zH4n6DhdsHh/bGzBmsewJnPu1xsyY0wQOFNs=; b=3OkAP40WAZriMxvkuWjQ26Efon0HUJXDSZm+VXkz2zdC42gcOfFI82TDOVwDLovC6r WPkOY8CAl1inQTaXfxGH7fiEP9lx7wZPa1e/h/pd3q73r2Asdm5dlArfr0Tspiv13VsD iosauzVaVSFIYrOplX255/YpAea1SptYjr80xbDF89Yr6NVSuE1oDzQVt46Zb044vOJy AWndpMf0xreFXkOdPZUoq5R2xQ/9/SVA5Iy0CUNz/qtvU7/trJyaDDWkCgfBEFMcG36s xwI3f7qL2s2EqxjPRZcaCVxnLAobr/JGqD8ILuv+Vs6i7Wu8qdFjj8hr96GtYesek8fU 0FWw== X-Gm-Message-State: AOAM533PGmxRgC6rCvQXUSxWS1TomzXr+eE2JfxUIffWs/ZOWUoe2iHC u+9Q/pltTHeSmasUNQ3YkNqf/8qxekBbiHxXBKhJfw== X-Google-Smtp-Source: ABdhPJyTgn6+uRrI+ipD2jsY1qIhZf35U/++/qi7b/L4vt6/9XULtwfeWGPSU219Ql2kbuVOWQA/5YECcad0sB1B9PI= X-Received: by 2002:a5d:6da4:: with SMTP id u4mr25831925wrs.82.1642614761924; Wed, 19 Jan 2022 09:52:41 -0800 (PST) MIME-Version: 1.0 References: <20211214204445.665580974@infradead.org> <20211214205358.701701555@infradead.org> <20211221171900.GA580323@dev-hv> In-Reply-To: From: Peter Oskolkov Date: Wed, 19 Jan 2022 09:52:30 -0800 Message-ID: Subject: Re: [RFC][PATCH 3/3] sched: User Mode Concurency Groups To: Peter Zijlstra Cc: Peter Oskolkov , mingo@redhat.com, tglx@linutronix.de, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-api@vger.kernel.org, x86@kernel.org, pjt@google.com, avagin@google.com, jannh@google.com, tdelisle@uwaterloo.ca Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: 1rqwbz7h5wxxs9ur66ygzs3e6d14o97w Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Qwr4sXWl; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf06.hostedemail.com: domain of posk@google.com designates 209.85.128.44 as permitted sender) smtp.mailfrom=posk@google.com X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: BA3A8180025 X-HE-Tag: 1642614763-523219 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Jan 19, 2022 at 1:00 AM Peter Zijlstra wrote: > > On Tue, Jan 18, 2022 at 10:19:21AM -0800, Peter Oskolkov wrote: > > > =========== signals and the general approach > > > > My version of the patchset has all of these things working. What it > > does not have, > > compared to the new approach we are discussing here, is runqueues per server > > and proper signal handling (and potential integration with proxy execution). > > > > Runqueues per server, in the LAZY mode, are easy to emulate in my patchset: > > nothing prevents the userspace to partition workers among servers, and have > > servers that "own" their workers to be pointed at by idle_server_tid_ptr. > > > > The only thing that is missing is proper treating of signals. But my patchset > > does ensure a single running worker per server, had pagefaults and preemptions > > sorted out, etc. Basically, everything works except signals. This patchet > > has issues with pagefaults, > > Already fixed pagefaults per: > > YeGvovSckivQnKX8@hirez.programming.kicks-ass.net Could you, please, post an updated RFC when you have a chance? Thanks! > > > worker timeouts > > I still have no clear answer as to what you actually want there. > > > , worker-to-worker context > > switches (do workers move runqueues when they context switch?), etc. > > Not in kernel, if they need to be migrated, userspace needs to do that. > > > And my patchset now actually looks smaller and simpler, on the kernel side, > > that what this patchset is shaping up to be. > > > > What if I fix signals in my patchset? I think the way you deal with signals > > will work in my approach equally well; I'll also use umcg_kick() to preempt > > workers instead of sending them a signal. > > > > What do you think? > > I still absolutely hate how long you do page pinning, it *will* wreck > things like CMA which are somewhat latency critical for silly things > like Android camera apps and who knows what else. > > You've also forgotten about this: > > YcWutpu7BDeG+dQ2@hirez.programming.kicks-ass.net > > That's not optional given how you're using page-pinning. Also, I think > we need at least one direct access to the page after getting the pin in > order to make it work. > > That also very much limits it to Anon pages. I can use the same mm/page pinning strategy as you do. But then our patchsets will be quite similar, I guess, with the difference being server wakeups with RUNNING workers vs "lazy" idle_server_tid_ptr. So OK, let's continue with your approach. If you could post a new RFC with the memory/paging fixes in it, I'll then add worker timeouts, as I outlined in a separate email ~ 30min ago, and continue with my integration/testing.