From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD6BEC433FE for ; Thu, 27 Jan 2022 12:20:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EF3716B0071; Thu, 27 Jan 2022 07:20:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EA3EB6B0072; Thu, 27 Jan 2022 07:20:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D92996B0073; Thu, 27 Jan 2022 07:20:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0166.hostedemail.com [216.40.44.166]) by kanga.kvack.org (Postfix) with ESMTP id C97496B0071 for ; Thu, 27 Jan 2022 07:20:01 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 63CD6181DA094 for ; Thu, 27 Jan 2022 12:20:01 +0000 (UTC) X-FDA: 79075973802.28.373C9A5 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) by imf27.hostedemail.com (Postfix) with ESMTP id 8E68F40005 for ; Thu, 27 Jan 2022 12:20:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=apFD4UwkGbnUlS6QRwwlqJtZYogyGpQVvXHYQdU89mo=; b=RAzvWVT3QoIcd1wSs2pQDzCr8b utwt3vbxz/RrDaxL58pAOfQR/+/GRKD2AzzRN0m54fa3sRpOt09Rm6VNBBVqbSBZm605853qYqtR4 UpyWDOeyi3RoCO4qadAapNaf4sA9e/nB576zo1ISzfWlUxXGyvGN1MaPL8614s0kKSb+e6y0BR4tS GHHU7Zx/O8rahpO3JDeJKFxaxc3IQMsA40kh64t7gay/9MGGba9oMddpbO4dy2/+vxMHBYd9fermu cpdlp69uG9ajzJpl5t/GrUVFf0VbK4x5fXiu653qop0byW2pNb6jDg5WhrH26JV355L9HrSmKBwJy EwjqrwaA==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nD3kf-004A8L-FF; Thu, 27 Jan 2022 12:19:45 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 6E101300268; Thu, 27 Jan 2022 13:19:43 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 1000) id 4608A2B520929; Thu, 27 Jan 2022 13:19:43 +0100 (CET) Date: Thu, 27 Jan 2022 13:19:43 +0100 From: Peter Zijlstra To: Tao Zhou Cc: mingo@redhat.com, tglx@linutronix.de, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-api@vger.kernel.org, x86@kernel.org, pjt@google.com, posk@google.com, avagin@google.com, jannh@google.com, tdelisle@uwaterloo.ca, mark.rutland@arm.com, posk@posk.io Subject: Re: [RFC][PATCH v2 5/5] sched: User Mode Concurency Groups Message-ID: References: <20220120155517.066795336@infradead.org> <20220120160822.914418096@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Stat-Signature: rkgay5taojfro348nhg1rt5tn6ym6i8e X-Rspam-User: nil Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=RAzvWVT3; spf=none (imf27.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org; dmarc=none X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 8E68F40005 X-HE-Tag: 1643286000-355294 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Jan 24, 2022 at 10:46:17PM +0800, Tao Zhou wrote: > Hi Peter, > > On Thu, Jan 20, 2022 at 04:55:22PM +0100, Peter Zijlstra wrote: > > [...] > > > +/* pre-schedule() */ > > +void umcg_wq_worker_sleeping(struct task_struct *tsk) > > +{ > > + struct umcg_task __user *self = READ_ONCE(tsk->umcg_task); > > + int ret; > > + > > + if (!tsk->umcg_server) { > > + /* > > + * Already blocked before, the pages are unpinned. > > + */ > > + return; > > + } > > + > > + /* Must not fault, mmap_sem might be held. */ > > + pagefault_disable(); > > + > > + ret = umcg_update_state(tsk, self, UMCG_TASK_RUNNING, UMCG_TASK_BLOCKED); > > + if (ret == -EAGAIN) { > > + /* > > + * Consider: > > + * > > + * self->state = UMCG_TASK_RUNNABLE | UMCG_TF_COND_WAIT; > > + * ... > > + * sys_umcg_wait(); > > + * > > + * and the '...' code doing a blocking syscall/fault. This > > + * ensures that returns with UMCG_TASK_RUNNING, which will make > > /UMCG_TASK_RUNNING/UMCG_TASK_RUNNABLE/ So the issue is that: self->state = UMCG_TASK_RUNNABLE | UMCG_TF_COND_WAIT; <#PF> umcg_sys_enter() umcg_pin_user_page() schedule() sched_submit_work() umcg_wq_worker_sleeping() umcg_update_state(tsk, self, UMCG_TASK_RUNNING, UMCG_TASK_BLOCKED) // -EAGAIN UMCG_DIE() Which is clearly not desirable. So this additinoal thing ensures that: umcg_update_state(tsk, self, UMCG_TASK_RUNNABLE, UMCG_TASK_BLOCKED) // 0 umcg_sys_exit() umcg_update_state(tsk, self, UMCG_TASK_BLOCKED, UMCG_TASK_RUNNABLE); umcg_enqueue_and_wake() umcg_notify_resume() umcg_wait() // must be UMCG_TASK_RUNNING here So when the pagefault finally does return, it will have: UMCG_TASK_RUNNING. Which will then make sys_umcg_wait() return -EAGAIN and around we go. > > + * sys_umcg_wait() return with -EAGAIN. > > + */ > > + ret = umcg_update_state(tsk, self, UMCG_TASK_RUNNABLE, UMCG_TASK_BLOCKED); > > + } > > + if (ret) > > + UMCG_DIE_PF("state"); > > + > > + if (umcg_wake_server(tsk)) > > + UMCG_DIE_PF("wake"); > > + > > + pagefault_enable(); > > + > > + /* > > + * We're going to sleep, make sure to unpin the pages, this ensures > > + * the pins are temporary. Also see umcg_sys_exit(). > > + */ > > + umcg_unpin_pages(); > > +}