From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f71.google.com (mail-oi0-f71.google.com [209.85.218.71]) by kanga.kvack.org (Postfix) with ESMTP id EFF3A6B0253 for ; Thu, 8 Dec 2016 07:58:46 -0500 (EST) Received: by mail-oi0-f71.google.com with SMTP id n184so715808359oig.1 for ; Thu, 08 Dec 2016 04:58:46 -0800 (PST) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01on0080.outbound.protection.outlook.com. [104.47.1.80]) by mx.google.com with ESMTPS id m20si14235092otd.277.2016.12.08.04.58.44 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 08 Dec 2016 04:58:45 -0800 (PST) Received: by mail-wm0-f46.google.com with SMTP id g23so216119249wme.1 for ; Thu, 08 Dec 2016 04:58:42 -0800 (PST) MIME-Version: 1.0 In-Reply-To: References: <52a0d9c3-5c9a-d207-4cbc-a6df27ba6a9c@suse.cz> From: Federico Reghenzani Date: Thu, 8 Dec 2016 13:58:19 +0100 Message-ID: Subject: Re: mlockall() with pid parameter Content-Type: multipart/alternative; boundary="001a114d46b8535454054325329c" Sender: owner-linux-mm@kvack.org List-ID: To: Vlastimil Babka , kirill@shutemov.name Cc: linux-mm@kvack.org --001a114d46b8535454054325329c Content-Type: text/plain; charset="UTF-8" Ok, these solutions are feasible but not very comfortable. I'll explain better what I'm going to do. I'm a developer of Barbeque Open Source Project that is run-time resource manager. It is basically composed of a daemon (barbeque) and a library (rtlib) linked with user applications. A user starts a process linked with rtlib that exchanges some information with Barbeque (e.g. it requests a performance goal). Barbeque is in charge of the assignment of resources trying to maintain the performance goals of all applications and predefined system requirements (e.g. temperatures and power consumption). When processes start, Barbeque tunes several parameters at run-time: create and set CGroups, select cpu governors and frequency, etc. In the case of a real-time process it decides the scheduling policy, the scheduling parameters, etc. Barbeque runs with root privileges, thus it has the CAP_SYS_NICE capability to enforce a RT scheduling policy on applications. The idea is to give to Barbeque the possibility to dinamically select if enforcing mlockall() or not for RT tasks, according to the available memory resources. I can do this using a sort of synchronization mechanism: Barbeque sets limits of the process and signal the rtlib to execute the mlockall() or the munlockall(), but I think it would be better to have a syscall that Barbeque can call directly without interfering with process execution. Yesterday I rapidly read the code of mlockall() and relative functions and I think that in order to add a pid parameter is maybe sufficient to convert the pid into a task struct and replace `current` with it. Probably, it will not be so easy. Tomorrow I'm going to read the code more in details and check if the implementation is actually easy and does not involve too much refactoring in the present code. Thank you, Federico 2016-12-07 21:01 GMT+01:00 Vlastimil Babka : > On 12/07/2016 05:33 PM, Federico Reghenzani wrote: > > > > > > 2016-12-07 17:21 GMT+01:00 Vlastimil Babka > >: > > > > On 12/07/2016 04:39 PM, Federico Reghenzani wrote: > > > Hello, > > > > > > I'm working on Real-Time applications in Linux. `mlockall()` is a > > > typical syscall used in RT processes in order to avoid page faults. > > > However, the use of this syscall is strongly limited by ulimits, so > > > basically all RT processes that want to call `mlockall()` have to > be > > > executed with root privileges. > > > > Is it not possible to change the ulimits with e.g. prlimit? > > > > > > Yes, but it requires a synchronization between non-root process and root > > process. > > Because the root process has to change the limits before the non-root > > process executes the mlockall(). > > Would it work if you did that between fork() and exec()? If you can > spawn them like this, that is. > > > Just to provide an example, another syscall used in RT tasks is the > > sched_setscheduler() that also suffers > > the limitation of ulimits, but it accepts the pid so the scheduling > > policy can be enforced by a root process to > > any other process. > > > > > > > > > What I would like to have is a syscall that accept a "pid", so a > process > > > spawned by root would be able to enforce the memory locking to > other > > > non-root processes. The prototypes would be: > > > > > > int mlockall(int flags, pid_t pid); > > > int munlockall(pid_t pid); > > > > > > I checked the source code and it seems to me quite easy to add this > > > syscall variant. > > > > > > I'm writing here to have a feedback before starting to edit the > code. Do > > > you think that this is a good approach? > > > > > > > > > Thank you, > > > Federico > > > > > > -- > > > *Federico Reghenzani* > > > PhD Candidate > > > Politecnico di Milano > > > Dipartimento di Elettronica, Informazione e Bioingegneria > > > > > > > > > > > > > -- > > *Federico Reghenzani* > > PhD Candidate > > Politecnico di Milano > > Dipartimento di Elettronica, Informazione e Bioingegneria > > > > -- > To unsubscribe, send a message with 'unsubscribe linux-mm' in > the body to majordomo@kvack.org. For more info on Linux MM, > see: http://www.linux-mm.org/ . > Don't email: email@kvack.org > -- *Federico Reghenzani* PhD Candidate Politecnico di Milano Dipartimento di Elettronica, Informazione e Bioingegneria --001a114d46b8535454054325329c Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Barbe= que runs with root privileges, thus it has the CAP_SYS_NICE capability
to enforce a RT scheduling policy on applications.

The idea is to give to Barbeque the possibility to dinamically selec= t if
enforcing mlockall() or not for RT tasks, according to the a= vailable memory
resources. I can do this using a sort of synchron= ization mechanism: Barbeque
sets limits of the process and signal= the rtlib to execute the mlockall() or the
munlockall(), but I t= hink it would be better to have a syscall that Barbeque can
call = directly without interfering with process execution.

Yesterday I rapidly read the code of mlockall() and relative functions a= nd I
think that in order to add a pid parameter is maybe sufficie= nt to convert the
pid into a task struct and replace `current` wi= th it. Probably, it will not=C2=A0
be so easy. Tomorrow I'm g= oing to read the code more in details and check if
the implementa= tion is actually easy and does not involve too much refactoring in
the present code.


Thank you,
Federico

2016-12-07 21:01 GMT+01:00 Vlastimil Babka <vbabka@suse.cz>:=
On 12/07/2016 05:33 PM,= Federico Reghenzani wrote:
>
>
> 2016-12-07 17:21 GMT+01:00 Vlastimil Babka <vbabka@suse.cz
> <mailto:vbabka@suse.cz= >>:
>
>=C2=A0 =C2=A0 =C2=A0On 12/07/2016 04:39 PM, Federico Reghenzani wrote:<= br> >=C2=A0 =C2=A0 =C2=A0> Hello,
>=C2=A0 =C2=A0 =C2=A0>
>=C2=A0 =C2=A0 =C2=A0> I'm working on Real-Time applications in L= inux. `mlockall()` is a
>=C2=A0 =C2=A0 =C2=A0> typical syscall used in RT processes in order = to avoid page faults.
>=C2=A0 =C2=A0 =C2=A0> However, the use of this syscall is strongly l= imited by ulimits, so
>=C2=A0 =C2=A0 =C2=A0> basically all RT processes that want to call `= mlockall()` have to be
>=C2=A0 =C2=A0 =C2=A0> executed with root privileges.
>
>=C2=A0 =C2=A0 =C2=A0Is it not possible to change the ulimits with e.g. = prlimit?
>
>
> Yes, but it requires a synchronization between non-root process and ro= ot
> process.
> Because the root process has to change the limits before the non-root<= br> > process executes the mlockall().

Would it work if you did that between fork() and exec()? If you can<= br> spawn them like this, that is.

> Just to provide an example, another syscall used in RT tasks is the > sched_setscheduler() that also suffers
> the limitation of ulimits, but it accepts the pid so the scheduling > policy can be enforced by a root process to
> any other process.
>
>
>
>=C2=A0 =C2=A0 =C2=A0> What I would like to have is a syscall that ac= cept a "pid", so a process
>=C2=A0 =C2=A0 =C2=A0> spawned by root would be able to enforce the m= emory locking to other
>=C2=A0 =C2=A0 =C2=A0> non-root processes. The prototypes would be: >=C2=A0 =C2=A0 =C2=A0>
>=C2=A0 =C2=A0 =C2=A0> int mlockall(int flags, pid_t pid);
>=C2=A0 =C2=A0 =C2=A0> int munlockall(pid_t pid);
>=C2=A0 =C2=A0 =C2=A0>
>=C2=A0 =C2=A0 =C2=A0> I checked the source code and it seems to me q= uite easy to add this
>=C2=A0 =C2=A0 =C2=A0> syscall variant.
>=C2=A0 =C2=A0 =C2=A0>
>=C2=A0 =C2=A0 =C2=A0> I'm writing here to have a feedback before= starting to edit the code. Do
>=C2=A0 =C2=A0 =C2=A0> you think that this is a good approach?
>=C2=A0 =C2=A0 =C2=A0>
>=C2=A0 =C2=A0 =C2=A0>
>=C2=A0 =C2=A0 =C2=A0> Thank you,
>=C2=A0 =C2=A0 =C2=A0> Federico
>=C2=A0 =C2=A0 =C2=A0>
>=C2=A0 =C2=A0 =C2=A0> --
>=C2=A0 =C2=A0 =C2=A0> *Federico Reghenzani*
>=C2=A0 =C2=A0 =C2=A0> PhD Candidate
>=C2=A0 =C2=A0 =C2=A0> Politecnico di Milano
>=C2=A0 =C2=A0 =C2=A0> Dipartimento di Elettronica, Informazione e Bi= oingegneria
>=C2=A0 =C2=A0 =C2=A0>
>
>
>
>
> --
> *Federico Reghenzani*
> PhD Candidate
> Politecnico di Milano
> Dipartimento di Elettronica, Informazione e Bioingegneria
>

--
To unsubscribe, send a = message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.= =C2=A0 For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=3Dmailto:"dont@kvack.org"> email@kva= ck.org </a>



--
=
Federico Reghenzani
PhD Candidate
Politecnico di Milano
Dipartimento di Elettronica, Informazione e Bioingegneria=

--001a114d46b8535454054325329c-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org