How to raise ulimit hard limit for real time priority programmatically with setuid or capability CAP_SYS_RESOURCE? - ulimit

Question

I would like to run a program under the linux SCHED_FIFO real-time class. I would prefer to keep the user's hard limit for RTPRIO set to 0, and to programmatically raise the hard limit just for the single process. It is broadly claimed that if I grant the process CAP_SYS_RESOURCE to allow it raise the hard limit, e.g. , man setrlimit 2:
The soft limit is the value that the kernel enforces for the corresponding resource. The hard limit acts as a ceiling for the soft limit: an unprivileged process may only set its soft limit to a value in the range from 0 up to the hard limit, and (irreversibly) lower its hard limit. A privileged process (under Linux: one with the CAP_SYS_RESOURCE capability) may make arbitrary changes to either limit value.
However, I can't seem to get this to work for me. Here is test code:
#include <stdio.h>
#include <sched.h>
#include <errno.h>
#include <string.h>
#include <sys/resource.h>
#define PRIORITY (50)
int main(int argc, char **argv) {
struct sched_param param;
struct rlimit rl;
int e, min_fifo, max_fifo;
min_fifo = sched_get_priority_min(SCHED_FIFO);
max_fifo = sched_get_priority_max(SCHED_FIFO);
printf("For policy SCHED_FIFO min priority is %d, max is %d.\n",
min_fifo, max_fifo);
if ((min_fifo>PRIORITY)||(max_fifo<PRIORITY)) {
printf("Desired priority of %d is out of range.\n", PRIORITY);
return 1;
}
if (getrlimit(RLIMIT_RTPRIO, &rl) != 0) {
e = errno;
printf("Failed to getrlimit(): %s.\n", strerror(e));
return 1;
}
printf("RTPRIO soft limit is %d, hard is %d.\n",
(int) rl.rlim_cur, (int) rl.rlim_max);
// Adjust hard limit if necessary
if (rl.rlim_max < PRIORITY) {
rl.rlim_max = PRIORITY;
if (setrlimit(RLIMIT_RTPRIO, &rl) != 0) {
e = errno;
printf("Failed to raise hard limit for RTPRIO to %d: %s.\n",
(int) rl.rlim_max, strerror(e));
return 1;
}
printf("Raised hard limit for RTPRIO to %d.\n", (int) rl.rlim_max);
}
// Adjust soft limit if necessary
if (rl.rlim_cur < PRIORITY) {
rl.rlim_cur = PRIORITY;
if (setrlimit(RLIMIT_RTPRIO, &rl) != 0) {
e = errno;
printf("Failed to raise soft limit for RTPRIO to %d: %s.\n",
(int) rl.rlim_cur, strerror(e));
return 1;
}
printf("Raised soft limit for RTPRIO to %d.\n", (int) rl.rlim_cur);
}
// Set desired priority with class SCHED_FIFO
param.sched_priority = PRIORITY;
if (sched_setscheduler(0, SCHED_FIFO, &param) != 0) {
e = errno;
printf("Setting policy failed: %s.\n", strerror(e));
return 1;
} else {
printf("Set policy SCHED_FIFO, priority %d.\n", param.sched_priority);
}
return 0;
}
This works as expected without special privilege with a hard limit of 99:
$ ./rtprio
For policy SCHED_FIFO min priority is 1, max is 99.
RTPRIO soft limit is 0, hard is 99.
Raised soft limit for RTPRIO to 50.
Set policy SCHED_FIFO, priority 50.
$
It works as expected with hard limit of 0 using sudo:
$ sudo ./rtprio
For policy SCHED_FIFO min priority is 1, max is 99.
RTPRIO soft limit is 0, hard is 0.
Raised hard limit for RTPRIO to 50.
Raised soft limit for RTPRIO to 50.
Set policy SCHED_FIFO, priority 50.
$
However it does not work as expected when setuid root:
$ sudo chown root ./rtprio
$ sudo chgrp root ./rtprio
$ sudo chmod ug+s ./rtprio
$ ls -l ./rtprio
-rwsrwsr-x 1 root root 8948 11月 28 12:04 ./rtprio
$ ./rtprio
For policy SCHED_FIFO min priority is 1, max is 99.
RTPRIO soft limit is 0, hard is 0.
Failed to raise hard limit for RTPRIO to 50: Operation not permitted.
It also unexpectedly fails with capability CAP_SYS_RESOURCE as well as with all capabilities:
$ sudo setcap cap_sys_resource=eip ./rtprio
$ getcap ./rtprio
./rtprio = cap_sys_resource+eip
$ ./rtprio
For policy SCHED_FIFO min priority is 1, max is 99.
RTPRIO soft limit is 0, hard is 0.
Failed to raise hard limit for RTPRIO to 50: Operation not permitted.
$ sudo setcap all=eip ./rtprio
$ getcap ./rtprio
./rtprio =eip
$ ./rtprio
For policy SCHED_FIFO min priority is 1, max is 99.
RTPRIO soft limit is 0, hard is 0.
Failed to raise hard limit for RTPRIO to 50: Operation not permitted.
What am I missing here?
$ uname -srv
Linux 3.13.0-100-generic #147-Ubuntu SMP Tue Oct 18 16:48:51 UTC 2016
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.5 LTS
Release: 14.04
Codename: trusty
$ bash --version | head -1
GNU bash, version 4.3.11(1)-release (x86_64-pc-linux-gnu)

Answer 1

The fact that setuid root didn't work is the clue.
It turns out that the test program above was in a partition mounted withnosuid, and hence the setuid bits have no effect. If you don't trust the setuid bits in the partition, then you probably shouldn't trust the file capabilities either. And, indeed, it turns out that when mounting with nosuid the file capabilities are also ignored.
It seems that luks encrypted home directories tend to be mounted nosuid.
I'm leaving this question up because there are a lot of search engine hits for "linux capabilities nosuid" indicating that a lot of time has been wasted on this issue (but of course you don't know to search for this until you have figured it out).