Unable to set affinity for a thread scheduled by shed-deadline

[z-linlinlin]

Since affinity setting is currently disabled for tasks handled by sched-deadline, how can we restrict threads to certain cpus but using the policy sched-deadline ?

[z-linlinlin]

Is it possible to reset the "cpus_allowed" for every thread ( in the code of "cpus" ) ?

[vingu-linaro]

hi z-linlinlin,

As you mentioned above, we can't restrict cpus allowed for deadline scheduler so it's up to the writer of the json file use case to ensure that this will not happen.
as default cpumask is all cpus and we can set affinity per phase with a fall back to this default mask when not mentioned, you should be able to ensure that all cpus are allowed when setting deadline scheduling mode

[credp]

IIUC, deadline performs the admittance test only once when a task is changed to deadline class, which means we can't be changing affinity after thread creation. However I think it should be possible to use cpusets to restrict deadline tasks so long as you don't go changing them after creation - but you would need to add better cgroup support to rt-app first. See #49

From the deadline docs in-kernel:

5. Tasks CPU affinity
   =====================

-deadline tasks cannot have an affinity mask smaller that the entire
root_domain they are created on. However, affinities can be specified
through the cpuset facility (Documentation/cgroup-v1/cpusets.txt).

5.1 SCHED_DEADLINE and cpusets HOWTO

An example of a simple configuration (pin a -deadline task to CPU0)
follows (rt-app is used to create a -deadline task).

mkdir /dev/cpuset
mount -t cgroup -o cpuset cpuset /dev/cpuset
cd /dev/cpuset
mkdir cpu0
echo 0 > cpu0/cpuset.cpus
echo 0 > cpu0/cpuset.mems
echo 1 > cpuset.cpu_exclusive
echo 0 > cpuset.sched_load_balance
echo 1 > cpu0/cpuset.cpu_exclusive
echo 1 > cpu0/cpuset.mem_exclusive
echo $$ > cpu0/tasks
rt-app -t 100000:10000:d:0 -D5 (it is now actually superfluous to specify
task affinity)

[jlelli]

performs the admittance test only once when a task is changed to deadline class

Also when parameters are changed. And when cpuset masks change as well (e.g., a check is performed to see if shrinking a cpumask down still leaves enough bandwidth for already admitted tasks).

You are right that the only interface we have at the moment to deal with affinities is using cpusets.

Admission control (and affinity checks) can however be disabled by writing -1 in sched_rt_runtime_us.

[thymbahutymba]

I'm trying to run a deadline taskset into a cpuset made with 2 cores. The taskset is the following:

{
  "tasks": {
    "task4": {
      "runtime": 47000,
      "timer": {
        "period": 1000000,
        "mode": "absolute",
        "ref": "unique"
      }
    },
    "task3": {
      "runtime": 57000,
      "timer": {
        "period": 1000000,
        "mode": "absolute",
        "ref": "unique"
      }
    },
    "task2": {
      "runtime": 26000,
      "timer": {
        "period": 1000000,
        "mode": "absolute",
        "ref": "unique"
      }
    },
    "task0": {
      "runtime": 411000,
      "timer": {
        "period": 1000000,
        "mode": "absolute",
        "ref": "unique"
      }
    },
    "task1": {
      "runtime": 208000,
      "timer": {
        "period": 1000000,
        "mode": "absolute",
        "ref": "unique"
      }
    }
  },
  "global": {
    "duration": 2,
    "default_policy": "SCHED_DEADLINE",
    "pi_enabled": false,
    "lock_pages": false,
    "logdir": "./rtapp-log",
    "log_basename": "rt-app",
    "log_mode": "file",
    "gnuplot": false
  }
}

I create the cpuset as said before by @credp, the only difference is echo 2-3 > mycpuset/cpuset.cpus. Once I try to execute the taskset with ./rt-app Config.json i get the following error:

[rt-app] <crit> [0] sched_setattr returned -1
sched_setattr: failed to set deadline attributes: Device or resource busy

The same taskset into a different terminal works as expected. What am I doing wrong?

EDIT:
I just find out something strange. In short it works with N tasks with N <= cpus in /dev/cpuset/mycpuset/cpuset.cpus.
So if there is only one cpu in the cpuset it does not work. If I use, for example, 2-3 cpus in cpuset.cpus the first -deadline task starts correctly and I get sched_setattr: failed to set deadline attributes: Device or resource busy for all left tasks. Finally if I use 0-2 as cpus for the cpuset I'm able to start the first and the second tasks but getting the error mentioned above for all other tasks.

[jlelli]

So, is this fixed/not a bug as per discussion happening at #101 ?

[thymbahutymba]

Yes, I think so.