The tracee cannot trace the syscall event that the container is started before the tracee.

[caoliwu]

I'm trying to trace the syscall events of the app container using tracee, and I found that:

1.tracee can trace syscall event
（1）start tracee container
（2）start the app container
Tracee can trace the syscall event of the app container. Everything is good.

2.tracee can't trace syscall event
（1）start the app container
（2）start tracee container
Tracee get nothing. The command I executed to start the tracee container is:
docker run --name tracee_0 --rm -it --pid=host --privileged -v /etc/os-release:/etc/os-release-host:ro -e LIBBPFGO_OSRELEASE_FILE=/etc/os-release-host -v /usr/src:/usr/src:ro -v /lib/modules:/lib/modules:ro -v /tmp/tracee:/tmp/tracee:rw -v /opt/clw/:/opt/clw:rw aquasec/tracee:full --filter container --filter set=syscalls --output option:parse-arguments-fds

Anything wrong with my operation?

[geyslan]

@caoliwu thanks for using Tracee.

Based on your comment, it's not clear if you're running the app in the same container of tracee, so that information would be interesting to analyse your results.

For now, consider this filter options /dist/tracee --help filter:

  --filter container=new                                        | only trace events from newly created containers
  --filter container=ab356bc4dd554                              | only trace events from container id ab356bc4dd554
  --filter container                                            | only trace events from containers
  --filter c                                                    | only trace events from containers (same as above)
  --filter '!container'                                         | only trace events from the host

[caoliwu]

@geyslan thanks for your reply.

Tracee is an independent container, and app is also an independent container. App and tracee are different containers.

I run the app container first, then the tracee container. Then I log in to the app container and run cat a.txt. Then I check the output log of the tracee container. I find that the tracee container does not trace the system call for reading a.txt in the app container.

[yanivagman]

Testing locally with the same flags you provided ( --filter container --filter set=syscalls --output option:parse-arguments-fds) and I can't reproduce this issue - containers which were loaded before tracee are being traced and I can see all their syscalls.
The behavior you describe might happen if for some reason the cgroup dir is not mounted correctly in the tracee container, thus tracee can't find already existing containers. A solution to this might be to use --cgroupns=host in the docker command running tracee or simply mounting your cgroup path /sys/fs/cgroup into the tracee container

[geyslan]

A solution to this might be to use --cgroupns=host in the docker command running tracee or simply mounting your cgroup path /sys/fs/cgroup into the tracee container

Indeed.

@caoliwu on my rig I managed to make your test to work as @yanivagman suggested.

1. Run your app container previously.

2. Run tracee docker with --cgroupns=host:

docker run --name tracee_0 --rm -it --cgroupns=host --pid=host --privileged -v /etc/os-release:/etc/os-release-host:ro -e LIBBPFGO_OSRELEASE_FILE=/etc/os-release-host -v /usr/src:/usr/src:ro -v /lib/modules:/lib/modules:ro -v /tmp/tracee:/tmp/tracee:rw -v /opt/clw/:/opt/clw:rw aquasec/tracee:full --filter container --filter comm=cat --output option:parse-arguments-fds

INFO: probing tracee capabilities...
INFO: starting tracee...
TIME             CONTAINER_ID  UID    COMM             PID/host        TID/host        RET              EVENT                     ARGS

3. Run your cat command from your app container and see the asked output in tracee container:

15:22:28:090702  9f79635b6acc  0      cat              7      /31204   7      /31204   0                sched_process_exec        cmdpath: /bin/cat, pathname: /bin/busybox, dev: 128, inode: 3548804, ctime: 1684888941326478707, inode_mode: 33261, interpreter_pathname: <nil>, interpreter_dev: <nil>, interpreter_inode: <nil>, interpreter_ctime: <nil>, argv: [cat /etc/passwd], interp: /bin/cat, stdin_type: S_IFCHR, stdin_path: /dev/pts/0, invoked_from_kernel: 0, env: <nil>
15:22:31:897027                0      systemd          1      /1       1      /1       0                container_remove          runtime: docker, container_id: 9f79635b6acc9ee7ca517b585e5040d54c105d9e6b00c82d22a71d4e06711bf9

[caoliwu]

@yanivagman The minimum requirement for the --cgroupns=host parameter is API 1.41. Currently, the API queried by docker version in my environment is 1.39. For some reason, I can't upgrade the Docker version. Is there any other way to test and verify it?