Mococular

[antoinefalisse]

Brainstorm on monocular pipeline:

Questions:

• One scaled model per motion?

Expected hurdles:

• Have a container with tensorflow (for augmenter) and pytorch (monocular), but I'm sure workable

1st approach:

• Frontend: When hitting new session, get option to go single-camera or multi-camera (default) route
  • This passes flag at session level: monocular
• Core:
  • Pose detection: Another loop waiting for monocular jobs (ie, same as OpenPose and HRNet)
  • Add big if statement in main.py to support Monocular code / Have main_monocular.py

[suhlrich]

Core:

• I'm not sure we want core and monocular to be in the same repo. The dependencies in mono are already a mess. We could definitely try, and it would help keep maintenance easier (e.g., adding osim models), but even the python versioning is not the same.

2nd approach

• frontend: same as 1st
• core:
• • another docker for mono (like mmpose/openpose), app.py saves a video in a folder and the mono docker runs it. Pros: easier for dependency conflicts, cons: harder to pass error information, more maintenance of redundant codes

[antoinefalisse]

I think we are talking about the same thing. Most of the moco stuff would happen in the mono docker, so the conflicts would be limited with the OpenCap docker.

[AlbertoCasasOrtiz]

I think what you are proposing should work, so as a summary:

1. In the viewer, select "monocular" or "multicamera".
2. If "monocular" selected, flag the session (maybe in metadata?) or in a new column in the database.
3. The current docker containers should ignore everything flagged as "monocular", as we would have new docker container processing only "monocular" trials.

I am a little disconnected from the monocular functionality so I have a question: Is calibration present in the case of "monocular"? Should we have the option of "callibration" for processing only calibration and neutral in "monocular" mode? Is calibration present in the case of "monocular"?

Regarding code redundancy, I think that is something we should try to address as soon as possible to avoid future issues as happened in the past in the frontend (fixing something on a file and the same error appearing in another file).