Skip to content

On gaze deployment to audio-visual cues of social interactions

Notifications You must be signed in to change notification settings

phuselab/GazeDeploy

Folders and files

NameName
Last commit message
Last commit date

Latest commit

a1e5efe · Mar 10, 2021

History

15 Commits
Aug 31, 2020
May 25, 2020
Aug 31, 2020
Aug 31, 2020
Aug 31, 2020
Mar 10, 2021
Aug 31, 2020
Mar 10, 2021
May 25, 2020
Aug 31, 2020
May 25, 2020
May 25, 2020
Aug 31, 2020
May 25, 2020
Feb 12, 2021
Aug 31, 2020
May 25, 2020
May 25, 2020

Repository files navigation

On gaze deployment to audio-visual cues of social interactions

Giuseppe Boccignone, Vittorio Cuculo¹, Alessandro D'Amelio¹, Giuliano Grossi¹, Raffaella Lanzarotti¹
¹ PHuSe Lab - Dipartimento di Informatica, Università degli Studi di Milano

Paper Boccignone, G., Cuculo, V., D’Amelio, A., Grossi, G., & Lanzarotti, R. (2020). On gaze deployment to audio-visual cues of social interactions. IEEE Access, 1–25.

https://ieeexplore.ieee.org/document/9184838

simulation

Requirements

python 3.6
pip install -r requirements.txt

Executing the demo

To simulate from the model:

  1. Do speaker identification and build face maps:
#sh build_face_maps.sh path/to/video vidName path/to/output
sh build_face_maps.sh data/videos/012.mp4 012 speaker_detect/output/
  1. Run the follwing command (it is assumed that low-level saliency maps (see Credits) are already computed, if you want to compute it on your own, you may want to use something like this)
python3 Simulate_model.py

Credits

  • Data: FindWhoToLookAt --->Repo, Paper

  • Speaker Detection Pipeline: SyncNet ---> Repo, Paper

  • Space-time Visual Saliency Detection: Code, Paper

Reference

If you use this code or data, please cite the paper:

@article{Boccignone2020,
  doi = {10.1109/access.2020.3021211},
  url = {https://doi.org/10.1109/access.2020.3021211},
  year = {2020},
  publisher = {Institute of Electrical and Electronics Engineers ({IEEE})},
  pages = {1--25},
  author = {Giuseppe Boccignone and Vittorio Cuculo and Alessandro D{\textquotesingle}Amelio and Giuliano Grossi and Raffaella Lanzarotti},
  title = {On gaze deployment to audio-visual cues of social interactions},
  journal = {{IEEE} Access}
}

About

On gaze deployment to audio-visual cues of social interactions

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published