Unifi-Protect-Camera-Motion

This Homebridge plugin allows you to add your Unifi Protect Cameras (and their Motion Sensors) to Homekit. It adds smart detection by using a machine learning model to detect specific classes of objects in the camera view.

How it Works

This plugin will automatically discover all the Unifi cameras from your Protect installation, and provide the following sensors for each one it finds:

• Camera, for viewing live RTSP streams
• Motion sensor, for sending push-notifications when motion or one of the desired objects have been detected
• A Switch, for easily enabling and disabling motion detection (on by default and after a Homebridge restart)
• A Switch, to trigger a motion event manually, forcing a rich notification
• (if enabled) A switch, that acts as a doorbell trigger, to manually trigger a rich doorbell notification

Motion Events and object detection

The plugin uses the Unifi Protect API to get motion events on a per camera basis. When motion has been detected one of the two methods below will be used to generate a motion notification in Homekit:

• The basic method: The "score" of the Unifi Protect motion event. (Which currently has a bug and is 0 as long as the motion is ongoing.)
• The advanced method: Object detection by use of yolov5. (recommended) This logic/model runs on-device, and no data will be sent to any online/external/cloud source or service. It is based on the coco ssd project.

Installation

Before installing this plugin, please make sure all the prerequisites have been met first. Consult the readme and the wiki before proceeding.

In short, the main dependencies are:

General:

• Python 3 and pip3 must be installed and on the path!

• Raspberry Pi / Ubuntu / Debian Linux:

  • Install: sudo apt-get install build-essential libcairo2-dev libpango1.0-dev libjpeg-dev libgif-dev librsvg2-dev
  • Install libgl: sudo apt install libgl1

• Mac OS:

  • Install via Homebrew: brew install pkg-config cairo pango libpng jpeg giflib librsvg

• Linux:

  • Install g++: sudo apt install g++
  • Install libgl: sudo apt install libgl1

• Other OSes:

  • See the node-canvas documentation
  • See the node-gyp documentation => install any dependencies for your OS!

Next, to install this plugin simply type:

sudo npm install homebridge-unifi-protect-camera-motion -g --unsafe-perm=true

Next, open the config.json that contains your Homebridge configuration, and add a block like the following one to the platforms array:

{  
    "platform": "UnifiProtectMotion", 
    "name": "Unifi protect cameras & motion sensors", 
    "unifi": { 
        "controller": "https://protect-ip:controller-ui-port", 
        "controller_rtsp": "rtsp://protect-ip:controller-rtsp-port", 
        "username": "username", 
        "password": "password", 
        "excluded_cameras": [
            "id-of-camera-to-exclude-1",
            "id-of-camera-to-exclude-2"
        ],
        "motion_interval": 5000, 
        "motion_repeat_interval": 30000, 
        "motion_score": 0, 
        "enhanced_motion": true, 
        "enhanced_motion_score": 50, 
        "enhanced_classes": ["person"], 
        "enable_motion_trigger": true,
        "enable_doorbell_for": [
            "id-of-camera-to-act-as-doorbell-1",
            "id-of-camera-to-act-as-doorbell-1"
        ],
        "save_snapshot": true,
        "debug": false, 
        "debug_network_traffic": false,
    },
    "upload_gphotos": false,
    "googlePhotos": {
        "auth_clientId": "CLIENT-ID",
        "auth_clientSecret": "CLIENT-SECRET",
        "auth_redirectUrl": "http://localhost:8888/oauth2-callback"
    },
    "mqtt_enabled": false,
    "mqtt": {
        "broker": "mqtt://broker-ip",
        "username": "MQTT-BROKER-USERNAME",
        "password": "MQTT-BROKER-PASSWORD",
        "topicPrefix": "motion/cameras"
    }
}  

You can verify the correctness of your config file by using jsonlint. The config must be valid or Homebridge will fail to restart correctly. If you are using Homebridge Config X, it will do its best to alert you to any syntax errors it finds.

General configuration fields

Field	Type	Required	Default value	Description
platform	string	yes	/	UnifiProtectMotion
name	string	yes	/	Name of the plugin that shows up in the Homebridge logs
unifi	object	yes	/	Wrapper object containing the unifi configuration
upload_gphotos	boolean	no	false	Contains a boolean indicating whether or not to upload each detection to a google photos album. When using enhanced mode the image is annotated with the class/score that was detected.
mqtt_enabled	boolean	no	false	Set this to true to enable MQTT support. Additional configuration required!
videoProcessor	string	no	ffmpeg	Contains the path to an custom FFmpeg binary

Unifi configuration fields

Field	Type	Required	Default value	Description
controller	string	yes	/	Contains the URL to the CloudKey or UDM with UnifiOS, or as legacy the URL to the Unifi Protect web UI, including port (no / or /protect/ at the end!)
controller_rtsp	string	yes	/	Contains the base URL to be used for playing back the RTSP streams, as seen in the RTSP configuration (no / at the end)
username	string	yes	/	Contains the username that is used to login in the web UI
password	string	yes	/	Contains the password that is used to login in the web UI
excluded_cameras	no	string[]	[]	An array that contains the IDs of the cameras which should be excluded from the enumeration in Homekit, all available IDs are printed during startup
motion_interval	number	yes	/	Contains the interval in milliseconds used to check for motion, a good default is 5000 milliseconds
motion_repeat_interval	number	no	/	Contains the repeat interval in milliseconds during which new motion events will not be triggered if they belong to the same ongoing motion, a good default is 30000 to 60000 milliseconds. This will prevent a bunch of notifications for events which are longer than the motion_interval! Omit this field to disable this functionality
motion_score	number	yes	/	Contains the minimum score in % that a motion event has to have to be processed, a good default is 50%, set this to 0 when using enhanced motion!
enhanced_motion	boolean	yes	/	Enables or disables the enhanced motion & object detection detection with Tensorflow
enhanced_motion_score	number	sometimes	/	This field is required if the enhanced_motion field is set to true and contains the minimum score/certainty in % the enhanced detection should reach before allowing an motion event to be triggered
enhanced_classes	string[]	sometimes	[]	This field is required if the enhanced_motion field is set to true and contains an array of classes (in lowercase) of objects to dispatch motion events for. The array should not be empty when using the enhanced detection! Look in look in src/coco/classes.ts for all available classes
enable_motion_trigger	boolean	no	false	Contains a boolean that when set to true will enable an extra button for each camera to manually trigger a motion notification
enable_doorbell_for	string[]	no	[]	Contains the id of the cameras for which the doorbell functionality should be enabled, all available IDs are printed during startup
save_snapshot	boolean	no	false	Contains a boolean indicating whether or not to save each detection to a jpg file in the .homebridge directory. When using enhanced mode the image is annotated with the class/score that was detected.
debug	boolean	no	false	Contains a boolean indicating whether or not to enable debug logging for the plugin and FFmpeg
debug_network_traffic	boolean	no	false	Contains a boolean indication whether or not to enable logging of all network requests

Google Photos configuration

Field	Type	Required	Default value	Description
auth_clientId	string	sometimes	/	This field is required when the upload_gphotos is set to true. Fill in the Client ID you generated for OAuth2 authentication
auth_clientSecret	string	sometimes	/	This field is required when the upload_gphotos is set to true. Fill in the Client Secret you generated for OAuth2 authentication
auth_redirectUrl	string	sometimes	/	Fill in 'http://localhost:8888/oauth2-callback' as a default, if you change this value to something else, also change it when creating the OAuth2 credentials! The port should always be 8888!

To enable the upload to Google Photos functionality please read the relevant wiki article

MQTT configuration

Field	Type	Required	Default value	Description
broker	string	no	/	This field is required when the enabled field is set to true. Fill in your MQTT broker url, without the port
username	string	no	/	This field contains the username for the MQTT broker connection, if any
password	string	no	/	This field contains the password for the MQTT broker connection, if any
topicPrefix	string	no	/	This field contains the optional topic prefix. Each motion event will be dispatched under topicPrefix/cameraName

Camera configuration

• Make sure each of your Unifi cameras has at least one RTSP stream enabled. However, I suggest enabling all available qualities for the best user experience as the plugin will choose the most appropriate one based on the request coming from Homekit.
  • To enable an RTSP stream: Login on the Protect web UI and go the settings of the camera and open the 'manage' tab
    Make sure all your cameras have the same port for the RTSP stream!
    For optimal results it is best to assign a static ip to your cameras
    

How to add the cameras to your Homekit setup

As per 0.4.1 the enumerated cameras and accompanying switches/triggers will show up automatically, You don't need to add them in manually anymore! If you add your Homebridge instance to the Home app the cameras will automatically be there.

Upgrade notice

If you are upgrading from a pre 0.4.1 the cameras you previously had in the Home app will no longer work and will have to be removed! Tap on a camera preview to open the camera feed, click the settings icon and scroll all the way to the bottom, there select Remove camera from home.

How to enable rich notifications (with image preview)

• Go to the settings of the camera view in the Home app
• Each camera has an accompanying motion sensor
• Enable notifications for the camera
• Whenever motion has been detected you will get a notification from the home app with a snapshot from the camera

Tested with

• Ubuntu VM with node 16
• UDM Pro with Protect 2.0.x
  
  
• 2x Ubiquiti UniFi Video UVC-G3-AF - PoE Camera
  
  
  
• 2x Ubiquiti Unifi Video UVC-G3-Flex - PoE Camera
  
  
  

Limitations, known issues & TODOs

Limitations

• Running this plugin on CPUs that do not support AVX (Celerons in NAS systems, ...) is not supported because there are no prebuilt Tensorflow binaries. Compiling Tensorflow from scratch is out of scope for this project!
  • Run it on a Raspberry Pi or machine with macOS / Windows / Linux (Debian based)
• Unifi Protect has a snapshot saved for every event, and there is an API to get these (with Width & Height), but the actual saved image is pretty low res and is scaled up to 1080p. Using the Anonymous snapshot actually gets a full resolution snapshot which is better for object detection.
• There is no way to know what motion zone (from Unifi) a motion has occurred in. This information is not present is the response from their API.
• The enhanced object detection using CoCo is not perfect and might not catch all the thing you want it to. It should do fine in about 95% of cases though.

TODOs

• Add more unit and integration tests (Ongoing)
• Add support for MQTT (Ongoing)
• Upgrade tfjs-node, now held back because newer versions (Upgrade to 2.x.x in future release) (Done)
• Implement required changes to make this work with Unifi OS
• Figure out how to get higher res streams on iPhone (only iPad seems to request 720p streams) (Done)
• Extend documentation & wiki (Done)
• Add support for two-way audio (Done)

Plugin development

• Checkout the git repo
• Run npm install in the project root folder
• Create a dummy config.json file under resources/test-config/
• use npm run watch to automatically watch for changes and restart Homebridge if needed, you can also add a remote debugger on port 4444 to debug the code.
• use npm run homebridge to start a Homebridge instance that points to the local config that does not auto-reload when changes are saved.

Credits

A big thanks to the developers of Homebridge Camera FFmpeg and Homebridge-unifi-protect for their contributions and valuable insights in how to get things working!

Disclaimer

This plugin is provided free of charge and without any warranty of its functionality.
The creator cannot be held responsible for any damages, missed motion notifications that cause damage or harm.