Smart Lock is a face recognition door lock we built and installed in Ars Futura office. It consists of four parts: Door API, Camera, Face Recognition API and Engine. Check out the blog post for more details.
Door API is a component which provides /unlock endpoint for unlocking the door.
This component is built for unlocking doors with electric strike.
Doors with electric strike are controlled by applying electric current to strike which unlocks (fail-secure) or locks it
(fail-safe). For setting up Door API you need Raspberry Pi and
5V relay. Connect Raspberry Pi to relay and connect relay
to door electric strike circuit to relay NC (normally closed) contact. Door API is implemented as
Flask server. /unlock endpoint activates relay through Raspberry Pi GPIO pins,
when relay is activated it connects door electric strike circuit which applies current to it and unlocks it
(fail-secure case). Check out api.py to see full implementation.
Here's a good tutorial on how to control relay using Raspberry Pi, in this tutorial author uses relay NO (normally open) contact, Door API assumes NC contact, but the logic for controlling relay is pretty much the same.
WARNING: If you are working with alternating current, be extremely cautious! Don't try anything without the supervision of trained professional!
Once you have all hardware setup, you can run Door API by running following commands on Raspberry Pi:
cd door-api
pip install -r requirements.txt
python api.py --pin GPIO_PIN_INDEXor just run it with Docker in one easy step (Docker installation):
docker run --name smart-lock-door-api --privileged --restart=always -d -p 80:80 arsfutura/smart-lock-door-api:latest --pin GPIO_PIN_INDEXOnce you have Door API up and running POST request to http://DOOR_IP/unlock should unlock the door. Door API will
return 202 if unlock request is successfully accepted.
api.py --help
usage: Simple Flask api for unlocking door with electric strike lock connected with Raspberry Pi via GPIO pins.
-p PIN [-d LOCK_DELAY] [-h]
required arguments:
-p PIN, --pin PIN Index of GPIO pin (in GPIO.BOARD mode) which controls
relay.
optional arguments:
-d LOCK_DELAY, --lock-delay LOCK_DELAY
How much time should we activate relay.
-h, --help show this help message and exit
Camera provides video stream which can be easily connected to programmatically.
This component is built for providing video stream on top of Raspberry Pi with camera module.
You can run Camera component by running following commands on Raspberry Pi (make sure you configure camera before using it):
cd camera
pip install -r requirements.txt
python stream.pyor just run it with Docker in one easy step (Docker installation):
docker run --name smart-lock-camera --privileged --restart=always -d -p 80:80 arsfutura/smart-lock-camera:latestOnce you have Camera up and running you can access live camera stream on http://CAMERA_IP and MJPG stream on
http://CAMERA_IP/stream.mjpg.
Face Recognition API provides /face-recognition
endpoint for classifying people in images.
Check out face-recognition repository for instructions on how to generate Face Recognition API. Also, if your are interested in inner workings of face-recognition, check out this blog post which explains it.
Engine is a component which connects everything together and makes Smart Lock work.
Engine connects to Camera, does classification on camera frames using Face Recognition API and unlocks the door using
Door API if a known person is recognised on a camera frame. Actual engine algorithm is a little bit more complicated,
check engine.py for details. Engine logs all actions and frames which trigger unlocks for debugging
purposes, logs are located in user-defined folder (check --log-path argument).
You can run Engine component by running following commands:
pip install -r engine/requirements.txt
python -m engine.engine --threshold THRESHOLD --camera-url http://CAMERA_IP/stream.mjpg --face-recognition-api-url http://FACE_RECOGNITION_IP:5000/face-recognition --door-api-url http://DOOR_IP/unlockor just run it with Docker in one easy step:
docker run --name smart-lock-engine --mount source=smart-lock-logs,target=/app/log --restart=always -d arsfutura/smart-lock-engine
--threshold THRESHOLD
--camera-url http://CAMERA_IP/stream.mjpg
--face-recognition-api-url http://FACE_RECOGNITION_IP:5000/face-recognition
--door-api-url http://DOOR_IP/unlockEngine depends on Camera, Door API and Face Recognition API but these 3 components don't have to be the exact ones described here. Camera stream can really be any video stream which is supported by OpenCV VideoCapture. Door API can be any HTTP POST endpoint, engine doesn't care what it does. Engine does assume that Face Recognition API is from face-recognition library, but again, it treats it as a HTTP POST endpoint, if you implement same response as Face Recognition API, you can have anything behind that endpoint. Basically, instead of unlocking, you could set up Engine to send you a message when some person appears in video stream.
engine.py --help
usage: engine.py --camera-url CAMERA_URL --face-recognition-api-url
FACE_RECOGNITION_API_URL --door-api-url DOOR_API_URL
--threshold THRESHOLD [--fps FPS]
[--haar-file-path HAAR_FILE_PATH] [--log-path LOG_PATH]
[--block-time BLOCK_TIME] [-h]
Face recognition engine which hooks to camera stream, performs face
recognition on stream frames and sends unlocks door request if a familiar face
is recognised.
required arguments:
--camera-url CAMERA_URL
URL of video stream.
--face-recognition-api-url FACE_RECOGNITION_API_URL
URL of face recognition endpoint.
--door-api-url DOOR_API_URL
URL of smart lock door-api which unlocks door.
--threshold THRESHOLD
Number from 0 to 1 which represents minimal confidence
which need to be met to unlock the door when face is
recognised from a video stream. Default values is 0.8
(We need to be at least 80% sure that we recognised
some person to unlock the door for him).
optional arguments:
--fps FPS How many frames per second will engine be processing.
This parameter affects engine performance the most,
setting too high fps can dramatically slow down the
engine. Default value od 2 should be good enough in
most situations.
--haar-file-path HAAR_FILE_PATH
Path to Haar Cascade (simple and fast face detector)
xml file.
--log-path LOG_PATH Directory path where logs will be saved. Default is
"log".
--block-time BLOCK_TIME
After door is unlocked engine will stop processing
frame for block-time seconds. By default, engine will
stop processing frames for 6 seconds after unlock.
-h, --help show this help message and exit
Data collector is convenience component for collecting data. It logs every frame which recognises a person where
min_confidence <= person_confidence <= max_confidence.
This component was built for collecting additional data in cases where Face Recognition API doesn't assign high-enough confidence to people. Let's say you set your Engine threshold to 0.8 (80%) and there are some people who consistently get assigned confidences in range of 60%-80%, rarely enough to break the threshold and get door unlocked, this means that you lack data for those people (issue could also be that you underfitted or overfitted your model, but let's assume that's not the case here). In order to gather more data, you run data collector with min confidence of 0.6 and max confidence of 0.8, collect those frames that you need (manually choose frames that are good, don't take blurry frames or ones where person is not front facing the camera) and retrain your model, this should boost your model performance.
You can run Data collector by running following commands:
pip install -r data-collector/requirements.txt
python -m data-collector.collector -min MIN_CONFIDENCE -max MAX_CONFIDENCE --camera-url http://CAMERA_IP/stream.mjpg --face-recognition-api-url http://FACE_RECOGNITION_IP:5000/face-recognitionor just run it with Docker in one easy step:
docker run --name smart-lock-data-collector --mount source=data,target=/app/log --restart=always -d arsfutura/smart-lock-data-collector
-min MIN_CONFIDENCE
-max MAX_CONFIDENCE
--camera-url http://CAMERA_IP/stream.mjpg
--face-recognition-api-url http://FACE_RECOGNITION_IP:5000/face-recognition Little tip for Docker Mac users, run following command to backup volumes from Docker VM to host machine:
docker run --rm -it -v /path/to/folder/on/your/mac:/backup -v /var/lib/docker:/docker alpine:edge tar cfz /backup/volumes.tgz /docker/volumes/Now you have backup of Docker volumes in /path/to/folder/on/your/mac.