In this Code Pattern we'll provide a web application that has the ability to capture frames from streaming cameras, and analyze each frame using IBM Maximo Visual Inspection. As analysis results are received, they can be grouped into positive or negative categories.
This code pattern is targeted towards users who have access to live streams or CCTV cameras, and would like to apply object detection and image classification to their live camera feeds.
- IBM Maximo Visual Inspection. This is an image analysis platform that allows you to build and manage computer vision models, upload and annotate images, and deploy apis to analyze images and videos. Sign up for a trial account of IBM Maximo Visual Inspection here. This link includes options to provision a IBM Maximo Visual Inspection instance either locally on in the cloud.
- User accesses web application and provides login credentials for both the Camera system and IBM Maximo Visual Inspection. User also fills out form to select model and classify images as positive or negative.
- Node.js backend connects to camera RTSP stream and forwards to frontend web application
- As frontend plays live video, user clicks "Capture Frame" or "Start Interval"
- Captured frames are forwarded to IBM Maximo Visual Inspection backend for analysis.
- Analysis results are rendered in web app, and grouped by user-defined positive/negative labels.
- An account on IBM Marketplace that has access to IBM Maximo Visual Inspection. This service can be provisioned here
You will also need to train and deploy a custom model beforehand. This can be done following the steps in this video
-
Docker - Can be used to run the application in a virtual container. If running via docker, the remaining prerequisites can be bypassed, and you can skip ahead to the steps labeled docker in step 2
-
ffmpeg - Package used here to connect to remote RTSP streams
# Linux
apt install ffmpeg
# Macbook OS X
brew install ffmpeg
- Node.js
Skip to Steps if you already have node.js installed on your system.
If expecting to run this application locally, please install Node.js and NPM. Windows users can use the installer at the link here.
If you're using Mac OS X or Linux, and your system requires additional versions of node for other projects, we'd suggest using nvm to easily switch between node versions. Install nvm with the following commands.
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.11/install.sh | bash# Place next three lines in ~/.bash_profile
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completionnvm install v8.9.0
nvm use 8.9.0Follow these steps to setup and run this Code Pattern.
git clone https://github.com/IBM/maximo-streaming-video-analysis
Navigate to the project folder
cd maximo-streaming-video-analysis
Local Deployment Install frontend dependencies
cd maximo-streaming-video-analysis
cd frontend
npm install
Start frontend
npm run serve
In a separate terminal tab, install backend dependencies
cd maximo-streaming-video-analysis
cd backend
npm install
Start backend
PORT=3000 npm start
Docker Start Backend
docker run -it -p 3000:3000 kkbankol/maximo-live /bin/bash -c "cd /maximo-streaming-video-analysis/backend && PORT=3000 npm start"
Start frontend
# frontend
docker run -it -p 8080:8080 kkbankol/maximo-live /bin/bash -c "cd /maximo-streaming-video-analysis/frontend && npm run serve"
Access the web application at localhost:8080.
Click "Login" and provide the url, username, and password for the Maximo Visual Inspection instance. These credentials should have been provided in your welcome letter.
Next, click "Configure Model" in the Menu. Select your custom model. Then, select objects or classes associated with that model that you'd like to observe in the dashboard. These selected categories can be grouped as either negative or positive.
After selecting the relevant model categories, we can then being streaming a video to the application for analysis. Currently, this app supports the following video sources. To stop a video from playing, click "Stop Stream".
- Webcam - This is the default method, and requires no configuration
- RTSP - RTSP is a streaming protocol commonly used by CCTV cameras. To connect to an RTSP stream, click the "Stream RTSP" button, provide the URL and the login credentials (if applicable).
- Youtube - To playback a Youtube video for analysis, click the "Stream RTSP" button and enter the Youtube URL.
- Local Video File - Click the "Upload Video" button and drag and drop your video file.
After clicking "Submit" in the "Stream RTSP" or "Upload File" modal, the video should immediately begin playing in the browser.
Once the video begins playing, click the "Capture Frame" button to analyze a single frame with the configured model. Click the "Capture Frame Interval" button to analyze a frame every second for 10 seconds.
As results are received that match the model configuration, they will then be rendered in the "Inference Results" section on the right hand side like so.
Clicking on one of the inference results will show a popup modal with detailed information for that particular inference, such as the identified class/object, heatmap/bounding boxes, and confidence score.
This code pattern is licensed under the Apache Software License, Version 2. Separate third party code objects invoked within this code pattern are licensed by their respective providers pursuant to their own separate licenses. Contributions are subject to the Developer Certificate of Origin, Version 1.1 (DCO) and the Apache Software License, Version 2.