Starting with OpenCV

Contributor: Siddharth Saxena, Jayant Agarwal, Mayank Mittal

Image Processing

Image processing is a vital part of VARUN. It provides it with vision which is an indispensable part of an Autonomous robot which helps it to interact with its environment. For Image Processing, OpenCV libraries is a no brainer.

OpenCV is released under a BSD license and hence it’s free for both academic and commercial use. It has C++, C, Python and Java interfaces and supports Windows, Linux, Mac OS, iOS and Android. Most importantly it is really easy to integrate OpenCV libraries with ROS. It provides various inbuilt features such as opening and closing of images, applying Gaussian filters to the image, converting images from one format to another like RBG to HSV or Grayscale etc.

Through the worksheets available here, you can learn some basic principles of image processing and apply them on OpenCV. These include point operations, morphology, digital filters, feature detectors (mainly canny) and image transforms. These worksheets are an exhaustive source of information covering vast amount of information with links to various basic terms.

Then set up OpenCV 2.4.9 in Ubuntu through the github repository. This sets up OpenCV and checks it by running a simple OpenCV program of showing the output of webcam on screen on a window. After setting it up, you may start with OpenCV tutorials on your favorite language C++ or Python available here and here respectively.

Though the tutorials are written for working with Visual Studio 2012 but they work with equal ease on Ubuntu in a very similar way to ROS. Thus it becomes highly convenient to integrate it with ROS and making various publishers and subscribers.

The tutorials are as organized as it gets, it consists of 12 lessons from the very basics of OpenCV that is opening of an image to advanced features like tracking an object and shape detection. It also shows glimpses of the power of OpenCV like Histogram Equalization, smoothing and blurring of images. All the material is more than enough to get started doing some basic IP tasks.

Example of Ball Detection

One of the things you can do using OpenCV is tracking a ball using the inbuilt functions and find its radius, which in turn can help in estimating the distance of the object from the camera. For this, you would need to find the relation between the radius of the ball in pixels on camera image and distance from the camera lens. This relation can be found by experimentation.

As an exercise, try to write the code for ball detection. You may use the following hints.

1. Make an ROS publisher which takes a video from any webcam in RBG format and then thresholds it
2. Use inbuilt functions to reduce the noise from the foreground, for example, the salt noise, pepper noise
3. Find the contours of the image
4. Find the circle which enclose the different contours and out of them find the one with the largest radius
5. Average out the radius of 5 to 10 nearby radii and then asks the user for actual distance of the ball from the screen
6. Finally publish the average radius and, distance between camera plane and center of the ball

Now for finding distances via Stereo Input, a popular method used is Triangularization. A few resources are as follow: https://courses.cs.washington.edu/courses/cse455/09wi/Lects/lect16.pdf http://www.dis.uniroma1.it/~iocchi/stereo/triang.html

Now for detecting the pattern, i.e. a cross or square for the marker dropper task, Neural Networks seem like the best solution hence this tutorial about deep learning can be quite useful (at least chapter 1)