This repository implements how to compute depth from stereo images. This is being tested on three different datasets, each containing two images of the same sceanrio but different camera angles. The baseline of the stereo setup is given.
python depth.py
For feature matching, we first need to locate features. Here, SIFT (Scale Invariant Feature Transform) is used. This detector is scale and rotation invariant. The algorithm is stated below:
- Convert images into grayscale.
- Compute SIFT features for both the images.
- Use Brute Force matcher to match the features. It uses L2 NORM distance for the same.
- Sort the matches in ascending order of least distance and for this project purpose only 150 matches were used.
- Inbuilt function of cv2.BFmatcher.match gives list of ob- jects. All points were collected from this object.
We use 8 point algorithm to estimate fundamental matrix. The algorithm is stated as follows:
- From the matches that are computed, 8 pairs are taken to estimate initial fundamental matrix.
- Before computing initial F matrix, we normalize image coordinates and compute is done using equations.
3)All of the pairs from both the images are tested against this fundamental matrix x ′ ∗ F ∗ x T and error was computed.
4)If the error is below some threshold value then that point pair will be inlier and corresponding Fundamental matrix will be considered as good.
- The above steps were repeated for 1000 iterations and outliers were discarded.
If we have calibrated cameras, we can get essential matrix from which we can easily estimate pose.
The Essential matrix is composed of Rotation and Translation of camera. If we decompose Essential matrix using singular value decomposition then we get four camera pose configurations.
To compute disparity, we need to have both the images in the same plane so that rotation does not come in to picture. To accomplish this, we will rectify the image pairs and also verify that they are coplanar by drawing epipolar lines. If epipolar lines are parallel then they are considered as coplanar. Inbuilt function of cv2.stereoRectifyUncalibrated() is used to get homography matrices of both the images.
