A demonstration of the Metropolis-Hastings MCMC algorithm that infers the parameters of a 3D line from noisy 2D images taken by perspective cameras and using their projection (u,v) plane.
Table of Contents
This project uses the Metropolis-Hastings Markov Chain Monte Carlo (MCMC) algorithm to infer the parameters of a given 3D line through the use of one and two perspective cameras' projection (u,v) plane. The images captured by the cameras contain noise, modeled as Gaussian, which is added to each point on the line.
In this problem, from the noisy images captured by the cameras, sample points are extracted from the images at different intervals, and these points serve as inputs to the Metropolis-Hastings algorithm.
The algorithm begins by generating initial point proposals by sampling them from a given prior distribution, and then iteratively samples new points based on the camera images and extracted sample points. Metropolis-Hastings algorithm in theory will always converge on a distribution, which is why it is being used to infer the parameters of the 3D line from the noisy images.
Initially, the result of the 3D line's parameters inferred from using one camera only manages to properly converge on two of the line's axes, which makes sense as it is hard to infer the depth axis of the line's points from a 2D noisy image taken by one perspective camera.
To solve this problem and to help the algorithm converge on the line's parameters faster, a second camera with a different position is used. Metropolis-Hastings is then run again but now with the input of both cameras, and finally, you can see in the result that all three axes of the 3D line were converged upon and that time of convergence was quicker than with only the use of one camera.
Note: In order to view the project's objective, you can read ista421ML-f2021-final-project-OptionA-MCMC.pdf. If you want to see the output alongside the conclusions drawn from the project, you can read final_project_OptionA_answers.pdf.
To get a local copy up and running follow these simple example steps.
The project uses the following dependencies. It is necessary to install them before running the script.
- numpy
pip install numpy
- scipy
pip install scipy
- matplotlib
pip install matplotlib
- Clone the repo
git clone https://github.com/empobla/Inference-of-a-3D-Line.git
- Enter the code directory
cd code - Run the script
python index.py
When running index.py, you will notice that initially your console seems to be doing nothing. This is normal, as it is running the Metropolis-Hastings algorithm's first 10% of samples (out of 50,000 total samples) for the first camera. This is a slow process, so just have patience. You will see progress updates in the console as follows:
M-H is 0.1 done
M-H is 0.2 done
M-H is 0.3 done
M-H is 0.4 done
M-H is 0.5 done
M-H is 0.6 done
M-H is 0.7 done
M-H is 0.8 done
M-H is 0.9 doneOnce these samples are done, the script will show you plots with the approximated 3D line. In addition, these plots will be saved automatically under the figures/ directory.
 |
 |
 |
After showing the plots, the script will proceed to run the Metropolis-Hastings algorithm taking into account two cameras instead of one. This, as the previous process, is also a slow process. The script will show you it's progress in the console as follows:
M-H T5 is 0.1 done
M-H T5 is 0.2 done
M-H T5 is 0.3 done
M-H T5 is 0.4 done
M-H T5 is 0.5 done
M-H T5 is 0.6 done
M-H T5 is 0.7 done
M-H T5 is 0.8 done
M-H T5 is 0.9 doneOnce this is done, the script will show you new plots with the approximated 3D line. You will notice that the program managed to better approximate the 3D line in less samples due to the use of two cameras. These plots will also be saved automatically under the figures/ directory.
 |
 |
 |
 |
All rights are reserved. Modification or redistribution of this code must have explicit consent from the owner.
Emilio Popovits Blake - Contact
Project Link: https://github.com/empobla/Inference-of-a-3D-Line