Tip2_DynSLAM

1.Clone the repository if you haven't already:
  git clone --recursive https://github.com/AndreiBarsan/DynSLAM
  **Some files(submodule) under the src dir need to be downloaded by yourself.**
  **eigen3 can be download from the 'Eigen website'**
  
2.Download opencv 2.4.9; cmake&make it; 
If you have installed e.g. opencv 3 previous, you should change sth in the CMakeLists
find_package(OpenCV REQUIRED) -> find_package(OpenCV 2 REQUIRED) 
This makes your project to know which version of opencv you'd like to use

3.Pangolin

4.Cuda 8
4.1 NVIDIA driver
  - Independencies 
  ``sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler  --no-install-recommends libboost-all-dev  libopenblas-dev liblapack-dev libatlas-base-dev libgflags-dev libgoogle-glog-dev liblmdb-dev

  - https://www.nvidia.cn/Download/index.aspx?lang=cn 
    Download according to your environment.
    
    
  - Drag the download under the 'Home' dir.
  
  - clean all files relates to Nvidia
    sudo apt-get --purge remove nvidia-*
    sudo apt autoremove
  
  
  - shut down graphic interface
    sudo /etc/init.d/lightdm stop
    
    press alt+ctrl+f1 
    
  - Install driver
    sudo chmod  a+x NVIDIA-Linux-x86_64-410.78.run  #adjust to your own file
    sudo ./NVIDIA-Linux-x86_64-410.78.run -no-x-check -no-nouveau-check -no-opengl-files
    
    Select Continue, yes, ok
    
    Error: kernel blabla fail. 
    Reason: install linus using UEFI.  Disable the Secure Boot in bios
    
    reinstall
    
    Then, Would you like to run the nvidia-xconfigutility to automatically update your x configuration so that the NVIDIA x driver ...
    Select Yes
    
    
    check whether install successfully
    cat /proc/driver/nvidia/version  
    OR
    sudo nvidia-smi

   start graphic interface
   sudo /etc/init.d/lightdm start\
   
4.2 Install Cuda 8.0
   download cuda 8.0 tool kit CUDA Toolkit 8.0 (download .runfile)
   
   install:
   sudo sh cuda_8.0.61_375.26_linux.run 
   
   Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 375.26?
   no, we have installed one.
   
   Install the CUDA 8.0 Toolkit?
   yes
   
   else yes
   
   sudo gedit ~/.bashrc
   
   add
   export PATH=/usr/local/cuda-8.0/bin${PATH:+:${PATH}}
   export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
   at the end.
   
   source ~/.bashrc

   test:
   nvcc --version
   
 4.3 install cuda samples.
   sudo apt-get install freeglut3 freeglut3-dev
   cd ~/NVIDIA_CUDA-8.0_Samples/common
   gedit findgllib.mk
   change [UBUNTU_PKG_NAME = "nvidia-418"] according to your driver version, save
   cd ~/NVIDIA_CUDA-8.0_Samples  
   make -j12 
   
   cd bin/x86_64/linux/release/  
   test: ./deviceQuery      
   

5. install nvidia-docker2
   sudo apt-get update
   sudo apt-get -y install apt-transport-https ca-certificates curl software-properties-common
   
   curl -fsSL http://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | sudo apt-key add -
   
   sudo add-apt-repository "deb [arch=amd64] http://mirrors.aliyun.com/docker-ce/linux/ubuntu $(lsb_release -cs) stable"
   
   sudo apt-get -y update
   sudo apt-get -y install docker-ce
   
   sudo groupadd docker
   sudo gpasswd -a $USER docker
   newgrp docker
   docker ps
   docker version
   
   docker volume ls -q -f driver=nvidia-docker | xargs -r -I{} -n1 docker ps -q -a -f volume={} | xargs -r docker rm -f
   sudo apt-get purge -y nvidia-docker
   sudo apt autoremove
   
   curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
        sudo apt-key add -
        
   distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
   
   curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
   sudo tee /etc/apt/sources.list.d/nvidia-docker.list

   sudo apt-get update
   
   sudo apt-get install -y nvidia-docker2
   sudo pkill -SIGHUP dockerd
  
   test:sudo docker run --runtime=nvidia --rm nvidia/cuda nvidia-smi
   
   
   
6.Install the prerequisites (Ubuntu example):
sudo apt-get install libxmu-dev libxi-dev freeglut3 freeglut3-dev glew-utils libglew-dev libglew-dbg libpthread-stubs0-dev binutils-dev libgflags-dev libpng++-dev libeigen3-dev
 
7.build DynSLAM project
mkdir build && cd build && cmake .. && make -j$(nproc)

error:
- 1.error: ‘setw’ was not declared in this scope 
sol: add #include <iomanip>

- 2.error: ‘resize’ is not a member of ‘cv’
  ‘cvtColor’ is not a member of ‘cv’ 
  ‘putText’ is not a member of ‘cv’ 
sol:add #include "opencv2/imgproc/imgproc.hpp"
        #include "opencv2/opencv.hpp"

- 3.error nvcc fatal   : redefinition of argument 'std'
find the location of the file showing this error.  Modify the cmakelist to delete -std=c++11 (may also need modify the CMakeLists.txt of DynSLAM)
do 
cmake ..
make

- 4.error: ‘imread’ is not a member of ‘cv’
sol:#include "opencv2/imgproc/imgproc.hpp"
    #include "opencv2/opencv.hpp"

- 5.error: ‘glDrawFrustrum’ is not a member of ‘pangolin’
  pangolin API change name glDrawFrustrum to glDrawFrustum
  just find the function and change it.
  
  Now everything works fine!!!!!
  
  
8.test DynSLAM
  - 1. Download the demo sequence 
  https://drive.google.com/uc?export=download&confirm=Nnbd&id=1V-I4Tle7MNbmnf2qRe6aTpjxOld2M2i8
  
  - 2.Extract that to a directory, and run DynSLAM on it (the mkdir circumvents a silly bug): 
  make a csv file in the root dir of the project:mkdir -p csv 
  then run:
  ./build/DynSLAMGUI --use_dispnet --dataset_root=path/to/extracted/archive --dataset_type=kitti-odometry
  e.g. ./build/DynSLAMGUI --use_dispnet --dataset_root=mini-seq-06 --dataset_type=kitti-odometry


-------------------------------------------------------------------------------------------------------------------
## Environment setting: MNC

1. Clone the MNC repository:

git clone --recursive https://github.com/daijifeng001/MNC.git

2. Install Python packages: numpy, scipy, cython, python-opencv, easydict, yaml.

 - sudo apt install cython -- user (version 0.25 works fine, if not
 
 python -m pip install cython==0.25.1 --user
 
 cython --version for checking the version.)
 
 [change default mode to python 2 if u r using python 3:
 
 sudo update-alternatives --install /usr/bin/python python /usr/bin/python2 100

 sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 50]
 
 - pip2 install numpy --user
 - pip2 install scipy --user
 - pip2 install opencv-python --user
 - pip2 install easydict --user
 - pip2 install PyYAML --user


3. Build the Cython modules and the gpu_nms, gpu_mask_voting modules by:
  
  - cd $MNC_ROOT/lib
  - make

4.Install Caffe and pycaffe dependencies (see: Caffe installation instructions for official installation guide)

  http://caffe.berkeleyvision.org/installation.html

  (Use the modifed Caffe provided by MNC)
  
  cd to caffe-mnc
  
  cp Makefile.config.example Makefile.config
  
  - Note
   # In your Makefile.config, make sure to have this line uncommented
   WITH_PYTHON_LAYER := 1
   # CUDNN is recommended in building to reduce memory footprint
   USE_CUDNN := 1 
   
   
  - if you enable cudnn, plz Install CUDNN
    1.https://www.jianshu.com/p/e22866b72f43
    
    2. Then do,
    cd /usr/local/cuda-8.0/lib64
    sudo rm -rf libcudnn.so libcudnn.so.5
    sudo ln -s libcudnn.so.5.0.5 libcudnn.so.5 (Modify it to your version)
    sudo ln -s libcudnn.so.5 libcudnn.so
    sudo ldconfig -v
    
    ref: https://blog.csdn.net/huang826336127/article/details/86670925
    
    When you use the caffe with cudnn and encouter some errors, here is the solution:
    https://blog.csdn.net/u011070171/article/details/52292680
    
  - Install BLAS and other Prerequisites mentioned by the official website
    
    (https://iqbalnaved.wordpress.com/2016/05/08/installing-caffe-and-pycaffe-on-ubuntu-14-04/)
  
  
  - install GLog & GFlags
    1. gflags
    git clone https://github.com/gflags/gflags.git
    cd gflags
    mkdir build && cd build
    cmake .. -DGFLAGS_NAMESPACE=google -DCMAKE_CXX_FLAGS=-fPIC ..
    make -j4
    sudo make install
    
    2. glog
    cd ~
    sudo apt-get install autoconf automake libtool
    git clone https://github.com/google/glog
    ./autogen.sh
    ./configure
    make -j8
    sudo make install
    
    
    
  - cd to caffe-MNC
  - cp Makefile.config.example Makefile.config
    # Adjust Makefile.config (for example, if using Anaconda Python, or if cuDNN is desired)
    make all -j4
    
    -------------------------------------------------------------------------------------------
    error:
    fatal error: hdf5.h
    
    solution:
    Makefile.config: modify INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
    to
    INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/

    Makefile: modify LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_hl hdf5
    to
    LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_serial_hl hdf5_serial

   ---------------------------------------------------------------------------------------------
   error:
   src/caffe/common.cpp: In function ‘void caffe::GlobalInit(int*, char***)’:
   src/caffe/common.cpp:45:5: error: ‘::gflags’ has not been declared
   ::gflags::ParseCommandLineFlags(pargc, pargv, true);
   
   solution:
    In files:
    caffe/include/caffe/common.hpp
    caffe/examples/mnist/convert_mnist_data.cpp
    
    Comment out the ifndef

    // #ifndef GFLAGS_GFLAGS_H_
    namespace gflags = google;
    // #endif  // GFLAGS_GFLAGS_H_
    ----------------------------------------------------------------------------------------------
    error:
    undefined reference to `cv::imdecode(cv::_InputArray const&, int)'

    solution:
    Having this error because I installed opencv3 (to check version: pkg-config --modversion opencv)
    
    Modify Makefile.config:
    
    # Uncomment if you're using OpenCV 3
    OPENCV_VERSION := 3
    
    # Whatever else you find you need goes here.
    #INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
    #INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/
    #LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
    INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include /usr/lib/x86_64-linux-gnu/hdf5/serial/include
    LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial

    
    and
    
    Modify Makefile:
    (this one may not need if you get errors when you modify this)
    LIBRARIES += glog gflags protobuf leveldb snappy \        lmdb boost_system hdf5_hl hdf5 m \        opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs
    -----------------------------------------------------------------------------------------------
    error:
    recipe for target '.build_release/src/caffe/proto/caffe.pb.cc' failed
    download and install protobuf from
    https://github.com/google/protobuf/releases/download/v3.4.1/protobuf-cpp-3.4.1.tar.gz
    
    or do 
    sudo apt-get install libprotobuf-dev protobuf-compiler
    
    ----------------------------------------------------------------------------------------------


    
    make clean
    make all -j16
    make pycaffe
    
    -----------------------------------------------------------------------------------------------
    errors:
    NVCC src/caffe/layers/cudnn_lcn_layer.cu nvcc fatal   : Unsupported gpu architecture 'compute_20' Makefile:594: recipe for target '.build_release/cuda/src/caffe/layers/cudnn_lcn_layer.o' failed make: *** [.build_release/cuda/src/caffe/layers/cudnn_lcn_lay
    In Makefile.config, comment it according to your version of cuda

  # CUDA architecture setting: going with all of them.
  # For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility.
  # For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility.
  # For CUDA >= 9.0, comment the *_20 and *_21 lines for compatibility.
  CUDA_ARCH := #-gencode arch=compute_20,code=sm_20 \
                #-gencode arch=compute_20,code=sm_21
                -gencode arch=compute_30,code=sm_30 \
                -gencode arch=compute_35,code=sm_35 \
                -gencode arch=compute_50,code=sm_50 \
                -gencode arch=compute_52,code=sm_52 \
                -gencode arch=compute_60,code=sm_60 \
                -gencode arch=compute_61,code=sm_61 \
                -gencode arch=compute_61,code=compute_61

    
   -----------------------------------------------------------------------------------------------  
   errors:
   fatal error: numpy/arrayobject.h: No such file or directory
    #include <numpy/arrayobject.h>
   
   solution: 
   sudo apt-get install python-numpy then make clean, make pycaffe (version 1.16.4)
    
    [For this project, we dont need to do this.
    make test -j16
    
    -----------------------------------------------------------------------------------------------
    error:
    src/caffe/test/test_smooth_L1_loss_layer.cpp:11:35: fatal error: caffe/vision_layers.hpp
    
    solution:
    find the file
    $CAFFE_ROOT/src/caffe/test/test_smooth_L1_loss_layer.cpp 
    comment the 11th line
    -----------------------------------------------------------------------------------------------
    ]

    
  
    Demo
    cd to MNC dir
    
    First, download the trained MNC model.
    ./data/scripts/fetch_mnc_model.sh
    
    Run the demo:
    cd $MNC_ROOT
    ./tools/demo.py
    
    solution for errors:
    sudo apt-get install python-skimage
    sudo apt-get install python-protobuf
    sudo pip install matplotlib
      
   ImportError: cannot import name _validate_lengths 
   (Not support this version of scikit-image)
   reinstall using
   - pip install -U scikit-image
      
    monitor gpu per second:
   -  watch -n 1 nvidia-smi 
    
    
    
    
    If you got errors like out of memory, modify the mnc_config.py file, (limitation of your GPU memory)
    change all the value that relates to 'size' to a smaller one.
    
    - run on kitti datasets:
    `` ./tools/demo.py --input /home/user1/Mengqi/DynSLAM/datasets/tracking/training/image_02/0002 --output /home/user1/Mengqi/DynSLAM/datasets/tracking/training/seg_output
    
    
    -----------------------------------------------------------------------------------------------------------
    
    Install libelas-tooling
    
    - download the project from
      http://www.cvlibs.net/download.php?file=libelas.zip
    
    - cd to the dir
    
    - cmake.
    
    - make 
     ----------------------------------------------------------------------------------------------------------
      error:
      error: ‘gflags’ has not been declared
      
      In files:
      src/kitti2klg.cpp:
      
      add:
      namespace gflags = google; 
      
     ---------------------------------------------------------------------------------------------------------
      Run './elas demo' => computes disparity maps for images from the 'img' directory



- Install Nvidia docker from:
  https://github.com/NVIDIA/nvidia-docker#quick-start
  
  
  

- Download dispnet from
  https://github.com/AndreiBarsan/dispnet-flownet-docker/tree/4f4d19b0edf01587cd0f29aa48b00d1459c9f6f7

  put it in the preprossing file.
  
  do make in the dir
  
  



-----------------------------------------------------------------------------------------------------------------------------
for DynSLAM,
preprocessing kitti tracking datasets:
- ./scripts/preprocess-sequence.sh kitti-tracking ./datasets/tracking/ training 2

Error:
docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock

sudo adduser $USER docker
newgrp docker


depth:
./process-kitti-tracking.sh ../../datasets/tracking testing 10

something wrong with pip, (error: main cant find)
So I comment in Dockerfile: (cuz I have got this environment set)
#RUN pip install --upgrade pip && \
#    pip install Cython numpy scipy scikit-image matplotlib h5py nose pandas \
#    protobuf python-gflags Pillow six python-dateutil pyyaml \
#    easydict opencv-python