Skip to content

Latest commit

 

History

History
103 lines (72 loc) · 4.37 KB

README.md

File metadata and controls

103 lines (72 loc) · 4.37 KB
Raw

yolact_ros

The yolact_ros is based on Yolact and is integrated with ROS(Robot Operating System)

Yolact(You Only Look At CoefficienTs) with ROS and Webcam

Requirement

  • ROS(Kinetic)
  • GPU supporting CUDA

Installation

  • Set up a Python3 environment.
  • Install Pytorch 1.0.1 (or higher) and TorchVision.
    • conda install pytorch==1.0.1 torchvision==0.2.2 cudatoolkit=9.0 -c pytorch
  • Install some other packages: 
    # Cython needs to be installed before pycocotools
pip install cython
pip install opencv-python pillow pycocotools matplotlib 

conda install opencv
pip install pyyaml
pip install rospkg
  • Clone this repository and enter it: 
    git clone https://github.com/dbolya/yolact.git
cd yolact
  • If you'd like to train YOLACT, download the COCO dataset and the 2014/2017 annotations. Note that this script will take a while and dump 21gb of files into ./data/coco. 
    sh data/scripts/COCO.sh
  • If you'd like to evaluate YOLACT on test-dev, download test-dev with this script. 
    sh data/scripts/COCO_test.sh

Weight

Image Size Backbone FPS mAP Weights
550 Resnet50-FPN 42.5 28.2 yolact_resnet50_54_800000.pth Mirror
550 Darknet53-FPN 40.0 28.7 yolact_darknet53_54_800000.pth Mirror
550 Resnet101-FPN 33.0 29.8 yolact_base_54_800000.pth Mirror
700 Resnet101-FPN 23.6 31.2 yolact_im700_54_800000.pth Mirror

To evalute the model, put the corresponding weights file in the ./weights directory and run one of the following commands.

In my workspace, After downloaded the weight, copy and paste to the trained_model path.

Yolact ros with Visual grasping

Yolact(You Only Look At CoefficienTs) grasp

Conda env:

  • torch11py36(Custom computer)
  • py36_ros(NSCL computer)

Capture image with customized config (In my case, Nobrand)

rosrun yolact_ros yolact_capture_img.py  --trained_model=/home/geonhee-ml/rl_ws/src/yolact_ros/src/yolact/weight/yolact_base_1234_100000.pth  --score_threshold=0.3 --top_k=100 --image=/home/geonhee-ml/rl_ws/src/yolact_ros/src/yolact/image/116.jpg

Launch

./realsense

roscore

(torch11py36) conda env

rosrun yolact_ros yolact_tcp_img.py  --trained_model=/home/geonhee-ml/rl_ws/src/yolact_ros/src/yolact/weight/yolact_base_1234_100000.pth  --score_threshold=0.3 --top_k=100 --image=/home/geonhee-ml/rl_ws/src/yolact_ros/src/yolact/image/116.jpg

Save image from realsense through ros serveice

rosrun yolact_ros yolact_save_img.py 

rosservice call /save_image "data: false"

Run ros server for getting call and sending instance information

rosrun yolact_ros yolact_ros_server.py  --trained_model=/home/geonhee-ml/rl_ws/src/yolact_ros/src/yolact/weight/yolact_base_1234_100000.pth  --score_threshold=0.3 --top_k=100