🩺 Polyp Detection using Detectron2 (Mask R-CNN)

Instance Segmentation of Colonic Polyps using Detectron2 and Mask R-CNN (ResNet-50 FPN) trained on the CVC-ClinicDB dataset.

This repository implements a complete deep learning pipeline:

• Dataset download from Kaggle
• Mask → COCO annotation conversion
• Detectron2 dataset registration
• Model training (Mask R-CNN R50-FPN)
• Inference & visualization
• Model export and configuration saving

Without segmentation (bgr image)

Model predicting polyps on the image

---

📌 Project Overview

Early detection of colonic polyps is essential for preventing colorectal cancer.
This project builds a full instance segmentation pipeline using Detectron2 to automatically detect and segment polyps from colonoscopy images.

Pipeline Workflow

1. Download dataset from Kaggle
2. Convert ground truth masks to COCO format
3. Register dataset in Detectron2
4. Train Mask R-CNN model
5. Perform inference on unseen images
6. Visualize predictions

---

🗂 Dataset

Dataset: CVC-ClinicDB
Source: Kaggle
https://www.kaggle.com/datasets/balraj98/cvcclinicdb

Dataset Structure

PNG/
 ├── Original/
 └── Ground Truth/

• Total images: 612
• Training split: 80%
• Validation split: 20%
• Classes: 1 (Polyp)

Ground truth masks are converted into COCO polygon segmentation format before training.

---

🛠 Tech Stack

• Framework: Detectron2 (v0.6)
• Deep Learning: PyTorch (2.5)
• Backbone: Mask R-CNN ResNet-50 FPN
• CUDA: 12.x
• Image Processing: OpenCV
• Visualization: Matplotlib
• Environment: Google Colab / Local GPU setup

---

⚙️ Installation

1️⃣ Clone Repository

git clone https://github.com/your-username/polyp-detection-detectron2.git
cd polyp-detection-detectron2

2️⃣ Install Dependencies

pip install pyyaml==5.1
pip install opencv-python
pip install matplotlib
pip install scikit-learn
pip install kaggle
pip install 'git+https://github.com/facebookresearch/detectron2.git'

Requirements:

• Python ≥ 3.8
• CUDA-enabled GPU recommended

---

🔄 Data Preparation

Download Dataset (Kaggle API)

kaggle datasets download balraj98/cvcclinicdb
unzip cvcclinicdb.zip

Convert Masks to COCO Format

The notebook performs:

• Contour extraction using cv2.findContours
• Polygon conversion
• Bounding box calculation
• COCO JSON generation
• Train/Validation split (80/20)

Output structure:

COCO_annotations/
 ├── train/
 │    └── annotations.json
 └── val/
      └── annotations.json

---

🧠 Model Configuration

Model configuration:

COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml

Key training parameters:

cfg.SOLVER.BASE_LR = 0.00025
cfg.SOLVER.MAX_ITER = 1000
cfg.SOLVER.IMS_PER_BATCH = 2
cfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 256
cfg.MODEL.ROI_HEADS.NUM_CLASSES = 1

---

🚀 Training

trainer = DefaultTrainer(cfg)
trainer.resume_or_load(resume=False)
trainer.train()

Outputs:

models/
 ├── model_1000_iter.pth
 └── config-1k_iter.yaml

TensorBoard monitoring supported:

tensorboard --logdir models/

---

🔍 Inference

cfg.MODEL.WEIGHTS = "model_1000_iter.pth"
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.45

predictor = DefaultPredictor(cfg)
outputs = predictor(image)

Predictions include:

• Segmentation masks
• Bounding boxes
• Confidence scores

Visualization performed using Detectron2 Visualizer.

---

📊 Results

• Training Images: 489
• Validation Images: 123
• Single-class instance segmentation
• Accurate mask overlays on colonoscopy frames

(Extend this section with mAP / IoU metrics if available.)

---

🧪 Use Cases

• AI-assisted colonoscopy
• Automated colorectal cancer screening
• Clinical decision support systems
• Medical imaging research
• Real-time endoscopic detection systems

---

📁 Project Structure

polyp-detection-detectron2/
│
├── dataset/
├── COCO_annotations/
│   ├── train/
│   └── val/
├── models/
├── notebook/
│   └── polyp_detection_using_detectron2.ipynb
└── README.md

---

🔮 Future Improvements

• Add COCOEvaluator metrics (mAP)
• Cross-dataset validation (e.g., Kvasir-SEG)
• Hyperparameter optimization
• Data augmentation
• Real-time video inference
• Deploy via Streamlit or FastAPI
• Docker containerization

---

👨‍💻 Contributor

Anirudha Kumar Sahu
Ph.D in Cancer Biology, BITS Pilani

---

📚 References

• Detectron2 – Facebook AI Research
• Mask R-CNN (He et al., 2017)
• CVC-ClinicDB Dataset

---

📜 License

Dataset license: As specified on Kaggle

---

⭐ If you find this project useful, consider starring the repository.