This awesome list is under construction. If you have anything to recommend or any suggestions, please feel free to contact us via e-mail (zxq@mail.dlut.edu.cn) or directly push a PR.
< Last updated: 30/05/2024 >
- Awesome-UniCDSeg-Segmentation
Awesome List for Unified Context-dependent Concept Segmentation (UniCDSeg)
- In philosophy and cognitive science, concepts usually contain the context-independent (CI) and context-dependent (CD) concepts, as described by the psychologist and cognitive scientist Barsalou.
- The most important difference between context-dependent (CD) and context-independent (CI) concepts is whether the background environment (spatial context) plays a decisive role in the definition of the concept. The CD concepts mean that the target is not cognizable without its background environment. But, the context-independent (CI) concepts still have its clear definition even without a background environment.
- The CI concept such as bear and bus can be well understood by relying only on the foreground. While the CD concept is the complete opposite, salient and shadow objects require a specific background to highlight the expression of saliency and shadow concepts. Similarly, it is impossible to determine whether the lesions are polyp or COVID-19 infection without background information. Just because of this property, we need to ensure the semantic certainty of visual prompts when unifying modeling.
- For the CI concept segmentation field (that is, semantic segmentation), popular datasets have multiple concept annotations for a single image. The trained CI models can well distinguish different concepts in a unified model.
- For the CD concept segmentation field, existing works explore the in-domain modeling, resulting in repetitive structure design, inefficient data utilization, and limited multi-domain generalization. And, the CI concept unified models cannot well solve the CD concept tasks.
| Dataset Name | CD Concept Type | Year | Publication | Links |
|---|---|---|---|---|
| DUTS | Salient Object Segmentation | 2017 | CVPR | Paper |
| DUT-OMRON | Salient Object Segmentation | 2013 | CVPR | Paper |
| PASCAL-S | Salient Object Segmentation | 2014 | CVPR | Paper |
| HKU-IS | Salient Object Segmentation | 2015 | CVPR | Paper |
| ECSSD | Salient Object Segmentation | 2015 | CVPR | Paper |
| CHAMELEON | Camouflaged Object Segmentation | 2018 | - | Paper |
| CAMO | Camouflaged Object Segmentation | 2019 | CVIU | Paper |
| COD10K | Camouflaged Object Segmentation | 2020 | CVPR | Paper |
| NC4K | Camouflaged Object Segmentation | 2021 | CVPR | Paper |
| UCF | Shadow Segmentation | 2010 | CVPR | Paper |
| ISTD | Shadow Segmentation | 2018 | CVPR | Paper |
| SBU | Shadow Segmentation | 2016 | ECCV | Paper |
| Trans10K | Transparent Object Segmentation | 2020 | ECCV | Paper |
| Trans10Kv2 | Transparent Object Segmentation | 2021 | IJCAI | Paper |
| ETIS-LaribPolypDB | Colonoscopy Polyp Segmentation | 2014 | IJCARS | Paper |
| CVC-ColonDB | Colonoscopy Polyp Segmentation | 2015 | TMI | Paper |
| CVC-ClinicDB | Colonoscopy Polyp Segmentation | 2015 | CMIG | Paper |
| Kvasir | Colonoscopy Polyp Segmentation | 2017 | ACM MMSys | Paper |
| CVC-300 | Colonoscopy Polyp Segmentation | 2017 | JHE | Paper |
| COVID-19 data | COVID-19 Lung Infection | 2020 | TMI | Paper |
| BUSI | Breast Lesion Segmentation | 2020 | Data in Brief | Paper |
| ISIC17-20 | Skin Lesion Segmentation | 2017-2020 | - | Website |
- Pixel Accuracy (PA) is calculated based on the binarized prediction mask and ground-truth:
- F-measure is a metric that comprehensively considers both precision and recall:
- weighted F-measure is proposed to improve the metric F-measure. It assigns different weights (ω) to precision and recall across different errors at different locations, considering the neighborhood information:
- S-measure evaluates the spatial structure similarity by combining the region-aware structural similarity Sr and the object-aware structural similarity So:
- E-measure can jointly capture image level statistics and local pixel matching information:
- IOU is the most common metric for evaluating classification accuracy:
- Dice is a statistic used to gauge the similarity of two samples and become the most used metric in validating medical image segmentation:
- Balanced error rate (BER) is a common metric to evaluate shadow detection performance, where shadow and non-shadow regions contribute equally to the overall performance without considering their relative areas:
- MAE measures the average absolute difference between the prediction and the ground truth pixel by pixel:
| Year | Model | Publication | Title | Links |
|---|---|---|---|---|
| 2024 | GateNetv2 | IJCV | Towards Diverse Binary Segmentation via A Simple yet General Gated Network Xiaoqi Zhao; Youwei Pang; Lihe Zhang; Huchuan Lu; Lei Zhang |
Paper/Code |
| Year | Model | Publication | Title | Links |
|---|---|---|---|---|
| 2024 | MVG | arXiv | Inspiring the Next Generation of Segment Anything Models: Comprehensively Evaluate SAM and SAM 2 with Diverse Prompts Towards Context-Dependent Concepts under Different Scenes Xiaoqi Zhao, Youwei Pang, Shijie Chang, Yuan Zhao, Lihe Zhang, Huchuan Lu, Jinsong Ouyang, Georges El Fakhri, Xiaofeng Liu |
Paper/Code |
| 2024 | MVG | arXiv | Medical Vision Generalist: Unifying Medical Imaging Tasks in Context Sucheng Ren, Xiaoke Huang, Xianhang Li, Junfei Xiao, Jieru Mei, Zeyu Wang, Alan Yuille, Yuyin Zhou |
Paper/Code |
| 2024 | Universal Model | MIA | Universal and Extensible Language-Vision Models for Organ Segmentation and Tumor Detection from Abdominal Computed Tomography Jie Liu, Yixiao Zhang, Kang Wang, Mehmet Can Yavuz, Xiaoxi Chen, Yixuan Yuan, Haoliang Li, Yang Yang, Alan Yuille, Yucheng Tang, Zongwei Zhou |
Paper/Code |
| 2024 | Spider | ICML | Spider : A Unified Framework for Context-dependent Concept Segmentation Xiaoqi Zhao; Youwei Pang; Wei Ji; Baicheng Sheng; Jiaming Zuo; Lihe Zhang; Huchuan Lu |
Paper/Code |
| 2024 | GateNetv2 | IJCV | Towards Diverse Binary Segmentation via A Simple yet General Gated Network Xiaoqi Zhao; Youwei Pang; Lihe Zhang; Huchuan Lu; Lei Zhang |
Paper/Code |
| 2024 | VSCode | CVPR | VSCode: General Visual Salient and Camouflaged Object Detection with 2D Prompt Learning Ziyang Luo; Nian Liu; Wangbo Zhao; Xuguang Yang; Dingwen Zhang; Deng-Ping Fan; Fahad Khan; Junwei Han |
Paper/Code |
| 2023 | SegGPT | ICCV | SegGPT: Towards Segmenting Everything in Context Xinlong Wang; Xiaosong Zhang; Yue Cao; Wen Wang; Chunhua Shen; Tiejun Huang |
Paper/Code |
| 2023 | UniverSeg | ICCV | UniverSeg: Universal Medical Image Segmentation Victor Ion Butoi; Jose Javier Gonzalez Ortiz; Tianyu Ma; Mert R. Sabuncu; John Guttag; Adrian V. Dalca |
Paper/Code |
| 2023 | EVP | CVPR | Explicit Visual Prompting for Low-Level Structure Segmentations Weihuang Liu; Xi Shen; Chi-Man Pun; Xiaodong Cun |
Paper/Code |
| 2023 | M2SNet | arXiv | M2SNet: Multi-scale in Multi-scale Subtraction Network for Medical Image Segmentation Xiaoqi Zhao; Hongpeng Jia; Youwei Pangl; Long Lv; Feng Tian; Lihe Zhang; Weibing Sun; Huchuan Lu |
Paper/Code |