HER-Seg: Holistically Efficient Segmentation for High-Resolution Medical Images

ArXiv Paper

Qing Xu¹ Zhenye Lou² Chenxin Li³ Yue Li^1,4 Xiangjian He^1✉

Tesema Fiseha Berhanu¹ Rong Qu⁴ Wenting Duan⁵ Zhen Chen⁶

¹UNNC   ²Sichuan University   ³CUHK   ⁴University of Nottingham   ⁵Univeristy of Lincoln   ⁶HKISI, CAS  

---

📰News

• [2025.04.08] The pre-print paper is now available! Fine-tuning the model with 1024×1024 image resolution and 16 batch sizes only needs 9.4GB GPU memory (~1 GTX1080ti or 1 RTX3060), 4 batch sizes only needs 2.6GB GPU memory (~1 GTX1060 3G). We hope everyone can enjoy our model.
• [2025.04.08] The checkpoint pre-trained on 0.1% SA-1B dataset is now available!

🛠Setup

git clone https://github.com/xq141839/HER-Seg.git
cd HER-Seg
conda create -f HER-Seg.yaml

Key requirements: Cuda 11.8+, PyTorch 2.0+

📚Data Preparation

• Distillation Pretraining (Optional): https://ai.meta.com/datasets/segment-anything-downloads/ ID: sa_000000.tar
• Dermoscopy: https://challenge.isic-archive.com/data/#2018
• X-ray: https://www.kaggle.com/code/nikhilpandey360/lung-segmentation-from-chest-x-ray-dataset/input
• Fundus: https://refuge.grand-challenge.org/
• Ultrasound: https://www.kaggle.com/datasets/jarintasnim090/udiat-data
• Microscopy: https://www.kaggle.com/competitions/data-science-bowl-2018
• CT: https://github.com/Beckschen/TransUNet/blob/main/datasets/README.md
• Colonoscopy: https://figshare.com/articles/figure/Polyp_DataSet_zip/21221579?file=37636550

The data structure is as follows.

HER-Seg
├── datasets
│   ├── image_1024
│     ├── ISIC_0000000.png
|     ├── ...
|   ├── mask_1024
│     ├── ISIC_0000000.png
|     ├── ...
|   ├── isic_data_split.json
|   ├── ...

🎪Segmentation Model Zoo

We provide all pre-trained models here.

Teacher Model	Pretraining Data	Checkpoints
SAM	0.1% SA-1B	Link
SAM2	0.1% SA-1B	TBA
SAM	1% SA-1B	TBA
SAM2	1% SA-1B	TBA

📜Citation

If you find this work helpful for your project,please consider citing the following paper:

@misc{xu2025hrmedsegunlockinghighresolutionmedical,
      title={HRMedSeg: Unlocking High-resolution Medical Image segmentation via Memory-efficient Attention Modeling}, 
      author={Qing Xu and Zhenye Lou and Chenxin Li and Xiangjian He and Rong Qu and Tesema Fiseha Berhanu and Yi Wang and Wenting Duan and Zhen Chen},
      year={2025},
      eprint={2504.06205},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2504.06205}, 
}