SAM-Adapter: Adapting SAM in Underperformed Scenes
A research implementation that adapts Meta's Segment Anything Model to downstream image-segmentation tasks, such as camouflaged-object and polyp detection, using adapters and prompts.
SAM-Adapter-PyTorch is the official code release for adapting Meta AI's Segment Anything Model, SAM, to downstream segmentation tasks where it otherwise underperforms, using lightweight adapters and prompts rather than retraining the whole model. It was presented at ICCV 2023, and the authors report strong results using the ViT-H version of SAM, with configurations also provided for the smaller ViT-L and ViT-B backbones for GPUs with less memory, at some cost to accuracy.
The codebase includes training and evaluation scripts driven by a YAML configuration file and needs a downloaded pretrained SAM checkpoint and dataset placed in specific local folders; the authors report training on four A100 GPUs due to SAM's memory requirements. Later branches in the same repository add support for adapting the newer SAM2 and SAM3 backbones. It is written in Python, requires PyTorch 1.13, and demonstrated the approach on medical polyp-segmentation data as well as its original camouflaged-object detection task.