PRISM
A research toolbox for training and evaluating multimodal large language models under continual instruction tuning, where a model learns new tasks in sequence without forgetting earlier ones.
PRISM is a plug-in, reproducible toolbox for training and evaluating multimodal large language models under multimodal continual instruction tuning, the problem of learning a sequence of tasks without catastrophic forgetting of earlier ones. A single entry point orchestrates sequential task training, inference and evaluation across multiple benchmarks and continual-learning methods.
New continual-learning strategies can be added as independent plugins without changing the underlying model code, and it currently supports three benchmarks and twelve continual-learning methods spanning rehearsal, regularization, prompt-based, LoRA-based and mixture-of-experts approaches, alongside a zero-shot baseline. It is written in Python and MIT licensed, and accompanies a series of published research papers.