CosyVoice3 LoRA Fine-Tuning

This toolkit adds PEFT integration for CosyVoice3's Qwen2-based LLM backbone, selective layer freezing with configurable unfreezing, LoRA-aware checkpoint save and load that stores adapters instead of full weights, and overfitting detection during training, since the upstream training code only saves checkpoints and never gates on overfitting.

It was validated on an RTX 3090 Ti (24GB) training on the IMDA NSC Singaporean-English dataset, reaching a best checkpoint at epoch 12 with 2.16 million trainable parameters, 0.44 percent of the 506-million-parameter full model, an 8.3MB checkpoint versus 4GB for full fine-tuning, and roughly 1.8 times faster training at 47 percent less peak VRAM than the project's earlier full-SFT run. Standard PyTorch and merged-weight vLLM inference were validated end to end.

A preregistered long-form quality test found the LoRA adapter materially worse than the base model on word-error rate under matched conditions, so the project states perceptual ranking between them is still pending.