DGX Spark Unsloth Lossless Speedup
A set of optimizations for Unsloth on NVIDIA DGX Spark hardware that speeds up LoRA and full fine-tuning of Qwen3.5 models by roughly 7 to 8 times, with a loss curve verified bit-identical to the unoptimized baseline within BF16 precision.
This project takes stock Unsloth running on NVIDIA DGX Spark from barely usable to 7.67x faster LoRA fine-tuning and 8.35x faster full fine-tuning on a real multimodal Qwen3.5-2B model, verifying the loss curve stays bit-identical to the unoptimized reference within BF16 precision; measured throughput reached 3,461 tokens per second for LoRA and 3,173 tokens per second for full fine-tuning on 24,315 real chat examples.
On the same training job, the optimized setup on DGX Spark hardware the user already owns matches the wall-clock time of a rented H100 GPU while costing nothing per run beyond electricity, against an estimated cost of roughly $50 for the equivalent H100 rental.
A three-command, zero-config wizard builds an optimized Docker image on first run, then walks through picking a model, validating data, a time estimate, training and merging, producing a checkpoint ready to serve. Built with Python, CUDA and Triton. MIT licensed.