NVIDIA DRIVE Thor (Tegra264) LLM Deployment Notes
Field notes recording how to cross-compile and deploy llama.cpp GPU inference on an NVIDIA DRIVE Thor automotive domain controller, including a troubleshooting guide and measured throughput at each optimization step.
This repository documents a real deployment of llama.cpp on an NVIDIA DRIVE Thor (Tegra264) automotive domain controller, from cross-compilation through NVFP4 quantization tuning. It records the hardware specifications, the steps taken, and measured results such as tokens-per-second at each stage, because official documentation for this board only covers the DriveOS SDK.
A troubleshooting guide indexed by symptom covers about 30 problems encountered across cross-compilation, model conversion, inference tuning and system or mounting issues, with the original error text included to make them searchable.
The notes cover benchmarks for several quantized models, a kernel optimization for NVFP4 matrix-vector multiplication, and a revised comparison of speculative-decoding drafting schemes, all measured directly on the hardware.