# NVIDIA DRIVE Thor (Tegra264) LLM Deployment Notes

Field notes recording how to cross-compile and deploy llama.cpp GPU inference on an NVIDIA DRIVE Thor automotive domain controller, including a troubleshooting guide and measured throughput at each optimization step.

This repository documents a real deployment of llama.cpp on an NVIDIA DRIVE Thor (Tegra264) automotive domain controller, from cross-compilation through NVFP4 quantization tuning. It records the hardware specifications, the steps taken, and measured results such as tokens-per-second at each stage, because official documentation for this board only covers the DriveOS SDK.

A troubleshooting guide indexed by symptom covers about 30 problems encountered across cross-compilation, model conversion, inference tuning and system or mounting issues, with the original error text included to make them searchable.

The notes cover benchmarks for several quantized models, a kernel optimization for NVFP4 matrix-vector multiplication, and a revised comparison of speculative-decoding drafting schemes, all measured directly on the hardware.

Website: https://github.com/isenlink/thor-fp8-llm