Deploying Large Language Models Locally: A Chinese Tutorial
A comprehensive Chinese-language tutorial on deploying large language models locally, covering how transformer inference and quantization work, choosing hardware, and comparing deployment tools such as LM Studio, Ollama, llama.cpp, vLLM and TensorRT-LLM.
This guide explains local LLM deployment from first principles for readers with no machine-learning or high-performance-computing background, first covering how tokens, transformers, prefill and decode, KV caching, prefix caching and speculative decoding work, and then what actually determines inference speed: memory capacity, bandwidth, compute, the difference between latency and throughput, how much memory a model actually needs, and how mixture-of-experts models can have large total parameter counts but small active parameter counts.
Further sections cover numeric precision and quantization formats such as FP8, FP4, GGUF, GPTQ and AWQ and which GPU architectures support which precision; how to choose a computer for a given budget between a discrete-GPU route and a large-unified-memory route; parallelism strategies such as data, tensor, pipeline and expert parallelism and how to choose among them in vLLM; how to choose a deployment tool for a given use case, from LM Studio for a first graphical setup through Ollama, llama.cpp, vLLM, TensorRT-LLM, NVIDIA NIM and OpenVINO; and how to estimate a three-year cost of running a given model locally versus paying per token.
It also lists common misconceptions, such as assuming a 20 GB model file will always fit in 24 GB of VRAM, or that four GPUs' capacity and performance simply add together.