DeepSeek V4 Flash on 8 GB RAM — CPU-only, NVMe-backed
A research fork that runs a 78.62 GiB DeepSeek V4 Flash GGUF model on a Linux laptop with only 7.7 GiB of RAM and no GPU, using memory-mapped NVMe demand paging instead of loading the full model into memory.
This fork demonstrates CPU-only execution of a DeepSeek V4 Flash GGUF checkpoint far larger than the machine's physical RAM, by keeping the weights file-backed on NVMe storage and letting Linux fault model pages into memory on demand rather than claiming the full 78.62 GiB is resident at once. A controlled public benchmark measured a 5.33-second cold first-token time, a maximum resident set size of about 6.2 GiB, and zero process swaps.
It adds CPU support for GGUFs whose dense attention projections are stored as Q4_K, and an opt-in CPU demand-paging mode for systems where the model file is much larger than physical RAM. Ongoing research measures weight-stationary and expert-union execution strategies for mixture-of-experts layers with controlled, reproducible experiment records; the project explicitly documents what its results do not claim, including production-grade inference speed or normal chat latency.