Qwen35.java
A zero-dependency, single-file inference engine for Qwen 3.5 language models written in pure Java, supporting model sizes from 0.8B to 35B (including a mixture-of-experts variant), several GGUF quantizations, and GraalVM native-image compilation.
Qwen35.java is a single-file, dependency-free Java implementation of inference for the Qwen 3.5 model family, based on the earlier llama3.java project. It supports every tested Qwen 3.5 size - 0.8B, 2B, 4B, 9B, 27B and the 35B-A3B mixture-of-experts variant - reading GGUF-format checkpoints in F16, BF16, F32 and several K-quant and legacy quantizations, with matrix-vector kernels built on Java's Vector API and optional GraalVM Native Image compilation with ahead-of-time model preloading for fast time-to-first-token.
It offers both a chat mode and a single-prompt mode from the command line, and runs directly through jbang with no manual git clone or model download step: jbang fetches and caches the roughly 5 GB model file itself on first run. A Makefile is also provided to build a plain jar.
It requires Java 21 or later for the MemorySegment-based memory-mapping feature the loader depends on.