afd-plugin
An experimental vLLM plugin that adds Attention-FFN Disaggregation, splitting attention and feed-forward computation across separate GPU or Ascend NPU workers without modifying vLLM itself.
afd-plugin is an external plugin for the vLLM inference engine that implements Attention-FFN Disaggregation (AFD): running the attention and feed-forward portions of a model on separate, role-specific workers. It installs through vLLM's general_plugins entry point and an --additional-config flag, and provides its own worker classes, model runners, model wrappers, connectors, configuration validation, and compatibility shims, targeting vLLM v0.26.0 for CUDA and Ascend NPU deployments without changing the vLLM source tree.
Current support includes AFD-aware wrappers for DeepSeek V2/V3/V3.2 and Qwen3 and Qwen3.5/3.6 MoE model families, eager and FULL_DECODE_ONLY graph execution, native dual-batch overlap with two ubatches on CUDA, and a synchronous Ascend execution path, along with connector-driven execution across role workers. The project describes itself as still experimental and in need of more large-scale, cross-hardware testing.