LingJing Embodied Intelligence
An open simulation stack for LLM and vision-language-model driven embodied agents in large 3D environments, combining a FastAPI and LangGraph backend with a Cesium-powered Unreal Engine 5.5 plugin for heterogeneous simulated vehicles, drones, robots and other actors.
LingJing is a simulation stack for embodied AI agents that combines a FastAPI and LangGraph backend with a Cesium-powered Unreal Engine 5.5 plugin, connecting heterogeneous simulators, geospatial assets, model services and agents through a shared urban physics environment. The backend handles task orchestration, LLM and vision-language-model calls, Gym-style environment loops, communication with the engine, and data recording, while the Unreal plugin provides the 3D world, equipment actors, cameras, UI, image capture and telemetry.
Each simulation step runs a perceive-think-act-communicate loop and records its outputs for evaluation and visualization, supporting centralized, hub-and-spoke and broadcast multi-agent coordination patterns, a Gym-style reset, step, run and close lifecycle with pluggable evaluators, and both OpenAI-compatible and Anthropic-compatible model providers. A mock engine lets the backend be developed without Unreal Engine installed, and the world can include drones, vehicles, robot dogs, ships and other task actors.
The backend runs on Python 3.10 or newer with a virtual environment and a local configuration file; large Unreal assets are distributed separately through Hugging Face, with only source code, configuration templates, tests and compact example data kept in this repository.