# autoresearch-qwen

An autonomous research loop that has an AI agent iterate on a fixed vision-language model's training code, evaluate each change against the official DocVQA benchmark, and keep only changes that produce a real measured improvement.

autoresearch-qwen scopes agentic training research to one concrete, reproducible contract: the base vision-language model, the official DocVQA dataset splits, and the evaluator are all fixed, while an agent is expected to edit only the training script. Each iteration downloads the dataset and model snapshot, trains with the agent's current training code, evaluates with the fixed evaluator against the full validation split using mean ANLS as the metric, and keeps or discards the change based on that real, unmoved score.

It maintains two branches for different hardware: a main branch targeting NVIDIA and CUDA multi-GPU setups using distributed training with DeepSpeed configs, and a second branch targeting Apple Silicon through MPS, preserved from a formerly separate repository. The project is explicitly inspired by an existing autoresearch pattern but narrowed to this one public vision-language benchmark.

It is a Python project released under the MIT license.

Website: https://github.com/wadeKeith/autoresearch-qwen