Back to directory

AI Research Platform

A self-evolving AI security research platform that publishes code-fix tasks through a honeypot, evaluates submitted fixes in a Docker sandbox with cheat detection, and exports the results as training data for LoRA fine-tuning.

AI Research Platform

AI Research Platform is a closed-loop AI security research system built from three components: a honeypot that publishes security-relevant coding tasks and captures how an AI submits a fix, an evaluation engine that runs each submission in a Docker sandbox and checks it against six kinds of cheating, and a training ground that turns the results into a standard dataset format and runs a LoRA fine-tuning pipeline.

The workflow publishes a task, an AI submits a code fix, the sandbox executes it, the evaluation engine scores it on multiple dimensions, failing cases are exported as training data, and a new fine-tuning round follows. The current automated agent in the repository generates fixes from a template; connecting a real model API would complete an unattended loop.

Anyone can contribute by submitting a fix through a GitHub issue template, which triggers an automatic CI evaluation, or by running an AI agent against the published tasks directly.

More in AI evals

See all 269
  • Provenance gate

    github.com

    A deterministic authorization gateway that sits between an AI agent and its tools, tracing where each parameter of a side-effect call came from, and blocking the call or asking a human when the source does not check out.

  • JevBench

    benchmarkheaven.com

    A reproducible benchmark from Benchmark Heaven that ranks Jev-class typed decision models on intelligence, calibration, speed, and cost, with a published harness, scoring rules, and results.

  • A co-evolutionary adversarial red-teaming tool for AI agents that uses quality-diversity evolutionary search to autonomously breed edge-case scenarios, testing whether an agent with tool access can be made to hallucinate, deadlock or leak funds.