Open Interface
A desktop application that lets an LLM such as GPT-4o control the computer directly, planning and executing keyboard and mouse actions from a plain-language request.
Open Interface is a desktop application that controls a computer using an LLM backend such as GPT-4o or Gemini. It sends the user's request to the model, which works out the steps needed, then Open Interface executes those steps by simulating keyboard and mouse input, sending the model updated screenshots as it goes so it can course-correct if something did not work as expected.
Demonstrated tasks include solving a daily word puzzle, drafting a meal plan in a document, and writing a simple web app. It ships prebuilt binaries for macOS, both Apple Silicon and Intel, Linux, and Windows, and on macOS it requires Accessibility and Screen Recording permissions to operate the keyboard, mouse, and take progress screenshots; it can also be run from source with Python 3.12.
It is written in Python and released under the GPL-3.0 license.