Chatty
A browser-based chat interface for open-source language models that runs entirely on-device using WebGPU, including local retrieval-augmented question answering over uploaded files.
Chatty is a browser application that runs large language models such as Gemma, Llama 2, Llama 3 and Mistral locally on a user's own hardware using WebGPU, so no server-side processing or account is required and the app keeps working offline once a model has downloaded. Uploaded files, including PDFs and non-binary code files, can be queried locally using Transformers.js embeddings and an in-memory vector store, without leaving the browser.
Other features include persisted chat history, exporting conversations as JSON or markdown, voice input, response regeneration, and light and dark modes. It is a Next.js application, requires a WebGPU-capable browser such as Chrome or Edge by default, with experimental support in Firefox Nightly, and can be run locally or tried on its hosted demo site.