TheWhisper
Optimized, open-weight Whisper speech-to-text models with streaming inference engines for NVIDIA GPUs and on-device Apple Silicon, aimed at low-latency real-time transcription.
TheWhisper re-releases Whisper speech-to-text models as open Hugging Face weights with a flexible chunk size, in place of the original model's fixed thirty-second window, with fine-tuned variants supporting ten, fifteen, twenty and thirty second inference windows. It pairs these weights with a high-performance inference engine for NVIDIA GPUs and a separate CoreML engine for macOS and Apple Silicon aimed at very low power draw, alongside a streaming implementation for both.
The repository includes a local REST API with example frontends in plain JavaScript and Electron, a companion Electron note-taking demo application built on top of it, and a tutorial for building a similar on-device desktop application. Benchmarks cover latency, memory use, power draw and speech-recognition accuracy.
It is aimed at low-latency, low-power, scalable streaming transcription for uses such as real-time captioning, live meetings, voice interfaces and edge deployments, and is written in Python and released under the MIT license.