Getting Started with OmniVoice-Studio – KDnuggets

# Introduction

 
You paste a paragraph of text into ElevenLabs, press Generate, and watch the character counter tick down. The free tier is gone before you finish testing. The Creator plan is $22 a month. The Pro plan is $99. And every audio file you generate leaves your machine and ends up on their servers, which matters the moment your content is sensitive, proprietary, or simply yours.

OmniVoice Studio is built on a different premise: everything runs on your hardware. Voice cloning, video dubbing, real-time dictation, voice design — all of it local, all of it free for personal use, no API key required, no usage counter. The project describes itself as “the open-source ElevenLabs alternative,” and that’s accurate, though the language coverage alone makes the comparison interesting: ElevenLabs supports 32 languages. OmniVoice Studio supports 646.

The project has accumulated 7.1k GitHub stars and 1.1k forks. The latest release, v0.2.7, shipped May 3, 2026, includes pre-built installers for macOS, Windows, and Linux. This article covers the full path from install to your first generated audio.

Beta notice: OmniVoice Studio is in active beta. Things can break between releases. For the most current fixes, cloning from source and running bun run desktop-prod is the recommended path over pre-built installers.

# What OmniVoice Studio Is and Why It Was Built

 
The simplest framing is this: OmniVoice Studio gives you a professional voice AI desktop app that never phones home. No accounts, no subscriptions, no cloud calls during inference. Your reference audio, your scripts, your generated files — they stay on your machine.

Here is how the feature set and pricing compare directly to ElevenLabs:

Under the hood, OmniVoice Studio is a Tauri desktop application — a Rust-based framework that wraps a React frontend and a FastAPI backend with 97 API endpoints. Persistent state lives in SQLite. The AI pipeline is built on four open-source components that do the actual work:

1. WhisperX handles transcription, word-level speech recognition, and alignment.
2. Demucs (from Meta) handles vocal isolation, separating speech from music and background noise.
3. OmniVoice from k2-fsa is the zero-shot diffusion text-to-speech (TTS) engine — the model that makes cloning work from a 3-second clip across 646 languages.
4. Pyannote handles speaker diarization, identifying who said what in a multi-speaker recording, which is what makes automatic voice assignment in the dubbing pipeline possible.

GPU acceleration is auto-detected at launch. You don’t configure anything: OmniVoice reads your hardware and routes accordingly to CUDA (NVIDIA), MPS (Apple Silicon), ROCm (AMD), or CPU. If you have under 8 GB VRAM, the TTS model offloads to the CPU automatically during transcription. The pipeline still runs, just slower.

# System Requirements

 
Before installing, check that your machine meets the minimum specs. The app will run below these, but you will notice it.

One thing worth knowing: you do not need a GPU to use OmniVoice Studio. The entire pipeline runs on CPU. TTS synthesis is roughly 3x slower without a GPU, and transcription of long videos will take longer, but for short voice clones and dictation, the CPU path is perfectly usable. Apple Silicon Macs are the sweet spot for GPU-less users; the app automatically picks MLX-optimized Whisper and TTS backends that use the Apple Neural Engine and Metal Performance Shaders, giving roughly 2x the throughput of the CPU path.

# Installing OmniVoice Studio

 
Pick the section for your operating system and follow it from top to bottom. The install sequence is the same across platforms: clone, install frontend dependencies with Bun, and launch. The differences are in the prerequisites.

// Installing on macOS

Prerequisites:

• macOS 12 (Monterey) or newer — Apple Silicon or Intel
• Python 3.11+
• Bun (the JavaScript runtime used to build the frontend)
• Xcode Command Line Tools
• FFmpeg

Install them in order:

# 1. Install Python via Homebrew (or use pyenv if you manage multiple versions)
brew install python@3.11

# 2. Install Bun
curl -fsSL https://bun.sh/install | bash

# 3. Install Xcode Command Line Tools
xcode-select --install

# 4. Install FFmpeg (used by the dubbing and capture pipelines)
brew install ffmpeg

Then clone and run:

# Clone the repository
git clone https://github.com/debpalash/OmniVoice-Studio.git
cd OmniVoice-Studio

# Install frontend dependencies
bun install

# Launch the app
bun run desktop-prod

The first launch is slower than every subsequent one. It builds the Tauri shell, creates the Python virtual environment via uv, syncs all Python dependencies, and downloads model weights — approximately 2.4 GB. The splash screen shows live progress for each step. Once it completes, the full UI opens.

Pre-built DMG users: Download the latest DMG from the Releases page, mount it, and drag OmniVoice Studio into /Applications. If the first launch shows “app is damaged and can’t be opened,” that is macOS Gatekeeper reacting to an unsigned app. The developer-ID signing and notarization pipeline is tracked for v0.4. For now, clear the quarantine attribute with a single terminal command:

# Remove the Gatekeeper quarantine attribute.
# Run this once after installing. The app is open source -- verify the
# SHA-256 checksum on the Releases page against the .dmg.sha256 file
# before running this if you want to confirm the download is clean.
xattr -cr "/Applications/OmniVoice Studio.app"

// Installing on Windows

Prerequisites:

• Windows 10 (21H2 or newer) or Windows 11, x64
• Python 3.11+
• Microsoft C++ Build Tools (required by pyannote.audio and occasional torch wheel rebuilds)
• Bun
• FFmpeg

Install them from a regular (non-admin) PowerShell:

# Install Python via winget
winget install Python.Python.3.11

# Install Microsoft C++ Build Tools
# Download from https://visualstudio.microsoft.com/visual-cpp-build-tools/
# Select "Desktop development with C++" workload during install

# Install Bun
powershell -c "irm bun.sh/install.ps1 | iex"

# Install FFmpeg via winget
winget install Gyan.FFmpeg

Then clone and run (still in PowerShell):

git clone https://github.com/debpalash/OmniVoice-Studio.git
cd OmniVoice-Studio
bun install
bun run desktop-prod

Windows-specific note (Triton/torch.compile OOM): On Windows, certain TTS engines (notably CosyVoice paths) trigger torch.compile kernel compilation on the first synthesis call. On machines with under 16 GB VRAM, this can OOM before any audio renders, surfacing as OutOfMemoryError: CUDA out of memory. The fix is in Settings → Performance: toggle “Disable torch.compile (Windows)” on. From the command line, set the environment variable before launching:

# Disable torch.compile to avoid OOM on first synthesis (Windows only).
# This falls back to the eager-mode kernel path -- slightly slower peak
# throughput, but the engine actually loads on low-VRAM machines.
$env:TORCH_COMPILE_DISABLE = "1"
bun run desktop-prod

Pre-built MSI users: Download the latest MSI from the Releases page, run the installer, and find OmniVoice Studio in the Start menu.

// Installing on Linux

Prerequisites (Debian/Ubuntu):

# Install Python
sudo apt install python3.11

# Install Bun
curl -fsSL https://bun.sh/install | bash

# Install FFmpeg
sudo apt install ffmpeg

# Install GTK/WebKit dependencies required by the Tauri desktop shell
sudo apt install \
  libwebkit2gtk-4.1-dev \
  libayatana-appindicator3-dev \
  librsvg2-dev \
  libssl-dev \
  libxdo-dev \
  build-essential

Prerequisites (Fedora):

sudo dnf install python3.11 ffmpeg-free
curl -fsSL https://bun.sh/install | bash
sudo dnf install webkit2gtk4.1-devel libappindicator-gtk3-devel librsvg2-devel openssl-devel

Clone and run:

git clone https://github.com/debpalash/OmniVoice-Studio.git
cd OmniVoice-Studio
bun install
bun run desktop-prod

Pre-built AppImage users:

# Download the AppImage from the Releases page, then:
chmod +x OmniVoice.Studio_*.AppImage
./OmniVoice.Studio_*.AppImage

# If you see a white screen on Fedora 44+ or Ubuntu 24.04, set this:
WEBKIT_DISABLE_COMPOSITING_MODE=1 ./OmniVoice.Studio_*.AppImage

The white screen on newer distros is a compositing regression in WebKitGTK 2.44/2.46. v0.3+ of the AppImage autodetects this and sets the flag automatically. The manual environment variable path is the fallback for source installs.

Pre-built .deb users:

sudo apt install ./OmniVoice.Studio_*.amd64.deb
omnivoice-studio

Docker (backend only):

For headless server use or team deployments where the desktop GUI isn’t needed, OmniVoice Studio ships a Docker path that runs just the FastAPI backend and its 97 API endpoints:

# Clone the repo if you haven't already
git clone https://github.com/debpalash/OmniVoice-Studio.git
cd OmniVoice-Studio

# Build and start the backend container
docker compose -f deploy/docker-compose.yml up

The backend API is then available at http://localhost:8000. The full API reference lives in the repo’s docs/ directory. This path is useful when integrating OmniVoice capabilities into a pipeline without running a desktop session.

# Setting Up Your Hugging Face Token

 
This step is optional for basic use, but required for two features: speaker diarization (the pyannote/speaker-diarization-3.1 model is gated on Hugging Face) and the larger voice-design engines.

You need a free Hugging Face account and a read token. Once you have one:

Option 1 — Through the app (recommended): Open Settings → API Keys, paste your hf_... token, and save. The app writes it to OmniVoice’s encrypted SQLite store and to the canonical huggingface_hub location, so every subprocess the app spawns picks it up automatically.

Option 2 — Environment variable:

# macOS / Linux -- add to ~/.zshrc or ~/.bashrc
export HF_TOKEN=hf_your_token_here
source ~/.zshrc

# Windows PowerShell -- writes to user-scope environment
# Use this, not setx. setx truncates values over 1024 chars and
# doesn't propagate to the current shell session.
[Environment]::SetEnvironmentVariable("HF_TOKEN", "hf_your_token_here", "User")

You also need to accept the model terms on the Hugging Face model page before downloading. Visit pyannote/speaker-diarization-3.1 and accept the gated model access request. This is a one-time step.

# Cloning a Voice

 
Voice cloning is the core feature and the one most people install OmniVoice for. The model powering it is OmniVoice from k2-fsa — a diffusion-based TTS system trained on 646 languages that operates zero-shot, meaning there is no fine-tuning step. You provide a reference clip at inference time, and the model adapts to the speaker’s voice on the fly.

How to clone a voice:
Navigate to the Voice Clone tab. You have two options for the reference audio: record directly in the app by clicking the microphone button, or upload an existing audio file. Either way, 3 to 10 seconds of clear speech is enough.

Then:

1. Upload or record your reference audio clip.
2. Select the target language from the dropdown (646 available).
3. Type or paste the text you want synthesized in the text field.
4. Click Generate.

OmniVoice processes locally, generates the audio, and plays it back for preview. You can export to MP3, WAV, or FLAC from the export button.

What makes a good reference clip: Background noise is the biggest quality killer. A clip recorded in a quiet room, with the speaker speaking naturally, will clone better than a noisy excerpt from a phone call. Avoid clips with background music; if that’s all you have, run it through the Vocal Isolation tab first (covered below) to strip the background before using it as a reference.

If the output sounds slightly off — robotic consonants, wrong rhythm — try a longer or different reference clip before assuming an engine issue. The zero-shot model is sensitive to the quality of the reference audio.

# Dubbing a Video

 
The dubbing pipeline is the most complex thing OmniVoice Studio does, and watching it run end-to-end is genuinely impressive. You give it a video — either a YouTube URL or a local file — and it transcribes the speech, translates it to your target language, clones the original speaker voices, synthesizes the dubbed audio in the cloned voices, and muxes everything back into an MP4. Locally. No upload.

The pipeline uses WhisperX for transcription, Pyannote for speaker diarization (identifying which voice belongs to which speaker), the OmniVoice model for synthesis, and Demucs to separate the original speech from background audio so the background can be preserved under the dubbed track.

How to dub a video:

1. Navigate to the Dub tab. Paste a YouTube URL or click the upload button to select a local file. Choose the target language. Click Start Dub.
2. The progress bar shows each stage as it runs: download (for YouTube), transcription, diarization, translation, synthesis, and mux. For a 5-minute video on a machine with a GPU, the full pipeline typically takes 8 to 12 minutes. CPU-only will take longer.
3. When complete, the dubbed MP4 is available in the Projects panel alongside the SRT subtitle file, the isolated stems, and the original transcription.
4. Batch queue: If you have multiple videos, drop them all into the queue by clicking Add to Queue for each one, then click Run All. The conductor processes them sequentially using GPU execution with a live progress bar per job. You can add more jobs while the queue is running.

# Designing a Voice

 
Voice design is for creating a new voice from scratch when you don’t have a reference clip. Navigate to the Voice Design tab, and you’ll find sliders and controls for gender, age, accent, pitch, speed, emotion, and dialect.

The design process is iterative: adjust the controls, hit Preview, listen, adjust again. The A/B Comparison button lets you lock one voice configuration as version A, tune the controls further to create version B, and toggle between them while the same sample text plays, so you’re comparing voices directly rather than relying on memory.

Once you’re satisfied with a voice, save it to your Voice Gallery with a name and tags. Saved voices are then available as targets in the Voice Clone tab — you can synthesize new audio in a saved designed voice without needing a reference clip each time.

# Using the Dictation Widget

 
The dictation widget is a system-wide transcription tool that works from any application without switching windows. The global hotkey is Cmd+Shift+Space on macOS and Ctrl+Shift+Space on Windows and Linux.

Press the hotkey from any app — your code editor, a browser text field, a notes app, anywhere. A small frameless floating window appears. Start speaking. The widget streams your speech through WhisperX’s ASR engine over a local WebSocket connection, transcribes in real time, auto-pastes the result into whatever application was focused before the widget opened, and disappears.

The entire flow — trigger, speak, paste — takes a few seconds. There is no window switching, no copy-paste step.

1. Configuring the hotkey: If Cmd+Shift+Space conflicts with another application on your system, open Settings → Dictation and change the hotkey binding to any combination that doesn’t clash. The new binding takes effect immediately without restarting the app.
2. What the widget does not do: It does not keep a transcription history. Each activation transcribes and pastes, then discards the audio. For longer transcription sessions where you want to review and edit a full transcript, use the main Transcription tab instead — that records to a file and shows a full editable transcript.

# Choosing a TTS Engine

 
OmniVoice Studio ships six TTS engines. The default, OmniVoice, covers 600+ languages and handles voice cloning and instructed generation. The others exist for specific reasons, and switching takes ten seconds via Settings → TTS Engine or the OMNIVOICE_TTS_BACKEND environment variable.

For English-only use on a machine without a GPU, KittenTTS and MOSS-TTS-Nano run near real-time on CPU. For Apple Silicon, switching to MLX-Audio gives you the fastest inference available on M-series hardware using the Apple Neural Engine directly. CosyVoice 3 is the choice when you want instructed generation — describing the voice style in natural language rather than dialing sliders.

Switch via the environment variable if you want to set it system-wide:

# Run OmniVoice Studio with CosyVoice 3 as the active TTS backend.
# Valid values: omnivoice, cosyvoice, mlx-audio, voxcpm2, moss-tts-nano, kittenTTS
export OMNIVOICE_TTS_BACKEND=cosyvoice
bun run desktop-prod

# Windows equivalent
$env:OMNIVOICE_TTS_BACKEND = "cosyvoice"
bun run desktop-prod

Adding a custom engine: OmniVoice uses a built-in backend registry. To plug in your own TTS engine, subclass TTSBackend in backend/services/tts_backend.py and add it to the _REGISTRY dictionary at the bottom of that file. The README documents the interface as roughly 50 lines of Python. The CONTRIBUTING guide covers the full development setup.

# Using OmniVoice Studio via the MCP Server

 
OmniVoice Studio ships a Model Context Protocol (MCP) server, which means you can call its TTS and dubbing capabilities from Claude Desktop, Cursor, or any MCP-compatible client — without opening the desktop app at all.

This is useful when you want to generate voice audio from inside an AI coding session or automate voice generation as part of a pipeline that’s already running in an MCP-capable tool.

The MCP server is exposed on localhost:8765 when OmniVoice Studio is running. To connect it to Claude Desktop, add the following to your claude_desktop_config.json:

{
  "mcpServers": {
    "omnivoice": {
      "command": "npx",
      "args": ["-y", "@omnivoice/mcp-server"],
      "env": {
        "OMNIVOICE_API_URL": "http://localhost:8765"
      }
    }
  }
}

Once connected, Claude Desktop can call OmniVoice tools directly. For example, you can type “Generate audio of this paragraph in a female voice with a British accent,” and Claude will route the request to OmniVoice’s local API, synthesize the audio, and return the file path.

The MCP server exposes the core capabilities — TTS generation, voice cloning with a reference file, and dubbing job creation — as named tools that any MCP client can discover and invoke. See the docs/ directory in the repo for the full tool schema.

# Conclusion

 
OmniVoice Studio makes a practical case for local-first voice AI. Not because cloud tools are bad, but because 646 languages, no usage meter, and audio that never leaves your machine add up to something genuinely different. The setup — one install sequence, a 2.4 GB model download, and an optional Hugging Face token — is a one-time investment. Everything after that is just using the tool.

It’s in active beta, and some edges are rough. But the core pipeline — cloning, dubbing, dictation, design — works, the community is responsive, and releases have been coming regularly since launch. For developers, content creators, and researchers who work with audio and care about where their data goes, it’s worth having locally.
 
 

Shittu Olumide is a software engineer and technical writer passionate about leveraging cutting-edge technologies to craft compelling narratives, with a keen eye for detail and a knack for simplifying complex concepts. You can also find Shittu on Twitter.