There are some obvious signs that can instantly differentiate between regular and advanced AI users. One, for instance, is the use of voice AI for daily tasks. While majority users still toil away on their keyboard for the perfect prompt, a person proficient in the use of AI now simply speaks to it. A well-put ask within a conversation saves you time, efforts, and often delivers better results than a standalone text. Despite these advantages, Voice AI has largely been limited to the elite. OpenAI now plans to change that with three new real-time voice models in the API.
The three new audio models: GPT-Realtime-2, GPT-Realtime-Translate, and GPT-Realtime-Whisper, are meant to help developers create voice apps that can listen, reason, translate, transcribe, and take action while the conversation is still happening. OpenAI describes them as “a new generation of real-time voice models” that can work as people speak.
Here, we shall explore the 3 models in detail and understand why they can change the use of AI as we know it. But before we begin, here is what you need to know about real-time voice models.
What Are Realtime Voice Models?
Real-time voice models are AI models that can understand and respond to speech while the conversation is still happening.
Normally, voice AI works in steps. First, it records your audio. Then it converts speech to text. Then another model reads the text and prepares an answer. Then another system converts that answer back into speech. This works, but it can feel slow and unnatural. Real-time voice models reduce that gap.
They are built to listen, understand, and respond almost instantly. So instead of waiting for the full sentence or full audio file to finish, the AI can process speech as it comes in. This makes the conversation feel more natural, especially when users pause, interrupt, change direction, or ask follow-up questions.
In simple terms, real-time voice models make AI conversations feel like speaking to an actual assistant. And that very experience is what OpenAI is targeting with its new launches.
New OpenAI Voice Models
OpenAI has launched three new audio models in the API: GPT-Realtime-2, GPT-Realtime-Translate, and GPT-Realtime-Whisper. Together, they are built for apps where AI needs to work while a person is speaking. That means the AI can hold a conversation, understand context, translate speech, transcribe live audio, and even use tools during the interaction. OpenAI says these models are meant to help developers build voice experiences that feel more natural and can “take action in real time.”
Again, this matters because voice AI is moving beyond simple commands. A useful voice agent should not just hear words and reply. It should understand what the person wants, remember the context, handle corrections, use tools, and respond naturally. OpenAI says the goal is to move real-time audio from simple “call-and-response” systems to voice interfaces that can actually do work as the conversation unfolds.
Each of the 3 OpenAI voice models solves a specific part of that ambition.
GPT-Realtime-2
GPT-Realtime-2 is the main conversational voice model. It is built for voice agents that need to talk naturally, understand context, handle interruptions, and take action during a live conversation.
For example, a customer support agent built on GPT-Realtime-2 could understand a user’s problem, ask follow-up questions, check order details using a tool, and respond while the call is still going on.
GPT-Realtime-Translate
As the name suggests, GPT-Realtime-Translate is built for live speech translation. It can take speech in one language and translate it into another language while the person is still speaking. A demo shared by OpenAI shows the model in action, and I dare say it seems a revolutionary aid for translation needs in live conversations or addresses.
You can understand how this can be useful for global meetings, travel apps, multilingual customer support, education platforms, and live events where people need near-instant translation.
GPT-Realtime-Whisper
GPT-Realtime-Whisper is built for live transcription. It converts speech into text in real time instead of waiting for the full audio file to finish. Meaning you will see the words typed in front of you almost as soon as you have spoken them.
This can help with live captions, meeting transcripts, call notes, classroom recordings, interviews, and any app where spoken words need to become usable text quickly.
OpenAI Voice Models: Key Features
Just from their capabilities listed above, we can imagine how useful these 3 OpenAI voice models can turn out to be. Yet, there are many more features that enhance this utility.
1. Voice Agents That Can Take Action
GPT-Realtime-2 is built for voice agents that do more than reply. It can reason through a request, call tools, handle corrections, and continue the conversation while work is happening. OpenAI says this moves voice AI towards systems that can “actually do work.”
2. Better Handling of Interruptions and Corrections
Real conversations are not clean. People pause, change their minds, interrupt, or correct themselves. GPT-Realtime-2 is designed to handle these moments better, so the conversation does not break every time the user changes direction. OpenAI says it has “stronger recovery behavior” for such cases.
3. Longer Context for Complex Tasks
OpenAI has increased the context window from 32K to 128K for GPT-Realtime-2. In simple terms, the model can remember and work with more information during longer conversations. This is useful for complex voice workflows like support calls, travel planning, healthcare conversations, or workplace assistants.
4. Live Translation Across Languages
GPT-Realtime-Translate can translate speech from 70+ input languages into 13 output languages while keeping pace with the speaker. This makes it useful for multilingual customer support, global meetings, live events, education, and creator platforms.
5. Live Transcription While People Speak
GPT-Realtime-Whisper can convert speech into text while the person is still speaking. This can power live captions, meeting notes, call transcripts, classroom notes, and faster follow-up workflows.
6. More Control Over Tone and Reasoning
Developers can control how the voice agent sounds and how much reasoning effort it uses. For example, the model can sound calm during a support issue, empathetic when a user is frustrated, or more upbeat while confirming a task. Developers can also choose reasoning levels from minimal to x-high, depending on the task.
OpenAI Voice Models: Use-cases
Based on these abilities above, OpenAI’s 3 new voice models are sure to act as an absolute boon for the following tasks:
1. Customer Support Agents
A company can build voice agents that answer customer calls, understand the issue, ask follow-up questions, check order or account details, and complete basic actions during the call.
2. Live Meeting Translation
Teams working across countries can use GPT-Realtime-Translate to translate conversations while people are speaking. This can make global meetings easier without waiting for manual translation later.
3. Live Captions and Transcripts
GPT-Realtime-Whisper can be used to create live captions for calls, webinars, classes, interviews, and events. It can also turn the conversation into searchable text.
4. Travel and Booking Assistants
A travel app can use real-time voice models to help users search flights, compare hotels, change bookings, or ask travel questions through a natural voice conversation.
5. Healthcare Call Assistants
Healthcare providers can use voice agents to help with appointment scheduling, patient intake, follow-up calls, or basic information collection. The final medical judgement must still stay with doctors and trained staff.
6. Workplace Voice Assistants
Companies can build internal voice assistants that help employees find files, summarise meetings, create task lists, update records, or pull information from internal systems.
Pricing and Availability
All three models: GPT-Realtime-2, GPT-Realtime-Translate, and GPT-Realtime-Whisper, are available through OpenAI’s Realtime API. Developers can also test them in the OpenAI Playground before building them into apps.
- GPT-Realtime-2: $32 per 1M audio input tokens, $0.40 per 1M cached input tokens, and $64 per 1M audio output tokens.
- GPT-Realtime-Translate: $0.034 per minute.
- GPT-Realtime-Whisper: $0.017 per minute.
Conclusion
OpenAI’s new real-time voice models clearly show where voice AI is heading next.
It is no longer just about asking a question and getting a spoken reply. With the new GPT voice models, developers can now build voice apps that are more action-oriented in nature. All of this, within the context of a seamless conversation.
In practicality, imagine this as a support call becoming faster. A meeting becoming multilingual. A classroom getting live transcripts. A travel app being more conversational. A workplace assistant moving from text chat to natural speech.
Of course, this does not mean every voice agent will suddenly become perfect. Developers will still need strong guardrails, clear user disclosure, privacy controls, and human review in sensitive areas like healthcare, finance, and legal support.
But the direction is clear. From a passive speech interaction to active real-time assistance, and OpenAI wants to be at the helm of it.
Login to continue reading and enjoy expert-curated content.
