Google just took Gemini 3.5 Transcribe generally available inside the Gemini API, and it is not a single model update. It is two purpose-built speech-to-text models built on Gemini's audio understanding, and they land right as more builders are wiring voice into their products, often stitched together with the same AI SDK powering most teams' AI features. If you are choosing a transcription API for a new feature, here is what actually shipped and how to think about it next to Whisper.
Key Takeaways
- Google shipped two distinct models: gemini-3.5-transcribe for high-accuracy, non-streaming transcription and gemini-3.5-transcribe-live for sub-second streaming, instead of one generic endpoint.
- Independent testing puts word-error rates around 2.6% for non-streaming and 4.0% for streaming audio, with automatic language detection across more than 85 languages.
- Built-in speaker diarization, word-level timestamps, and custom vocabulary biasing for up to 1,000 terms replace the glue code most teams currently bolt onto Whisper.
- Smart transcription mode auto-strips filler words and self-corrections, cutting the manual post-processing step most transcription pipelines still need.
- Third-party platforms including LiveKit, Pipecat, and Vercel have already integrated the Live API, a signal the streaming ecosystem is not starting from zero.
- For product managers, the two-model split turns a single transcription checkbox into two distinct specs, and vocabulary biasing lets you scope jargon-heavy verticals like legal or medical calls without a fine-tuning project.
Learn this hands-on
Become a 10x PM by learning how to use Claude Code in your daily work as a Product Manager, through 3 highly efficient live sessions of 1h30. Join the Claude Code for PMs live cohort.
Two Models, Not One
Google shipped gemini-3.5-transcribe for high-accuracy, non-streaming transcription and gemini-3.5-transcribe-live for low-latency, bidirectional streaming over WebSockets through the Live API. That split matters for builders: a meeting-notes tool and a live voice assistant have very different latency needs, and Google chose to ship two models instead of forcing everyone through one generic endpoint.
Both models are built on Gemini's broader audio-understanding capabilities rather than adapted from an older speech-recognition stack, which according to Google gives them stronger context awareness than conventional transcription systems.
Accuracy: Where the Numbers Land
Independent coverage cites word-error rates around 2.6% for non-streaming audio and 4.0% for streaming audio, which puts Gemini 3.5 Transcribe in competitive territory with dedicated voice-AI providers (9to5Google). The models handle automatic, utterance-based language detection across more than 85 languages, and they are built to deal with regional accents and dialects without the app having to declare a language up front.
Whisper, by contrast, ships as a single model without a dedicated real-time streaming variant, native speaker diarization, or built-in vocabulary biasing. Most teams running Whisper in production bolt those pieces on themselves with separate libraries or services. Gemini 3.5 Transcribe folds them into the API call, which is the real difference for builders: it is less about a single accuracy number and more about how much glue code you avoid writing. It is the same specialization trend we tracked when comparing Gemini 3.0 Pro against Claude Sonnet and when we put GPT-5.6 side by side with Claude Fable 5: providers are shipping purpose-built variants instead of one generalist model.
The Features That Actually Matter for Builders
Three additions stand out in the changelog:
- Speaker diarization and word-level timestamps, so the API can attribute segments to individual speakers and pinpoint when each word was said. That is the feature meeting-transcript, call-center, and captioning tools need most.
- Smart transcription mode, which automatically strips filler words and self-corrections and applies punctuation and formatting, cutting down on manual post-processing.
- Custom vocabulary biasing for up to 1,000 domain-specific terms, useful for jargon-heavy audio like legal, medical, or technical calls.
Real-Time Streaming via the Live API
Gemini 3.5 Transcribe Live targets interactive voice apps that need sub-second latency. It supports both interim and finalized transcription events, plus multiple Voice Activity Detection (VAD) strategies, so you can control how aggressively the model segments speech mid-conversation. Third-party platforms including Agora, Fishjam, LangChain, LiveKit, Pipecat, Vercel, and Vision Agents have already integrated the Live model through the Gemini Live API, which is a meaningful signal: the ecosystem around it is not starting from zero. It is the same competitive dynamic we saw when Gemini Omni Flash reached general availability against GPT-4o: shipping ecosystem integrations early now matters as much as the benchmark itself.
Where to Access It
The models are available through Google AI Studio and Google Antigravity as part of the Gemini API. Pre-recorded transcription runs through the model endpoint directly, and continuous streaming runs through the Live API. The same underlying technology is also rolling out in first-party Google products, including the Gemini app on macOS and the Rambler feature on Android, and is reportedly coming to Chrome. If you are wiring an API like this into your own app rather than waiting for a first-party integration, learning to ship fast with Claude Code is the quickest way to get it tested end to end.
Gemini 3.5 Transcribe for Product Managers
If you are a PM scoping a voice feature rather than shipping the code yourself, the two-model split changes how you write the spec. A "transcribe this recorded call" feature and a "live voice agent" feature are not the same integration, and now they map to two distinct APIs with different latency and pricing tradeoffs instead of one generic transcription checkbox.
It also changes what you can ask for in a PRD. Speaker diarization and word-level timestamps used to be a build-it-yourself line item; now they are a parameter. Custom vocabulary biasing means you can spec accurate transcription for a jargon-heavy vertical (legal, medical, support calls) without a fine-tuning project. That is worth flagging in any discovery synthesis or teardown involving voice or call-center tooling this quarter, especially when the target is a production-ready app you are shipping end to end.
Product manager and want to work like this? This is exactly what we teach in Claude Code for PMs, our live cohort for product teams: 3 live sessions of 90 minutes over 2 weeks. Every PM ships a real feature, builds their own agent, and gets personalized written feedback.
The Bottom Line
Gemini 3.5 Transcribe reaching general availability is less about beating Whisper on a single benchmark and more about consolidation. Diarization, timestamps, vocabulary biasing, and a dedicated low-latency streaming model are now one API surface instead of five separate integrations. For builders adding voice or transcription to a product today, that is the practical reason to evaluate it before defaulting to the tool you already know, the same way it is worth reevaluating your model stack whenever a capability shift like Claude Sonnet 5's 1M-token context window changes what is possible.
Source: Gemini API changelog
