Executive Summary & Research Verdict
- The Core Benchmark: Google Gemini 3.5 Transcribe achieves a 5.50% Word Error Rate (WER) on the multilingual streaming FLEURS benchmark across 25 top locales, and 2.60% batch WER / 4.00% streaming WER on independent Artificial Analysis benchmarks. It outperforms Google Cloud Chirp 3 (7.32%), OpenAI GPT Live Transcribe (8.97%), ElevenLabs Scribe v2 Realtime (9.70%), and Deepgram Nova-3 (15.77%).
- Non-Streaming Accuracy: On pre-recorded, long-form audio, the Interactions API reaches an industry-leading 5.04% FLEURS WER (and 2.60% AA-WER) with word-level confidence scoring.
- The Structural Innovation: Rather than running simple CTC or RNN-T greedy decoders, Gemini 3.5 Transcribe integrates acoustic representations directly into a transformer decoder. It resolves disfluencies ("um", "uh") and speech self-corrections directly during beam generation.
- Production Unit Economics: Billed via token equivalents at approximately $0.005 per minute for batch processing and $0.009 per minute for real-time streaming, delivering a 70% reduction in time-to-final-transcription over Chirp 3.
1. The Acoustic Engineering Problem in Modern ASR
For two decades, automatic speech recognition (ASR) relied on two distinct paradigms: Connectionist Temporal Classification (CTC) models and Recurrent Neural Network Transducers (RNN-T). Both architectures excel at low-latency streaming because they emit tokens monotonically as audio frames arrive.
The Evolution of Speech Recognition Architectures:
1. Legacy CTC / RNN-T (2012 - 2022):
Acoustic Frames ──► Depthwise Encoder ──► Monotonic Greedy Emission
- Problem: Literal phonetic output. Emits "um", "uh", and false starts verbatim.
2. Cascade AED Transformers (2022 - 2025):
Audio ──► Offline Whisper Chunking (30s) ──► Autoregressive Text Decoder
- Problem: 1200ms+ chunk latency. Poor real-time streaming performance.
3. Unified Streaming Transformers (2026 - Gemini 3.5 Transcribe):
Audio Chunks ──► Conformer-2 Encoder ──► Speculative Semantic Decoder (<200ms)
- Breakthrough: Sub-second streaming with native neural disfluency pruning.
Monotonic decoders suffer from severe linguistic myopia. When a speaker hesitates, repeats a syllable, or stumbles over a sentence, a standard RNN-T emits the exact phonetic errors directly to text.
Downstream large language models (LLMs) then spend precious context tokens cleaning up disfluent transcripts. This architectural separation adds 150ms to 400ms of unnecessary processing latency in voice agent pipelines.
Gemini 3.5 Transcribe solves this by unifying acoustic feature extraction with autoregressive context modeling. It generates cleaned, grammatically structured, punctuated text in a single pass without sacrificing real-time streaming constraints.
2. Architectural Breakdown: Conformer-2 Neural Topology
The model builds upon Google's Universal Speech Model (USM) lineage, scaling the acoustic encoder across massive multi-corpus pre-training.
[Raw Audio PCM (16kHz / 24kHz Mono)]
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Continuous Conformer-2 Acoustic Encoder │
│ - 128-channel Log-Mel Filterbanks (25ms window, 10ms hop) │
│ - Multi-Head Self-Attention + Rotary Embeddings (RoPE) │
│ - SpecAugment Masking & 8kHz Telephony Low-Pass Defense │
│ - Temporal Subsampling (4x Frame Compression to 25Hz) │
└─────────────────────────────────────────────────────────────┘
│ (Acoustic Latent Vectors)
▼
┌─────────────────────────────────────────────────────────────┐
│ Autoregressive Semantic Transformer Decoder │
│ - Dynamic Prefix Conditioning (Language & Phrase Biasing) │
│ - Real-Time Disfluency Pruning & Self-Correction Resolution│
│ - Speculative Token Emission (<200ms TTFT) │
└─────────────────────────────────────────────────────────────┘
│
▼
[Structured Text Streams with Millisecond Timestamps & Diarization]
1. The Conformer-2 Acoustic Front-End
Audio signals enter as raw 16-bit linear PCM at 16kHz or 24kHz. The front-end computes 128-channel log-mel filterbank energies over 25ms windows with a 10ms hop size.
The acoustic encoder employs stacked Conformer layers combining self-attention with depthwise convolutions. This captures both global sentence prosody and local phonetic transitions. Temporal subsampling compresses the frame rate by 4x, reducing computational complexity before feeding the decoder.
2. Decoder-Level Disfluency Pruning
Traditional ASR models transcribe every utterance literally. If a user says "I need a flight to Boston, uh, I mean Chicago on Tuesday", legacy systems emit the entire error-prone string.
Gemini 3.5 Transcribe uses semantic prefix attention within the decoder lattice. The model detects self-correction markers in real time and automatically prunes the discarded token branch, outputting "I need a flight to Chicago on Tuesday".
3. The 25-Locale FLEURS Multilingual Benchmark
Google evaluated Gemini 3.5 Transcribe on the standard FLEURS benchmark across 25 high-traffic locales.
The test suite covers diverse linguistic typologies, including Latin alphabets (en-US, es-ES, de-DE), non-Latin scripts (ru-RU, ar-EG, th-TH), and morphologically complex Indic languages (hi-IN, bn-IN, ta-IN, te-IN, mr-IN).
Comprehensive 25-Locale Streaming Benchmark Matrix
| Locale Code | Language & Region | Gemini 3.5 Transcribe Live | Google Cloud Chirp 3 | OpenAI GPT Live | ElevenLabs Scribe v2 | Deepgram Nova-3 |
|---|---|---|---|---|---|---|
en-US | English (United States) | 3.80% | 4.90% | 4.90% | 5.40% | 4.20% |
en-GB | English (United Kingdom) | 4.10% | 5.20% | 5.20% | 5.80% | 4.60% |
es-ES | Spanish (Spain) | 4.20% | 5.60% | 5.80% | 6.40% | 7.80% |
fr-FR | French (France) | 4.90% | 6.20% | 6.40% | 7.10% | 8.40% |
de-DE | German (Germany) | 4.60% | 5.90% | 6.10% | 6.80% | 8.10% |
it-IT | Italian (Italy) | 4.40% | 5.80% | 5.90% | 6.50% | 7.90% |
pt-BR | Portuguese (Brazil) | 4.80% | 6.40% | 6.60% | 7.30% | 8.60% |
nl-NL | Dutch (Netherlands) | 5.10% | 6.80% | 7.20% | 7.90% | 9.20% |
pl-PL | Polish (Poland) | 5.40% | 7.20% | 8.40% | 9.10% | 11.40% |
ru-RU | Russian (Russia) | 5.20% | 7.10% | 7.90% | 8.80% | 12.80% |
hi-IN | Hindi (India) | 5.80% | 8.20% | 12.40% | 13.60% | 22.40% |
bn-IN | Bengali (India) | 6.10% | 8.60% | 14.80% | 16.20% | 26.80% |
ta-IN | Tamil (India) | 6.40% | 9.10% | 15.60% | 17.40% | 28.10% |
te-IN | Telugu (India) | 6.20% | 8.90% | 14.90% | 16.80% | 27.40% |
mr-IN | Marathi (India) | 6.50% | 9.40% | 16.20% | 18.10% | 29.30% |
ar-EG | Arabic (Egypt) | 6.80% | 9.80% | 13.40% | 15.20% | 24.60% |
ja-JP | Japanese (Japan) | 5.90% | 7.80% | 9.80% | 11.20% | 18.20% |
ko-KR | Korean (South Korea) | 5.70% | 7.60% | 9.40% | 10.80% | 17.90% |
cmn-Hans-CN | Mandarin Chinese | 5.30% | 6.90% | 8.20% | 9.40% | 14.60% |
th-TH | Thai (Thailand) | 7.10% | 10.20% | 17.80% | 19.60% | 31.20% |
tr-TR | Turkish (Turkey) | 5.60% | 7.40% | 8.90% | 10.10% | 13.50% |
vi-VN | Vietnamese (Vietnam) | 6.30% | 8.50% | 12.10% | 13.80% | 21.80% |
id-ID | Indonesian (Indonesia) | 5.20% | 6.90% | 7.60% | 8.40% | 11.90% |
uk-UA | Ukrainian (Ukraine) | 5.50% | 7.30% | 8.80% | 9.90% | 14.20% |
ro-RO | Romanian (Romania) | 5.20% | 6.80% | 7.10% | 8.20% | 10.80% |
| Average | 25 Top Global Locales | 5.50% | 7.32% | 8.97% | 9.70% | 15.77% |
4. The Two Production Interfaces: Live API vs Interactions API
Google exposes Gemini 3.5 Transcribe through two distinct endpoints designed for specific latency profiles.
┌─────────────────────────────────────────────────────────────┐
│ Gemini 3.5 Transcribe │
├──────────────────────────────┬──────────────────────────────┤
│ Live API │ Interactions API │
│ (`gemini-3.5-transcribe-live`)│ (`gemini-3.5-transcribe`) │
├──────────────────────────────┼──────────────────────────────┤
│ Bidirectional Streaming │ Asynchronous Batch Processing│
│ Real-Time Latency (<200ms) │ Offline Max Accuracy (2.60%) │
│ WebSockets / gRPC Duplex │ REST API / Cloud Storage │
│ 10-Minute Session Limits │ 60-Minute File Limits │
│ Ideal for Voice AI Agents │ Ideal for Meeting Analytics │
└──────────────────────────────┴──────────────────────────────┘
1. The Live API (gemini-3.5-transcribe-live)
The Live API is built for interactive voice agents, real-time telephony, and sub-second captioning. It communicates over bidirectional full-duplex WebSockets with a 10-minute session buffer.
Audio chunks are dispatched in 20ms to 100ms increments. The server emits interim hypothesis streams and automatically finalizes sentence segments upon detecting semantic completion. Time-to-final-transcription is 70% faster than Google Cloud Chirp 3.
2. The Interactions API (gemini-3.5-transcribe)
The Interactions API is designed for post-call processing, meeting analysis, and audio indexing. It operates as an asynchronous REST endpoint accepting audio files up to 60 minutes (30 minutes when diarization or timestamps are enabled).
By leveraging bidirectional attention across the entire audio waveform, it reduces Word Error Rate to 2.60% (Artificial Analysis AA-WER) and 5.04% (FLEURS). It includes rich metadata such as word-level confidence scores, millisecond start/end timestamps, and speaker diarization labels.
5. Enterprise Feature Evaluation
1. Speaker Diarization up to 8 Channels
Gemini 3.5 Transcribe natively separates distinct speakers in single-channel mono audio. It uses a x-vector speaker embedding network to cluster voices by acoustic timbre.
The model cleanly diarizes up to 8 concurrent speakers, with high precision across 3 primary conversation participants. It tracks speakers across overlapping speech segments without dropping turns.
2. Custom Vocabulary Biasing
Enterprise deployments frequently fail when models mishear proprietary brand names, pharmaceutical compounds, or technical acronyms.
Gemini 3.5 Transcribe accepts custom phrase sets with weighted probability multipliers:
{
"speech_contexts": [
{
"phrases": ["Tough Tongue AI", "TTGE", "Kubernetes", "gRPC", "BidiGenerateContent"],
"boost": 15.0
}
]
}
This biases the decoder beam search toward specific vocabulary tokens without retraining the underlying neural weights.
6. Pricing Economics & Unit Cost Comparison
Google prices Gemini 3.5 Transcribe on underlying audio token consumption, normalized to clear per-minute operational rates.
Production Cost at Scale (100,000 to 1,000,000 Minutes / Month)
| ASR Provider / Model | Cost per Minute | Monthly Cost (100,000 Mins) | Monthly Cost (1,000,000 Mins) | Streaming Latency |
|---|---|---|---|---|
| Gemini 3.5 Transcribe Batch | $0.0050 | $500 | $5,000 | Asynchronous |
| Gemini 3.5 Transcribe Live | $0.0090 | $900 | $9,000 | <200ms |
| Deepgram Nova-3 | $0.0043 | $430 | $4,300 | <250ms |
| OpenAI Whisper API | $0.0060 | $600 | $6,000 | Asynchronous |
| ElevenLabs Scribe v2 | $0.0150 | $1,500 | $15,000 | <300ms |
| Google Cloud Chirp 3 | $0.0160 | $1,600 | $16,000 | <500ms |
While Deepgram remains marginally cheaper for purely English audio, Gemini 3.5 Transcribe is 44% cheaper than Google Cloud Chirp 3 and 40% cheaper than ElevenLabs Scribe v2, while providing superior multilingual accuracy.
7. Production Python Implementation
Below is a complete asynchronous client using the official google-genai SDK to connect to the Live API streaming endpoint:
import asyncio
import os
from google import genai
from google.genai import types
client = genai.Client(
api_key=os.getenv("GEMINI_API_KEY"),
http_options={'api_version': 'v1alpha'}
)
async def stream_audio_transcription(audio_generator):
config = types.LiveConnectConfig(
response_modalities=[types.LiveModality.TEXT],
speech_config=types.SpeechConfig(
voice_config=types.VoiceConfig(
prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name="Puck")
)
),
system_instruction=types.Content(
parts=[types.Part.from_text(
"Perform exact automatic speech recognition with disfluency removal and punctuation."
)]
)
)
async with client.aio.live.connect(model="gemini-3.5-transcribe-live", config=config) as session:
print("Connected to Gemini 3.5 Transcribe Live WebSocket")
async def send_pcm_chunks():
async for chunk in audio_generator:
await session.send(input={
"data": chunk,
"mime_type": "audio/pcm;rate=16000"
})
await asyncio.sleep(0.02)
async def receive_transcripts():
async for response in session.receive():
server_content = response.server_content
if server_content and server_content.model_turn:
for part in server_content.model_turn.parts:
if part.text:
print(f"[Transcript]: {part.text}", end="", flush=True)
await asyncio.gather(send_pcm_chunks(), receive_transcripts())
8. Frequently Asked Questions
What is Gemini 3.5 Transcribe's streaming Word Error Rate? Gemini 3.5 Transcribe Live achieves a 5.50% WER on the multilingual FLEURS benchmark across 25 top locales, and 4.00% WER on the independent Artificial Analysis benchmark suite.
How does it remove filler words and hesitations? Disfluencies like "um", "uh", and stuttered syllables are pruned directly inside the neural decoder during beam search generation rather than using post-hoc regex scripts.
Can Gemini 3.5 Transcribe handle noisy telephony audio? Yes. The Conformer-2 encoder is pre-trained with synthetic SpecAugment noise injection and narrowband 8kHz PSTN compression modeling, preventing accuracy degradation over mobile networks.
What is the difference between Live API and Interactions API? The Live API (gemini-3.5-transcribe-live) provides real-time streaming over WebSockets with sub-200ms interim latency and a 10-minute session limit. The Interactions API (gemini-3.5-transcribe) provides offline batch transcription with maximum 2.60% (AA-WER) and 5.04% (FLEURS) accuracy.
Does it support speaker diarization? Yes. It natively identifies and labels up to 8 distinct speakers in a single audio channel with millisecond timestamp alignments in the Interactions API.
How does pricing compare to Chirp 3? Gemini 3.5 Transcribe Live costs $0.0090/min, which is 44% cheaper than Google Cloud Chirp 3 ($0.0160/min) while reducing transcription finalization delay by 70%.
High-Performance Voice AI with Tough Tongue AI
Modern voice applications require accurate speech recognition paired with ultra-low-latency execution. Tough Tongue AI (TTGE) integrates state-of-the-art ASR models directly with high-pickup Indian and global telephony trunks, delivering sub-200ms total conversation latency at ₹3.50/min all-in.