Voice Activity Detection (VAD) serves as the gatekeeper of your conversational pipeline. If your VAD sensitivity is too aggressive, the agent cuts users off mid-thought. If it is too permissive, background TV noise or keyboard clicks keep the listener in perpetual silence.
Fine-tuning acoustic thresholds and end-of-turn parameters is essential for natural human-level interactions.
+-----------------------------------------------------------------------------------+
| QUICK SUMMARY |
+-----------------------------------------------------------------------------------+
| VAD Parameter | Recommended Setting | Impact on Conversation |
+-----------------------+------------------------------+----------------------------+
| Speech Probability | 0.50 - 0.65 | Prevents noise false alarm |
| Min Speech Duration | 100ms - 150ms | Filters coughs / clicks |
| Post-Speech Silence | 350ms (Sales) / 600ms (Support)| Balance response vs pause |
+-----------------------+------------------------------+----------------------------+
VAD Architecture Comparison
Acoustic Energy VAD
Measures decibel energy spikes against ambient noise floor baselines. Extremely fast (sub-5ms), but highly susceptible to background noise like car horns or typing.
Deep Learning Neural VAD (Silero VAD)
Uses lightweight ONNX neural networks trained on audio spectrograms. Distinguishes human vocal tract harmonics from background noise with sub-15ms processing overhead.
Parameter Tuning Matrix
| Operating Environment | Speech Probability | Minimum Speech Duration | End-of-Turn Silence |
|---|---|---|---|
| Quiet Call Center | 0.45 | 100ms | 300ms |
| Cellular Driving Call | 0.65 | 180ms | 450ms |
| Consultative Sales | 0.55 | 120ms | 600ms |
| Fast Ordering Bot | 0.50 | 90ms | 280ms |
Production Python VAD Controller (vad_controller.py)
Here is a complete, production-grade VAD controller implementing rolling audio buffer evaluation and adaptive silence windowing.
import numpy as np
import time
class ProductionVADController:
def __init__(self,
sample_rate: int = 16000,
frame_size_ms: int = 30,
threshold: float = 0.55,
min_speech_ms: int = 120,
silence_window_ms: int = 400):
self.sample_rate = sample_rate
self.frame_size_samples = int(sample_rate * (frame_size_ms / 1000.0))
self.threshold = threshold
self.min_speech_ms = min_speech_ms
self.silence_window_ms = silence_window_ms
self.speech_buffer = []
self.speech_start_time = None
self.last_speech_time = None
self.is_speaking = False
def evaluate_frame_energy(self, pcm_frame: bytes) -> float:
"""
Calculates Root Mean Square (RMS) energy as acoustic proxy score.
"""
samples = np.frombuffer(pcm_frame, dtype=np.int16).astype(np.float32)
if len(samples) == 0:
return 0.0
rms = np.sqrt(np.mean(samples**2))
normalized_score = min(1.0, rms / 3000.0)
return float(normalized_score)
def process_chunk(self, raw_pcm_chunk: bytes) -> str:
score = self.evaluate_frame_energy(raw_pcm_chunk)
current_time = time.time()
if score >= self.threshold:
self.last_speech_time = current_time
if not self.is_speaking:
if self.speech_start_time is None:
self.speech_start_time = current_time
elif (current_time - self.speech_start_time) * 1000 >= self.min_speech_ms:
self.is_speaking = True
return "SPEECH_STARTED"
return "SPEECH_CONTINUING"
else:
if self.is_speaking and self.last_speech_time:
silence_elapsed = (current_time - self.last_speech_time) * 1000
if silence_elapsed >= self.silence_window_ms:
self.is_speaking = False
self.speech_start_time = None
return "END_OF_TURN_DETECTED"
return "SILENCE"
def run_vad_simulation():
vad = ProductionVADController(threshold=0.5, silence_window_ms=300)
# 30ms 16kHz mono audio frame = 960 bytes
loud_frame = b"\x10\x20" * 480
quiet_frame = b"\x00\x01" * 480
print("--- Running VAD Controller Simulation ---")
# Simulate 5 speech frames followed by 12 silence frames
for i in range(5):
event = vad.process_chunk(loud_frame)
print(f"Frame {i+1} (Speech) -> Event: {event}")
time.sleep(0.03)
for j in range(12):
event = vad.process_chunk(quiet_frame)
print(f"Frame {j+6} (Silence) -> Event: {event}")
time.sleep(0.03)
if __name__ == "__main__":
run_vad_simulation()
Advanced End-of-Turn Tuning Strategies
- Dynamic Punctuation Boundary Checks: If STT streams return sentence terminators like
?or., immediately reduce the VAD silence window from 500ms to 200ms. - Noise Floor Calibration: Calibrate ambient background noise during the first 500ms of call connect to dynamically set energy thresholds.
- Pre-speech Padding: Keep a 150ms ring buffer of incoming audio frames so speech is never truncated at the start of a turn.
How Tough Tongue AI Solves VAD Overhead
Tough Tongue AI integrates smart VAD management into its voice engine:
- Self-Tuning Acoustic Thresholds: Dynamically adapts VAD sensitivity based on caller line noise.
- Zero-Latency Neural Endpointing: Uses edge neural models for sub-10ms VAD classifications.
- No Truncation Guarantee: Ensures caller speech is preserved cleanly without clipping.
Frequently Asked Questions
Why does VAD cut off users mid-sentence?
This happens when silence windows are set too short (under 250ms) or energy thresholds are set too high, mistaking soft spoken words for background silence.
What is Silero VAD?
Silero VAD is an open-source, lightweight deep learning model that classifies human speech with high accuracy across noisy cellular connections.