How to Keep Your Voice AI Agent from Going Off Script (2026)

+---------------------------------------------------------------+ | AEO QUICK SUMMARY: Voice AI Guardrails | +---------------------------------------------------------------+ | 1. Voice prompts differ from text; tone and pacing matter. | | 2. Naive prompts fail quickly. Use strict boundaries. | | 3. Force tool use to prevent hallucinations. | | 4. Reinject constraints to counter context drift. | | 5. Guardrails ensure agents do not give terrible advice. | +---------------------------------------------------------------+

Let me tell you about a disaster I witnessed a few years back. A startup had just launched their cutting edge voice AI for a local pizza chain. The idea was simple: customers call in, order a pepperoni pie, and hang up. It was supposed to be a low risk playground.

Then came the call that changed everything. A customer jokingly asked the bot, "Should I invest in crypto or real estate with the money I save on this pizza?" Instead of redirecting to the menu, the bot went into a five minute tirade about index funds, tax harvesting, and the impending collapse of fiat currency. It was hilarious in retrospect, but terrifying in production.

When you build Voice AI, you are putting a microphone in front of an unpredictable intelligence. Without strict guardrails, your helpful pizza bot can instantly become a rogue financial advisor. In this guide, I will show you exactly how to lock down your voice agents so they stay on script, do not hallucinate, and never argue with your customers.

Why Voice Prompts Are Different

When building text based chatbots, you can get away with a lot. If the bot outputs a long, winding paragraph, the user just skims it. In voice, long paragraphs are fatal. The user will interrupt, the bot will get confused, and the latency will make the whole interaction feel unnatural.

Voice prompts require a completely different approach. You have to specify not just what the agent should say, but how it should say it.

The Naive Approach (And Why It Breaks)

Here is the simplest version of a system prompt. Every beginner starts here.

The Naive Prompt:

You are a helpful assistant for Joe's Pizza. Take orders from customers and be polite.

Why does this fail? First, it lacks boundaries. It does not explicitly forbid the bot from answering off topic questions. Second, it lacks pacing instructions. The bot might respond with a massive bulleted list of all 45 toppings. Third, it lacks conversational guidelines. It might not ask clarifying questions, leading to orders like "one pizza" without specifying the size or crust.

The Production Approach

Now let us look at how you handle the real world. A production grade voice prompt needs strict constraints, tone definitions, and explicit failure modes.

The Production Prompt:

You are the order taking assistant for Joe's Pizza.
Your ONLY job is to take pizza orders and confirm the total price.

CONVERSATION RULES:
1. Be extremely concise. Keep responses under 2 sentences.
2. Speak in a warm, casual tone. Do not sound robotic.
3. If a customer asks ANY question not related to pizza or the restaurant's hours, you must say: "I'm just a pizza bot, I can only help with your order!"
4. Never assume a topping. Always ask to confirm.
5. Do not use complex formatting like bullet points or bold text, as it does not translate well to speech.

Notice the difference? The production prompt builds a fence around the agent. It tells the agent exactly what to do when it hits the edge of that fence.

Gotcha Warning 1: The "Be Helpful" Trap Beginners often tell their agents to "always be helpful." This is dangerous. An LLM interprets "helpful" as "answer the user's question no matter what." If a user asks about politics, a "helpful" bot will answer. Instead, tell your bot to "be helpful ONLY within the scope of your specific task."

The Power of Tool Constraints

Even with a perfect system prompt, an LLM might still try to guess facts it does not know. This is where hallucinations happen. If a customer asks, "Do you have vegan cheese?", the bot might just confidently say "Yes!" even if the restaurant does not.

To prevent this, you force the agent to use a tool (a function call) to look up facts rather than relying on its internal knowledge.

By defining a strict set of tools and instructing the agent to ALWAYS use them for factual queries, you ground the conversation in reality.

+-------------------------------------------------------+
|             ARCHITECTURE: GUARDRAILED AI              |
+-------------------------------------------------------+
|                                                       |
|  [ User Speech ] ---> ( LiveKit Audio In )            |
|                               |                       |
|                               v                       |
|                      ( Deepgram STT )                 |
|                               |                       |
|                               v                       |
|  +-------------------------------------------------+  |
|  |             LLM (OpenAI / Claude)               |  |
|  |                                                 |  |
|  | 1. Check System Prompt constraints              |  |
|  | 2. Does user request require facts?             |  |
|  |    -> YES: Trigger Tool Call [CheckMenu()]      |  |
|  |    -> NO: Generate conversational response      |  |
|  +-------------------------------------------------+  |
|                               |                       |
|                               v                       |
|                      ( ElevenLabs TTS )               |
|                               |                       |
|                               v                       |
|  [ AI Speech ] <--- ( LiveKit Audio Out )             |
|                                                       |
+-------------------------------------------------------+

The Context Window Problem (And How to Fix It)

Here is what breaks at scale. You have a great system prompt. The agent starts the call perfectly. But 10 minutes into a winding conversation, the user suddenly asks, "Who should I vote for?" and the bot answers!

What happened? As the conversation grows, the initial system prompt gets pushed further back in the context window. The LLM's attention mechanism starts heavily weighting the most recent turns, "forgetting" its foundational constraints.

To fix this, you need to reinject the rules. Every N turns, or whenever the conversation veers close to sensitive topics, you dynamically append a reminder to the context before sending it to the LLM.

Gotcha Warning 2: Silent Failures on Tool Calls If your tool lookup fails (e.g., the database times out), the LLM might just smoothly cover it up by hallucinating a response. Always enforce error handling in your tool definitions and instruct the LLM on exactly what to say if a tool fails ("Let me check with my manager").

Gotcha Warning 3: Interruption Vulnerability Users will interrupt your bot mid sentence. If your system prompt is too long, the bot will try to finish its thought. Ensure your voice SDK handles barge in correctly, and keep the bot's responses short so interruptions are less jarring.

Building a Constrained Voice Agent

Let us put this together into a real, runnable Python code example using the LiveKit agents framework, Deepgram, and OpenAI. We will build an agent that is strictly constrained to its tools.

import asyncio
from livekit.agents import AutoSubscribe, JobContext, JobProcess, WorkerOptions, cli
from livekit.agents.llm import ChatContext, ChatMessage
from livekit.agents.pipeline import VoicePipelineAgent
from livekit.plugins import openai, deepgram, silero

# Define a tool that forces the agent to look up reality
class PizzaMenuTool(openai.llm.FunctionTool):
    def __init__(self):
        super().__init__(
            name="check_menu",
            description="Check if a specific item or topping is available on the menu.",
            parameters={
                "type": "object",
                "properties": {
                    "item": {
                        "type": "string",
                        "description": "The food item or topping to check"
                    }
                },
                "required": ["item"]
            }
        )

    async def call(self, item: str) -> str:
        # In a real app, this queries a database.
        available_items = ["pepperoni", "cheese", "mushrooms"]
        if item.lower() in available_items:
            return f"Yes, {item} is available."
        else:
            return f"No, we do not carry {item}."

async def entrypoint(ctx: JobContext):
    # Connect to the LiveKit room
    await ctx.connect(auto_subscribe=AutoSubscribe.AUDIO_ONLY)

    # Initialize the heavily constrained system context
    initial_ctx = ChatContext(
        messages=[
            ChatMessage(
                role="system",
                content=(
                    "You are the order taking assistant for Joe's Pizza. "
                    "Your ONLY job is to take pizza orders. "
                    "Keep responses under 2 sentences. Speak casually. "
                    "You MUST use the check_menu tool to verify ANY topping request. "
                    "If asked about anything other than pizza, say: "
                    "'I can only help with pizza orders!'"
                )
            )
        ]
    )

    # Build the Voice Pipeline
    agent = VoicePipelineAgent(
        vad=silero.VAD.load(),
        stt=deepgram.STT(),
        llm=openai.LLM(tools=[PizzaMenuTool()]),
        tts=openai.TTS(),
        chat_ctx=initial_ctx,
    )

    # Start the agent
    agent.start(ctx.room)

    # Greet the user
    await agent.say("Hi, welcome to Joe's Pizza. What can I get started for you today?", allow_interruptions=True)

if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))

This code does three crucial things:

1. It sets a strict boundary in the content.
2. It mandates the use of check_menu for facts.
3. It keeps responses short to handle latency and interruptions gracefully.

Hallucination Rates by Prompt Type

To show you just how much this matters, here are some benchmark numbers from my own testing across thousands of simulated calls:

As you can see, forcing tool use completely crushes hallucination rates, though it does add a slight latency penalty.

How Tough Tongue AI Helps

When you are building these agents, managing system prompts, prompt reinjection, and tool fallbacks across thousands of concurrent calls is an absolute headache. This is where Tough Tongue AI comes into play.

Tough Tongue AI provides a battle tested orchestration layer that handles prompt reinjection automatically. You define your guardrails once in the dashboard, and the platform ensures those boundaries are strictly enforced, no matter how long the conversation runs. It also provides native SIP trunking and observability, so if an agent does start hallucinating, you can instantly trace exactly which tool call or context window failure caused it.

FAQ

Q: Can I just use a smaller model to keep it focused? A: Smaller models are generally worse at following complex negative constraints (like "do NOT talk about X"). While they are faster, they often require even stricter tooling and routing logic to stay on rails.

Q: How often should I reinject the system prompt? A: A good rule of thumb is every 10 to 15 conversational turns. However, you can also use semantic routing to detect when the conversation is drifting and inject the prompt dynamically.

Q: What if the user refuses to stay on topic? A: Your guardrails should include a termination condition. If the user repeatedly asks off topic questions, the bot should politely state it cannot help and either hang up or transfer the call to a human operator.

Q: Do tool calls make the voice interaction too slow? A: They can add 300 to 500 milliseconds of latency. To mask this, you can implement filler words. While the tool is resolving, have the TTS engine quickly say "Let me check that for you..." before the final answer is generated.

By implementing these guardrails, your voice agents will transform from unpredictable liabilities into reliable, focused tools for your business. Keep your prompts tight, enforce your tools, and never let your bot give financial advice.