Setting up an after-hours AI Voice Agent

An AI Voice Agent can keep your business responsive even when your team is offline. This article shows you how to create, configure, test, and activate an after-hours AI Voice Agent that answers routine customer questions and, when it can't help, collects the caller's details and preferred callback time so your reps can pick up right where it left off.

Note: Throughout this article, we'll use an example based on ACME, an e-commerce business with support available Monday to Friday, 9am to 6pm. Outside those hours, ACME wants an AI Voice Agent to answer common questions and, if it can't resolve a query, collect the caller's details and a preferred callback time.

How it works

This setup can be adapted to suit virtually any use case. For the ACME example, a typical call flow looks like this:

1. A customer calls the company number outside business hours with a question about the returns process.
2. The AI Voice Agent greets the customer and draws on its configured knowledge base to answer the question.
3. If the agent can't answer the question, it offers to arrange a callback with a support team member during business hours.
4. If the caller agrees, the agent collects a few details, including their preferred callback time.
5. The call ends, and the details are logged so the support team can follow up when they're next available.

Note: This guide assumes you already have business hours configured in the call distribution of the phone number where you'll place the AI Voice Agent. For information on how to configure business hours, please see Smartflows widgets: The Time Rule widget

Step 1: Create your AI Voice Agent

For this step, we will use the automated Agent Builder tool, which allows you to describe the desired behavior of your AI Voice Agent in plain language. You can also incorporate call transcripts, so the tool surfaces your most relevant calls based on your described goal and uses them to inform the AI agent's configuration.

Agent Builder is a quick-start tool. You'll still be able to edit the agent afterward, and we recommend reviewing it thoroughly.

Steps:

1. Navigate to AI Agents > AI Voice Agents from the left-hand navigation menu.
2. Click Create an AI Voice Agent and select Inbound.
3. Describe what you want the AI Voice Agent to do in plain language. For the ACME example: "Create a friendly customer support agent that helps customers with FAQ outside of business hours."

Note: Using call transcripts requires at least 300 calls in your account to activate. If you'd rather create your AI Voice Agent manually, you can select that option when prompted.

Tip: Add further details to your prompt to make it more specific to your business or use case.

Step 2: Configure your AI Voice Agent

Now that the AI agent is created, it must be configured. If you used the Agent Builder, the basic configuration will be populated but you will need to review:

• Agent profile: sets the agent's core identity.
• Company information: defines how the agent refers to your business.
• Call script (greeting and farewell messages): defines the messages callers hear at the start and end of a call.
• Call transfer rules: disable these, since there's no available team after hours.

For more help with these settings, please refer to Configuring your AI Voice Agent.

Keeping our aforementioned use case in mind, there are 3 configuration settings that we will now go into further detail with. 

Knowledge base

The knowledge base is where you provide the information your agent uses to answer caller questions. It consists of one or more knowledge sources connected to the agent.

If you used Agent Builder with call transcripts, a concise FAQ knowledge source will already have been created based on your previous conversations and aligned with your prompt. Open and review this knowledge source. You can add directly to it, or connect additional knowledge sources.

Tip: Only include information specific to your use case. Avoid trying to cover everything, stick to your most frequently asked questions about your most popular topics. Overloading your knowledge base can slow down responses and reduce accuracy.

Conversational guidelines

Conversational guidelines shape how the agent conducts a conversation: refining tone, defining phrase responses, and directing how it handles edge cases. This is also where you set the if/then logic that gives your agent the ability to reason within the boundaries you've set.

By default, the AIVA won't attempt to answer questions outside its knowledge scope. For our use case, we'll build on that behaviour by defining how it informs the customer and what happens next using if/then logic: if the agent can't answer the question, it lets the customer know, then asks if they'd like a callback. If yes, it moves on to collecting their details. If no, it thanks them for calling and ends the conversation politely:

Unable To Answer

• If you are not able to answer the caller’s question, inform them with the phrase “Sorry, that question is outside the scope of my knowledge”
• Ask them “Would you like me to arrange a member of our friendly support team to call you back during business hours to answer your question” 
• If YES, immediately proceed to asking intake questions 
• If NO, say “You can also find support resources on our website at acme.com” and politely end the call. 

Tip: You can build upon these Conversational Guidelines with additional business-specific phrases, tone adjustments or general behaviour directions specific to your use case

Intake questions

Intake questions define the specific details the agent collects once a caller agrees to a callback. Once collected, these details are logged in a structured format and passed to your support team, so they have what they need to follow up efficiently.

Tip: Ask no more than five intake questions. Asking too many can lengthen the conversation and frustrate the caller, so stick to what your team actually needs for context.

For the ACME use case, the intake questions we will be collecting are: 

• What is your name?
• Please provide a brief description of your query.
• What day and time best suits you for our support team to contact you?

Note: Depending on your use case, intake questions are also a good way to capture information such as order ID, account number, or other identifying details.

Step 3: Test your after-hours agent

Once configuration is complete, test your agent thoroughly using the Test call panel on the right-hand side of the agent editor before activating it in your call flow. This simulates a real call experience without firing any call activity to your team or triggering actions in your CRM.

Run through three scenarios to confirm the agent behaves as expected:

• Happy path: ask a question you know is within the agent's knowledge scope and confirm it answers correctly. Then ask a question outside its scope and confirm the agent recognizes this, offers a callback, and successfully collects the caller's details.
• Fuzzy path: ask a vague or ambiguous question to see how the agent handles uncertainty, and whether it asks clarifying questions or defaults to offering a callback.
• Unhappy path: push the agent with challenging conditions, such as interrupting mid-response, giving contradictory details, or asking it to do something outside its scope. Confirm it handles these gracefully rather than breaking down or giving incorrect information.

If you encounter unexpected results, identify the issue, make the necessary adjustments, and repeat testing until you're confident the agent is ready to serve customers.

Step 4: Activate your after-hours agent

To activate your after-hours agent, add it to the call flow of the number where you want it to handle requests. Since this agent's purpose is to answer calls after hours only, add it to the At any other time branch of your time-rule (business hours) widget. This replaces any current voicemail configuration.

Steps:

1. Go to Aircall Dashboard > Numbers in the left-hand menu and select the relevant number.
2. Click Edit in the top right corner to open the call flow editor.
3. On the At any other time branch, click the + icon to open the widget menu and select AI Voice Agent.
4. Select your after-hours AI Voice Agent.
5. Click Publish. Your agent is now live.

Important: After deployment, monitor your agent's performance by reviewing calls and using the agent performance dashboard. For details, please see Track AI Voice Agent's performance.

Best practices

• Keep your knowledge base focused on your most frequently asked questions and popular topics rather than trying to cover everything.
• Limit intake questions to five or fewer to keep the callback flow quick and frustration-free.
• Test across happy, fuzzy, and unhappy path scenarios before going live.
• Disable call transfer rules for after-hours agents, since no team is available to receive transfers.
• Monitor performance regularly after launch to catch and resolve any unexpected behavior.