How to test an AI Agent

This article explains how to test an AI Agent before making it live. It covers the required test configuration, the steps to follow during testing, and what to check to ensure the Agent is working as expected. By following this guide, you can identify and fix any issues in the conversation flow, module behaviour, and responses before publishing the Agent.

Prerequisites for Testing

Before testing the AI Agent, make sure the following are completed:

• Ensure all required modules are fully configured and no module shows a warning.
• Confirm that the Agent is in Draft or Ready to Use state.
• Select the channel you want to test (for example: Chat, SMS, or Voice), based on what is configured for the Agent.
• Verify that all necessary configurations are in place, such as:
  • Data Collection fields
  • Screening setup (if applicable)
  • FAQ content access
  • Closing and Feedback settings
• If your Agent uses any variables or data, make sure valid test data is available.

Testing Steps

• Open the drafted AI Agent
• Click on Test from the top bar

• You need to complete the Testing Configuration for testing the created outbound AI Agents by filling in the details. However, it's optional for the inbound agents if you wish to test the writebacks.

• In the Job Title, search and select the job that you want to use for testing. This ensures the Agent uses the correct job-related data during the conversation.
• In Choose Candidate, search and select a candidate to simulate the interaction. The Agent will use this candidate’s details during the test.
• Enable Enable write back during test if you want the data captured during testing (such as responses or updates) to be written back to the system.
• If you do not want any test data to be saved, keep the write back option disabled.
• Once all details are selected, proceed to start testing the Agent by clicking on Save. This will take you to the Testing screen where you can test.

Important Note: Only use a designated test candidate profile if you’re toggling on the enable writeback during testing to avoid overwriting real candidate data.

Testing the Voice Agent

• Select Voice from the Channels panel to start testing the voice experience.
• Click Start Call to begin the test call with the AI Agent.
• Speak as a user would and go through the conversation naturally.
• Observe how the Agent:
  • Responds to your inputs
  • Handles pauses and interruptions
  • Continues the conversation flow

Adjust Speech Settings (Optional)

You can fine-tune how the Agent sounds and responds during the test:

• Name & Pronunciation
  Add a name and define how it should be pronounced during the conversation.
• Responsiveness
  Adjust how quickly the Agent responds after you finish speaking.
• Interruption Sensitivity
  Control how easily the Agent can be interrupted while speaking.
• Voice Speed
  Adjust how fast or slow the Agent speaks.
• Voice Temperature
  Control how expressive and varied the Agent’s voice sounds.

• Test different scenarios by speaking clearly, interrupting, or pausing to ensure the Agent behaves as expected.
• Make adjustments to the settings if required and re-test the call.

Testing the Chat (Web) Agent

• Select Chat (Web) from the Channels panel to start testing the web chat experience.
• Use the chat window shown on the screen to interact with the AI Agent.
• Start the conversation by typing a response in the message box and clicking the send button.
• Go through the conversation step by step, just like an actual user would.
• During testing, verify that:
  • The Greeting message is displayed correctly
  • Questions are asked as expected
  • Responses are clear and relevant
  • The conversation flows smoothly across modules
• Try different scenarios:
  • Provide expected answers
  • Enter incomplete or unexpected inputs
  • Ask questions to test FAQ responses
• Observe how the Agent handles:
  • User inputs and follow-up questions
  • Data collection and validations
  • Fallback responses
• Review the Closing Message and Feedback Collection at the end of the conversation.
• If needed, go back to the Build section, update the configuration, and test again. 

Testing the Chat (SMS) Agent

• Select Chat (SMS) from the Channels panel to test the SMS conversation flow.
• Use the chat interface to simulate an SMS conversation with the AI Agent.
• Start the conversation by typing a message and sending it, similar to how a user would respond over SMS.
• Go through the conversation step by step and verify that:
  • Messages are clear and concise (suitable for SMS)
  • Questions are asked correctly
  • Responses are easy to understand
• Test different scenarios:
  • Provide expected responses
  • Send short or incomplete replies
  • Ask questions to check FAQ handling
• Observe how the Agent handles:
  • Back-and-forth messaging
  • Data collection through SMS
  • Fallback responses when inputs are unclear
• Review the Closing Message and ensure it is appropriate for SMS communication.
• Make changes if required and re-test to confirm the updates. 

Note: You can update the test setup at any time by clicking Testing Configuration from the top right. This allows you to change the Job Title, Candidate, or write back settings and re-run the test as needed.

Next Steps - Publish & Use

• Once you have completed testing and are satisfied with the Agent’s behaviour, go back and review the overall configuration one final time.
• If everything looks correct, proceed to Publish the Agent.
• To trigger the AI Agent, it must be added to an Automation Workflow
• Your AI Agent is now ready to start interacting with users and supporting your workflows.