Call Scoring

⚠️ Call Scoring is currently in beta.

Call Scoring is the newest addition to Aircall AI, available in all AI-applicable languages (English, French, Spanish, German, Dutch, and Italian).

Call Scoring allows sales and customer support managers to create custom scorecards that use a mix of AI and human-evaluated criteria.

Call Scoring also adds a new Scorecard tab to the Conversation Center of the Dashboard, where admins can see performance metrics across all of the calls they've scored, helping to quickly pin-point areas for improvement.

Access

Users subscribed to the Aircall AI add-on will have access to Call scoring in all AI-applicable languages (English, French, Spanish, German, Dutch, and Italian). Users can evaluate any call with a recording in any language, but only calls with transcripts will benefit from AI automated evaluations.

Users cannot evaluate voicemails.

Functionality

Scorecard creation

Scorecards are a collection of questions that can be grouped into categories. Categories represent areas of performance that the manager is evaluating and will be used to show performance results in the Scorecard Stats analysis.

Scorecards have a title, an optional description, and a language setting. While users can create custom questions in any language, this setting will inform the language of the fixed AI questions they can select to build their scorecard, as well as the language of the AI answers.

There are two types of questions: custom and AI-powered. Each question has a question text, an answer type, an optional answer guide, and a weighting system.

Users can create custom scorecards where they evaluate the call manually. Users can also leverage AI questions to create the scorecard. When AI questions are used, every call is graded automatically for that question, and AI fills in the score and explanation for the answer, thus speeding up review times.

The answer types are:

• yes or no choice
• a 5-rating qualitative scale ( very poor-> excellent)

Each of these answer choices also has a not applicable option that the user can manually enable. Not applicable describes situations where the question could not be assessed for various reasons (no data in conversation, no need, etc).

Each question has a weight, represented by points, that users can manually set. The points of each question are summed to represent the global points for the scorecard, which are normalized to 100%. In addition, a category will also sum up the points of the questions grouped underneath.

The answer guide is beneficial because it ensures everyone has the same objective understanding of how the evaluation is done.

Custom scorecards can be created from any conversation page by going to the Evaluation tab or from the Scorecard tab.

Scoring mechanism

Users can manually define the weight or questions, represented by points the points, however the weights of the choices for the questions are fixed (i.e., users can not set the “No” choice to 100%. It will always be 0%).

Scorecard Management

All users with Admin and/or Supervisor roles can create and edit a scorecard. A scorecard has two states: draft and published. When in draft mode, the scorecard is not yet available to run call evaluations based on it. When published, it becomes visible for call evaluations.

A user can edit a published scorecard. They can edit the scorecard name and description, add/delete questions and/or categories, edit the category title, and change the points for each question. The question text can not be edited. When publishing the changes, users should be aware that the changes will impact the scorecard analytics moving forward. At the same time, conversations evaluated in the past will be based on the previous version.

Users can delete a scorecard, which will disappear from the list of available scorecards. Evaluations made based on the deleted scorecard will be kept. Aggregations of the respective scorecard can no longer be accessed because the scorecard was deleted.

Conversation Review

When reviewing a conversation, managers can navigate to the Evaluation tab on the conversation page and select the desired scorecard. For the custom questions, the users will have to select the answer choice based on which the question score will be calculated, and they can optionally fill in the explanation for the choice.

If the scorecard contains AI-fixed questions, AI will automatically fill in the choices, scores, and explanations when selecting it. AI will leverage the conversation transcript as the only source of information to answer the questions. Users can always edit the AI scores and answers. The AI answers are filled in the AI question language.

Users can fill in a general feedback comment section for the global score, which is automatically calculated based on the choices and scores for each question.

Users can delete an evaluation and use the same scorecard or another one to run a new evaluation.

Aggregation

When clicking on a published scorecard in the Scorecard tab, users can access Scorecard stats.

This section shows the global score per scorecard, calculated by averaging the global scores for all the evaluated calls with the specific scorecard for the selected time period. Users have a date filter to select the period based on which they want to see the scores. The evaluations included in the selected period are based on the call dates, not the evaluation dates.

Users can see how many conversations were evaluated and for how many agents. They can also access the list of conversations included in the scorecard's global score.

The stats for each scorecard also show the global scores per category.

Permission and Sharing

All users with Admin and Supervisor roles can create, edit, and delete a scorecard, evaluate a conversation, and view the scorecard stats.

Users can share the conversation page's ULR link to share an evaluation with an Agent role, and the evaluated agent will have access.