Understanding automated Call Scoring

Automated call scoring enables you to evaluate calls using AI Assist Pro based on customizable criteria—without needing to trigger evaluations manually. By setting up automation rules, you can ensure consistent, scalable call quality monitoring across your organization.

Important: Automated call scoring requires AI Assist Pro licenses for both the admin configuring the automation and the agent whose calls will be evaluated. Only scorecards with AI-evaluable questions are supported.

How automation rules work

Automation rules define which calls will be automatically evaluated. You can create tailored rules using the following filters:

• Users: Individual agents or entire teams
• Phone numbers: Specific lines or all company numbers
• Call direction: Inbound, outbound, or both
• Call duration: Minimum and/or maximum length

• Call tags: call contains all of the selected tags

The automation engine runs on two events:

• Call ends

  • CALL_CONVERSATION_UPDATED

• Call tags updated

  • TAG_CREATED

  • TAG_DELETED

For each event, the engine scans your active rules from top to bottom and applies the first matching rule. The scorecard attached to that rule is then evaluated by AI. If an evaluation already exists for that call, it is replaced. Only one evaluation is stored per call at any time.

Setting up automation rules

Step 1: Access automation rules

1. Go to Aircall Dashboard > AI Assist > Scorecards.
2. Click the Automation rules tab.
3. Select Create automation to start a new rule.

Step 2: Configure rule criteria

• Scorecard selection: Choose the scorecard to apply. Only scorecards with AI-evaluable questions support automation.
• User targeting: Select individual users or full teams. Use All users to apply the rule across the organization.
• Number assignment: Apply the rule to all numbers or limit it to specific lines. You can filter by line type or campaign.
• Call direction
  • Inbound: Calls initiated by customers
  • Outbound: Calls initiated by agents
• Duration requirements: Set minimum and/or maximum call lengths for the rule to apply.

• Call tags: Apply the scorecard when the call contains any of the selected tags.

Step 3: Set rule priority

When multiple rules could apply to the same call:

• Only the highest priority rule will execute.
• Drag and reorder rules to adjust priority.
• Lower-priority matches will be ignored.

Tag-based triggers for automated call scoring

This feature enhances Automated Call Scoring by introducing tag-based triggers. Previously, automation rules could apply scorecards based on criteria such as users, numbers, call direction, or duration. With this update, scorecards can also be applied automatically when a call includes specific tags.

Because tags are often added after a call ends, the automation engine will now re-evaluate rules each time call tags are updated, not just at the time the call ends.

Supervisors can now ensure automated evaluations are aligned with existing workflows. For example, a call tagged “Escalation” can automatically trigger a QA scorecard, while a “Demo” tag can trigger a Sales scorecard. This improvement increases the accuracy and relevance of automated scoring coverage.

New trigger type: Call tags

Admins can now configure automation rules to apply a scorecard when any tag in a defined list is present on a call.

• Example: If a call has the tag “Escalation” or “High Priority”, apply the Support QA scorecard.

Trigger timing

• Rules are re-evaluated whenever tags are updated after a call ends.
• If tag conditions are met post-call, the scorecard is applied retroactively.
• If another scorecard has already been applied, it will be overridden.

Evaluation handling

• When multiple rules could apply (for example, a tag-based rule and an existing rule), the system respects the priority order as it does today.
• Only one rule will fire per call to prevent duplicate evaluations.

Managing automation rules

In the Automation rules tab, you can view and manage all your rules.

• View rule names, descriptions, active status, criteria summaries, and priority order.
• Edit or disable rules
  • Click the ellipsis (⋯) to edit a rule’s configuration.
  • Use the Active/Inactive toggle to pause or resume automation without deleting the rule.

Note: Active rules will evaluate matching calls. Inactive rules remain saved but will not trigger evaluations.

Rule examples

Viewing automated results

After a call is completed, evaluations are available in:

• The call record, displaying the scorecard and evaluation results
• Scorecard analytics, showing team performance over time

Note: Automation rules only apply to calls that occur after the rule is created. Past calls are not retroactively evaluated.

AI Scorecard Evaluator tool

Use the AI Scorecard Evaluator to improve the quality of your call scoring questions and ensure they are AI-compatible.

What it can do

• Evaluate a single question or an entire scorecard
• Rate each question’s AI scoring difficulty (easy, medium, or hard)
• Suggest improved wording for clarity and accuracy
• Recommend a complete scoring guide (e.g. Yes/No or Very Poor → Excellent)

Try the AI Scorecard Evaluator (ChatGPT tool)

Note: This tool runs on OpenAI’s ChatGPT. Do not paste sensitive customer data, personally identifiable information (PII), or confidential business information into the tool. Learn more in [OpenAI’s guide](https://openai.com/enterprise-privacy).

You will need a ChatGPT account to access the tool, but will not need a paid license to use it.

How to get the best results

• Format questions in Yes/No or ranged style (e.g. 1–5, Poor to Excellent)
• Include your scoring guide to help the tool provide better suggestions
• If no scoring guide is provided, the tool will suggest one

FAQs

Can I run multiple automations on the same call?
No. Only the highest priority rule that matches the call will be executed.

What happens if a rule has no matching calls?
The rule stays active, but won’t generate evaluations until matching calls occur.

Can I pause an automation without deleting rules?
Yes. Use the Active/Inactive toggle to temporarily disable a rule.

Do automation rules work with manual questions?
No. Automation only applies to AI-evaluable questions in scorecards.