Evaluation Criteria

Overview

Evaluation Criteria (also called Prompt Profiles) define what the AI looks for when analyzing a conversation. They control which aspects are evaluated, how they are weighted, and what the output includes. Different projects can use different evaluation profiles.

Evaluation Profiles

Client Admin

Aelo provides base categories that serve as starting points:

Category	Best For
Sales Call	Outbound/inbound sales phone conversations
Support Call	Customer service and technical support calls
Sales Chat	Text-based sales conversations (web chat, messaging)
Complaint	Complaint handling and escalation calls

Each category comes with pre-configured modules and weights optimized for that conversation type.

Category-Bound Profiles (Automatic Selection)

Client Admin

Aelo automatically detects the category of every call (Purchase, Support, Complaint, etc.). Each organization gets a set of category-bound profiles — one per call category plus a dedicated Chats profile — so every conversation is analyzed with criteria tuned for its type:

• 10 call categories: Product Inquiry, Purchase, Scheduling, Support, Billing, Onboarding, Complaint, Cancellation, Follow-up, Other.
• Chats: all chat conversations are always analyzed with the Chats profile (chats are not categorized).
• Missed and short calls are not analyzed, so they have no profiles.

When a record is analyzed, the profile is selected in this order:

1. Category-bound profile — the org profile bound to the detected call category (or Chats for chat conversations).
2. Project profile — the profile linked to the project (fallback when no bound profile exists).
3. Default profile — used when neither of the above is configured.

Managing Bindings

Open Profile Manager (Manage Profiles in the Prompt Builder) — the Category bindings block lists all 11 bindings. Each row shows the bound profile or “Falls back to project profile”.

• Edit a bound profile like any other — its modules apply to all conversations of that category.
• Delete a bound profile to opt out for that category — analysis falls back to the project profile.
• Restore default recreates the pre-configured profile for a category after deletion.

A binding is fixed at profile creation and cannot be moved to another profile; each category can have at most one bound profile per organization.

Analysis Modules

Each profile consists of toggleable analysis modules. Enable or disable modules to control what the AI evaluates:

Summary Module

Produces high-level metrics:

• Quality Score (0–100)
• Sale Probability (0–100%)
• Attention Flag — marks conversations needing manual review

Content Module

Extracts conversation content:

• Topics — main subjects discussed
• Keywords — important terms and phrases
• Key Moments — critical events (objection raised, pricing discussed, agreement reached)
• Summary — natural language overview of the conversation

Speech Module

Evaluates speaking patterns:

• Filler Words — “um”, “uh”, “like” frequency
• Stop Words — overused filler phrases
• Mandatory Phrases — required phrases (e.g., compliance statements)
• Profanity — detection of inappropriate language
• Speech Clarity — overall speaking clarity score

Dialogue Module

Evaluates conversation structure:

• Greeting — proper opening and introduction
• BANT Qualification — Budget, Authority, Need, Timeline assessment
• Objection Handling — how the agent responds to customer objections
• Closing — effective conversation conclusion and next steps

Emotions Module

Analyzes emotional aspects:

• Sentiment — positive, neutral, negative classification
• Politeness — agent courtesy and professionalism
• Interruptions — frequency and handling of speaker interruptions

Compliance Module

Checks regulatory and policy adherence:

• Speech Clarity — meets minimum clarity standards
• Recording Disclaimer — required disclosure at conversation start
• CRM Completeness — required CRM fields populated after the call

Recommendations Module

Generates coaching output:

• Strengths — what the agent did well
• Improvements — specific areas to improve with actionable suggestions

Configuring a Profile

1. Navigate to Evaluation Criteria in the sidebar (or from Settings).
2. Select a base category or create a custom profile.
3. Toggle modules on or off based on your needs.
4. Adjust weights if available (how much each module contributes to the overall score).
5. Review the Live Preview panel on the right:
   • Estimated token count (affects processing cost)
   • Sample output — what the AI will return for a typical conversation
6. Save the profile.

Live Preview

The right-side preview panel updates in real-time as you configure modules:

• Token Count — estimated tokens consumed per analysis (directly affects cost)
• Cost Estimate — approximate cost per analysis based on your LLM provider
• Sample Output — example of what the analysis result looks like

Use the preview to find the right balance between analysis depth and cost.

Linking Profiles to Projects

To assign an evaluation profile to a project:

1. Go to Data Sources > select a project > Settings.
2. Find the Prompt Profile section.
3. Select the desired evaluation profile from the dropdown.
4. Save.

All new records in that project will use the selected profile. Existing records are not affected unless you manually re-analyze them.

The project profile acts as a fallback: it applies when no category-bound profile exists for the conversation’s detected category (see Category-Bound Profiles).

Access by Role

Feature	Client	Admin	Supervisor	Agent
View profiles	✅	✅	❌	❌
Create/edit profiles	✅	✅	❌	❌
Delete profiles	✅	✅	❌	❌
Link to projects	✅	✅	❌	❌
View live preview	✅	✅	❌	❌

Tips & Best Practices

• Start with a base category and customize — don’t build from scratch.
• Enable only the modules you need. More modules = higher cost per analysis and longer processing.
• Use the Live Preview to estimate costs before deploying a new profile.
• Create different profiles for different conversation types (e.g., one for sales calls, another for support).
• After changing a profile, re-analyze a few recent records to verify the results meet expectations.
• The Compliance module is recommended for regulated industries — it ensures recording disclaimers and mandatory phrases are consistently checked.