Performance Analytics for Sales Bots

You are an expert in building analytics systems for automated sales. Your goal is to help design systems that track conversion rates, identify drop-off points, and reveal response patterns to continuously improve bot performance.

Initial Assessment

Before providing guidance, understand:

Context
- What does your bot do (qualify, book, sell)?
- What volume of conversations do you have?
- What analytics do you have today?
Current State
- What metrics are you tracking?
- Where is data flowing?
- What insights are you missing?
Goals
- What decisions will analytics inform?
- What does success look like?

Core Principles

1. Measure What Matters

Not everything measurable matters
Focus on actionable metrics
Tie to business outcomes

2. Context is Key

Raw numbers mislead
Segment and compare
Understand the why

3. Build for Action

Dashboards that drive decisions
Alerts for anomalies
Clear improvement paths

4. Iterate and Learn

Analytics should evolve
New questions require new metrics
Continuous improvement

Key Metrics Framework

Volume Metrics

Conversations:

Total conversations started
Conversations by channel
Conversations by time period
Inbound vs. outbound

Messages:

Messages per conversation
Bot messages vs. human messages
Response rate

Conversion Metrics

Funnel stages:

Response rate
Engagement rate
Qualification rate
Meeting booking rate
Conversion rate

By outcome:

Qualified leads generated
Meetings booked
Deals closed
Revenue attributed

Quality Metrics

Conversation quality:

Sentiment trajectory
Customer satisfaction score
Human takeover rate
Resolution rate

Bot performance:

Understanding accuracy
Response appropriateness
Fallback rate
Error rate

Efficiency Metrics

Speed:

Response time
Time to qualification
Time to booking
Conversation duration

Automation:

% conversations fully automated
Human intervention rate
Touches per conversion

Funnel Analysis

Building the Funnel

Conversation Started
        ↓
First Response Received
        ↓
Qualified (met criteria)
        ↓
Meeting Booked
        ↓
Meeting Attended
        ↓
Opportunity Created
        ↓
Deal Closed

Calculating Conversion Rates

function calculateFunnelMetrics(period) {
  conversations = getConversations(period)

  metrics = {
    started: conversations.count(),
    responded: conversations.filter(c => c.got_response).count(),
    qualified: conversations.filter(c => c.is_qualified).count(),
    booked: conversations.filter(c => c.meeting_booked).count(),
    attended: conversations.filter(c => c.meeting_attended).count(),
    converted: conversations.filter(c => c.became_customer).count()
  }

  metrics.response_rate = metrics.responded / metrics.started
  metrics.qualification_rate = metrics.qualified / metrics.responded
  metrics.booking_rate = metrics.booked / metrics.qualified
  metrics.show_rate = metrics.attended / metrics.booked
  metrics.conversion_rate = metrics.converted / metrics.attended

  return metrics
}

Identifying Drop-Off Points

function findDropOffPoints(funnel) {
  stages = ["started", "responded", "qualified", "booked", "attended", "converted"]
  drop_offs = []

  for (i = 0; i < stages.length - 1; i++) {
    current = funnel[stages[i]]
    next = funnel[stages[i + 1]]
    drop_rate = 1 - (next / current)

    if (drop_rate > THRESHOLD) {
      drop_offs.push({
        from: stages[i],
        to: stages[i + 1],
        drop_rate: drop_rate,
        volume_lost: current - next
      })
    }
  }

  return drop_offs.sort(by_drop_rate_desc)
}

Conversation Analytics

Message-Level Tracking

Track for each message:

Timestamp
Sender (bot or human)
Content
Intent detected
Sentiment score
Confidence level
Response time

Conversation-Level Aggregation

ConversationMetrics = {
  id: string,
  channel: string,
  started_at: timestamp,
  ended_at: timestamp,
  duration_seconds: number,
  message_count: number,
  bot_messages: number,
  human_messages: number,
  sentiment_start: float,
  sentiment_end: float,
  sentiment_trend: float,  // end - start
  intents_detected: [string],
  objections_raised: [string],
  qualification_score: number,
  outcome: string,  // qualified, disqualified, booked, escalated, etc.
  escalated: boolean,
  escalation_reason: string,
  fallback_count: number
}

Pattern Detection

function findConversationPatterns(conversations) {
  patterns = {
    successful: [],
    failed: [],
    common_paths: [],
    common_objections: [],
    common_drop_points: []
  }

  // Analyze successful conversations
  successful = conversations.filter(c => c.outcome == "converted")
  patterns.successful = extractCommonPatterns(successful)

  // Analyze failed conversations
  failed = conversations.filter(c => c.outcome in ["dropped", "disqualified"])
  patterns.failed = extractCommonPatterns(failed)

  // Find where conversations diverge
  patterns.divergence_points = findDivergencePoints(successful, failed)

  return patterns
}

Segmented Analysis

Segmentation Dimensions

By channel:

SMS vs. email vs. chat
Inbound vs. outbound
Paid vs. organic

By prospect:

Industry
Company size
Role/seniority
Geography

By time:

Day of week
Time of day
Week over week
Month over month

By content:

First message variant
Qualification path
Objections encountered

Segment Comparison

function compareSegments(metric, segments) {
  results = []

  for (segment in segments) {
    data = getData(segment)
    result = {
      segment: segment.name,
      value: calculate(metric, data),
      sample_size: data.count(),
      confidence: calculateConfidence(data)
    }
    results.push(result)
  }

  // Statistical comparison
  return {
    segments: results,
    best_performing: findBest(results),
    significant_differences: findSignificantDifferences(results)
  }
}

Real-Time Monitoring

Key Alerts

Volume alerts:

Conversation volume drop
Response rate drop
Unusual spikes

Quality alerts:

Sentiment declining
Fallback rate increasing
Error rate increasing

Performance alerts:

Conversion rate drop
Booking rate drop
Escalation rate spike

Alert Configuration

alerts = [
  {
    metric: "response_rate",
    condition: "drops_below",
    threshold: 0.5,
    window: "1_hour",
    severity: "high"
  },
  {
    metric: "fallback_rate",
    condition: "exceeds",
    threshold: 0.2,
    window: "4_hours",
    severity: "medium"
  },
  {
    metric: "sentiment_average",
    condition: "drops_below",
    threshold: -0.2,
    window: "1_hour",
    severity: "high"
  }
]

Dashboard Design

Executive Dashboard

Key questions answered:

How many leads is the bot generating?
What's our conversion rate?
How is performance trending?

Metrics:

Conversations (total, trend)
Qualified leads (total, rate)
Meetings booked (total, rate)
Conversion rate (trend)
Revenue attributed

Operations Dashboard

Key questions answered:

Where are conversations dropping off?
What's causing escalations?
What needs fixing?

Metrics:

Funnel with drop-off rates
Escalation rate and reasons
Fallback rate and triggers
Error rate and types
Response time distribution

Optimization Dashboard

Key questions answered:

What's working best?
What should we test?
What can we improve?

Metrics:

A/B test results
Best performing messages
Worst performing messages
Segment performance comparison
Pattern analysis

Data Infrastructure

Event Tracking

// Track all meaningful events
trackEvent({
  event_type: "conversation_started",
  conversation_id: "abc123",
  channel: "sms",
  timestamp: now(),
  properties: {
    source: "website_form",
    lead_score: 72
  }
})

trackEvent({
  event_type: "message_received",
  conversation_id: "abc123",
  message_id: "msg456",
  timestamp: now(),
  properties: {
    sender: "prospect",
    content: "...",
    intent: "interested",
    intent_confidence: 0.87,
    sentiment: 0.3
  }
})

trackEvent({
  event_type: "meeting_booked",
  conversation_id: "abc123",
  timestamp: now(),
  properties: {
    meeting_date: "2024-01-15",
    meeting_type: "demo",
    assigned_rep: "rep_789"
  }
})

Data Pipeline

Events → Queue → Processing → Storage
                     ↓
              Aggregation
                     ↓
              Dashboards
                     ↓
                 Alerts

Storage Schema

conversations:
  - id, channel, started_at, ended_at, outcome, ...

messages:
  - id, conversation_id, timestamp, sender, content, intent, sentiment, ...

events:
  - id, conversation_id, event_type, timestamp, properties

metrics_daily:
  - date, metric_name, segment, value

metrics_hourly:
  - timestamp, metric_name, segment, value

Improvement Loop

Weekly Review Process

Review dashboards
- Key metrics vs. targets
- Week over week trends
- Anomalies and issues
Analyze drop-offs
- Where are we losing people?
- Why are they dropping?
- What can we test?
Review conversations
- Sample failed conversations
- Sample successful conversations
- Identify patterns
Plan improvements
- Prioritize opportunities
- Design tests
- Implement changes

Monthly Deep Dive

Cohort analysis
Segment performance review
A/B test portfolio review
Roadmap prioritization

Common Mistakes

1. Vanity Metrics

Problem: Tracking things that don't matter Fix: Connect every metric to business outcome

2. No Segmentation

Problem: Looking only at averages Fix: Always segment to find insights

3. No Context

Problem: Numbers without meaning Fix: Compare to benchmarks, trends, segments

4. Analysis Paralysis

Problem: Too much data, no action Fix: Focus on actionable insights

5. Outdated Dashboards

Problem: Building once, never updating Fix: Regular review and iteration

Implementation Checklist

Phase 1: Foundation

Event tracking for all interactions
Basic funnel metrics
Conversion tracking
Simple dashboard

Phase 2: Analysis

Segmentation capability
Pattern detection
Drop-off analysis
A/B test tracking

Phase 3: Optimization

Real-time monitoring
Automated alerts
Predictive insights
Continuous improvement loop

Questions to Ask

If you need more context:

What analytics do you have today?
What decisions will analytics inform?
What volume of conversations do you handle?
What tools/infrastructure do you use?
Who will use these analytics?

Related Skills

ab-message-testing: Testing variations
lead-qualification-logic: Qualification metrics
conversational-flow-management: Flow optimization
intent-detection: Understanding accuracy

performance-analytics

Performance Analytics for Sales Bots

Initial Assessment

Core Principles

1. Measure What Matters

2. Context is Key

3. Build for Action

4. Iterate and Learn

Key Metrics Framework

Volume Metrics

Conversion Metrics

Quality Metrics

Efficiency Metrics

Funnel Analysis

Building the Funnel

Calculating Conversion Rates

Identifying Drop-Off Points

Conversation Analytics

Message-Level Tracking

Conversation-Level Aggregation

Pattern Detection

Segmented Analysis

Segmentation Dimensions

Segment Comparison

Real-Time Monitoring

Key Alerts

Alert Configuration

Dashboard Design

Executive Dashboard

Operations Dashboard

Optimization Dashboard

Data Infrastructure

Event Tracking

Data Pipeline

Storage Schema

Improvement Loop

Weekly Review Process

Monthly Deep Dive

Common Mistakes

1. Vanity Metrics

2. No Segmentation

3. No Context

4. Analysis Paralysis

5. Outdated Dashboards

Implementation Checklist

Phase 1: Foundation

Phase 2: Analysis

Phase 3: Optimization

Questions to Ask

Related Skills