Feedback Analyzer

Overview

feedback-analyzer evaluates skill effectiveness through analysis of usage data, feedback, metrics, and outcomes.

Purpose: Data-driven understanding of what works and what doesn't

The 4 Analysis Operations:

Collect Usage Data - Gather metrics on skill usage and effectiveness
Measure Effectiveness - Quantify impact and ROI of skills
Analyze Trends - Identify patterns in usage and effectiveness
Extract Insights - Generate actionable insights from data

When to Use

After skills have been used (have usage data)
Measuring toolkit ROI and impact
Understanding which skills provide most value
Identifying underutilized skills
Data-driven improvement decisions

Operations

Operation 1: Collect Usage Data

Purpose: Gather data on how skills are used

Data Sources:

Build times (how long to build skills?)
Usage frequency (which skills used most?)
Effectiveness metrics (do skills achieve purposes?)
Quality scores (from reviews)
User feedback (satisfaction, issues)

Process:

Identify data sources
Collect available metrics
Document usage patterns
Organize data for analysis

Output: Usage data collection

Time: 30-60 minutes

Operation 2: Measure Effectiveness

Purpose: Quantify skill impact and ROI

Metrics:

Time savings (vs without tool)
Quality improvements (before/after)
Efficiency gains (percentage faster)
Usage rate (frequency of use)
Satisfaction (user ratings)

Process:

Define effectiveness criteria
Calculate metrics
Compare to baseline or targets
Assess ROI

Output: Effectiveness measurements with evidence

Time: 45-90 minutes

Operation 3: Analyze Trends

Purpose: Identify patterns in effectiveness over time

Process:

Plot metrics over time
Identify trends (improving/degrading/stable)
Find correlations
Detect anomalies

Output: Trend analysis with insights

Time: 45-90 minutes

Operation 4: Extract Insights

Purpose: Generate actionable insights from data

Process:

Synthesize findings
Identify high-impact insights
Make recommendations
Prioritize actions

Output: Data-driven insights and recommendations

Time: 30-60 minutes

Example Analysis

Effectiveness Analysis: Development Toolkit
===========================================

Usage Data (Skills 1-23):
- Build times: 2h - 20h (mean: 6.8h)
- Efficiency: 35% - 97% faster than baseline (mean: 85%)
- Quality: 100% pass rate (5/5 structure)

Effectiveness Metrics:
- Time Saved: 392 hours total (85% reduction)
- Quality: Maintained (100% Grade A)
- Completion: 100% (all 23 finished)
- ROI: 392h saved / 68h invested = 576% ROI

Trends:
✅ Improving: Efficiency compounds (72% → 97%)
✅ Stable: Quality consistent (all 5/5)
⚠️ Plateau: Efficiency plateaus ~85-90% for simple skills

Insights:
1. Toolkit highly effective (576% ROI, 85% efficiency)
2. Quality maintained despite speed (100% pass rate)
3. Efficiency plateaus at 85-90% (cannot exceed certain minimum times)
4. Complex skills still benefit (35-50% faster)

Recommendations:
1. Continue using toolkit (proven effective)
2. Expect 85-90% efficiency for simple/medium skills
3. Adjust estimates for complex skills (30-50% faster, not 85%)
4. Focus on quality maintenance (already excellent)

Quick Reference

Operation	Focus	Time	Output
Collect Usage Data	Gather metrics	30-60m	Data collection
Measure Effectiveness	Quantify impact, ROI	45-90m	Effectiveness metrics
Analyze Trends	Patterns over time	45-90m	Trend analysis
Extract Insights	Actionable insights	30-60m	Recommendations

Integration: Uses data from skill-evolution-tracker, analysis skill

feedback-analyzer provides data-driven understanding of toolkit effectiveness for evidence-based improvement decisions.