AgentHub - Multi-Agent DAG Orchestration

Category: Engineering / AI Agents Maintainer: Claude Skills Team

Overview

AgentHub provides patterns and tools for orchestrating multiple AI agents as a directed acyclic graph (DAG). Instead of one agent doing everything sequentially, AgentHub lets you decompose complex tasks into sub-tasks, assign each to a specialized agent, define dependencies between them, and merge their outputs into a coherent result.

The core insight: complex tasks decompose better than they scale. A 10-step sequential task run by one agent hits context limits and quality degradation. Five parallel agents with clear scopes and a merge step produce better results faster.

Sub-Skills

This skill uses compound sub-skill architecture. Each sub-skill in skills/ handles a stage of the orchestration lifecycle:

Sub-Skill	File	Purpose
Init	`skills/init.md`	Initialize a multi-agent workflow definition
Run	`skills/run.md`	Execute a defined workflow end-to-end
Spawn	`skills/spawn.md`	Spawn individual agents within a workflow
Board	`skills/board.md`	Dashboard showing agent status and progress
Eval	`skills/eval.md`	Evaluate agent outputs for quality and consistency
Merge	`skills/merge.md`	Merge outputs from multiple agents into final result
Status	`skills/status.md`	Show workflow execution status and health

Sub-Skill Flow

Init ──> Run ──> Spawn (parallel) ──> Eval ──> Merge
                      │                            │
                    Board ◄──── Status ◄───────────┘

Lifecycle: Init defines the workflow DAG, Run orchestrates execution, Spawn creates individual agents, Board provides real-time visibility, Eval checks output quality, Merge combines results, and Status reports overall health.

Scripts

Script	Purpose
`scripts/dag_analyzer.py`	Analyze DAG definitions for cycles, unreachable nodes, and bottlenecks
`scripts/board_manager.py`	Manage agent task boards with status tracking
`scripts/result_ranker.py`	Rank and merge outputs from multiple agents
`scripts/session_manager.py`	Manage orchestration sessions and state

Core Concepts

Workflow DAG

A workflow is a directed acyclic graph where:

Nodes are agent tasks with a defined scope, inputs, and expected outputs
Edges are dependencies: agent B cannot start until agent A completes
Root nodes have no dependencies and start immediately
Terminal nodes have no dependents and feed into the merge step

┌──────────┐     ┌──────────┐     ┌──────────┐
│ Research  │────>│ Analysis │────>│  Merge   │
│  Agent    │     │  Agent   │     │  Agent   │
└──────────┘     └──────────┘     └──────────┘
                       ▲
┌──────────┐           │
│ Data      │──────────┘
│ Agent     │
└──────────┘

Workflow Definition Format

{
  "name": "market-analysis",
  "description": "Comprehensive market analysis for product launch",
  "agents": {
    "researcher": {
      "task": "Research competitor landscape and market size",
      "inputs": ["product_description"],
      "outputs": ["competitor_list", "market_size"],
      "dependencies": []
    },
    "data_collector": {
      "task": "Collect pricing and feature data from competitors",
      "inputs": ["competitor_list"],
      "outputs": ["pricing_data", "feature_matrix"],
      "dependencies": ["researcher"]
    },
    "analyst": {
      "task": "Analyze positioning opportunities and pricing strategy",
      "inputs": ["pricing_data", "feature_matrix", "market_size"],
      "outputs": ["positioning_report", "pricing_recommendation"],
      "dependencies": ["data_collector", "researcher"]
    },
    "writer": {
      "task": "Write executive summary combining all findings",
      "inputs": ["positioning_report", "pricing_recommendation"],
      "outputs": ["executive_summary"],
      "dependencies": ["analyst"]
    }
  },
  "config": {
    "max_parallel": 3,
    "timeout_per_agent": 300,
    "retry_on_failure": true,
    "quality_threshold": 0.7
  }
}

Agent States

State	Description
`PENDING`	Waiting for dependencies to complete
`READY`	All dependencies met, queued for execution
`RUNNING`	Currently executing
`COMPLETED`	Finished successfully
`FAILED`	Failed after all retries
`SKIPPED`	Skipped due to upstream failure
`EVALUATING`	Output being evaluated for quality

Execution Strategy

Topological sort the DAG to determine execution order
Identify parallel groups: nodes with no inter-dependencies run simultaneously
Execute root nodes first (no dependencies)
Chain results: completed node outputs become inputs for dependents
Evaluate outputs at quality gates
Merge terminal outputs into final result

Workflows

Workflow 1: Define and Validate

1. Define agents with tasks, inputs, outputs, dependencies
2. Run dag_analyzer.py to validate:
   - No cycles in the dependency graph
   - All referenced inputs are produced by upstream agents
   - No unreachable nodes
   - Critical path length is acceptable
3. Estimate execution time based on agent count and dependencies

Workflow 2: Execute Orchestration

1. Load workflow definition
2. Initialize session (session_manager.py)
3. Topological sort to determine execution order
4. For each parallel group:
   a. Spawn agents (up to max_parallel)
   b. Monitor progress on board
   c. Collect outputs on completion
   d. Evaluate outputs against quality threshold
5. Pass outputs to downstream agents as inputs
6. Merge final outputs
7. Generate execution report

Workflow 3: Evaluate and Iterate

1. Collect all agent outputs
2. Run quality evaluation (eval sub-skill)
3. Rank outputs by quality score (result_ranker.py)
4. If any output below threshold:
   a. Retry the agent with adjusted instructions
   b. Or flag for human review
5. Merge passing outputs into final result

Common Patterns

Fan-Out / Fan-In

Multiple independent agents work in parallel, then a single agent merges results:

Task A ──┐
Task B ──┼──> Merge
Task C ──┘

Pipeline

Sequential agents where each transforms the previous output:

Extract ──> Transform ──> Load ──> Validate

Reducer

Multiple agents produce competing outputs, ranked and best one selected:

Agent 1 ──┐
Agent 2 ──┼──> Rank ──> Best Output
Agent 3 ──┘

Validator Chain

Each agent validates the previous agent's work:

Generate ──> Review ──> Fix ──> Approve

Best Practices

Small, focused agent scopes -- each agent should have a single clear objective
Explicit inputs/outputs -- never rely on implicit shared state between agents
Quality gates between stages -- evaluate before passing outputs downstream
Timeout per agent -- prevent runaway agents from blocking the workflow
Retry with context -- when retrying a failed agent, include the failure reason
Merge strategy documented -- how competing or complementary outputs combine
Critical path awareness -- optimize the longest dependency chain first
Idempotent agents -- agents should produce the same output given the same input

Common Pitfalls

Pitfall	Why It Happens	Fix
Cycle in DAG	Agent A depends on B which depends on A	Run dag_analyzer.py before execution
Output format mismatch	Agent B expects JSON, Agent A produces markdown	Define explicit output schemas per agent
Single bottleneck agent	One agent depends on everything	Restructure DAG to parallelize dependencies
Lost context between agents	Outputs too terse for downstream use	Require structured output with context preservation
Quality degradation in merge	Naive concatenation loses coherence	Use a dedicated merge agent with synthesis instructions
Runaway execution time	No timeouts, retry loops	Set timeout_per_agent and max retries

Troubleshooting

Problem	Cause	Solution
Workflow hangs at agent N	Dependency not met or agent timeout	Check board for PENDING agents; verify upstream completed; check timeout config
Merged output is incoherent	No merge strategy defined	Use the merge sub-skill with explicit synthesis instructions
Agent produces wrong format	Input/output contract unclear	Define JSON schemas for agent inputs and outputs
DAG validation fails with cycle	Circular dependency in definition	Use dag_analyzer.py to identify the cycle; restructure the dependency chain
Quality eval fails everything	Threshold too strict for task complexity	Lower threshold or add a revision step before eval

Success Criteria

DAG validation passes on every workflow definition before execution
Parallel execution utilization above 60% -- agents running in parallel most of the time
Quality gate pass rate above 80% -- agent outputs meet threshold on first attempt
End-to-end execution time within 2x critical path -- parallelization delivers real speedup
Zero lost outputs -- every agent's output is captured and available for merge/review
Merge coherence score above 0.7 -- final merged output reads as a unified deliverable

Scope and Limitations

This skill covers:

Multi-agent workflow design with DAG dependency graphs
Agent spawning, monitoring, and lifecycle management
Output quality evaluation and ranking
Result merging strategies for coherent final deliverables

This skill does NOT cover:

Individual agent design or prompt engineering (see agent-designer)
Agent memory and self-improvement (see self-improving-agent)
Infrastructure for running agents (compute, scheduling, deployment)
Real-time streaming communication between agents

Integration Points

Skill	Integration	Data Flow
`agent-designer`	Defines individual agent capabilities that become DAG nodes	Agent specs flow in; execution results flow back for agent tuning
`self-improving-agent`	Each agent can use self-improvement patterns to get better	Session feedback from orchestration feeds into agent learning loops
`prompt-engineer-toolkit`	Agent task prompts benefit from prompt engineering	Optimized prompts improve individual agent quality within the DAG
`context-engine`	Manages what context each agent sees	Context retrieval provides relevant inputs to each spawned agent
`observability-designer`	Monitors workflow execution and agent health	Agent state transitions and timing metrics feed into dashboards

agenthub