orchestration
Multi-Project Orchestration
Plan Format
An orchestration plan is a JSON structure that defines projects, tasks, dependencies, and execution order:
{
"planId": "orch-abc123",
"summary": "E-commerce platform with React frontend and Node.js backend",
"createdAt": "2024-01-15T10:00:00Z",
"projects": [
{
"name": "ecommerce-frontend",
"path": "./ecommerce-frontend",
"techStack": "React + TypeScript + TailwindCSS",
"description": "Customer-facing web application"
},
{
"name": "ecommerce-backend",
"path": "./ecommerce-backend",
"techStack": "Node.js + Express + PostgreSQL",
"description": "REST API server"
}
],
"tasks": [
{
"id": "t1",
"project": "ecommerce-frontend",
"title": "Initialize React project",
"description": "Create React app with TypeScript, configure routing and TailwindCSS",
"dependsOn": [],
"complexity": "low",
"estimatedMinutes": 10
},
{
"id": "t2",
"project": "ecommerce-backend",
"title": "Initialize backend project",
"description": "Set up Express server with TypeScript, configure PostgreSQL connection",
"dependsOn": [],
"complexity": "low",
"estimatedMinutes": 10
},
{
"id": "t3",
"project": "ecommerce-backend",
"title": "Implement authentication API",
"description": "JWT auth with login, register, refresh endpoints",
"dependsOn": ["t2"],
"complexity": "medium",
"estimatedMinutes": 30
},
{
"id": "t4",
"project": "ecommerce-frontend",
"title": "Build auth UI",
"description": "Login/register pages, auth context, token management",
"dependsOn": ["t1", "t3"],
"complexity": "medium",
"estimatedMinutes": 25
}
],
"executionOrder": [
["t1", "t2"],
["t3"],
["t4"]
]
}
Rules for planning:
- Each phase in
executionOrderruns in parallel - Tasks within a phase must have all
dependsOnsatisfied by prior phases - Each project gets one terminal — tasks for the same project execute sequentially
- Task descriptions must be detailed enough for a sub-agent to execute independently
Inter-Agent Communication Protocol
Sub-agents (project-workers) communicate status using markers written to terminal output:
Status Markers
| Marker | Format | Purpose |
|---|---|---|
[PROGRESS:terminal-id] |
[PROGRESS:term-123] 45% - Implementing auth middleware |
Progress update with percentage and description |
[DONE:terminal-id] |
[DONE:term-123] Authentication API complete with tests |
Task completed successfully |
[ERROR:terminal-id] |
[ERROR:term-123] PostgreSQL connection refused |
Unrecoverable error |
[QUESTION:terminal-id] |
[QUESTION:term-123] Should auth use JWT or session cookies? |
Needs user decision |
Writing markers from sub-agent
The project-worker agent writes markers by echoing them in the terminal:
echo "[PROGRESS:$TERMINAL_ID] 30% - Database schema created"
echo "[DONE:$TERMINAL_ID] Task complete: user authentication implemented"
echo "[ERROR:$TERMINAL_ID] Cannot connect to database — check connection string"
echo "[QUESTION:$TERMINAL_ID] Frontend framework preference: Next.js or Vite+React?"
Reading markers from orchestrator
The meta-orchestrator reads terminal output and parses markers:
# Read last 50 lines from a sub-agent's terminal
OUTPUT=$(aiter terminal read --id terminal-456 --lines 50)
# Parse with jq — look for markers in output lines
echo "$OUTPUT" | jq -r '.data.lines[]' | grep -E '\[(PROGRESS|DONE|ERROR|QUESTION):'
Dispatch Workflow
Step 1: Create or find the project
# Check if project already exists
EXISTING=$(aiter project list | jq -r ".data[] | select(.name == \"$PROJECT_NAME\") | .id")
if [ -z "$EXISTING" ]; then
# Create new project
PROJECT_ID=$(aiter project create --path "$PROJECT_PATH" --name "$PROJECT_NAME" | jq -r '.data.id')
else
PROJECT_ID=$EXISTING
fi
Step 2: Create a terminal in the project
TERM_RESULT=$(aiter terminal create --projectId $PROJECT_ID)
TERMINAL_ID=$(echo $TERM_RESULT | jq -r '.data.terminalId')
sleep 2 # Wait for shell initialization
Step 3: Send the task prompt to the sub-agent
Write the full task template to the terminal, which the AI CLI running in that terminal will process:
aiter terminal write --id $TERMINAL_ID --text "$(cat <<'TASK_EOF'
You are a project-worker sub-agent. Execute the following task:
**Task:** Initialize React project with TypeScript
**Project:** ecommerce-frontend
**Terminal ID:** terminal-456
**Requirements:**
- Create React app with Vite + TypeScript
- Install and configure TailwindCSS
- Set up React Router for navigation
- Create basic project structure (pages/, components/, hooks/, utils/)
**Communication Protocol:**
- Report progress: echo "[PROGRESS:terminal-456] <percentage>% - <description>"
- Report completion: echo "[DONE:terminal-456] <summary of what was done>"
- Report errors: echo "[ERROR:terminal-456] <error description>"
- Ask questions: echo "[QUESTION:terminal-456] <question>"
Begin working now.
TASK_EOF
)"
Step 4: Monitor progress
# Poll loop — check every 5 seconds
while true; do
OUTPUT=$(aiter terminal read --id $TERMINAL_ID --lines 30 | jq -r '.data.lines[]')
if echo "$OUTPUT" | grep -q "\[DONE:$TERMINAL_ID\]"; then
echo "Task completed!"
break
elif echo "$OUTPUT" | grep -q "\[ERROR:$TERMINAL_ID\]"; then
ERROR=$(echo "$OUTPUT" | grep "\[ERROR:$TERMINAL_ID\]" | tail -1)
echo "Error detected: $ERROR"
break
elif echo "$OUTPUT" | grep -q "\[QUESTION:$TERMINAL_ID\]"; then
QUESTION=$(echo "$OUTPUT" | grep "\[QUESTION:$TERMINAL_ID\]" | tail -1)
echo "Question: $QUESTION"
# Present to user for decision
break
fi
sleep 5
done
Task Template
Full template for dispatching to a project-worker:
You are a project-worker sub-agent executing a task within a larger orchestration plan.
## Task Details
- **Task ID:** {task.id}
- **Title:** {task.title}
- **Project:** {task.project}
- **Terminal ID:** {terminal_id}
- **Description:** {task.description}
- **Dependencies completed:** {list of completed dependency tasks and their outcomes}
## Context
{Additional context from the orchestrator: shared resources, API contracts, design decisions}
## Communication Protocol
Report your status using these exact markers:
1. **Progress updates** (every significant milestone):
echo "[PROGRESS:{terminal_id}] <0-100>% - <what you just completed>"
2. **Completion** (when task is fully done):
echo "[DONE:{terminal_id}] <one-line summary of deliverables>"
3. **Errors** (when blocked and cannot continue):
echo "[ERROR:{terminal_id}] <what went wrong and what you tried>"
4. **Questions** (when you need a decision):
echo "[QUESTION:{terminal_id}] <specific question with options if applicable>"
## Rules
- Work autonomously — do not wait for instructions after starting
- Report progress at least every 5 minutes of work
- If a dependency artifact is missing, report [ERROR] immediately
- Test your work before reporting [DONE]
- Include relevant file paths in your [DONE] message
Begin working now.
State Tracking
Orchestration state is persisted to .aiter/memory/orchestration.md so it survives session restarts and can be read by the heartbeat agent.
Orchestration State File Format
---
type: orchestration-state
planId: orch-abc123
created: 2024-01-15T10:00:00Z
status: in_progress
---
## Plan Summary
E-commerce platform with React frontend and Node.js backend
## Projects
| Name | Project ID | Terminal ID | Status |
|------|-----------|-------------|--------|
| ecommerce-frontend | project-111 | terminal-aaa | active |
| ecommerce-backend | project-222 | terminal-bbb | active |
## Execution State
### Phase 1 (completed)
- **t1** [DONE]: Initialize React project — terminal-aaa
- Completed: 2024-01-15T10:05:00Z
- Output: Vite + React + TypeScript configured, TailwindCSS installed
- **t2** [DONE]: Initialize backend project — terminal-bbb
- Completed: 2024-01-15T10:06:00Z
- Output: Express + TypeScript + PostgreSQL configured
### Phase 2 (in_progress)
- **t3** [IN_PROGRESS 65%]: Implement authentication API — terminal-bbb
- Started: 2024-01-15T10:07:00Z
- Last update: Implementing JWT refresh token logic
### Phase 3 (pending)
- **t4** [PENDING]: Build auth UI — terminal-aaa
- Blocked by: t3
Updating orchestration state
Use the Edit tool to update specific sections of orchestration.md:
- When a task starts: change
[PENDING]to[IN_PROGRESS 0%] - On progress: update the percentage and last update line
- On completion: change to
[DONE]with timestamp and output - On error: change to
[ERROR]with error description - When all tasks complete: change top-level
statustocompleted
Notification Integration
Send notifications at key orchestration milestones:
# Phase completed
aiter notify send --message "Orchestration '$PLAN_SUMMARY': Phase $PHASE_NUM completed ($COMPLETED/$TOTAL tasks done)" --priority normal
# Task error
aiter notify send --message "Orchestration error in task '$TASK_TITLE': $ERROR_MSG" --priority high
# All tasks complete
aiter notify send --message "Orchestration '$PLAN_SUMMARY' completed successfully! All $TOTAL tasks done." --priority normal
# Send to IM channel if configured
aiter message send --channel $CHANNEL_ID --text "**Orchestration Update:** Phase 2 complete. Starting Phase 3." --format markdown
Error Recovery
| Error | Recovery |
|---|---|
| Terminal closed unexpectedly | Recreate terminal in same project, re-dispatch failed task |
| Sub-agent stuck (no progress >10 min) | Read terminal output, check for interactive prompts, send Enter or answer |
| Dependency task failed | Skip dependent tasks, report error to user with options |
| AiTer disconnected | Wait 10 seconds, retry aiter status info, escalate if still down |
| Project creation failed | Check path exists, check permissions, try alternative path |