AgentObservability

Fail

Audited by Gen Agent Trust Hub on Feb 16, 2026

Risk Level: HIGHDATA_EXFILTRATIONCOMMAND_EXECUTION
Full Analysis
  • Data Exposure & Exfiltration (HIGH): The skill exposes the entire agent interaction history over the network.
  • Evidence: apps/client/vite.config.ts sets the host to 0.0.0.0, making the dashboard accessible on all network interfaces.
  • Evidence: apps/server/src/index.ts starts a Bun server on port 4000 without any authentication or authorization mechanisms.
  • Risk: Anyone on the same network (including via Tailscale as suggested in the config) can access the dashboard and view sensitive event payloads containing private code, credentials, or system information captured by the agent.
  • Persistence Mechanisms (HIGH): The skill configures itself to run persistently on the host system.
  • Evidence: AUTO-START.md and launchd/install.sh describe the creation of macOS LaunchAgents (com.pai.observability-server.plist and com.pai.observability-client.plist) in ~/Library/LaunchAgents/.
  • Risk: This ensures that the monitoring and network-exposed services are automatically started on every boot, increasing the window of vulnerability for data exposure.
  • Indirect Prompt Injection Surface (MEDIUM): The dashboard ingests and displays untrusted data from agent interactions.
  • Ingestion Points: apps/server/src/file-ingest.ts reads JSONL files generated by the capture-all-events.ts hook.
  • Boundary Markers: None. The system parses and displays the payload of events directly.
  • Capability Inventory: The server maintains a SQLite database (themes.db) and provides APIs for theme management which could be targets for schema confusion or injection.
  • Sanitization: The client uses Vue 3, which provides some protection against XSS, but raw event payloads are displayed in pre tags, potentially allowing for UI-based deception if an agent processes malicious external content.
  • Unsafe Command Execution (LOW): The setup process requires making scripts executable and modifying system-wide agent settings.
  • Evidence: SETUP.md instructs the user to chmod +x scripts and append commands to settings.json that execute the hook on every tool use.
Recommendations
  • AI detected serious security threats
Audit Metadata
Risk Level
HIGH
Analyzed
Feb 16, 2026, 02:55 AM