self-play
Self-Play Schema Probing
You are running the self-play loop for the ethpandaops/panda project. This finds schema ambiguities by asking the same question N times with different personas and checking if the generated queries agree on which tables to use. When they disagree, you use schema introspection to determine the correct tables and write the fix autonomously.
The primary metric is average entropy across all probes. Lower is better (0 = perfect agreement). This number should trend down over time as you add examples and runbooks.
Prerequisites
The panda repo must be at the working directory. The probe infrastructure lives in tests/eval/.
Server: Build first, then the probe runner auto-starts a local server on :2481:
make build # builds panda-server binary
Dependencies: The evaluator LLM needs OPENROUTER_API_KEY set in the environment.
The Loop
More from ethpandaops/panda
query
Query Ethereum network data via ethpandaops CLI or MCP server. Use when analyzing blockchain data, block timing, attestations, validator performance, network health, or infrastructure metrics. Provides access to ClickHouse (blockchain data), Prometheus (metrics), Loki (logs), and Dora (explorer APIs).
5investigate
Debug Ethereum devnet or network issues. Use when diagnosing finality delays, network splits, offline nodes, client bugs, or general network health problems. Works for both local Kurtosis devnets and remote hosted deployments.
4install-mcp
>-
3install-panda
>-
2create-runbook
Extract a reusable runbook from a successful investigation or troubleshooting session. Use after completing a multi-step diagnosis that could help future investigations.
1create-module
Add a new datasource module to ethpandaops/panda. Triggers on: add module, new module, create module, add plugin, new plugin, create plugin, add datasource.
1