EngFlow Artifact Download

Download test logs and artifacts from EngFlow CI invocations (mesolite.cluster.engflow.com) using engflow_artifacts.py (co-located in this skill directory).

Prerequisites

The engflow_auth CLI must be installed and authenticated:

# One-time login (opens browser for Google SSO)
engflow_auth login mesolite.cluster.engflow.com

# Verify authentication
engflow_auth export mesolite.cluster.engflow.com

Quick Reference

Use run.sh to invoke the script — it manages a local venv with dependencies automatically. The script is at .claude/skills/engflow-artifacts/run.sh:

Command	Purpose
targets <ID>	List failed targets in an invocation
list <ID> --target <LABEL>	List per-shard artifacts for a target
download <ID> --target <LABEL> --shard N	Download artifacts for a shard
download <ID> --target <LABEL> --shard N --run M	Download a specific run
blob <HASH> <SIZE>	Download a specific CAS blob
url '<FULL_URL>'	Download from a full EngFlow invocation URL

Typical Investigation Workflow

EF=.claude/skills/engflow-artifacts/run.sh

# 1. Find what failed
./$EF targets daa807a0-3589-40a5-94b5-3440c7490d6a

# 2. List shards for the failed target
./$EF list daa807a0-3589-40a5-94b5-3440c7490d6a \
  --target '//pkg/sql/ttl:ttl_test'

# 3. Download the failing shard's logs
./$EF download daa807a0-3589-40a5-94b5-3440c7490d6a \
  --target '//pkg/sql/ttl:ttl_test' --shard 100 --outdir /tmp/engflow-test

# 4. Read the test log
cat /tmp/engflow-test/test.log

# 5. Inspect extracted CockroachDB server logs from outputs.zip
ls /tmp/engflow-test/

Per-Shard Artifacts

Every shard has:

• test.xml — JUnit XML test results (referenced via CompactDigest: raw 32-byte SHA-256 hash)
• test.log — stdout/stderr from the test run (referenced via bytestream:// URL)

Some shards also have:

• outputs.zip — CockroachDB server logs captured during the test
• manifest — lists files in outputs.zip

How It Works

• engflow_auth export provides a JWT token.
• gRPC-web calls via curl to EngFlow's internal v1alpha ResultStore API (GetTarget, GetTargetLabelsByStatus).
• Protobuf parsing via google.protobuf with a reverse-engineered .proto schema (resultstore.proto). Responses are properly deserialized — shard numbers and run numbers come from actual protobuf fields, not byte-pattern heuristics.
• CAS blob downloads via curl with the auth token as a cookie.
• outputs.zip is auto-extracted after download.

Each test target has multiple shards (parallel test splits) and runs (repeated executions). Use --shard to select a shard and --run to select a run (defaults to run 1). Shard numbers are 1-based and match the testReportShard parameter in EngFlow URLs.

EngFlow's gRPC-web endpoint requires HTTP/2, so the script uses curl (which negotiates HTTP/2 via ALPN) rather than Python HTTP libraries that only speak HTTP/1.1. Dependencies are installed automatically by run.sh.

Extracted outputs.zip Structure

logTestName123456/
  test.log                                    # short log
  test.{node-id}.roach.{timestamp}.log        # full structured log
  test-health.log                             # health channel
  test-storage.log                            # storage channel
  test-kv-distribution.log                    # KV distribution channel

URL Structure

Invocation page: https://mesolite.cluster.engflow.com/invocations/default/{INVOCATION_ID}

With filters: ?testReportRun=1&testReportShard={N}&testReportAttempt=1&treeFilterStatus=failed#targets-{BASE64_TARGET}

Tips

• Blob hashes are SHA-256. Sizes are in bytes.
• Use list to find which shard has outputs.zip (not all shards produce one).
• If engflow_auth export fails, re-authenticate with engflow_auth login mesolite.cluster.engflow.com.
• Default output directory is /tmp/engflow-artifacts.
• --all flag on targets shows both passed and failed targets.