writing-evals

Risky tool definition detected

Malicious tool definition detected

dataset: process.env.AXIOM_DATASET, // flagSchema, include: ['**/*.eval.{ts,js}'], exclude: ['**/node_modules/**', '**/dist/**', '**/build/**'], // Uncomment to track token usage per eval run: // instrumentation: (env) => setupInstrumentation(env), timeoutMs: 60_000, }, });

Tool passed security scan

Risky tool definition detected

Check if they've already done this before suggesting it. ### Option 1: OAuth login (recommended) ```bash # Login via browser OAuth npx axiom auth login # Verify authentication npx axiom auth status # Switch organizations (if multiple) npx axiom auth switch # Logout npx axiom auth logout ``` ### Option 2: Environment variables Store in `.env` at the project root: ```bash AXIOM_URL="https://api.axiom.co" AXIOM_TOKEN="API_TOKEN" AXIOM_DATASET="DATASET_NAME" AXIOM_ORG_ID="ORGANIZATION_ID" ``` The CL

**Before writing online eval code, always read the SDK's bundled docs first** — they match the installed version and contain the latest API, parameters, and patterns: ```bash cat node_modules/axiom/dist/docs/evals/online/functions/onlineEval.md ``` --- ## Common Pitfalls | Problem | Cause | Solution | |---------|-------|----------| | "All flag fields must have defaults" | Missing `.default()` on a leaf field | Add `.default(value)` to every leaf in flagSchema | | "Union types not supported" | Us

Malicious tool definition detected

Requires the sre skill's # axiom-query script and a configured Axiom deployment (~/.config/axiom-sre/config.toml).

Tool passed security scan

Malicious tool definition detected

}); // if (span && result.usage) { // span.setAttribute('gen_ai.usage.input_tokens', result.usage.prompt_tokens); // span.setAttribute('gen_ai.usage.output_tokens', result.usage.completion_tokens); // span.setAttribute('gen_ai.request.model', 'gpt-4o-mini'); // span.setAttribute('gen_ai.response.model', result.model); // } // return result.choices[0].message.content; }, ``` --- ## CLI Options ``` axiom eval [target] [options] Arguments: target file, directory, glob, or eval name (default: ".") O

Malicious tool definition detected

{ 'X-AXIOM-ORG-ID': orgId } : {}), }, }); provider = new NodeTracerProvider({ resource: resourceFromAttributes({ [ATTR_SERVICE_NAME]: 'my-app-evals' }), spanProcessors: [new BatchSpanProcessor(exporter)], }); provider.register(); initAxiomAI({ tracer: trace.getTracer('my-app-tracer'), redactionPolicy: RedactionPolicy.AxiomDefault }); return { provider }; };

Tool passed security scan