Telemetry Querying Skill

Use this skill as the entry point for any investigation, debugging, or data question that may be answered from telemetry data. It helps you decide where the relevant signal lives (metrics, logs, traces, RUM) and tells you which reference files to load before querying.

Loading References

Before querying, load the reference files for the chosen pillar:

Pillar	Load these files
Logs	references/dataprime-reference.md + references/logs-querying.md
Spans / Traces	references/dataprime-reference.md + references/spans-querying.md
Metrics	references/promql-guidelines.md + references/metrics-querying.md
RUM (frontend)	references/dataprime-reference.md + references/logs-querying.md + references/rum-querying.md + references/rum-fields.md
DataPrime syntax only	references/dataprime-reference.md

---

Safety

All query commands (cx logs, cx spans, cx metrics, cx dataprime, cx search-fields) are read-only and work in --read-only mode. They never modify data and can be run freely without --yes.

---

Quick Routing Guide

Use this table for obvious cases where one pillar is the clear first choice:

Question Type	First Choice	Fallback
UI behavior, page load, frontend errors	RUM	Traces (if backend-related)
Endpoint latency, throughput, error rates	Metrics	Traces (for per-request detail)
Service-to-service dependencies, request flow	Traces	Logs (for debug output)
Specific error messages, stack traces	Logs	Traces (for request context)
Infrastructure health (CPU, memory, disk)	Metrics	-
Business events (purchases, signups)	Depends - see Discovery Workflow	-

For ambiguous questions (e.g., "How much money did users spend last week?"), the signal could live in any pillar. Follow the Discovery Workflow below.

---

Discovery Workflow

When the answer could reside in multiple pillars, run discovery in parallel to find the best source.

Step 1: Search Metrics

Check if a relevant metric exists:

cx metrics search --name '*transaction*'
cx metrics search --name '*payment*'
cx metrics search --name '*revenue*'
cx metrics search --description "total purchase amount"

If a matching metric is found, load references/promql-guidelines.md + references/metrics-querying.md and continue.

Step 2: Search Log and Span Fields

Use semantic field search to find relevant DataPrime paths:

cx search-fields "transaction amount" --dataset logs
cx search-fields "payment total" --dataset spans
cx search-fields "purchase value" --dataset logs --limit 10

Requirements: cx search-fields needs a Coralogix API key or OAuth on the active profile. If credentials are missing, prompt the user to run cx profiles add.

If matching fields are found:

• For logs: load references/dataprime-reference.md + references/logs-querying.md
• For spans: load references/dataprime-reference.md + references/spans-querying.md

Step 3: Search the Codebase

When discovery results are ambiguous or you need to validate what a metric/field actually represents, search the codebase:

• Look for metric registration code (e.g., prometheus.NewCounter, metrics.record)
• Look for log statements that emit the field (e.g., logger.info("transaction", ...))
• Look for span attributes (e.g., span.setAttribute("purchase.amount", ...))

This confirms the semantic meaning and helps you choose the right pillar.

Step 4: Choose and Query

Based on discovery results, pick the pillar with the clearest signal, load its reference files (see Loading References), then query.

---

Fallback and Pivoting

If your initial route yields no results, pivot to another pillar.

Example pivot paths:

• Metrics empty → try traces (per-request data) or logs (event records)
• Logs empty → try traces (structured span attributes) or metrics (aggregated counters)
• Traces empty → try logs (text-based debug output)

Do not stop after one failed attempt. Try at least two pillars before concluding the data does not exist.

---

CLI Commands Reference

Command	Purpose	When to Use
cx schema	Output the full command tree as JSON	Discover all available commands and their flags
cx metrics search --name <pattern>	Find metrics by name	First step for metrics discovery
cx metrics search --description <text>	Semantic metric search	When you know what you want but not the name
cx search-fields "<text>" --dataset logs	Find log fields by description	Discovery for log-based questions
cx search-fields "<text>" --dataset spans	Find span fields by description	Discovery for trace-based questions
cx spans "filter $l.serviceName == '<service>'" --limit 10	Search spans by service	When investigating a specific service
cx dataprime list	List DataPrime commands/functions	When building log or span queries

---

Examples

Example 1: Business Question (Ambiguous Source)

Question: "How much money did people spend on the platform last week?"

Approach:

1. Search metrics: cx metrics search --name '*revenue*' and cx metrics search --name '*transaction*'
2. Search log fields: cx search-fields "transaction amount" --dataset logs
3. Search span fields: cx search-fields "payment total" --dataset spans
4. If a metric like payment_total_usd exists, load metrics references and run a range query
5. If only logs have the data, load logs references and use DataPrime aggregation
6. If traces have purchase.amount attribute, load spans references

Example 2: Latency Question (Clear First Choice)

Question: "What's the average latency of the checkout route?"

Approach:

1. First try metrics: cx metrics search --name '*checkout*latency*' or cx metrics search --name '*http*duration*'
2. If a histogram metric exists, load metrics references and use histogram_quantile
3. If no metric, fall back to traces: load spans references and aggregate span durations

Example 3: Frontend Performance (RUM)

Question: "Why is the dashboard page loading slowly for users?"

Approach:

1. This is clearly a RUM question - load references/rum-querying.md + references/rum-fields.md + references/logs-querying.md + references/dataprime-reference.md
2. Query web vitals and page load times
3. If RUM shows backend calls are slow, pivot to spans references for the API calls

Example 4: Error Investigation (Logs + Traces)

Question: "Why are users getting 500 errors on the payment endpoint?"

Approach:

1. Check error rate metrics → load metrics references
2. Search for error logs → load logs references
3. Get traces for failed requests → load spans references
4. Cross-reference: find trace IDs in logs, then fetch full traces for root cause

---

Beyond Investigation

Not every question is answered by querying data. If the user's intent is operational rather than investigative, route to the appropriate workflow skill:

User Intent	Route To
Reducing costs, checking usage, TCO policies	cx-cost-optimization
Incident triage, SLO breaching, who got paged	cx-incident-management
Setting up monitoring, webhooks, notifications	cx-observability-setup
Configuring parsing rules, enrichments, E2M	cx-data-pipeline
Access audit, API keys, user management	cx-platform-admin
Creating or managing dashboards	cx-create-dashboard

---

Key Principles

• Load references before querying: check the Loading References table first
• Discover before querying: always run search/discovery to find the right source
• Parallel discovery: for ambiguous questions, search metrics, logs, and spans concurrently
• Validate with code: when unsure what a metric or field represents, check the codebase
• Pivot on failure: if one pillar is empty, try another before giving up