Agent Pattern Verification

Purpose

Verify AI agent code for common anti-patterns that can cause infinite loops, runaway retries, tool mismatches, and context overflow. All analysis happens locally.

When to Use

Trigger this skill when the user asks to:

• "verify agent patterns"
• "check agent loops"
• "verify tools"
• "check retry limits"
• "verify agent safety"

Note: For full verification including security, quality, and language-specific checks, tell the user to say "verify agent".

Process

Step 1: Detect Agent Framework

Identify the agent framework by checking imports in Python/TypeScript files:

Import Pattern	Framework
from langgraph or import langgraph	LangGraph
from crewai or import crewai	CrewAI
from autogen or import autogen	AutoGen
from langchain or import langchain	LangChain
Direct openai/anthropic SDK only	Custom

Also check for framework config files: langgraph.json, crew.yaml.

Step 2: Locate Agent Files

Find files to analyze:

Priority files:

• graph.py, graph.ts - Agent workflow definitions
• tools.py, tools.ts, tools/*.py, tools/*.ts - Tool implementations
• state.py, state.ts - State schemas
• prompts.py, prompts/*.md, system.md - Prompt templates
• agent.py, agent.ts - Main agent logic

Directories to check:

• src/agent/, agent/, src/, project root
• lib/, app/, packages/

Exclude from analysis:

• skills/ directory — these are skill definitions, not agent system prompts

Step 3: Run Pattern Checks

Check Tiers

• [PATTERN] — Mechanical check. Apply exactly as written.
• [HEURISTIC] — Judgment required. Mark findings clearly.

Tag every finding with [P] for pattern or [H] for heuristic.

---

3.1 [PATTERN] Loop Safety

Apply mechanically. Do not pass a loop because it "looks like it might terminate."

Pattern to find	Pass condition	Severity
while True: in Python	A break statement exists within the same block scope	⚠️ Warning if absent
for { } in Go	A break or return exists within the block	⚠️ Warning if absent
while (true) in TS/JS	A break or return exists within the block	⚠️ Warning if absent
Function calls itself recursively	A non-recursive return path exists (base case), OR a depth/counter parameter is present	⚠️ Warning if absent

[HEURISTIC] Fallback: Unrecognized Loop Patterns

After applying the pattern table, also scan for:

• Loops where termination depends entirely on external/runtime state with no timeout
• Generator functions that yield indefinitely without documented exit
• Event/polling loops without timeout parameters
• Recursive call chains across multiple functions without depth tracking

Flag as ⚠️ Warning: "Potential unbounded loop not matching known patterns — verify termination condition manually"

---

3.2 [PATTERN] Retry Limit Enforcement

Apply mechanically. If required parameter is absent, flag as ❌ Issue.

Python — Decorator-based:

Library/Pattern	Required parameter	Fail condition
@retry (tenacity)	stop=stop_after_attempt(n) or stop=stop_after_delay(n)	stop= absent
@backoff.on_exception	max_tries=n	max_tries= absent

Python — HTTP client retry:

Library/Pattern	Required parameter	Fail condition
urllib3.Retry(...)	total=n where n > 0	total= absent or total=0
HTTPAdapter(max_retries=Retry(...))	Retry object must have total=n	total= absent
httpx.HTTPTransport(retries=n)	retries=n where n > 0	retries= absent or retries=0

Python — AWS SDK (boto3):

Library/Pattern	Required parameter	Fail condition
Config(retries={...})	max_attempts > 1	max_attempts absent or ≤ 1

Note: boto3 without explicit retry config uses SDK defaults (3 attempts) — do not flag absence.

JavaScript/TypeScript:

Library/Pattern	Required parameter	Fail condition
retry(...) (async-retry)	retries: n in options	retries: absent
pRetry(...) (p-retry)	retries: n in options	retries: absent

Custom retry loops (all languages):

Pattern to find	Pass condition	Fail condition
Loop + try/except + continue	Integer counter with max check	No counter → ❌ Issue

[HEURISTIC] Fallback: Unrecognized Retry Patterns

After applying pattern tables, scan for:

• Functions/decorators with "retry" in name not in tables above
• Imported modules with "retry" in package name (e.g. stamina, aiohttp_retry)
• Loops with sleep + exception handling + re-invocation without visible counter
• Config keys like max_retries, retry_count, attempts

Flag as ⚠️ Warning: "Potential retry pattern not matching known libraries — verify retry bounds manually"

---

3.3 [PATTERN] Tool Registry Consistency

Step 1: Collect defined tools

Scan tool definition files. A name found by any pattern counts as registered.

Python — decorator patterns:

Pattern	How to extract name
@tool (LangChain) on def	Function name below decorator
@function_tool (OpenAI Agents SDK) on def	Function name below decorator
@tool(name="...")	Use name= argument value

Python — dict/list patterns:

Pattern	How to extract name
{"type": "function", "function": {"name": "..."}} (OpenAI)	Value of "name" inside "function"
{"name": "...", "input_schema": {...}} (Anthropic)	Top-level "name"
{"name": "...", "description": "...", "parameters": {...}}	Top-level "name"
ToolNode([func1, func2, ...]) (LangGraph)	Each function name in list
tools = [func1, func2] / TOOLS = [...]	Each identifier in list

TypeScript/JavaScript:

Pattern	How to extract name
{ type: "function", function: { name: "..." } } (OpenAI)	name: inside function:
tool({ description: "...", parameters: z.object({...}) })	The const variable name
new DynamicTool({ name: "...", ... }) (LangChain.js)	Value of name:
zodFunction({ name: "...", ... })	Value of name:

Step 2: Collect tool references from prompts

Scan .md, .txt, prompts.py for backtick-quoted identifiers naming capabilities.

Step 3: Cross-reference

Finding	Severity
Reference not in definition list	❌ Issue (hallucinated tool)
Defined tool not in any prompt	⚠️ Warning (undocumented tool)

[HEURISTIC] Tools never bound to LLM

Find where tools are defined and where LLM is invoked. If tools exist but are never connected to the LLM call, flag as ❌ Issue: "Tools defined but never connected to LLM invocation"

[HEURISTIC] Fallback: Unrecognized Tool Definitions

Scan for tool-like structures:

• Dicts with both "description" and "parameters" keys
• Functions with structured docstrings (name, params, return)
• Variables named tools, tool_list, available_tools, functions
• Classes with run(), execute(), or __call__() methods

Include in count and note: "Tool detected via heuristic — verify this is an intended agent tool."

---

3.4 [PATTERN] Context Size Awareness

Formula: token_estimate = len(file_content_chars) / 4

Content	⚠️ Warning threshold	❌ Issue threshold
System prompt file	> 4,000 tokens	> 8,000 tokens
Single tool description	> 500 tokens	> 1,000 tokens
All tool descriptions combined	> 2,000 tokens	> 4,000 tokens

Exclude: skills/ directories (loaded on demand, not embedded)

[HEURISTIC] Fallback: Borderline and Non-Standard

• Estimates within 20% of threshold → flag with tokenizer recommendation
• Dynamic prompts (f-strings, .format()) → flag if template alone is large
• Multiple concatenated prompts → estimate combined size
• Prompts with includes → note effective size may be larger

---

3.5 [HEURISTIC] Explicit Tool Listing

Check system prompts for:

• Headers like "Available Tools", "You have access to"
• Tool capability descriptions

Flag if tools are defined but not documented in system prompt.

---

3.6 [PATTERN] LangGraph Graph Cycle Analysis

(Only when LangGraph is detected)

Detection steps:

a. Find graph file (graph.py, graph.ts, or file with StateGraph/MessageGraph)

b. Build edge map:

• workflow.add_edge(source, dest) — unconditional edge
• workflow.add_conditional_edges(source, fn, mapping) — extract destinations from mapping

c. Identify cycles: nodes reachable from themselves

d. For each cycle, check if END (or "__end__") is reachable via conditional edge

Condition	Severity
Cycle exists, END reachable via conditional	✅ Pass
Cycle exists, no path to END	❌ Issue
Graph has no END node	❌ Issue
Node has no outgoing edges and is not END	⚠️ Warning (dead-end)

Example — infinite cycle (❌ Issue):

workflow.add_edge("agent", "tools")
workflow.add_edge("tools", "agent")  # no path to END

Example — cycle with exit (✅ Pass):

workflow.add_conditional_edges("agent", should_continue, {
    "continue": "tools",
    "end": END
})
workflow.add_edge("tools", "agent")

[HEURISTIC] Fallback: Non-LangGraph Graphs

Scan for graph-like control flow:

• State machines with transition tables
• Custom routing with implicit cycles
• LangGraph.js (camelCase methods)
• CrewAI/AutoGen agent handoffs
• Adjacency lists without termination path

Flag as ⚠️ Warning: "Potential cyclic control flow — verify termination condition exists"

---

Step 4: Generate Report

# Agent Pattern Verification Report

**Project:** [name or path]
**Date:** [current date]
**Framework detected:** [LangGraph | CrewAI | AutoGen | LangChain | Custom | None]
**Files analyzed:** [count]

## Summary

✅ X checks passed | ⚠️ Y warnings | ❌ Z issues

## Loop Safety

- [x] All loops have termination conditions
- [ ] ⚠️ Potential unbounded loop at `[file:line]`

## Retry Limits

- [x] All retry mechanisms have explicit limits
- [ ] ❌ Missing retry limit at `[file:line]`

## Tool Consistency

- [x] Tool registry found: X tools defined
- [ ] ❌ Y hallucinated tool references
- [ ] ⚠️ Z undocumented tools

## Context Size

- [x] System prompt within limits (~X tokens)
- [ ] ⚠️ System prompt exceeds recommended size

## Findings

> `[P]` = pattern-matched · `[H]` = heuristic

### ✅ Passing
- `[P]` [Check]: [confirmation]

### ⚠️ Warnings
- `[P|H]` [Check]: [description]
  - **Location:** [file:line]
  - **Suggestion:** [how to fix]

### ❌ Issues
- `[P|H]` [Check]: [description]
  - **Location:** [file:line]
  - **Rule:** [which rule violated]
  - **Fix:** [specific remediation]

## Recommendations

1. [Priority recommendation]
2. [Additional improvements]

---

For full verification including security, quality, and language-specific checks, say "verify agent".