skill-creator
Audited by Runlayer on Feb 25, 2026
Malicious tool definition detected
Tool: LICENSE.txt [1/2] Description: Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1.
Tool: LICENSE.txt [2/2] Description: this License.
Malicious tool definition detected
Tool: SKILL.md [1/5] Description: --- name: skill-creator description: Create new skills, improve existing skills, and measure skill performance.
Tool: SKILL.md [2/5] Description: you haven't heard (and how could you, it's only very recently that it started), there's a trend now where the power of Claude is inspiring plumbers to open up their terminals, parents and grandparents to google "how to install npm". On the other hand, the bulk of users are probably fairly computer-literate.
Tool: SKILL.md [3/5] Description: explain to the model why things are important in lieu of heavy-handed musty MUSTs. Use theory of mind and try to make the skill general and not super-narrow to specific examples. Start by writing a draft and then look at it with fresh eyes and improve it.
Tool: SKILL.md [4/5] Description: Save outputs to: workspace/v<N>/runs/run-<R>/outputs/ ``` #### Step 2: Grade Assertions Spawn grader subagents (or grade inline — see "Without subagents" below). Update task to `reviewing` stage.
Tool: SKILL.md [5/5] Description: and grade sequentially in the main loop.
Malicious tool definition detected
Tool: agents/analyzer.md [1/2] Description: # Post-hoc Analyzer Agent Analyze blind comparison results to understand WHY the winner won and generate improvement suggestions. ## Role After the blind comparator determines a winner, the Post-hoc Analyzer "unblids" the results by examining the skills and transcripts.
Tool: agents/analyzer.md [2/2] Description: these parameters in your prompt: - **benchmark_data_path**: Path to the in-progress benchmark.json with all run results - **skill_path**: Path to the skill being benchmarked - **output_path**: Where to save the notes (as JSON array of strings) ## Benchmark Process ### Step 1: Read Benchmark Data 1.
Malicious tool definition detected
Tool: agents/comparator.md Description: # Blind Comparator Agent Compare two outputs WITHOUT knowing which skill produced them. ## Role The Blind Comparator judges which output better accomplishes the eval task. You receive two outputs labeled A and B, but you do NOT know which skill produced which.
Malicious tool definition detected
Tool: agents/executor.md Description: # Executor Agent Execute an eval prompt using a skill and produce a detailed transcript. ## Role The Executor runs a single eval case: load the skill, execute the prompt with staged input files, and document everything in a transcript.
Malicious tool definition detected
Tool: agents/grader.md [1/2] Description: # Grader Agent Evaluate expectations against an execution transcript and outputs. ## Role The Grader reviews a transcript and output files, then determines whether each expectation passes or fails.
Tool: agents/grader.md [2/2]
Malicious tool definition detected
Tool: references/benchmark-mode.md Description: # Benchmark Mode Reference **Requires subagents.** Benchmark mode relies on parallel execution of many independent runs. Without subagents, use Eval mode for individual eval testing instead.
Malicious tool definition detected
Tool: references/eval-mode.md Description: # Eval Mode Reference Eval mode runs skill evals and grades expectations. Enables measuring skill performance, comparing with/without skill, and validating that skills add value.
Malicious tool definition detected
Tool: references/mode-diagrams.md Description: # Mode Workflow Diagrams Visual representations of how each mode orchestrates building blocks.
Malicious tool definition detected
Tool: references/schemas.md [1/2] Description: # JSON Schemas This document defines the JSON schemas used by skill-creator-edge.
Tool: references/schemas.md [2/2] Description: 5.4 } }, "output_quality": { "A": { "score": 9, "strengths": ["Complete solution", "Well-formatted", "All fields present"], "weaknesses": ["Minor style inconsistency in header"] }, "B": { "score": 5, "strengths": ["Readable output", "Correct basic structure"], "weaknesses": ["Missing date field", "Formatting inconsistencies", "Partial data extraction"] } }, "expectation_results": { "A": { "passed": 4, "total": 5, "pass_rate": 0.80, "details": [ {"te
Malicious tool definition detected
Tool: scripts/aggregate_benchmark.py Description: #!/usr/bin/env python3 """ Aggregate individual run results into benchmark summary statistics.
Malicious tool definition detected
Tool: scripts/copy_skill.py Description: #!/usr/bin/env python3 """ Copy a skill directory with metadata tracking.
Malicious tool definition detected
Tool: scripts/init_json.py Description: #!/usr/bin/env python3 """ Initialize JSON files with the correct structure for skill-creator-edge.
Malicious tool definition detected
Tool: scripts/init_skill.py [1/2] Description: #!/usr/bin/env python3 """ Skill Initializer - Creates a new skill from template Usage: init_skill.py <skill-name> --path <path> Examples: init_skill.py my-new-skill --path skills/public init_skill.py my-api-helper --path skills/private init_skill.py custom-skill --path /custom/location """ import sys from pathlib import Path SKILL_TEMPLATE = """--- name: {skill_name} description: [TODO: Complete and informative explanation of what the skill does an
Tool: scripts/init_skill.py [2/2] Description: Path to created skill directory, or None if error """ # Determine skill directory path skill_dir = Path(path).resolve() / skill_name # Check if directory already exists if skill_dir.exists(): print(f"❌ Error: Skill directory already exists: {skill_dir}") return None # Create skill directory try: skill_dir.mkdir(parents=True, exist_ok=False) print(f"✅ Created skill directory: {skill_dir}") except Exception as e: print(f"❌ Error creating directory: {e
Malicious tool definition detected
Tool: scripts/package_skill.py Description: #!/usr/bin/env python3 """ Skill Packager - Creates a distributable .skill file of a skill folder Usage: python utils/package_skill.py <path/to/skill-folder> [output-directory] Example: python utils/package_skill.py skills/public/my-skill python utils/package_skill.py skills/public/my-skill ./dist """ import fnmatch import sys import zipfile from pathlib import Path from quick_validate import validate_skill # Patterns to exclude when packaging skills.
Malicious tool definition detected
Tool: scripts/prepare_eval.py Description: #!/usr/bin/env python3 """ Prepare environment for running a skill eval.
Malicious tool definition detected
Tool: scripts/quick_validate.py Description: #!/usr/bin/env python3 """ Quick validation script for skills - minimal version """ import sys import os import re import yaml from pathlib import Path def validate_skill(skill_path): """Basic validation of a skill""" skill_path = Path(skill_path) # Check SKILL.md exists skill_md = skill_path / 'SKILL.md' if not skill_md.exists(): return False, "SKILL.md not found" # Read and validate frontmatter content = skill_md.read_text() if not content.startswit
Malicious tool definition detected
Tool: scripts/validate_json.py [1/2] Description: #!/usr/bin/env python3 """ Validate JSON files produced by skill-creator-edge.
Tool: scripts/validate_json.py [2/2] Description: e: return False, [f"Invalid JSON: {e}"] # Infer type if not provided if json_type is None: json_type = infer_type(file_path) if json_type is None: return False, [f"Could not infer JSON type from filename.