Compile the Swift binary

cd scripts/ swiftc -O -o image_tagger image_tagger.swift


## Usage

### Analyze image → JSON

```bash
./scripts/image_tagger /path/to/photo.jpg

Output includes:

faces — bounding boxes, roll/yaw/pitch, landmarks (eyes, nose, mouth)
bodies — 18 skeleton joints with confidence scores
hands — 21 joints per hand (left/right)
text — OCR results with bounding boxes
labels — scene classification (desk, outdoor, clothing, etc.)
barcodes — QR codes, UPC, etc.
saliency — attention and objectness regions

Annotate image with boxes

python3 scripts/annotate_image.py photo.jpg output.jpg

Draws colored boxes:

🟢 Green: faces
🟠 Orange: body skeleton
🟣 Magenta: hands
🔵 Cyan: text regions
🟡 Yellow: rectangles/objects
Scene labels at bottom

Python integration

import subprocess, json

def analyze(path):
    r = subprocess.run(['./scripts/image_tagger', path], capture_output=True, text=True)
    return json.loads(r.stdout[r.stdout.find('{'):])

tags = analyze('photo.jpg')
print(tags['labels'])  # [{'label': 'desk', 'confidence': 0.85}, ...]
print(tags['faces'])   # [{'bbox': {...}, 'confidence': 0.99, 'yaw': 5.2}]

Example JSON Output

{
  "dimensions": {"width": 1920, "height": 1080},
  "faces": [{"bbox": {"x": 0.3, "y": 0.4, "width": 0.15, "height": 0.2}, "confidence": 0.99, "roll": -2, "yaw": 5}],
  "bodies": [{"joints": {"head_joint": {"x": 0.5, "y": 0.7, "confidence": 0.9}, "left_shoulder": {...}}, "confidence": 1}],
  "hands": [{"chirality": "left", "joints": {"VNHLKWRI": {"x": 0.4, "y": 0.3, "confidence": 0.85}}}],
  "text": [{"text": "HELLO", "confidence": 0.95, "bbox": {...}}],
  "labels": [{"label": "outdoor", "confidence": 0.88}, {"label": "sky", "confidence": 0.75}],
  "saliency": {"attentionBased": [{"x": 0.2, "y": 0.1, "width": 0.6, "height": 0.8}]}
}

Detection Capabilities

Feature	Details
Faces	Bounding box, confidence, roll/yaw/pitch angles, 76-point landmarks
Bodies	18 joints: head, neck, shoulders, elbows, wrists, hips, knees, ankles
Hands	21 joints per hand, left/right chirality
Text (OCR)	Recognized text with confidence and bounding boxes
Labels	1000+ scene/object categories (clothing, furniture, outdoor, etc.)
Barcodes	QR, UPC, EAN, Code128, PDF417, Aztec, DataMatrix
Saliency	Attention-based and objectness-based regions

Use Cases

Photo tagging — Auto-tag photos with detected objects/scenes
Posture monitoring — Track face/body position for ergonomics
Document scanning — Extract text from images
Security — Detect people in camera feeds
Accessibility — Describe image contents

vision-tagger

Vision Tagger

Requirements

Setup (one-time)