Omi Plugin Development Skill

This skill provides guidance for developing Omi plugins/apps, including webhook patterns, chat tools, and OAuth flows.

When to Use

Use this skill when:

• Creating new Omi plugins/apps
• Implementing webhook handlers
• Adding chat tools for LangGraph
• Setting up OAuth integrations
• Building prompt-based apps

Key Patterns

Plugin Types

1. Prompt-Based Apps

No server required - Just define prompts

• Chat prompts: Customize AI personality
• Memory prompts: Customize memory extraction

2. Integration Apps

Requires server endpoint - Webhook-based

• Memory triggers: Webhook on memory creation
• Real-time transcript: Webhook for live transcripts
• Chat tools: Custom tools for LangGraph
• Audio streaming: Raw audio processing

Webhook Patterns

Memory Creation Webhook

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class MemoryWebhook(BaseModel):
    id: str
    content: str
    category: str
    user_id: str

@app.post("/webhook/memory-created")
async def memory_created(webhook: MemoryWebhook):
    """Called when a memory is created."""
    # Process memory
    # Can create new memories via API
    # Can trigger actions
    return {"status": "processed"}

Real-time Transcript Webhook

@app.post("/webhook/transcript")
async def transcript_segment(segment: dict):
    """Called with live transcript segments."""
    text = segment.get("text")
    
    # Process in real-time
    if "hey omi" in text.lower():
        await trigger_action()
    
    return {"status": "received"}

Chat Tools

Creating a Chat Tool

from langchain.tools import tool

@tool
def my_custom_tool(query: str) -> str:
    """Description of what this tool does.
    
    Args:
        query: The search query
        
    Returns:
        Results as a string
    """
    result = perform_search(query)
    return json.dumps(result)

# Register in app configuration
CHAT_TOOLS = [my_custom_tool]

Usage: Tool becomes available in agentic chat path when app is enabled

OAuth Integration

Setting Up OAuth

from authlib.integrations.fastapi_oauth2 import OAuth2

oauth = OAuth2(
    client_id=os.getenv("CLIENT_ID"),
    client_secret=os.getenv("CLIENT_SECRET"),
    server_metadata_url="https://accounts.google.com/.well-known/openid-configuration",
)

@app.get("/auth")
async def auth():
    return await oauth.google.authorize_redirect(
        redirect_uri="https://your-app.com/callback"
    )

Plugin Configuration

App Manifest

{
  "id": "my-plugin",
  "name": "My Plugin",
  "description": "Plugin description",
  "capabilities": [
    "memory_trigger",
    "real_time_transcript",
    "chat_tools"
  ],
  "webhook_url": "https://your-app.com/webhook"
}

Common Tasks

Creating a New Plugin

1. Choose plugin type (prompt-based or integration)
2. Set up server (if integration app)
3. Implement webhook handlers
4. Register plugin in Omi app
5. Test with webhook.site first

Adding Chat Tools

1. Create tool function with @tool decorator
2. Write clear tool description
3. Register in app configuration
4. Tool becomes available when app enabled

Setting Up OAuth

1. Create OAuth app in provider (Google, Apple, etc.)
2. Configure redirect URIs
3. Implement OAuth flow in plugin
4. Store tokens securely

Best Practices

1. Error Handling: Handle webhook errors gracefully
2. Idempotency: Make webhooks idempotent
3. Rate Limiting: Implement rate limiting
4. Security: Verify webhook signatures
5. Documentation: Document your plugin API
6. Testing: Test with webhook.site first

Related Documentation

The docs/ folder is the single source of truth for all user-facing documentation, deployed at docs.omi.me.

• Plugin Introduction: docs/doc/developer/apps/Introduction.mdx - View online
• Integrations: docs/doc/developer/apps/Integrations.mdx - View online
• Chat Tools: docs/doc/developer/apps/ChatTools.mdx - View online
• OAuth: docs/doc/developer/apps/Oauth.mdx - View online
• Prompt-Based Apps: docs/doc/developer/apps/PromptBased.mdx - View online
• Audio Streaming: docs/doc/developer/apps/AudioStreaming.mdx - View online
• Submitting Apps: docs/doc/developer/apps/Submitting.mdx - View online
• Plugin Development: .cursor/rules/plugin-development.mdc

Related Cursor Resources

Rules

• .cursor/rules/plugin-development.mdc - Plugin development patterns
• .cursor/rules/plugin-apps-js.mdc - JavaScript plugin patterns
• .cursor/rules/backend-api-patterns.mdc - Backend API patterns
• .cursor/rules/backend-architecture.mdc - Backend architecture

Subagents

• .cursor/agents/plugin-developer/ - Uses this skill for plugin development

Commands

• /create-plugin - Uses this skill for plugin scaffolding
• /create-app - Uses this skill for app creation