together-dedicated-containers
Together Dedicated Containers
Overview
Use Dedicated Container Inference when the user needs a custom runtime, not just managed model hosting.
Core building blocks:
- Jig CLI for build and deployment
- Sprocket SDK for request handling inside the container
- Queue API for async jobs
When This Skill Wins
- Deploy a custom inference worker
- Bundle custom dependencies or runtime logic into a container
- Use queue-based async processing with progress tracking
- Run a specialized image, video, or multimodal pipeline
Hand Off To Another Skill
- Use
together-dedicated-endpointsfor standard model hosting without custom containers - Use
together-gpu-clustersfor full cluster ownership and orchestration control - Use
together-chat-completions,together-images, ortogether-videowhen a serverless product already covers the task
Quick Routing
- Minimal worker template
- Start with scripts/sprocket_hello_world.py
- Read references/sprocket-sdk.md
- Build, deploy, logs, queue, and secrets
- Queue submission and polling
- Start with scripts/queue_client.py or scripts/queue_client.ts
Workflow
- Confirm that the user truly needs a custom container runtime.
- Implement the worker with Sprocket's request lifecycle.
- Configure
pyproject.tomlfor image, runtime, autoscaling, and mounts. - Deploy with Jig.
- Submit jobs through the queue API and poll until completion.
High-Signal Rules
- Python scripts require the Together v2 SDK (
together>=2.0.0). If the user is on an older version, they must upgrade first:uv pip install --upgrade "together>=2.0.0". - Prefer dedicated endpoints over containers unless the runtime or pipeline is genuinely custom.
- Treat the worker contract and
pyproject.tomlas the source of truth for deployment behavior. - Parameterize deployment name, queue inputs, and resource sizing instead of hardcoding them.
- Queue-based jobs are asynchronous by default; account for polling and result retrieval in client code.
Resource Map
- Jig CLI: references/jig-cli.md
- Sprocket SDK: references/sprocket-sdk.md
- Python queue client: scripts/queue_client.py
- TypeScript queue client: scripts/queue_client.ts
- Worker template: scripts/sprocket_hello_world.py
Official Docs
More from togethercomputer/skills
together-chat-completions
Real-time and streaming text generation via Together AI's OpenAI-compatible chat/completions API, including multi-turn conversations, tool and function calling, structured JSON outputs, and reasoning models. Reach for it whenever the user wants to build or debug text generation on Together AI, unless they specifically need batch jobs, embeddings, fine-tuning, dedicated endpoints, dedicated containers, or GPU clusters.
40together-batch-inference
High-volume, asynchronous offline inference at up to 50% lower cost via Together AI's Batch API. Prepare JSONL inputs, upload files, create jobs, poll status, and download outputs. Reach for it whenever the user needs non-interactive bulk inference rather than real-time chat or evaluation jobs.
39together-images
Text-to-image generation and image editing via Together AI, including FLUX and Kontext models, LoRA-based styling, reference-image guidance, and local image downloads. Reach for it whenever the user wants to generate or edit images on Together AI rather than create videos or build text-only chat applications.
33together-fine-tuning
LoRA, full fine-tuning, DPO preference tuning, VLM training, function-calling tuning, reasoning tuning, and BYOM uploads on Together AI. Reach for it whenever the user wants to adapt a model on custom data rather than only run inference, evaluate outputs, or host an existing model.
32together-embeddings
Dense vector embeddings, semantic search, RAG pipelines, and reranking via Together AI. Generate embeddings with open-source models and rerank results behind dedicated endpoints. Reach for it whenever the user needs vector representations or retrieval quality improvements rather than direct text generation.
31together-evaluations
LLM-as-a-judge evaluation framework on Together AI. Classify, score, and compare model outputs, select judge models, use external-provider judges or targets, poll results and download reports. Reach for it whenever the user wants to benchmark outputs, grade responses, compare A/B variants, or operationalize automated evaluations.
31