Together Dedicated Endpoints

Overview

Use dedicated endpoints for managed single-tenant model hosting with predictable performance and no shared serverless pool.

Typical fits:

Use together-chat-completions for serverless chat inference
Use together-dedicated-containers for custom runtimes or nonstandard inference pipelines
Use together-gpu-clusters for raw infrastructure or cluster orchestration

Create and manage a standard endpoint
- Start with scripts/manage_endpoint.py or scripts/manage_endpoint.ts
- Read references/api-reference.md
Lifecycle tuning or troubleshooting
- Read references/api-reference.md
Deploy a fine-tuned model
- Start with scripts/deploy_finetuned.py
- Read references/dedicated-models.md
Upload and deploy a custom model
- Start with scripts/upload_custom_model.py
- Read references/dedicated-models.md
Hardware and sizing choices
- Read references/hardware-options.md

Confirm that the task needs dedicated hosting instead of serverless or containers.
Verify model eligibility and inspect available hardware.
Create the endpoint with explicit scaling and timeout settings.
Wait for readiness before sending inference traffic.
Stop or delete the endpoint when the workload no longer needs to run.

Python scripts require the Together v2 SDK (together>=2.0.0). If the user is on an older version, they must upgrade first: uv pip install --upgrade "together>=2.0.0".
Model eligibility and hardware availability are gating constraints; check them early.
Endpoint management uses endpoint IDs, while inference usually uses the endpoint name as model.
Autoscaling, auto-shutdown, prompt caching, and speculative decoding materially affect operations and cost.
For custom or fine-tuned models, do not skip the intermediate verification steps before deployment.