Vast.ai Hello World

Overview

Rent your first GPU instance on Vast.ai, run a PyTorch workload, and destroy the instance when done. Demonstrates the full lifecycle: search offers, create instance, connect via SSH, run a job, and tear down.

Prerequisites

Completed vastai-install-auth setup
Vast.ai account with credits ($1+ recommended for testing)
SSH key uploaded to Vast.ai (cloud.vast.ai > Account > SSH Keys)

Instructions

Step 1: Search for Available GPUs (CLI)

# Find cheap single-GPU offers sorted by price
vastai search offers 'num_gpus=1 gpu_ram>=8 inet_down>100 reliability>0.95' \
  --order 'dph_total' --limit 5

# Output columns: ID, GPU, VRAM, $/hr, DLPerf, Reliability, Location

Step 2: Search for Available GPUs (REST API)

curl -s -H "Authorization: Bearer $VASTAI_API_KEY" \
  "https://cloud.vast.ai/api/v0/bundles/?q=%7B%22num_gpus%22%3A%7B%22eq%22%3A1%7D%2C%22gpu_ram%22%3A%7B%22gte%22%3A8%7D%2C%22reliability2%22%3A%7B%22gte%22%3A0.95%7D%2C%22rentable%22%3A%7B%22eq%22%3Atrue%7D%7D&order=dph_total&limit=5" \
  | jq '.offers[:3] | .[] | {id, gpu_name, num_gpus, gpu_ram, dph_total, reliability2}'

Step 3: Create an Instance (CLI)

# Replace OFFER_ID with the ID from search results
vastai create instance OFFER_ID \
  --image pytorch/pytorch:2.2.0-cuda12.1-cudnn8-runtime \
  --disk 20 \
  --onstart-cmd "echo 'Instance ready'"

Step 4: Create an Instance (Python)

from vastai_client import VastClient

client = VastClient()

# Search for affordable RTX 4090 offers
offers = client.search_offers({
    "num_gpus": {"eq": 1},
    "gpu_name": {"eq": "RTX_4090"},
    "reliability2": {"gte": 0.95},
    "rentable": {"eq": True},
})

# Pick the cheapest offer
best = sorted(offers["offers"], key=lambda o: o["dph_total"])[0]
print(f"Best offer: {best['gpu_name']} at ${best['dph_total']:.3f}/hr (ID: {best['id']})")

# Create instance with PyTorch image
instance = client.create_instance(
    offer_id=best["id"],
    image="pytorch/pytorch:2.2.0-cuda12.1-cudnn8-runtime",
    disk_gb=20,
    onstart="nvidia-smi && python -c 'import torch; print(torch.cuda.is_available())'",
)
print(f"Instance created: {instance}")

Step 5: Monitor and Connect

# Check instance status (wait for 'running')
vastai show instances --raw | jq '.[] | {id, actual_status, ssh_host, ssh_port}'

# Connect via SSH once running
ssh -p SSH_PORT root@SSH_HOST

# On the instance: verify GPU access
nvidia-smi
python -c "import torch; print(f'CUDA available: {torch.cuda.is_available()}')"

Step 6: Run a Test Workload

# test_gpu.py — run this ON the rented instance
import torch
import time

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"Device: {device} ({torch.cuda.get_device_name(0)})")

# Simple matrix multiplication benchmark
size = 4096
a = torch.randn(size, size, device=device)
b = torch.randn(size, size, device=device)

torch.cuda.synchronize()
start = time.time()
c = torch.matmul(a, b)
torch.cuda.synchronize()
elapsed = time.time() - start

tflops = (2 * size**3) / elapsed / 1e12
print(f"Matrix multiply {size}x{size}: {elapsed:.3f}s ({tflops:.2f} TFLOPS)")
print("Hello World from Vast.ai!")

Step 7: Destroy the Instance

# IMPORTANT: Destroy to stop billing
vastai destroy instance INSTANCE_ID

# Verify it's gone
vastai show instances

Output

GPU instance rented and running on Vast.ai
SSH connection established to the remote GPU machine
PyTorch workload executed successfully with GPU acceleration
Instance destroyed (billing stopped)

Error Handling

Error	Cause	Solution
`No offers found`	Filters too strict	Relax GPU or reliability filters
`Insufficient funds`	Account balance too low	Add credits at cloud.vast.ai
`Instance failed to start`	Docker image pull failed	Use a smaller or more common image
`SSH connection refused`	Instance still loading	Wait 1-2 min for status `running`
`CUDA not available`	Driver mismatch	Use a CUDA-compatible Docker image

Resources

Next Steps

Proceed to vastai-local-dev-loop for development workflow setup.

Examples

Cheapest GPU test: Search with vastai search offers 'num_gpus=1' --order 'dph_total' --limit 1, create an instance with the ubuntu image, SSH in, run nvidia-smi, then destroy.

Specific GPU model: Filter for H100 with gpu_name=H100_SXM and reliability>0.99 for production-grade hardware. Expect $2.50-4.00/hr.

vastai-hello-world