LangChain Deploy Integration

Overview

Deploy LangChain applications to production using LangServe, Docker, and cloud platforms. Covers containerization of chains and agents, LangServe API deployment, and integration with LangSmith for production observability.

Prerequisites

LangChain application with chains/agents defined
Docker installed for containerization
LangSmith API key for production tracing
Platform CLI (gcloud, aws, or docker compose)

Instructions

Step 1: LangServe API Setup

# serve.py
from fastapi import FastAPI
from langserve import add_routes
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

app = FastAPI(title="LangChain API")

# Define your chain
prompt = ChatPromptTemplate.from_template("Answer: {question}")
chain = prompt | ChatOpenAI(model="gpt-4o-mini")

# Add LangServe routes
add_routes(app, chain, path="/chat")

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8000)  # 8000: API server port

Step 2: Dockerfile

FROM python:3.11-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

ENV LANGCHAIN_TRACING_V2=true
ENV LANGCHAIN_PROJECT=production

EXPOSE 8000  # 8000: API server port
CMD ["uvicorn", "serve:app", "--host", "0.0.0.0", "--port", "8000"]  # API server port

Step 3: Docker Compose for Development

version: "3.8"
services:
  langchain-api:
    build: .
    ports:
      - "8000:8000"  # 8000: API server port
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - LANGCHAIN_API_KEY=${LANGCHAIN_API_KEY}
      - LANGCHAIN_TRACING_V2=true
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]  # API server port
      interval: 30s

Step 4: Cloud Run Deployment

gcloud run deploy langchain-api \
  --source . \
  --region us-central1 \
  --set-secrets=OPENAI_API_KEY=openai-key:latest \
  --set-secrets=LANGCHAIN_API_KEY=langsmith-key:latest \
  --set-env-vars=LANGCHAIN_TRACING_V2=true \
  --min-instances=1 \
  --memory=1Gi

Step 5: Health Check with LangSmith

from langsmith import Client

async def health_check():
    try:
        client = Client()
        # Verify LangSmith connection
        client.list_projects(limit=1)
        return {"status": "healthy", "tracing": "enabled"}
    except Exception as e:
        return {"status": "degraded", "error": str(e)}

Error Handling

Issue	Cause	Solution
Import errors	Missing dependencies	Pin versions in requirements.txt
LangSmith timeout	Network issue	Set `LANGCHAIN_TRACING_V2=false` as fallback
Memory exceeded	Large context	Increase container memory, use streaming
Cold start slow	Heavy imports	Use gunicorn with preload

Examples

Production Requirements

langchain>=0.3.0
langchain-openai>=0.2.0
langserve>=0.3.0
langsmith>=0.1.0
uvicorn>=0.30.0
fastapi>=0.115.0

Resources

Next Steps

For multi-environment setup, see langchain-multi-env-setup.

Output

Configuration files or code changes applied to the project
Validation report confirming correct implementation
Summary of changes made and their rationale

langchain-deploy-integration