Dagster Integrations Index

Navigate 82+ Dagster integrations organized by Dagster's official taxonomy. Find AI/ML tools, ETL platforms, data storage, compute services, BI tools, and monitoring integrations.

When to Use This Skill vs. Others

If User Says...	Use This Skill/Command	Why
"which integration for X"	`/dagster-integrations`	Need to discover appropriate integration
"does dagster support X"	`/dagster-integrations`	Check integration availability
"snowflake vs bigquery"	`/dagster-integrations`	Compare integrations in same category
"best practices for X"	`/dagster-conventions`	Implementation patterns needed
"implement X integration"	`/dg:prototype`	Ready to build with specific integration
"how do I use dbt"	`/dagster-conventions` (dbt section)	dbt-specific implementation patterns
"make this code better"	`/dignified-python`	Python code review needed
"create new project"	`/dg:create-project`	Project initialization needed

Quick Reference by Category

Category	Count	Common Tools	Reference
AI & ML	6	OpenAI, Anthropic, MLflow, W&B	`references/ai.md`
ETL/ELT	9	dbt, Fivetran, Airbyte, PySpark	`references/etl.md`
Storage	35+	Snowflake, BigQuery, Postgres, DuckDB	`references/storage.md`
Compute	15+	AWS, Databricks, Spark, Docker, K8s	`references/compute.md`
BI & Visualization	7	Looker, Tableau, PowerBI, Sigma	`references/bi.md`
Monitoring	3	Datadog, Prometheus, Papertrail	`references/monitoring.md`
Alerting	6	Slack, PagerDuty, MS Teams, Twilio	`references/alerting.md`
Testing	2	Great Expectations, Pandera	`references/testing.md`
Other	2+	Pandas, Polars	`references/other.md`

Category Taxonomy

This index aligns with Dagster's official documentation taxonomy from tags.yml:

ai: Artificial intelligence and machine learning integrations (LLM APIs, experiment tracking)
etl: Extract, transform, and load tools including data replication and transformation frameworks
storage: Databases, data warehouses, object storage, and table formats
compute: Cloud platforms, container orchestration, and distributed processing frameworks
bi: Business intelligence and visualization platforms
monitoring: Observability platforms and metrics systems for tracking performance
alerting: Notification and incident management systems for pipeline alerts
testing: Data quality validation and testing frameworks
other: Miscellaneous integrations including DataFrame libraries

Note: Support levels (dagster-supported, community-supported) are shown inline in each integration entry.

Last verified: 2026-01-27

Finding the Right Integration

I need to...

Load data from external sources

SaaS applications → ETL (Fivetran, Airbyte)
Files/databases → ETL (dlt, Sling, Meltano)
Cloud storage → Storage (S3, GCS, Azure Blob)

Transform data

SQL transformations → ETL (dbt)
Distributed transformations → ETL (PySpark)
DataFrame operations → Other (Pandas, Polars)
Large-scale processing → Compute (Spark, Dask, Ray)

Store data

Cloud data warehouse → Storage (Snowflake, BigQuery, Redshift)
Relational database → Storage (Postgres, MySQL)
File/object storage → Storage (S3, GCS, Azure, LakeFS)
Analytics database → Storage (DuckDB)
Vector embeddings → Storage (Weaviate, Chroma, Qdrant)

Validate data quality

Schema validation → Testing (Pandera)
Quality checks → Testing (Great Expectations)

Run ML workloads

LLM integration → AI (OpenAI, Anthropic, Gemini)
Experiment tracking → AI (MLflow, W&B)
Distributed training → Compute (Ray, Spark)

Execute computation

Cloud compute → Compute (AWS, Azure, GCP, Databricks)
Containers → Compute (Docker, Kubernetes)
Distributed processing → Compute (Spark, Dask, Ray)

Monitor pipelines

Team notifications → Alerting (Slack, MS Teams, PagerDuty)
Metrics tracking → Monitoring (Datadog, Prometheus)
Log aggregation → Monitoring (Papertrail)

Visualize data

BI dashboards → BI (Looker, Tableau, PowerBI)
Analytics platform → BI (Sigma, Hex, Evidence)

Integration Categories

AI & ML

Artificial intelligence and machine learning platforms, including LLM APIs and experiment tracking.

Key integrations:

OpenAI - GPT models and embeddings API
Anthropic - Claude AI models
Gemini - Google's multimodal AI
MLflow - Experiment tracking and model registry
Weights & Biases - ML experiment tracking
NotDiamond - LLM routing and optimization

See references/ai.md for all AI/ML integrations.

ETL/ELT

Extract, transform, and load tools for data ingestion, transformation, and replication.

Key integrations:

dbt - SQL-based transformation with automatic dependencies
Fivetran - Automated SaaS data ingestion (component-based)
Airbyte - Open-source ELT platform
dlt - Python-based data loading (component-based)
Sling - High-performance data replication (component-based)
PySpark - Distributed data transformation
Meltano - ELT for the modern data stack

See references/etl.md for all ETL/ELT integrations.

Storage

Data warehouses, databases, object storage, vector databases, and table formats.

Key integrations:

Snowflake - Cloud data warehouse with IO managers
BigQuery - Google's serverless data warehouse
DuckDB - In-process SQL analytics
Postgres - Open-source relational database
Weaviate - Vector database for AI search
Delta Lake - ACID transactions for data lakes
DataHub - Metadata catalog and lineage

See references/storage.md for all storage integrations.

Compute

Cloud platforms, container orchestration, and distributed processing frameworks.

Key integrations:

AWS - Cloud compute services (Glue, EMR, Lambda)
Databricks - Unified analytics platform
GCP - Google Cloud compute (Dataproc, Cloud Run)
Spark - Distributed data processing engine
Dask - Parallel computing framework
Docker - Container execution with Pipes
Kubernetes - Cloud-native orchestration
Ray - Distributed computing for ML

See references/compute.md for all compute integrations.

BI & Visualization

Business intelligence and visualization platforms for analytics and reporting.

Key integrations:

Looker - Google's BI platform
Tableau - Interactive dashboards
PowerBI - Microsoft's BI tool
Sigma - Cloud analytics platform
Hex - Collaborative notebooks
Evidence - Markdown-based BI
Cube - Semantic layer platform

See references/bi.md for all BI integrations.

Monitoring

Observability platforms and metrics systems for tracking pipeline performance.

Key integrations:

Datadog - Comprehensive observability platform
Prometheus - Time-series metrics collection
Papertrail - Centralized log management

See references/monitoring.md for all monitoring integrations.

Alerting

Notification and incident management systems for pipeline alerts.

Key integrations:

Slack - Team messaging and alerts
PagerDuty - Incident management for on-call
MS Teams - Microsoft Teams notifications
Twilio - SMS and voice notifications
Apprise - Universal notification platform
DingTalk - Team communication for Asian markets

See references/alerting.md for all alerting integrations.

Testing

Data quality validation and testing frameworks for ensuring data reliability.

Key integrations:

Great Expectations - Data validation with expectations
Pandera - Statistical data validation for DataFrames

See references/testing.md for all testing integrations.

Other

Miscellaneous integrations including DataFrame libraries and utility tools.

Key integrations:

Pandas - In-memory DataFrame library
Polars - Fast DataFrame library with columnar storage

See references/other.md for other integrations.

References

Integration details are organized in the following files:

AI & ML: references/ai.md - AI and ML platforms, LLM APIs, experiment tracking
ETL/ELT: references/etl.md - Data ingestion, transformation, and replication tools
Storage: references/storage.md - Warehouses, databases, object storage, vector DBs
Compute: references/compute.md - Cloud platforms, containers, distributed processing
BI & Visualization: references/bi.md - Business intelligence and analytics platforms
Monitoring: references/monitoring.md - Observability and metrics systems
Alerting: references/alerting.md - Notifications and incident management
Testing: references/testing.md - Data quality and validation frameworks
Other: references/other.md - DataFrame libraries and miscellaneous tools

Using Integrations

Most Dagster integrations follow a common pattern:

Install the package:
```
pip install dagster-<integration>
```

Import and configure a resource:

from dagster_<integration> import <Integration>Resource

resource = <Integration>Resource(
    config_param=dg.EnvVar("ENV_VAR")
)

Use in your assets:

@dg.asset
def my_asset(integration: <Integration>Resource):
    # Use the integration
    pass

For component-based integrations (dbt, Fivetran, dlt, Sling), see the specific reference files for scaffolding and configuration patterns.

integrations-index

Dagster Integrations Index

When to Use This Skill vs. Others

Quick Reference by Category

Category Taxonomy

Finding the Right Integration

I need to...

Integration Categories

AI & ML

ETL/ELT

Storage

Compute

BI & Visualization

Monitoring

Alerting

Testing

Other

References

Using Integrations