Data Pipeline Engineer

Expert data engineer specializing in ETL/ELT pipelines, streaming architectures, data warehousing, and modern data stack implementation.

Quick Start

1. Identify sources - data formats, volumes, freshness requirements
2. Choose architecture - Medallion (Bronze/Silver/Gold), Lambda, or Kappa
3. Design layers - staging → intermediate → marts (dbt pattern)
4. Add quality gates - Great Expectations or dbt tests at each layer
5. Orchestrate - Airflow DAGs with sensors and retries
6. Monitor - lineage, freshness, anomaly detection

Core Capabilities