I will build scalable data pipelines using dagster, AWS, postgresql, and redshift

Are you looking for a reliable Data Engineer to build scalable, production-grade data pipelines?

I specialize in building modern data platforms using:

• Dagster (workflow orchestration & asset-based pipelines)
• PostgreSQL (source & metadata DB)
• Amazon S3 (data lake storage)
• Amazon Redshift (analytics warehouse)
• Python (ETL/ELT development)

What I Can Do For You

Build end-to-end ETL/ELT pipelines

Design Dagster assets & jobs

Load data from APIs / DBs S3 Redshift

Implement incremental pipelines (CDC, watermarking)

Optimize performance for millions of records

Handle schema evolution & data validation

Setup data partitioning (daily/hourly)

Create S3-based data lake architecture

Debug & fix existing pipelines

My Expertise Includes

• Dagster multi-asset pipelines
• PostgreSQL to Redshift migration
• S3 Parquet partitioning
• Incremental loads (no duplicates)
• Large-scale data ingestion (millions of rows)
• Data quality & validation
• Unit & integration testing
• Error handling & retries

Production-Ready Approach

I follow industry best practices:

• Modular code structure
• Logging & monitoring
• Retry & failure handling
• Idempotent pipelines
• CI/CD-ready design