I will create scalable data pipelines using azure data factory and databricks

India

I speak Kannada, Hindi, English

Big Data Engineer

Experienced Data Engineer with 7+ years of experience building scalable, cloud-based data pipelines and automating data workflows. I specialize in Python, SQL, Azure(ADLS Gen2, Azure Databricks, Azur...
About this Gig

I specialize in designing and developing end-to-end ETL/ELT pipelines using Azure Data Factory (ADF) and Databricks (PySpark). I integrate data from multiple sourcesAPIs, SQL databases, cloud storage, files, and moreinto clean, structured, and production-ready pipelines.


With strong expertise in ADF orchestration, Databricks Delta pipelines, and PySpark transformations, I can help you build robust workflows that are easy to maintain and optimized for performance. Whether you need a simple ingestion pipeline or a complete multi-source data solution, I deliver scalable, secure, and well-documented results.


What I offer:

Data ingestion from multiple sources

ADF pipelines, triggers, orchestration

Databricks Delta ETL/ELT pipelines

PySpark transformations

Data quality checks and validation

Streaming or batch pipelines

Documentation and best practices


Lets build a pipeline that delivers accurate, reliable, and production-grade data for your business!

Destination Platform:

Google BigQuery

Azure Synapse Analytics

Tools & Platforms:

AWS Glue DataBrew

Azure Data Factory

My Portfolio