I will build pyspark etl pipelines for batch and streaming data on AWS

Pakistan

I speak Urdu, English

1 order completed

Data Engineer for PySpark, AWS and QuickSight

I'm Muhammad Zeeshan, a Data Engineer working full-time on AWS data platforms. I build PySpark ETL pipelines (batch and streaming) on Amazon EMR, set up S3 data lakes, and turn the results into clear ...
About this Gig

Need a reliable PySpark job or a full ETL pipeline? I'm a full-time Data Engineer, and building batch and streaming pipelines with PySpark on AWS is my day job.


What I can do for you:

- Write PySpark ETL jobs: ingest, clean, transform, load

- Process large datasets in batch on Amazon EMR

- Build streaming jobs for real-time data

- Integrate your jobs with Apache Hive and Apache Hudi

- Set up or organise a data lake on Amazon S3

- Optimise or debug existing PySpark code


What you get:

- Clean, commented code

- Short notes on how to run and schedule it

- Replies usually within an hour


How it works:

1. Send me your data source, your target, and the transformation you need

2. I confirm scope, price, and timeline

3. I deliver the code and revise based on your feedback


Please message me before ordering so we can agree on scope first.

Expertise:

Big data

•

Data manipulation

•

ETL

•

Transformation

•

SQL

•

NoSQL

Technology:

Apache Hadoop

•

Apache Spark

•

Excel

•

Python

•

SQL

•

NoSQL