I will do big data tasks using apache hadoop superset kafka mongo clickhouse

Hello ! I'm a data engineer with an interest to scale and optimise data-pipelines.

This gig is about offering my Big-data services for Machine Learning and analytics with Apache Spark, Apache Hadoop, Apache Hive, Apache Kafka, Apache Airflow, superset , Spark SQL, and MongoDB , clickhouse.

I code in Python.

I enjoy transforming raw big-data(structured or unstructured ) into analytics , visuals or to train Highgly accurate ML models.

My prior project

• music-recommendation system on spotify,
• Personalized excel file search engine
• Amazon market-basket analysis
• Hadoop Cluster optimization
• Dijsktra algorithm using GraphX.

tools : shell-scripting,hadoop , pyspark , java + spark ,Scala + Spark , kafka and mongodb

While all these projects include streaming data , ETL , analytics ,ML aswell.

Additional, I can set up Spark clusters on VM or cloud with Mesos, Yarn, or standalone configurations.

please drop a text and discuss the task before placing order .

Thanks , looking forward to be your help in your next project :)