I will build or troubleshoot your kafka streaming data pipeline

Italy

I speak English, Portuguese

Senior Data Engineer Big Data Architect ML AI Engineer Developer

Hi, I'm Gilson, a senior data engineer and big data architect based in Italy. I design and build real-time data platforms for enterprise teams: Kafka and Airflow pipelines, cloud data warehouses, and ...
About this Gig

I will help you design, build, troubleshoot or improve a production Kafka data pipeline.


I have extensive experience with real-time data processing and event-driven systems, including environments where data needs to be processed continuously and reliably.


I can help with:

Kafka architecture

Event ingestion

Producers and consumers

Real-time transformations

Data integration

Python-based processing

Streaming and batch integration

Error handling

Data persistence

Pipeline troubleshooting

Performance and reliability improvements

Docker/Kubernetes deployment


I can work with your existing architecture or design a new solution from the beginning.


I have worked on real-time data platforms in demanding environments including iGaming, where streaming data supported the identification of suspicious player behaviour.

For complex systems, please contact me before ordering so we can define the scope.

Expertise:

Automations

•

Big data

•

Data acquisition

•

Data extraction

Technology:

Apache Kafka

•

Apache Spark

•

BigQuery

•

Python

•

SAS

My Portfolio

Other Data Engineering Services I Offer