Browse categories
Explore
Fiverr Pro
English
$
USD
Looks Like This Service Is On Hold
I will build scalable databricks lakehouse and spark pipelines
Brazil
I speak Portuguese, English, Spanish
GenAI Architect Senior Data Engineer Analytics Engineer Blockchain Advisor
Senior Data & GenAI Engineer | 20+ Yrs Exp | Ex-Mastercard & Palantir & CapGemini
Elite specialist in Modern Lakehouse (Databricks, Spark, Delta Lake) and GenAI Agents (MongoDB, Oracle). I design end...
About this Gig
Need a robust, enterprise-grade data foundation for your AI and Analytics?
With over 20 years of experience at Mastercard and Palantir, I specialize in designing and optimizing high-performance data architectures using Databricks and Apache Spark. I don't just move data; I build resilient Lakehouses that scale.
What I bring to your project:
- Medallion Architecture: Implementing Bronze, Silver, and Gold layers for maximum data quality.
- Cost Optimization: Tuning Spark clusters and pipelines to reduce your cloud spend.
- Delta Lake Expertise: Ensuring ACID transactions and time travel for your big data.
- 2026 Certified: Advanced credentials in Databricks Stream Processing and MLOps.
My Technical Stack:
- Databricks, Apache Spark (PySpark/SQL).
- Delta Lake, Unity Catalog (Governance).
- Azure Data Factory, GCP Dataflow.
- Python, Scala, and advanced SQL optimization.
Stop struggling with slow and expensive pipelines. Lets build a future-proof data ecosystem.
FAQ
Can you help us reduce our Databricks consumption costs?
Absolutely. My "Data Audit" includes a deep analysis of cluster utilization, job scheduling, and Spark code optimization to ensure you are only paying for the resources you actually need.
Do you implement the Medallion Architecture?
Yes. I specialize in building Bronze, Silver, and Gold layers using Delta Lake, ensuring data quality, ACID transactions, and a "single source of truth" for your analytics and AI.
Will you handle the transition to Unity Catalog?
Yes. I can manage the migration and setup of Unity Catalog to provide centralized access control, auditing, lineage, and data discovery across your workspace.
Can you integrate data from legacy systems like SAP or Oracle?
Yes. With 20+ years of experience, I am expert in building connectors and ingestion pipelines from diverse sources, including legacy ERPs, into a Modern Lakehouse.
Do you offer MLOps integration for the pipelines?
Yes. In the Premium package, I can set up production-ready pipelines that feed directly into MLflow or other MLOps frameworks to support your GenAI and Data Science teams.
Is post-project technical support available?
Yes. Every project comes with a standard support period. For long-term maintenance, custom monthly retainers can be arranged.
