Browse categories
Explore
Fiverr Pro
English
$
USD
I will build data engineering solutions using azure
Web scraper and Data Engineer
I bring 4+ years of professional Python web scraping experience, having built over 300 custom crawlers for diverse websites. I’m highly skilled in Scrapy, Selenium, Selenium Wire, Playwright, PyDoll, ...
Level 2
Has met high performance criteria and has a proven track record for meeting client expectations.
About this Gig
Looking for a data engineer to build reliable ETL pipelines on Azure? I design and implement end-to-end data solutions using Azure Data Factory (ADF), Azure Databricks, and Medallion Architecture to ingest, transform, and manage your data efficiently.
Services I Offer:
- Data Ingestion: APIs, CSV, Excel, Databases, Cloud Storage
- Data Transformation: Databricks (PySpark/SQL),ADF Data Flows (filter, aggregate, select, derived column, alter row, joins)
- ETL Pipelines with Medallion Architecture (Bronze Silver Gold)
- ADF Activities (Copy Data, ForEach, Lookup, Web, Stored Procedures)
- Scheduling & Triggers: Time-based, storage-based, and event-driven
Why Me?
- Strong expertise in ADF & Azure Databricks
- Optimized, scalable ETL pipelines
Tools & Platforms:
Azure Data Factory
•
Other
My Portfolio
FAQ
What do you need from me to get started?
I’ll need access to your Azure subscription (or a dedicated resource group), details about your data sources (APIs, databases, files, etc.), and your requirements for transformations and storage.
Can you set up both simple and complex ETL pipelines?
Yes! I can build anything from a basic ADF pipeline with scheduled triggers to advanced Azure Databricks ETL workflows using the Medallion Architecture (Bronze, Silver, Gold).
Do you also handle data transformations?
Absolutely. I use ADF Data Flows (filter, aggregate, joins, derived columns, alter rows, etc.) and Databricks (PySpark/SQL) for custom transformations.
Can you integrate with my existing databases or APIs?
Yes. I can ingest data from APIs, CSV, Excel, SQL Databases, and cloud storage into Azure, ensuring smooth integration with your existing systems.
