I will build python etl data pipelines for apis, sql and csv
About this Gig
Struggling with messy data, manual reporting, or unreliable data workflows that slow down business decisions?
I will build a Python ETL / ELT data pipeline to extract, clean, transform, and load data from APIs, CSV/Excel files, SQL databases, cloud storage, or CRMs for dashboards, reporting, analytics, ML, or AI-ready workflows.
You will get:
- ETL / ELT pipelines for automated data ingestion and transformation
- API, CSV, Excel, SQL database, and cloud storage integrations
- Batch pipelines scheduled hourly, daily, weekly, or custom
- Data warehouse loading into BigQuery, Redshift, PostgreSQL, Snowflake, or ClickHouse
- dbt models for staging, incremental loads, tests, and documentation
- Orchestration with Airflow, Cloud Composer, cron, or cloud schedulers
- Data quality checks, validation, logging, and error handling
Tech stack:
- Python, SQL, dbt
- Airflow, Airbyte
- Kafka, Debezium
- BigQuery, Redshift, Snowflake, ClickHouse
- MySQL, PostgreSQL, MongoDB
- AWS Glue, Lambda, S3, Kinesis, DMS
- GCP Dataflow, Pub/Sub, Datastream
Message me before ordering so I can understand your requirements.
FAQ
Do you work with Python ETL scripts?
Yes. I can create Python ETL scripts for data extraction, cleaning, transformation, validation, loading, logging, and error handling.
Can you automate my data pipeline on a schedule?
Yes. I can schedule your data pipeline to run hourly, daily, weekly, or on a custom schedule using Airflow, cron, Cloud Composer, cloud schedulers, or other orchestration tools.
