I will build a python data pipeline with tests and scheduling
Python Automation, Data Pipelines and Excel Solutions, Finance Background
About this Gig
Your data arrives in files, APIs and exports that do not agree with each other, and somebody is stitching it together by hand every month.
I build that as a pipeline instead: a repeatable job that ingests your sources, normalizes them to one schema, and produces the output you actually need - on a schedule, without anyone babysitting it.
How I work:
Every source row is accounted for. Rows that fail to map are FLAGGED, never silently dropped. You get a reconciliation showing counts before and after.
Failure cases handled first: retries on transient errors, and an alert when a source is unavailable rather than a quietly empty report.
Credentials live in environment secrets you control, never in the code.
Tests included, so a provider changing their format is caught rather than discovered three weeks later.
I build these with CI, type checking, automated deployment and scheduled execution as standard. Your delivery includes the project-specific code, its tests and its documentation, to the scope of your package.
Message me with your sources and what the output needs to look like, and I will tell you honestly whether this is a fit before you order.
Destination Platform:
Other
Tools & Platforms:
Other
FAQ
What data sources can you work with?
CSV, Excel, JSON, XML, SQL databases (Postgres, MySQL, SQLite) and REST APIs. If yours is something else, message me first and I will tell you honestly.
Will I be able to run it myself afterwards?
Yes. You get the source code and setup documentation, and the Premium package includes a handover session: one scheduled call of up to 45 minutes. The point is that you are not dependent on me.
What happens to rows that do not fit the schema?
They are flagged with a reason, never dropped. Silent row loss is the most common and most damaging defect in this kind of work, so I reconcile counts before and after and show you the difference.
Do you need access to our production systems?
No, and I would rather not have it. Exports or a read-only credential are enough for almost everything.
Can it run on a schedule without me doing anything?
Yes, from the Standard package up. It runs on a schedule in your own account - usually GitHub Actions, so there is no new subscription to buy. I build and test the scheduled job and hand you the configuration and a setup guide, so nothing depends on me.
What if a source changes format after delivery?
The tests will catch it. Within the revision window I will fix it; beyond that I am happy to quote a small maintenance job.

