I will do data cleaning, deduplication and formatting on messy data files
Gig Summary
Python Data Engineer: Web Scraping, ETL Pipelines and Data Enrichment
Level 1
Has met certain performance criteria and shows strong potential in the marketplace.
About this Gig
The file is 40,000 rows, three people have edited it, and the same company appears as Acme, ACME Ltd and Acme Limited. Fixing that by hand takes a week and introduces new errors on the way. Data cleaning done in code takes an afternoon and can be run again next month.
What I fix
- Duplicates, including near-matches that a spreadsheet formula will never catch
- Inconsistent dates, names, addresses, currencies and casing
- Merged or split columns, and files that need reshaping before they are usable
- Encoding, delimiters, and the rows that break on import
What comes back
- One clean file: CSV, Excel, JSON or a Google Sheet
- A changelog naming what was changed, merged or dropped, with the row counts
- The cleaning script, if you want to run it yourself next time
The largest single job I have run processed 100M+ JSON files into analysis-ready CSV.
Excel runs out before most of these jobs do. Anything past about 50,000 rows, or anything needing fuzzy matching, I do in Python and Pandas instead. Send me the file and tell me what done looks like. I will confirm the scope and quote before you order.
My Portfolio
FAQ
Can you merge several files into one?
Yes, and it is most of the work on a typical order. Different column names, different date formats, overlapping rows and one file with an extra column nobody documented. I map them to a single schema and tell you which rows came from where.
My file is 500,000 rows. Is that a problem?
No. Above about 50,000 rows the work moves out of a spreadsheet and into Python and Pandas, which is faster and repeatable rather than slower. Half a million rows is an ordinary job. Send a sample and I will confirm the tier before you order.
What counts as a duplicate?
You tell me, and that is the part worth agreeing before I start. Exact matches are easy. The useful work is fuzzy matching: Acme Ltd against ACME Limited, the same address written four ways, two records that differ only in whitespace. I show you the matches before merging.
Will you tell me what you changed?
Yes, every order comes with a changelog: rows in, rows out, what was merged, what was dropped and why, and which fields were standardised. You should not have to trust a clean file. You should be able to check it.
Do I get the cleaning script?
Included in Standard and Premium, in Python, commented, so you can run it on next month's export without ordering again. Basic delivers the cleaned file only, which is the right choice if this is genuinely a one-off.
Can you clean scraped data with broken encoding?
Yes, and it is a common one. Mojibake from the wrong encoding, HTML entities left in text fields, stray whitespace and newlines inside cells, numbers stored as text with currency symbols attached. Send a sample and I will tell you what is recoverable.
Can you analyse the data as well as clean it?
Summary tables, pivots and charts are included in Standard and Premium, and they are useful for sanity-checking the clean. If you need real statistical work or modelling, that is a different job and I would rather point you elsewhere than do it adequately.
Can this run every month automatically?
Yes, as a scheduled pipeline rather than a repeated order: the file lands, the script runs, the clean version appears where you need it. That is my data pipeline gig, not this one. Message me and I will send the link rather than charging you twelve times.
My data has personal information in it. Is that a problem?
Tell me before you send it. I can work on it, and I will delete the source files after delivery on request, but I would rather know upfront than find names and addresses in a file I did not expect them in. Pseudonymised or sampled data is fine for scoping.
How do I know which package I need?
Send the file first, or a sample of it. Some jobs look simple and need fuzzy matching across three sources; others look terrifying and are one find-and-replace. I will tell you which tier it is, or quote a custom offer, before you spend anything.
93 reviews for this Gig
| (83) | ||
| (8) | ||
| (0) | ||
| (1) | ||
| (1) |
Rating Breakdown
- Seller communication level
- Quality of delivery
- Value of delivery
Sort By
J jcummings72

United States
I had 74,000 duplicate contacts and he was able to scrub the file and condense to no duplicates with all of the information. I’ve been trying to resolve this for years. He was able to turn it around within 24 hours. Thanks again!
$50-$100
Price
4 days
Duration
Helpful?S 
soweilam01

United Arab Emirates
Ongoing collaborationHighly recommend this seller for any project involving word and or excel.
$50
Price
1 day
Duration
Helpful?S 
soweilam01

United Arab Emirates
Ongoing collaborationVery detailed oriented and thorough seller. Happy reterning customer.
$50
Price
4 days
Duration
Helpful?J 
johncharlin

India
very good job done
$50
Price
1 day
Duration
Helpful?A aclaudiof

Puerto Rico
Job was done close to expectations, but not complete. Unfortunately it was delivered too late, and I couldn't meet my client due date. Supplier suggested a 50% discount, that ant the end was not given, and never delivered on time, with lack of response due to a private family event.
$100-$200
Price
2 weeks
Duration
Helpful?
93 reviews for this Gig
| (83) | ||
| (8) | ||
| (0) | ||
| (1) | ||
| (1) |
Rating Breakdown
- Seller communication level
- Quality of delivery
- Value of delivery
Sort By
J jcummings72

United States
I had 74,000 duplicate contacts and he was able to scrub the file and condense to no duplicates with all of the information. I’ve been trying to resolve this for years. He was able to turn it around within 24 hours. Thanks again!
$50-$100
Price
4 days
Duration
Helpful?S 
soweilam01

United Arab Emirates
Ongoing collaborationHighly recommend this seller for any project involving word and or excel.
$50
Price
1 day
Duration
Helpful?S 
soweilam01

United Arab Emirates
Ongoing collaborationVery detailed oriented and thorough seller. Happy reterning customer.
$50
Price
4 days
Duration
Helpful?J 
johncharlin

India
very good job done
$50
Price
1 day
Duration
Helpful?A aclaudiof

Puerto Rico
Job was done close to expectations, but not complete. Unfortunately it was delivered too late, and I couldn't meet my client due date. Supplier suggested a 50% discount, that ant the end was not given, and never delivered on time, with lack of response due to a private family event.
$100-$200
Price
2 weeks
Duration
Helpful?

