I will build data validation and quality checks using pyspark snowflake and aws

India

I speak English

26 orders completed

Data Engineering, Data Analytics, Web Development, Automation, AI Development

I have 11 years of extensive experience in Data Analytical Programming, Automation, Data Quality Framework, REST APIs, Data Warehousing, Cloud Engineering and Web Development. I have expertise in bel...

Level 1

Has met certain performance criteria and shows strong potential in the marketplace.

About this Gig

Bad data leads to bad decisions. I help you detect, validate, and fix data issues by building scalable, production-ready data validation frameworks using modern data engineering tools.

What I Will Do

  • Data validation & quality checks
  • Source vs target data reconciliation
  • Row count, schema & data type validation
  • Null, duplicate & anomaly checks
  • Business rule validations
  • Automated validation pipelines
  • Detailed validation reports & logs

Tech Stack

  • PySpark large-scale data validation
  • Python custom rules & automation
  • Snowflake warehouse-level validation
  • SQL reconciliation & analytical checks
  • AWS S3, Glue, EC2, Lambda, CloudWatch, Redshift, RDS, Secrets Manager

Expertise:

Automations

Big data

ETL

Normalization

SQL

NoSQL

Technology:

Amazon Redshift

Apache Cassandra

Python

SQL

Snowflake

Other Data Engineering Services I Offer