I will build scalable data pipelines and data warehousing on gcp bigquery
Certified Google Cloud Data Engineer and Datawarehouse Specialist
About this Gig
Are you looking to harness the power of Google Cloud Platform to transform, store, and analyze your data efficiently? I specialize in designing and building robust data pipelines and data warehousing solutions on GCP.
Services I Offer:
1) Design and development of end-to-end data pipelines
2) Integration with various data sources (APIs, on-prem, cloud storage, etc.)
3) Data ingestion using Pub/Sub, Cloud Storage, or Cloud Functions
4) ETL/ELT implementation using Dataflow or Cloud Composer (Apache Airflow)
5) Data warehousing with BigQuery
6) Scheduled and event-driven workflows
7) Monitoring and logging setup for pipelines
8) BigQuery Stored Procedure, Views
9) Data Build Tool(DBT)
10) Integration with Youtube,Salesforce, Google sheets and and other API via Cloud Run
Other Data Engineering Services I Offer
FAQ
What tools or services on GCP do you use for building data pipelines?
I commonly use BigQuery, Cloud Storage, Pub/Sub, Cloud Dataflow, Cloud Functions. Depending on your use case, I’ll recommend the most cost-effective and scalable combination.
Can you integrate data from third-party APIs or on-premise sources?
Yes! I can connect GCP pipelines to REST APIs, flat files, SQL/NoSQL databases, or even on-premise sources using secure connectors and data transfer services.
Do you support real-time or streaming data pipelines?
Absolutely. I can build streaming pipelines using Pub/Sub and Dataflow for low-latency, real-time processing, including windowed aggregations, filtering, and enrichment.
What kind of transformations can you implement?
From simple SQL-based data transformations in BigQuery to complex event-driven and stream transformations in Dataflow/Apache Beam, I support a full range of ETL/ELT tasks.
Can you optimize my existing GCP pipeline or data warehouse?
Yes. I offer performance tuning, storage cost optimization, and review of your GCP usage to reduce latency and expenses.
