I will build scalable web crawler and scraper

India

I speak English

72 orders completed

Big Data Engineer at Amazon with 6+ years of experience in Hadoop, Spark, PySpark, Hive and Cloud platforms like AWS, GCP, Azure and DataBricks. Developed and scheduled highly efficient production ETL...
About this Gig

I'm a Big Data Engineer from Amazon with 5+ years of experience in data collection, modelling, and transformation.

If you're looking to crawl and scrape some websites, you're just a ping away from accomplishing it.


Web Crawling | Web Scraping | Data Extraction | Data Mining | Data Collection | Data Scraping


My Services (Including but not limited to):

  1. Web crawling - navigating over web pages, web pages with pagination/scroll more or load more contents.
  2. Web scraping - extracting content from web pages, image extraction, URL extraction and revisiting them.
  3. Data extraction from API endpoints.
  4. Collect data in different formats (Excel Sheet, CSV, XML, JSON, MySql, Parquet, ORC, etc).
  5. Crawl and Scrape any websites including E-Commerce websites (Shopify, WooComerce, etc), Business Directories Scraping, etc.,
  6. Download images from website.



Technology:

Python

Google Sheets

Excel

Scrapy

Beautiful soup

Information type:

Currency & stocks

Images

News & events

Technique:

Automated