TripAdvisor Scraper
Only for educational purposes. Use at own risk, it might violate TripAdvisor policies.
Dependencies
Install:
Tested configuration
- Python 2.7.6 + Scrapy 0.24.4
- Python 2.7.6 + Scrapy 0.20.2
Usage - Have fun!
cd tripadvisor-scraper/
Scrape and save data in JSON lines format:
scrapy crawl tripadvisor-restaurant -o output/result.json
For JSON format use:
scrapy crawl tripadvisor-restaurant -o output/result.json -t json
but pay attention it doesn’t scale well for large amounts of data since incremental (aka. stream-mode)