GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → montoyamoraga → scrapers

montoyamoraga / scrapers

Licence: MIT license
scrapers for building your own image databases

Programming Languages

python
139335 projects - #7 most used programming language

Labels

scraperscrapingseleniumscrape

Projects that are alternatives of or similar to scrapers

diffbot-php-client
[Deprecated - Maintenance mode - use APIs directly please!] The official Diffbot client library
Stars: ✭ 53 (+15.22%)
Mutual labels:  scraper, scraping, scrape
ha-multiscrape
Home Assistant custom component for scraping (html, xml or json) multiple values (from a single HTTP request) with a separate sensor/attribute for each value. Support for (login) form-submit functionality.
Stars: ✭ 103 (+123.91%)
Mutual labels:  scraper, scraping, scrape
Autoscraper
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
Stars: ✭ 4,077 (+8763.04%)
Mutual labels:  scraper, scraping, scrape
stweet
Advanced python library to scrap Twitter (tweets, users) from unofficial API
Stars: ✭ 287 (+523.91%)
Mutual labels:  scraper, scrape
Serpscrap
SEO python scraper to extract data from major searchengine result pages. Extract data like url, title, snippet, richsnippet and the type from searchresults for given keywords. Detect Ads or make automated screenshots. You can also fetch text content of urls provided in searchresults or by your own. It's usefull for SEO and business related research tasks.
Stars: ✭ 153 (+232.61%)
Mutual labels:  scraper, scraping
Instagram Scraper
scrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot
Stars: ✭ 2,209 (+4702.17%)
Mutual labels:  scraper, scrape
Geziyor
Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.
Stars: ✭ 1,246 (+2608.7%)
Mutual labels:  scraper, scraping
Colly
Elegant Scraper and Crawler Framework for Golang
Stars: ✭ 15,535 (+33671.74%)
Mutual labels:  scraper, scraping
Anime Dl
Anime-dl is a command-line program to download anime from CrunchyRoll and Funimation.
Stars: ✭ 190 (+313.04%)
Mutual labels:  scraper, scraping
Goose Parser
Universal scrapping tool, which allows you to extract data using multiple environments
Stars: ✭ 211 (+358.7%)
Mutual labels:  scraper, scraping
crawler-chrome-extensions
爬虫工程师常用的 Chrome 插件 | Chrome extensions used by crawler developer
Stars: ✭ 53 (+15.22%)
Mutual labels:  scraper, scraping
Phpscraper
PHP Scraper - an highly opinionated web-interface for PHP
Stars: ✭ 148 (+221.74%)
Mutual labels:  scraper, scraping
Udemycoursegrabber
Your will to enroll in Udemy course is here, but the money isn't? Search no more! This python program searches for your desired course in more than [insert big number here] websites, compares the last updated date, and gives you the download link of the latest one back, but you also have the choice to see the other ones as well!
Stars: ✭ 137 (+197.83%)
Mutual labels:  scraper, scraping
Linkedin Profile Scraper
🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.
Stars: ✭ 171 (+271.74%)
Mutual labels:  scraper, scraping
Seleniumcrawler
An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site
Stars: ✭ 117 (+154.35%)
Mutual labels:  scraper, scraping
Jsonframe Cheerio
simple multi-level scraper json input/output for Cheerio
Stars: ✭ 196 (+326.09%)
Mutual labels:  scraper, scraping
Scrape Linkedin Selenium
`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.
Stars: ✭ 239 (+419.57%)
Mutual labels:  scraper, scraping
google-scraper
This class can retrieve search results from Google.
Stars: ✭ 33 (-28.26%)
Mutual labels:  scraper, scraping
readability-cli
A CLI for Mozilla Readability. Get clean, uncluttered, ready-to-read HTML from any webpage!
Stars: ✭ 41 (-10.87%)
Mutual labels:  scraping, scrape
Django Dynamic Scraper
Creating Scrapy scrapers via the Django admin interface
Stars: ✭ 1,024 (+2126.09%)
Mutual labels:  scraper, scraping
View All Similar Projects ➔

scrapers

About

scrapers is a collection of free/libre open-source software written by Aarón Montoya-Moraga.

scrapers is both a tool for building databases and and educational resource for learning scraping.

scrapers is educational because it tries to be heavily documented, clean, and easy to follow.

scrapers performs the scraping in an explicit way, it shows you the browser going through the data, instead of running in the background, thus being very open in the way it works, which can be used for both documentation and live performance.

Technical details

All of these scrapers were written using Python, Selenium, and ChromeDriver.

Contents

  • bing-images
  • captcha
  • google-images
  • instagram
  • mugshots

Installation and prerrequisites

  • Install Python2 and Python3
  • Install Homebrew if in Mac
  • Install Chromedriver

Acknowledgements

License

MIT

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].