voliveirajr / Seleniumcrawler
Licence: gpl-3.0
An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site
Stars: ✭ 117
Programming Languages
python
139335 projects - #7 most used programming language
Projects that are alternatives of or similar to Seleniumcrawler
Scrape Linkedin Selenium
`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.
Stars: ✭ 239 (+104.27%)
Mutual labels: scraper, scraping, selenium, selenium-webdriver
Linkedin
Linkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy
Stars: ✭ 309 (+164.1%)
Mutual labels: scraper, scrapy, scraping, selenium-webdriver
Panther
A browser testing and web crawling library for PHP and Symfony
Stars: ✭ 2,480 (+2019.66%)
Mutual labels: scraping, selenium, selenium-webdriver
Udemycoursegrabber
Your will to enroll in Udemy course is here, but the money isn't? Search no more! This python program searches for your desired course in more than [insert big number here] websites, compares the last updated date, and gives you the download link of the latest one back, but you also have the choice to see the other ones as well!
Stars: ✭ 137 (+17.09%)
Mutual labels: scraper, scraping, selenium
InstaBot
Simple and friendly Bot for Instagram, using Selenium and Scrapy with Python.
Stars: ✭ 32 (-72.65%)
Mutual labels: scraping, selenium, scrapy
Email Extractor
The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url
Stars: ✭ 81 (-30.77%)
Mutual labels: scraper, scrapy, scraping
RARBG-scraper
With Selenium headless browsing and CAPTCHA solving
Stars: ✭ 38 (-67.52%)
Mutual labels: scraping, selenium, scrapy
Sillynium
Automate the creation of Python Selenium Scripts by drawing coloured boxes on webpage elements
Stars: ✭ 100 (-14.53%)
Mutual labels: scraper, selenium, selenium-webdriver
bots-zoo
No description or website provided.
Stars: ✭ 59 (-49.57%)
Mutual labels: scraper, scraping, selenium
scrapy facebooker
Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.
Stars: ✭ 22 (-81.2%)
Mutual labels: scraper, scraping, scrapy
kick-off-web-scraping-python-selenium-beautifulsoup
A tutorial-based introduction to web scraping with Python.
Stars: ✭ 18 (-84.62%)
Mutual labels: scraper, selenium, selenium-webdriver
Post Tuto Deployment
Build and deploy a machine learning app from scratch 🚀
Stars: ✭ 368 (+214.53%)
Mutual labels: scrapy, scraping, selenium
Django Dynamic Scraper
Creating Scrapy scrapers via the Django admin interface
Stars: ✭ 1,024 (+775.21%)
Mutual labels: scraper, scrapy, scraping
Cfselenium
A native Selenium WebDriver binding for ColdFusion
Stars: ✭ 77 (-34.19%)
Mutual labels: selenium, selenium-webdriver
Alipayspider Scrapy
AlipaySpider on Scrapy(use chrome driver); 支付宝爬虫(基于Scrapy)
Stars: ✭ 70 (-40.17%)
Mutual labels: scrapy, selenium
Spam Bot 3000
Social media research and promotion, semi-autonomous CLI bot
Stars: ✭ 79 (-32.48%)
Mutual labels: scraper, selenium
Wswp
Code for the second edition Web Scraping with Python book by Packt Publications
Stars: ✭ 112 (-4.27%)
Mutual labels: scrapy, selenium
Instaloctrack
An Instagram OSINT tool to collect all the geotagged locations available on an Instagram profile in order to plot them on a map, and dump them in a JSON.
Stars: ✭ 85 (-27.35%)
Mutual labels: scraper, selenium
seleniumcrawler
This is a Webcrawler based on Scrapy and Selenium frameworks
This spider crawls thru the directferries.com website in order to generate a json file with all tickets available for one of these directions Dublin-Liverpool / Liverpool-Dublin with departure tomorrow and returning in 3 days.
HOW TO EXECUTE:
Is required an environment with the following requirements installed: -Python 2.7 -Scrapy 0.18 -Selenium web-drivers
To execute the crawler the following command should be executed from the project path
scrapy crawl crawlermate_selenium -a category=[dublin or liverpool] -o [filename] -t json
for an example, to generate tickets for dublin to liverpool on items.json file you should execute scrapy crawl crawlermate_selenium -a category=dublin -o items.json -t json
References:
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].