GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → madeindjs → spider

madeindjs / spider

Licence: MIT License
Multithreaded Web spider crawler written in Rust.

Programming Languages

rust
11053 projects
shell
77523 projects

Labels

crawlerscraperreqwest

Projects that are alternatives of or similar to spider

covid-19
Current and historical coronavirus covid-19 confirmed, recovered, deaths and active case counts segmented by country and region. Includes csv, json and sqlite data along with an interactive website explorer.
Stars: ✭ 15 (-81.48%)
Mutual labels:  scraper
genius-lyrics-api
A library for fetching song lyrics & album art from genius.com 🎶🌈
Stars: ✭ 129 (+59.26%)
Mutual labels:  scraper
bing-ip2hosts
bingip2hosts is a Bing.com web scraper that discovers websites by IP address
Stars: ✭ 99 (+22.22%)
Mutual labels:  scraper
peeling-onions
A repository to store Deep Web (onion domain) crawler, scraper, and NLP tools for Tor network.
Stars: ✭ 18 (-77.78%)
Mutual labels:  scraper
kick-off-web-scraping-python-selenium-beautifulsoup
A tutorial-based introduction to web scraping with Python.
Stars: ✭ 18 (-77.78%)
Mutual labels:  scraper
metafetch
NodeJS package that fetches a given URL's title, description, images, links etc.
Stars: ✭ 21 (-74.07%)
Mutual labels:  scraper
Coinsta
A Python package for acquiring both historical and current data of cryptocurrencies
Stars: ✭ 47 (-41.98%)
Mutual labels:  scraper
ogcheckr-api
An api to check social media username availability on a variety of services
Stars: ✭ 18 (-77.78%)
Mutual labels:  scraper
evine
Interactive CLI Web Crawler
Stars: ✭ 140 (+72.84%)
Mutual labels:  scraper
scoopi-scraper
Scoopi Web Scraper is a heavy duty tool to extract data from HTML pages.
Stars: ✭ 18 (-77.78%)
Mutual labels:  scraper
Scraper-Projects
🕸 List of mini projects that involve web scraping 🕸
Stars: ✭ 25 (-69.14%)
Mutual labels:  scraper
patreon-scraper
WIP Patreon attachment download written in TypeScript
Stars: ✭ 25 (-69.14%)
Mutual labels:  scraper
fb-page-chat-download
Python script to download messages from a Facebook page to a CSV file
Stars: ✭ 51 (-37.04%)
Mutual labels:  scraper
metacritic api
PHP Metacritic API - Mirrored by my GitLab
Stars: ✭ 31 (-61.73%)
Mutual labels:  scraper
pysoundcloud
Scraping the Un–scrapable™
Stars: ✭ 63 (-22.22%)
Mutual labels:  scraper
wishlist
Read an Amazon wishlist programmatically with Python
Stars: ✭ 44 (-45.68%)
Mutual labels:  scraper
scrapy facebooker
Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.
Stars: ✭ 22 (-72.84%)
Mutual labels:  scraper
linkedinscraper
LinkedinScraper is an another information gathering tool written in python. You can scrape employees of companies on Linkedin.com and then create these employee names, titles and emails.
Stars: ✭ 22 (-72.84%)
Mutual labels:  scraper
sp-subway-scraper
🚆This web scraper builds a dataset for São Paulo subway operation status
Stars: ✭ 24 (-70.37%)
Mutual labels:  scraper
django-instagram
Instagram application for Django.
Stars: ✭ 73 (-9.88%)
Mutual labels:  scraper
View All Similar Projects ➔

Spider

crate version

Multithreaded web crawler written in Rust.

Getting Started

View the spider to get started. If you need the CLI go to the spider_cli.

You can get a working example at examples/example.rs and run it with.

cargo run --example example

Contribute

I am open-minded to any contribution. Just fork & commit on another branch.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].