1958 Open source projects that are alternatives of or similar to Scrapple

Simple but useful Python web scraping tutorial code.

📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.

PHP Library for detecting CMS

`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.

Papercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.

Faster requests on Python 3

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

A library that enables programmatic interaction with daft.ie. Daft.ie has nationwide coverage and contains about 80% of the total available properties in Ireland.

Web Scraping Craigslist's Engineering Jobs in NY with Scrapy

Simple Query Scraping with CSS and Go Reflection (MOVED to Gitlab)

Crawlera middleware for Scrapy

Scape top GitHub repositories and users based on keywords

Go cascadia package command line CSS selector

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

PHP Scraper - an highly opinionated web-interface for PHP

Powerful web scraping framework for Crystal

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

Chopper is a tool to extract elements from HTML by preserving ancestors and CSS rules

Tool to scrape linkedin

Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments

A Scrapy extension to log items coverage when the spider shuts down

🌎 🏨 Scrape Booking.com 🏨 🌎

Web Scraping Framework

extract data from html table

A Node.js library to easily manage and rotate a pool of web browsers, using any of the popular browser automation libraries like Puppeteer, Playwright, or SecretAgent.

⚽ Instantly find 🏆EURO 2016 live-streams & highlights, now a Web App!

Basic setup with random user agents and IP addresses for Python Scrapy Framework.

A test suite of common scraper detection techniques. See how detectable your scraper stack is.

码云仓库链接:AJay13/ECommerceCrawlers Github 仓库链接:DropsDevopsOrg/ECommerceCrawlers 项目展示平台链接:http://wechat.doonsec.com

With Selenium headless browsing and CAPTCHA solving

A Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.

A library to read a YML file with Xpath or CSS Selectors and extract data from HTML pages using them

Dynamic file detection tool based on crawler 基于爬虫的动态敏感文件探测工具

Simple and friendly Bot for Instagram, using Selenium and Scrapy with Python.

Web scraper for grabing data from Linkedin profiles or company pages (personal project)

Spin up Tor containers and then proxy HTTP requests via these Tor instances

Scrapy project for scraping data from IMDB with Movie Dataset including 58,623 movies' data.

web scraping tool for grailed.com

Scraping Ebay's products using Scrapy Web Crawling Framework

Proxy pool. Finds and checks proxies with rest api for querying results. Can find over 25k proxies in under 5 minutes.

🕸 List of mini projects that involve web scraping 🕸

Crawl all unique internal links found on a given website, and extract SEO related information - supports javascript based sites

Simplify your ETL processes with these hands-on data sanitation tips, tricks, and best practices

A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.

Build a Flask web application to help users retrieve key restaurant information and feature-based reviews (generated by applying market-basket model – Apriori algorithm and NLP on user reviews).

Crawly, a high-level web crawling & scraping framework for Elixir.

Scraping articles of medium and providing audio versions 📑 to 🔊 using django

A Scraper made 100% in Python using BeautifulSoup and Tor. It can be used to scrape both normal and onion links. Happy Scraping :)

🐤️ Lost Ark wait notifier

🕷️some python spiders with BeautifulSoup or scarpy

PTT 網路版爬蟲

API for scrapping common meme sites

No description or website provided.

Building a model to recognize incentives for landscape restoration in environmental policies from Latin America, the US and India. Bringing NLP to the world of policy analysis through an extensible framework that includes scraping, preprocessing, active learning and text analysis pipelines.

ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-020-03726-9

Apify SDK — The scalable web scraping and crawling library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy

📓 Minicurso de raspagem de dados web com Python ministrado na Semana de Tecnologia da FATEC Jundiaí

PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs