1891 Open source projects that are alternatives of or similar to Sasila

Simple but useful Python web scraping tutorial code.

Antch, a fast, powerful and extensible web crawling & scraping framework for Go

Elegant Scraper and Crawler Framework for Golang

Scrapy, a fast high-level web crawling & scraping framework for Python.

[Unmaintained] A simple and clean video/music/image downloader 👾

A scalable web crawler framework for Java.

Declarative web scraping

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

Distributed crawler powered by Headless Chrome

Transistor, a Python web scraping framework for intelligent use cases.

Crawly, a high-level web crawling & scraping framework for Elixir.

No description or website provided.

HTTP API for Scrapy spiders

🐾 Creeper - The Next Generation Crawler Framework (Go)

Tổng hợp script crawl dữ liệu từ các mạng xã hội & website tiếng Việt

新浪微博相册大图多线程爬虫。

自学入门 Python 优质中文资源索引，包含 书籍 / 文档 / 视频，适用于 爬虫 / Web / 数据分析 / 机器学习 方向

OWASP D4N155 - Intelligent and dynamic wordlist using OSINT

Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head

News, full-text, and article metadata extraction in Python 3. Advanced docs:

An Instagram bot developed using the Selenium Framework

ScrapingOutsourcing专注分享爬虫代码 尽量每周更新一个

네이버 뉴스 수집을 위한 도구

Go process used to crawl websites

Crawlera middleware for Scrapy

京东商品价格监控：监控用户设定商品价格，降价邮件/微信提醒。技术：Python爬虫/IP代理池/JS接口爬取/Selenium页面爬取

🎓 中国大学MOOC、学堂在线、网易云课堂、好大学在线、爱课程 MOOC 课程下载。

Free Web Scraping Tool with Java

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

Powerful web scraping framework for Crystal

Crawl all unique internal links found on a given website, and extract SEO related information - supports javascript based sites

🤖 A Versatile Telegram Bot

A test suite of common scraper detection techniques. See how detectable your scraper stack is.

《数据采集从入门到放弃》源码。内容简介：爬虫介绍、就业情况、爬虫工程师面试题 ；HTTP协议介绍； Requests使用 ；解析器Xpath介绍； MongoDB与MySQL； 多线程爬虫； Scrapy介绍 ；Scrapy-redis介绍； 使用docker部署； 使用nomad管理docker集群； 使用EFK查询docker日志

APIs for loginning some websites by using requests.

一个获取知乎用户主页信息的多线程Python爬虫程序。

B站用户爬虫 好耶~是爬虫

Easily crawl news portals or blog sites using Storm Crawler.

👨‍👩‍👦 Social account detection and extraction in Python, e.g. for crawling/scraping.

An open source web crawling platform

ProxyCrawl Python library for scraping and crawling

Apify SDK — The scalable web scraping and crawling library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

Universal scrapping tool, which allows you to extract data using multiple environments

Tutorial for web scraping / crawling with Node.js.

A Scrapy extension to log items coverage when the spider shuts down

[Deprecated - Maintenance mode - use APIs directly please!] The official Diffbot client library

PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs

A Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, ...). Including asynchronous networking support.

A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.

Crawl sites for RSS, Atom, and JSON feeds.

Screen scraping and web crawling framework

An interactive Command-Line Interface Build in NodeJS for downloading a single or multiple images to disk from URL

Web crawling and scraping framework for Golang

Papercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.

Continuous scalable web crawler built on top of Flink and crawler-commons

A framework for creating semi-automatic web content extractors

🐝 Web vertical crawler framework for fun

Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.