622 Open source projects that are alternatives of or similar to top-github-scraper

PHP Scraper - an highly opinionated web-interface for PHP

`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.

A framework for creating semi-automatic web content extractors

PHP Library for detecting CMS

Web Scraping Craigslist's Engineering Jobs in NY with Scrapy

A library that enables programmatic interaction with daft.ie. Daft.ie has nationwide coverage and contains about 80% of the total available properties in Ireland.

📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs

Node.js package to bypass CloudFlare's anti-bot JavaScript challenges

NBA Stats API via Basketball Reference

Papercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.

MetaData html scraper and parser for Node.js (supports Promises and callback style)

Fetch user's data across social media

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

A Router WiFi key recovery/cracking tool with a twist.

📓 Minicurso de raspagem de dados web com Python ministrado na Semana de Tecnologia da FATEC Jundiaí

Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments

Go cascadia package command line CSS selector

Powerful web scraping framework for Crystal

Apify SDK — The scalable web scraping and crawling library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, SHFE and news data crawlers on BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist

Basic setup with random user agents and IP addresses for Python Scrapy Framework.

Simple Query Scraping with CSS and Go Reflection (MOVED to Gitlab)

A Node.js library to easily manage and rotate a pool of web browsers, using any of the popular browser automation libraries like Puppeteer, Playwright, or SecretAgent.

Faster requests on Python 3

A library to read a YML file with Xpath or CSS Selectors and extract data from HTML pages using them

Web Scraping Framework

Web scraper for grabing data from Linkedin profiles or company pages (personal project)

Display Markdown formatted documents on your local web server using GitHub's Markdown rendering API and CSS to mimic the visuals of GitHub itself.

Transform unstructured document collections to structured Linked Data

CHeSF is the Chrome Headless Scraping Framework, a very very alpha code to scrape javascript intensive web pages

A Discord bot built to satisfy a multitude of needs

Download git-repo for any url

Find a github repository an its recent commits

Get the contributor stats summary from all repos of any given organization

Instagram Bot which when given a post url will spam mentions to increase the chances of winning. Win Instagram Giveaways!

Proxy pool. Finds and checks proxies with rest api for querying results. Can find over 25k proxies in under 5 minutes.

Web crawling and scraping framework for Golang

Tableau scraper python library. R and Python scripts to scrape data from Tableau viz

Crawl sites for RSS, Atom, and JSON feeds.

App to write journal digitally. Simple as that.

State of the art object detection in real-time using YOLOV3 algorithm. Augmented with a process that allows easy training of the classifier as a plug & play solution . Provides alert if an item in an alert list is detected.

Simple scripts that I'm using to automate the boring things.

🌍 Dataset that shows the Internet affordability by country (a shocking reality!)

GitHub workflow-optimizing bot by @zulip

GitHub CLI extension to display GitHub notifications

All-in-one Python (And now Go!) module to help solve captchas with Capmonster, 2captcha and Anticaptcha API's!

Larry 🐦 is a really simple Twitter bot generator that tweets random repositories from Github built in Go

Let's learn a deep look at the Android architecture

🚀 Create an automatic portfolio based on GitHub profile.

vue和element-ui搭建一個後台管理系統，使用github提供的api搞事情。輸入您的github賬號名自動幫你生成基本的github信息哦😯

GitHub notifications tracker for Telegram. Pushes GitHub notifications to Telegram.

Github iOS Client based on Github REST V3 API and GraphQL V4 API

Library to download subtitles from subscene.com

Full-Text Search the readme, description, homepage and URL of your GitHub starred repository. Use GitHub OAuth 2, React, Redux, Golang (server side), Elasticsearch, Redis.

Fixtures for all the octokittens

A command line program to download hentai videos and images from multiple websites

This repository contains very useful restful API's and functionalities in node-js containing many important tutorial code for mastering node-js, all tutorials have been published on medium.com, tutorials link is given below