310 Open source projects that are alternatives of or similar to Juriscraper

OWASP D4N155 - Intelligent and dynamic wordlist using OSINT

Python framework to scrape Pastebin pastes and analyze them

A ruby gem to check if the owner of a given email address or website is working for THE MAN (a.k.a verifies government domains).

Open source release of street center lines in Chicago.

PHP Library for detecting CMS

Get info from any web service or page

Grawler is a tool written in PHP which comes with a web interface that automates the task of using google dorks, scrapes the results, and stores them in a file.

SEO python scraper to extract data from major searchengine result pages. Extract data like url, title, snippet, richsnippet and the type from searchresults for given keywords. Detect Ads or make automated screenshots. You can also fetch text content of urls provided in searchresults or by your own. It's usefull for SEO and business related research tasks.

An experiment in open source at the Department of Defense.

🔥🐻 The Speakeasy Scripting Engine Which Combines Speed, Safety, and Simplicity

Simple scriptes for Level UP your scraping Skills, and source code for Level UP playlist on Youtube

📚 Documentation, rationale and accessibility guidance

Information on joining the government community — a collaborative community for sharing best practices in furtherance of open source, open data, and open government efforts.

A scalable web crawler framework for Java.

The web browser that's built for scraping.

Index page and integration layer for projects that related to Ukrainian Health Services government institution

📖 This tool is to download course from educative.io for offline usage. It uses your login credentials and download the course.

Use Pentaho's open source data integration tool (Kettle) to create Extract-Transform-Load (ETL) processes to update a Socrata open data portal. Documentation is available at http://open-data-etl-utility-kit.readthedocs.io/en/stable

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

legacy backend for Open States

USWDS website and documentation

The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url

Pick the most common user-agents on the Internet 👻

Nim library for querying HTML using CSS-selectors (like JavaScripts document.querySelector)

htmlSQL is a experimental PHP library which allows you to access HTML values by an SQL like syntax.

Caseflow is a web application that enables the tracking and processing of appealed claims at the Board of Veterans' Appeals.

Contains all the public APIs listed in Phantombuster's API store. Pull requests welcome!

Simple Query Scraping with CSS and Go Reflection (MOVED to Gitlab)

An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site

Pythonic HTML Parsing for Humans™

Scrapy, a fast high-level web crawling & scraping framework for Python.

Data.gov source code and issue tracker

Laravel package to collect your bank statements history. Currently support for parsing statements history from BCA, Mandiri, BNI, and MUAMALAT e-banking websites.

List of datasets to apply stats/machine learning/technology to the world of social good.

Python scraper for Language Pods such as Japanesepod101.com 👹 🗾 🍣 Compatible with Japanese, Chinese, French, German, Italian, Korean, Portuguese, Russian, Spanish and many more! ✨

Open source release of bike routes in Chicago.

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

Extract data or evaluate value from HTML/XML documents using XPath

Library with a set of tools for scraping information about Nintendo games and its prices across all regions (NA, EU and JP).

🕷 Google client for SERPS

Node.js package to bypass CloudFlare's anti-bot JavaScript challenges

Anime-dl is a command-line program to download anime from CrunchyRoll and Funimation.

Example end to end data engineering project.

Your will to enroll in Udemy course is here, but the money isn't? Search no more! This python program searches for your desired course in more than [insert big number here] websites, compares the last updated date, and gives you the download link of the latest one back, but you also have the choice to see the other ones as well!

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

USWDS visual design and UX assets

Data extraction of Google's COVID-19 Mobility Reports

微服务治理框架简介

Gather, curate, and feature stories of public servants and civic hackers using GitHub as part of their open government innovations

Linkedin Learning videos downloader

ASP.NET View State Decoder

scan for webcams on the internet

Open-source budget visualization framework.

PHP Scraper - an highly opinionated web-interface for PHP

Distributed crawler, database and web frontend for public directories indexing

Open source design and front-end development resources for creating Section 508 compliant, responsive, and consistent websites.

Provides easier interaction with Socrata open data portals http://dev.socrata.com. Users can provide a 'Socrata' data set resource URL, or a 'Socrata' Open Data API (SoDA) web query, or a 'Socrata' "human-friendly" URL, returns an R data frame. Converts dates to 'POSIX' format. Manages throttling by 'Socrata'.

The developer community designing and developing public digital services in Italy

Scraping statistics, predicting NBA player performance with neural networks and boosting algorithms, and optimising lineups for Draft Kings with genetic algorithm. Capstone Project for Machine Learning Engineer Nanodegree by Udacity.

A curated list of awesome puppeteer resources.