1961 Open source projects that are alternatives of or similar to D4n155

Search Google/Bing/Ecosia/DuckDuckGo/Yandex/Yahoo for a search term with a default set of websites, bug bounty programs or a custom collection.

Declarative web scraping

A small program that forwards searches from Cortana to your preferred browser and search engine.

Get (almost) original messages from google group archives. Your data is yours.

Universal scrapping tool, which allows you to extract data using multiple environments

🕷 Google client for SERPS

No description or website provided.

Easy way to brute-force web directory.

Open Cortana searches with your default browser.

Go Web Application Penetration Test

Crawly, a high-level web crawling & scraping framework for Elixir.

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

A Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, ...). Including asynchronous networking support.

Grawler is a tool written in PHP which comes with a web interface that automates the task of using google dorks, scrapes the results, and stores them in a file.

Crawl all unique internal links found on a given website, and extract SEO related information - supports javascript based sites

Amazingly fast response crawler to find juicy stuff in the source code! 😎🔥

Scrape and take screenshots of dynamic and static webpages

Crawlera middleware for Scrapy

Creates an XML-Sitemap by crawling a given site.

OSINT Swiss Army Knife

HostHunter a recon tool for discovering hostnames using OSINT techniques.

A framework for creating semi-automatic web content extractors

Simple but useful Python web scraping tutorial code.

Free Web Scraping Tool with Java

pagodo (Passive Google Dork) - Automate Google Hacking Database scraping and searching

自学入门 Python 优质中文资源索引，包含 书籍 / 文档 / 视频，适用于 爬虫 / Web / 数据分析 / 机器学习 方向

Antch, a fast, powerful and extensible web crawling & scraping framework for Go

Easy to use lightweight web crawler（易用的轻量化网络爬虫）

Elegant Scraper and Crawler Framework for Golang

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

Trino: Master your translations with command line!

Fawkes is a tool to search for targets vulnerable to SQL Injection. Performs the search using Google search engine.

This repository contains all the code I use in my YouTube tutorials.

GUI based offensive penetration testing tool (Open Source)

Google search results crawler, get google search results that you need

A tool for Windows to upload and manage files in Google Drive. It resumes uploads in case of an error or failure. Perfect for uploading large files or if your connection is unstable.

Verify that a request is from Google crawlers using Google's DNS verification steps

Papercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.

Lightweight package to query popular search engines and scrape for result titles, links and descriptions

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

Easily create XML sitemaps for your website.

一个灵活、友好的爬虫框架

Scrapy, a fast high-level web crawling & scraping framework for Python.

A Python Tool For google Hacking

Crawler (Bot) searching for credential leaks on different paste sites.

Automatically Launch Google Hacking Queries Against A Target Domain

swiss army knife for hackers

Distributed crawler powered by Headless Chrome

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Python framework to scrape Pastebin pastes and analyze them

Incredibly fast crawler designed for OSINT.

Google, Naver multiprocess image web crawler (Selenium)

POOPAK - TOR Hidden Service Crawler

Dark Web OSINT Tool

Yar is a tool for plunderin' organizations, users and/or repositories.

A scalable web crawler framework for Java.

Osintgram is a OSINT tool on Instagram. It offers an interactive shell to perform analysis on Instagram account of any users by its nickname

[Unmaintained] A simple and clean video/music/image downloader 👾

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

🕵️‍♂️ Investigate Google emails and documents.