Skallwar / Suckit
Licence: other
Suck the InTernet
Stars: ✭ 429
Programming Languages
rust
11053 projects
Labels
Projects that are alternatives of or similar to Suckit
Gazpacho
🥫 The simple, fast, and modern web scraping library
Stars: ✭ 525 (+22.38%)
Mutual labels: hacktoberfest, webscraping
Geeksforgeeksscrapper
Scrapes g4g and creates PDF
Stars: ✭ 124 (-71.1%)
Mutual labels: hacktoberfest, webscraping
Nytimes App
🗽 A Simple Demonstration of the New York Times App 📱 using Jsoup web crawler with MVVM Architecture 🔥
Stars: ✭ 246 (-42.66%)
Mutual labels: hacktoberfest, webscraping
Advanceddlsupport
Delegate-based C# P/Invoke alternative - compatible with all platforms and runtimes.
Stars: ✭ 419 (-2.33%)
Mutual labels: hacktoberfest
Cli Microsoft365
Manage Microsoft 365 and SharePoint Framework projects on any platform
Stars: ✭ 420 (-2.1%)
Mutual labels: hacktoberfest
Imgcat
a tool to output images as RGB ANSI graphics on the terminal
Stars: ✭ 425 (-0.93%)
Mutual labels: hacktoberfest
Mudlet
⚔️ A cross-platform, open source, and super fast MUD client with scripting in Lua
Stars: ✭ 427 (-0.47%)
Mutual labels: hacktoberfest
Marshmallow Sqlalchemy
SQLAlchemy integration with marshmallow
Stars: ✭ 426 (-0.7%)
Mutual labels: hacktoberfest
Deimosc2
DeimosC2 is a Golang command and control framework for post-exploitation.
Stars: ✭ 423 (-1.4%)
Mutual labels: hacktoberfest
Themer
themer is inspired by trevordmiller/nova and chriskempson/base16.
Stars: ✭ 4,483 (+944.99%)
Mutual labels: hacktoberfest
Digitalocean Cloud Controller Manager
Kubernetes cloud-controller-manager for DigitalOcean (beta)
Stars: ✭ 418 (-2.56%)
Mutual labels: hacktoberfest
Sync gateway
Manages access and synchronization between Couchbase Lite and Couchbase Server
Stars: ✭ 427 (-0.47%)
Mutual labels: hacktoberfest
Cortx
CORTX Community Object Storage is 100% open source object storage uniquely optimized for mass capacity storage devices.
Stars: ✭ 426 (-0.7%)
Mutual labels: hacktoberfest
Lavalink
Standalone audio sending node based on Lavaplayer.
Stars: ✭ 420 (-2.1%)
Mutual labels: hacktoberfest
Ngx Scanner
Angular (2+) QR code, Barcode, DataMatrix, scanner component using ZXing.
Stars: ✭ 420 (-2.1%)
Mutual labels: hacktoberfest
Ola
The Open Lighting Architecture - The Travel Adaptor for the Lighting Industry
Stars: ✭ 424 (-1.17%)
Mutual labels: hacktoberfest
SuckIT
SuckIT allows you to recursively visit and download a website's content to
your disk.
Features
- [x] Vacuums the entirety of a website recursively
- [x] Uses multithreading
- [x] Writes the website's content to your disk
- [x] Enables offline navigation
- [x] Offers random delays to avoid IP banning
- [ ] Saves application state on CTRL-C for later pickup
Options
| Option | Behavior |
|---|---|
-h, --help |
Displays help information |
-v, --verbose |
Activate Verbose output |
-d, --depth |
Specify the level of depth to go to when visiting the website |
-j, --jobs |
Number of threads to use |
-o, --output |
Output directory where the downloaded files are written |
-t, --tries |
Number of times to retry when the downloading of a page fails |
-u, --user-agent |
User agent to be used for sending requests |
-i, --include |
Specify a regex to include pages that match this pattern |
-e, --exclude |
Specify a regex to exclude pages that match this pattern |
-a, --auth |
Provide usernames and passwords for the downloader to use |
--dry-run |
Do everything without saving the files to the disk |
Example
A common use case could be the following:
suckit http://books.toscrape.com -j 8 -o /path/to/downloaded/pages/
Installation
As of right now, SuckIT does not work on Windows.
To install it, you need to have Rust installed.
-
Check out this link for instructions on how to install Rust.
-
If you just want to install the suckit executable, you can simply run
cargo install --git https://github.com/skallwar/suckit -
Now, run it from anywhere with the
suckitcommand.
Arch Linux
suckit can be installed from available AUR packages using an AUR helper. For example,
yay -S suckit
Want to contribute ? Feel free to open an issue or submit a PR !
License
SuckIT is primarily distributed under the terms of both the MIT license and the Apache License (Version 2.0)
See LICENSE-APACHE and LICENSE-MIT for details.
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].