All Projects → udhos → Jazigo

udhos / Jazigo

Licence: mit
Jazigo is a tool written in Go for retrieving configuration for multiple devices, similar to rancid, fetchconfig, oxidized, Sweet.

Programming Languages

go
31211 projects - #10 most used programming language

Projects that are alternatives of or similar to Jazigo

Govno
Backup your govno to S3! VNO protocol implementation in Go
Stars: ✭ 21 (-87.12%)
Mutual labels:  aws-s3, backup
s3-mongo-backup
Mongodb backups to S3
Stars: ✭ 18 (-88.96%)
Mutual labels:  backup, aws-s3
Ghbackup
Embarrassingly simple GitHub backup tool
Stars: ✭ 139 (-14.72%)
Mutual labels:  backup
Kafka Connect Storage Cloud
Kafka Connect suite of connectors for Cloud storage (Amazon S3)
Stars: ✭ 153 (-6.13%)
Mutual labels:  aws-s3
Kirby Autogit
⬢ Saves every change made via Kirby Panel to a Git repository
Stars: ✭ 147 (-9.82%)
Mutual labels:  backup
Cognito Backup Restore
AIO Tool for backing up and restoring AWS Cognito User Pools
Stars: ✭ 142 (-12.88%)
Mutual labels:  backup
Backup And Bcp For Aws
A collection of scripts & tooling that's executed from Lambda to backup your AWS Services such as Route53, EBS, RDS, EFS, etc into a S3 bucket allowing you to sync into Google Cloud for Business Continuity.
Stars: ✭ 151 (-7.36%)
Mutual labels:  backup
Shadowreader
Serverless load testing for replaying website traffic. Powered by AWS Lambda.
Stars: ✭ 138 (-15.34%)
Mutual labels:  aws-s3
Imapbackup
A Python script for incremental backups of IMAP mailboxes
Stars: ✭ 156 (-4.29%)
Mutual labels:  backup
Awsmobile Cli
CLI experience for Frontend developers in the JavaScript ecosystem.
Stars: ✭ 147 (-9.82%)
Mutual labels:  aws-s3
Aws Labs
step by step guide for aws mini labs. Currently maintained on : https://github.com/Cloud-Yeti/aws-labs Youtube playlist for labs:
Stars: ✭ 153 (-6.13%)
Mutual labels:  aws-s3
Elephant Shed
PostgreSQL Management Appliance
Stars: ✭ 146 (-10.43%)
Mutual labels:  backup
Piadvanced
This started as a custom install for my pihole!
Stars: ✭ 144 (-11.66%)
Mutual labels:  backup
0x4447 product s3 email
📫 A serverless email server on AWS using S3 and SES
Stars: ✭ 2,905 (+1682.21%)
Mutual labels:  aws-s3
Pg extractor
PG Extractor - Advanced PostgreSQL Dump Filter
Stars: ✭ 139 (-14.72%)
Mutual labels:  backup
Rsync Incremental Backup
Configurable bash script to send incremental backups of your data to a local or remote target
Stars: ✭ 150 (-7.98%)
Mutual labels:  backup
Wal G
Archival and Restoration for Postgres
Stars: ✭ 1,974 (+1111.04%)
Mutual labels:  backup
Mc
MinIO Client is a replacement for ls, cp, mkdir, diff and rsync commands for filesystems and object storage.
Stars: ✭ 1,962 (+1103.68%)
Mutual labels:  aws-s3
M Wiz
Install Metasploit And Repair Metasploit In Termux With Easy Steps
Stars: ✭ 148 (-9.2%)
Mutual labels:  backup
Are We Consistent Yet
Compare eventual consistency of object stores
Stars: ✭ 157 (-3.68%)
Mutual labels:  aws-s3

GoDoc Go Report Card Build Status gocover dev gocover store

Table of Contents

Created by gh-md-toc

About Jazigo

Jazigo is a tool written in Go for retrieving configuration for multiple devices, similar to rancid, fetchconfig, oxidized, Sweet.

Installation and usage are supposed to be dead simple. If you hit any surprising difficulty, please report.

Supported Platforms

Please send pull requests for new plataforms.

Features

  • Written in Go. Single executable file. No runtime dependency.
  • Straightforward usage: run the binary then point browser to web UI. Default settings should work out-of-the-box.
  • Tool configuration is automatically saved as YAML. However one is NOT supposed to edit configuration file directly.
  • Spawns multiple concurrent lightweight goroutines to quickly handle large number of devices.
  • Very easy to add support for new platforms. See the Cisco IOS model as example.
  • Backup files can be accessed from web UI.
  • See file differences directly from the web UI.
  • Support for SSH and TELNET.
  • Can directly store backup files into AWS S3 bucket.
  • Can call an external program and collect its output.

Requirements

Quick Start - Short version

This is how to boot up Jazigo very quickly:

With Go Modules (starting from Go 1.11)

git clone https://github.com/udhos/jazigo ;# clone outside of GOPATH
cd jazigo
go install ./jazigo
mkdir etc repo log
JAZIGO_HOME=$PWD ~/go/bin/jazigo

Without Go Modules (before Go 1.11)

go get github.com/udhos/jazigo
cd ~/go/src/github.com/udhos/jazigo
./build.sh
mkdir etc repo log
JAZIGO_HOME=$PWD ~/go/bin/jazigo

Quick Start - Detailed version

Installation and usage are supposed to be dead simple. If you hit any surprising difficulty, please report.

If you want to build from source code, start from step 1.

If you downloaded the executable binary file, start from step 5.

1. Setup GOPATH as usual

Example:

#export GOPATH=~/go ;# starting from Go 1.8, GOPATH is no longer required
mkdir ~/go

2. Get dependencies

go get github.com/icza/gowut/gwu
go get github.com/udhos/lockfile
go get github.com/udhos/equalfile
go get github.com/udhos/difflib
go get gopkg.in/yaml.v2
go get golang.org/x/crypto/ssh
go get github.com/aws/aws-sdk-go

3. Get source code

go get github.com/udhos/jazigo

4. Compile and install

go install github.com/udhos/jazigo/jazigo

5. Decide where to store config, backup, log and static www files

Example:

export JAZIGO_HOME=$HOME/jazigo
mkdir $JAZIGO_HOME
cd $JAZIGO_HOME
mkdir etc repo log www

Hint: By default, Jazigo looks for directories 'etc', 'repo', 'log', and 'www' under $JAZIGO_HOME. If left undefined, JAZIGO_HOME defaults to /var/jazigo. See command line options to fine tune filesystem locations.

6. Copy static files (CSS and images) to $JAZIGO_HOME/www

Example:

# If you have downloaded jazigo using 'go get':
cp ~/go/src/github.com/udhos/jazigo/www/* $JAZIGO_HOME/www

# Otherwise get static files from https://github.com/udhos/jazigo/tree/master/www
cd $JAZIGO_HOME/www
wget https://raw.githubusercontent.com/udhos/jazigo/master/www/fail-small.png
wget https://raw.githubusercontent.com/udhos/jazigo/master/www/ok-small.png
wget https://raw.githubusercontent.com/udhos/jazigo/master/www/jazigo.css
wget https://raw.githubusercontent.com/udhos/jazigo/master/www/GitHub-Mark-32px.png

7. Run jazigo once (see -runOnce option)

$GOPATH/bin/jazigo -runOnce

Watch messages logged to standard output for errors.

Hint: Since root privileges are usually not needed, run Jazigo as a regular user.

8. Run jazigo forever

$GOPATH/bin/jazigo -disableStdoutLog

9. Open the web interface

Point web browser at: http://localhost:8080/jazigo

Global Settings

You might want to adjust global settings. See the Jazigo admin window under http://localhost:8080/jazigo/admin.

maxconfigfiles: 120
holdtime: 12h0m0s
scaninterval: 10m0s
maxconcurrency: 20
maxconfigloadsize: 10000000

maxconfigfiles: This option limits the amount of files stored per device. When this limit is reached, older files are discarded.

holdtime: When a successful backup is saved for a device, the software will only contact that specific device again after expiration of the 'holdtime' timer.

scaninterval: The interval between two device table scans. If the device table is fully processed before the 'scaninterval' timer, the software will wait idly for the next scan cycle. If the full table scan takes longer than 'scaninterval', the next cycle will start immediately.

maxconcurrency: This option limits the number of concurrent backup jobs. You should raise this value if you need faster scanning of all devices. Keep in mind that if your devices use a centralized authentication system (for example, Cisco Secure ACS), the authentication server might become a bottleneck for high concurrency.

maxconfigloadsize: This limit puts restriction into the amount of data the tool loads from a file to memory. Intent is to protect the servers' memory from exhaustion while trying to handle multiple very large configuration files.

Importing Many Devices

You can use the Web UI to add devices, but it is not designed for importing a large number of devices.

The easiest way to include many devices is by using the command line option -deviceImport.

1. Build a device table using this format:

$ cat table.txt
#
# model   id   hostport      transports username password enable-password
#
cisco-ios lab1 router1905lab telnet     san      fran     sanjose
cisco-ios lab2 router3925lab telnet     san      fran     sanjose
junos     auto ex4200lab     ssh,telnet backup   juniper1 not-used
junos     auto 1.1.1.1:2222  ssh        backup   juniper1 not-used
$

Hint: The device id must be unique. You can generate a meaningful device id manually as you like. You can also let Jazigo create id's automatically by specifying the special id auto.

2. Then load the table with the option -deviceImport:

$ $GOPATH/bin/jazigo -deviceImport < table.txt

Using AWS S3

Quick recipe for using S3 bucket:

1. Create a bucket 'bucketname' on AWS region 'regionname'.

2. Authorize the client to access the bucket

An usual way is to create an IAM user, add key/secret, and put those credentials into ~/.aws/credentials:

$ cat ~/.aws/credentials
[default]
aws_access_key_id = key
aws_secret_access_key = secret

3. Run jazigo pointing its config and repository paths to S3 bucket ARN:

S3 bucket ARN: arn:aws:s3:regionname::bucketname/foldername

# Example
ARN=arn:aws:s3:regionname::bucketname/foldername
$GOPATH/bin/jazigo -configPathPrefix=$ARN/etc/jazigo.conf. -repositoryPath=$ARN/repo

Hint: You could point config and repository to distinct buckets.

Calling an external program

You can use the pseudo model run to call an external program to collect custom configuration.

Create a device using the model run, then specify the program arguments in the attribute runprog:

Example:

# This example calls: /bin/bash -c "env | egrep ^JAZIGO_"
runprog:
- /bin/bash
- -c
- env | egrep ^JAZIGO_

The external program invoked by the model run will receive its device authentication credentials as environment variables:

JAZIGO_DEV_ID=deviceid
JAZIGO_DEV_HOSTPORT=host[:port] -- port is optional
JAZIGO_DEV_USER=username
JAZIGO_DEV_PASS=password

The external program is expected to issue captured configuration to stdout and then to exit with zero exit status.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].