Logrange - streaming database

Logrange is highly performant streaming database for aggregating streams of records like application logs, system metrics, audit logs etc. from thousands of sources. Logrange provides an API to do operations from simple search to the data analysis and machine learning.

The product

• Size tolerant. Logrange is able to store as much data as written there. The only limitation is the storage space available for the database.
• Highly performant. Writing and reading millions of records per second (hundered megabytes or gigabytes of data).
• Write optimized. Logrange persists the raw data in realtime. All other data processing like indexing can be done later.
• Low latency. Data becomes available for read within milliseconds after it is written
• Scalable. Supporting tens of thousands different streams of records (terrabytes of the data)
• Highly available. In clustering solution, data could be replicated between logrange nodes. Logrange will support data and load distribution policies.
• Native for stream processing. Merging, filtering and search using LQL (Logrange Query Language)
• Open Source. Logrange is 100% open source. It can be used for building trustworthy data storages.
• Ready to use. Basic installation includes pre-configured log processing tools: collector, forwarder, CLI tool and Logrange database service.
• Easy installation either in a containerized (k8s or docker) or a custom environment.

Quick start

Logrange shipment includes 2 executables - logrange server and lr - the logrange client. In the quick start you can use precompiled binaries to try logrange out within 1 minute:

Step 1. Let's put everything into one dir

Make a directory and enter there:

mkdir lrquick
cd lrquick

Step 2. Install logrange server and run it

curl -s http://get.logrange.io/install | bash -s logrange -d ./bin
./bin/logrange start --base-dir=./data --daemon

Normally, you have to see something like Started. pid=12345

Step 3. Install logrange client and start collecting logs from the machine

curl -s http://get.logrange.io/install | bash -s lr -d ./bin
./bin/lr collect --storage-dir=./collector --daemon

The command above runs collector in background. It will send logs found in /var/log folder to the logrange server started in step 2.

Step 4. Connect to the server, using CLI tool.

./bin/lr shell
...

In the logrange shell, you can try select to retrieve collected data:

> select limit 10

Or try help command to find out what commands are available.

Quick stop

From the logrange folder (lrquick) type the following commands to stop collector and the logrange server:

./bin/lr stop-collect --storage-dir=./collector
./bin/logrange stop --base-dir=./data

Now, to clean up, just remove the lrquick folder:

cd ..
rm -rf ./lrquick/

Documentation

k8s Installation
The product

Getting Help

• Found a bug or thinking about a new feature? File an issue
• If you have any question or feedback regarding Logrange, send us e-mail to: [email protected]

Contact us

Whether you have problems with log aggregating, processing or analysis, or wanting to build a secure data aggregating solution.

Reach out to [email protected]

Join our gitter chat

License

Apache License 2.0, see LICENSE.

Acknowledgments

• GoLand IDE by JetBrains is used for the code development