All Projects → lensesio → lenses-go

lensesio / lenses-go

Licence: Apache-2.0 License
Lenses.io CLI (command-line interface)

Programming Languages

go
31211 projects - #10 most used programming language

Projects that are alternatives of or similar to lenses-go

lentes
Functional references for Clojure and ClojureScript
Stars: ✭ 80 (+135.29%)
Mutual labels:  lenses
pathex
Fastest way to access data in Elixir
Stars: ✭ 242 (+611.76%)
Mutual labels:  lenses
lenses-jdbc
Lenses.io JDBC driver for Apache Kafka
Stars: ✭ 20 (-41.18%)
Mutual labels:  lenses
guardian
Guardian is a tool for extensible and universal data access with automated access workflows and security controls across data stores, analytical systems, and cloud products.
Stars: ✭ 127 (+273.53%)
Mutual labels:  dataops
exercises-monocle
Scala Exercises for the Scala Optics library Monocle http://julien-truffaut.github.io/Monocle/
Stars: ✭ 12 (-64.71%)
Mutual labels:  lenses
dataops-platform-airflow-dbt
Build DataOps platform with Apache Airflow and dbt on AWS
Stars: ✭ 33 (-2.94%)
Mutual labels:  dataops
raccoon
Raccoon is a high-throughput, low-latency service to collect events in real-time from your web, mobile apps, and services using multiple network protocols.
Stars: ✭ 159 (+367.65%)
Mutual labels:  dataops
rivery cli
Rivery CLI
Stars: ✭ 16 (-52.94%)
Mutual labels:  dataops
program-imperatively-using-purescript
Program imperatively using PureScript, State monad and lenses
Stars: ✭ 23 (-32.35%)
Mutual labels:  lenses
shield
Shield is a role-based cloud-native user management system, identity & access proxy, and authorization server for your applications and API endpoints.
Stars: ✭ 158 (+364.71%)
Mutual labels:  dataops
cli
Polyaxon Core Client & CLI to streamline MLOps
Stars: ✭ 18 (-47.06%)
Mutual labels:  dataops
siren
Siren provides an easy-to-use universal alert, notification, channels management framework for the entire observability infrastructure.
Stars: ✭ 70 (+105.88%)
Mutual labels:  dataops
concave
🧐 Lens-like state management (for React).
Stars: ✭ 13 (-61.76%)
Mutual labels:  lenses
lenticular.ts
(Yet another) implementation of functional lenses in JavaScript/TypeScript.
Stars: ✭ 29 (-14.71%)
Mutual labels:  lenses
vulkn
Love your Data. Love the Environment. Love VULKИ.
Stars: ✭ 43 (+26.47%)
Mutual labels:  dataops
dagger
Dagger is an easy-to-use, configuration over code, cloud-native framework built on top of Apache Flink for stateful processing of real-time streaming data.
Stars: ✭ 238 (+600%)
Mutual labels:  dataops
putting-lenses-to-work
A presentation for BayHac 2017 on how I uses lenses at work
Stars: ✭ 73 (+114.71%)
Mutual labels:  lenses
optics.js
🔭 Lenses, Prisms and Traversals in JavaScript!
Stars: ✭ 46 (+35.29%)
Mutual labels:  lenses
datatile
A library for managing, validating, summarizing, and visualizing data.
Stars: ✭ 419 (+1132.35%)
Mutual labels:  dataops
versatile-data-kit
Versatile Data Kit (VDK) is an open source framework that enables anybody with basic SQL or Python knowledge to create their own data pipelines.
Stars: ✭ 144 (+323.53%)
Mutual labels:  dataops

Lenses Client (Go)

The Lenses REST API client written in Go.

Build Status report card chat

Installation

The only requirement is the Go Programming Language version 1.13+ and a Lenses Box of version 2.0 at least.

# If you have Go < 1.13 you may need to set GO111MODULE=on
$ go get -u github.com/lensesio/lenses-go/cmd/lenses-cli

This command will install both the client library for development usage and the CLI in $PATH (setup your $GOPATH/bin if you didn't already).

CLI

Lenses offers a powerful CLI (command-line tool) built in Go that utilizes the REST and WebSocket APIs of Lenses, to communicate with Apache Kafka and exposes a straight forward way to perform common data engineering and site reliability engineering tasks, such as:

  • Automate your CI/CD (continuous-integration and continuous-delivery)
  • Create topics/acls/quotas/schemas/connectors/processors
  • Change or retrieve configurations to store in github

Documentation

Please navigate to https://docs.lenses.io/dev/lenses-cli/ to learn how to install and use the lenses-cli.

Development

Build

lenses-go use go modules as dependency management system. For daily development workflow you need to use Makefile with certain actions.

Builds a binary based on your current OS system

make build

Builds binaries for all OS systems

make cross-build

Lint

We use the Golang golint for linting using:

make lint

Tests

If you want to run the tests, use the following:

make test

Clean

Clean all binaries and coverage files:

make clean

Client

Getting started

import "github.com/lensesio/lenses-go"

Authentication

// Prepare authentication using raw Username and Password.
//
// Use it when Lenses setup with "BASIC" or "LDAP" authentication.
auth := lenses.BasicAuthentication{Username: "user", Password: "pass"}
auth := lenses.KerberosAuthentication{
    ConfFile: "/etc/krb5.conf",
    Method:   lenses.KerberosWithPassword{
        Realm: "my.realm or default if empty",
        Username: "user",
        Password: "pass",
    },
}
auth := lenses.KerberosAuthentication{
    ConfFile: "/etc/krb5.conf",
    Method:   lenses.KerberosWithKeytab{KeytabFile: "/home/me/krb5_my_keytab.txt"},
}
auth := lenses.KerberosAuthentication{
    ConfFile: "/etc/krb5.conf",
    Method:   lenses.KerberosFromCCache{CCacheFile: "/tmp/krb5_my_cache_file.conf"},
}

Custom auth can be implement as well: Authenticate(client *lenses.Client) error, see client_authentication.go file for more.

Config

// Prepare the client's configuration based on the host and the authentication above.
currentConfig := lenses.ClientConfig{Host: "domain.com", Authentication: auth, Timeout: "15s", Debug: true}

// Creating the client using the configuration.
client, err := lenses.OpenConnection(currentConfig)
if err != nil {
    // handle error.
}

Read Config from any io.Reader or file

// ReadConfig reads and decodes Config from an io.Reader based on a custom unmarshaler.
// This can be useful to read configuration via network or files (see `ReadConfigFromFile`).
// Sets the `outPtr`. Retruns a non-nil error on any unmarshaler's errors.
ReadConfig(r io.Reader, unmarshaler UnmarshalFunc, outPtr *Config) error

// ReadConfigFromFile reads and decodes Config from a file based on a custom unmarshaler,
// `ReadConfigFromJSON` and `ReadConfigFromYAML` are the internal users,
// but the end-developer can use any custom type of decoder to read a configuration file
// with ease using this function, but keep note that the default behavior of the fields
// depend on the existing unmarshalers, use these tag names to map your decoder's properties.
//
// Accepts the absolute or the relative path of the configuration file.
// Sets the `outPtr`. Retruns a non-nil error if parsing or decoding the file failed or file doesn't exist.
ReadConfigFromFile(filename string, unmarshaler UnmarshalFunc, outPtr *Config) error

// TryReadConfigFromFile will try to read a specific file and unmarshal to `Config`.
// It will try to read it with one of these built'n formats:
// 1. JSON
// 2. YAML
TryReadConfigFromFile(filename string, outPtr *Config) error
// TryReadConfigFromHome will try to read the `Config`
// from the current user's home directory/.lenses, the lookup is based on
// the common configuration filename pattern:
// lenses-cli.json, lenses-cli.yml or lenses.json and lenses.yml.
TryReadConfigFromHome(outPtr *Config) bool

// TryReadConfigFromExecutable will try to read the `Config`
// from the (client's caller's) executable path that started the current process.
// The lookup is based on the common configuration filename pattern:
// lenses-cli.json, lenses-cli.yml or lenses.json and lenses.yml.
TryReadConfigFromExecutable(outPtr *Config) bool

// TryReadConfigFromCurrentWorkingDir will try to read the `Config`
// from the current working directory, note that it may differs from the executable path.
// The lookup is based on the common configuration filename pattern:
// lenses-cli.json, lenses-cli.yml or lenses.json and lenses.yml.
TryReadConfigFromCurrentWorkingDir(outPtr *Config) bool

// ReadConfigFromJSON reads and decodes Config from a json file, i.e `configuration.json`.
//
// Accepts the absolute or the relative path of the configuration file.
// Error may occur when the file doesn't exists or is not formatted correctly.
ReadConfigFromJSON(filename string, outPtr *Config) error

// ReadConfigFromYAML reads and decodes Config from a yaml file, i.e `configuration.yml`.
//
// Accepts the absolute or the relative path of the configuration file.
// Error may occur when the file doesn't exists or is not formatted correctly.
ReadConfigFromYAML(filename string, outPtr *Config) error

Example Code:

# file: ./lenses.yml
CurrentContext: main
Contexts:
  main:
    Host: https://<your-lenses-host-url>
    Kerberos:
      ConfFile: /etc/krb5.conf
      WithPassword:
        Username: the_username
        Password: the_password
        Realm: empty_for_default

Usage:

var config lenses.Config
err := lenses.ReadConfigFromYAML("./lenses.yml", &config)
if err != nil {
    // handle error.
}

client, err := lenses.OpenConnection(*config.GetCurrent())

Config contains tons of capabilities and helpers, you can quickly check them by navigating to the config.go source file.

API Calls

All lenses-go#Client methods return a typed value based on the call and an error as second output to catch any errors coming from backend or client, forget panics.

Go types are first class citizens here, we will not confuse you or let you work based on luck!

topics, err := client.GetTopics()
if err != nil {
    // handle error.
}

// Print the length of the topics we've just received from our Lenses Box.
print(len(topics))

Example on how deeply we make the difference here: Client#GetTopics returns []lenses.Topic, so you can work safely.

topics[0].ConsumersGroup[0].Coordinator.Host

Documentation

Detailed documentation can be found at godocs.

Versioning

License

Distributed under Apache Version 2.0 License, click here for more details.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].