All Projects → leboncoin → Avrocado

leboncoin / Avrocado

Licence: mit
Avrocado is a convenience library to handle Avro in Golang

Programming Languages

go
31211 projects - #10 most used programming language

Labels

Projects that are alternatives of or similar to Avrocado

AvroConvert
Apache Avro serializer for .NET
Stars: ✭ 44 (+109.52%)
Mutual labels:  avro
Ratatool
A tool for data sampling, data generation, and data diffing
Stars: ✭ 279 (+1228.57%)
Mutual labels:  avro
Avro4s
Avro schema generation and serialization / deserialization for Scala
Stars: ✭ 593 (+2723.81%)
Mutual labels:  avro
avro-schema-generator
Library for generating avro schema files (.avsc) based on DB tables structure
Stars: ✭ 38 (+80.95%)
Mutual labels:  avro
schema-registry-plugin
Gradle plugin to interact with Confluent Schema-Registry.
Stars: ✭ 60 (+185.71%)
Mutual labels:  avro
Choetl
ETL Framework for .NET / c# (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)
Stars: ✭ 372 (+1671.43%)
Mutual labels:  avro
kafka-compose
🎼 Docker compose files for various kafka stacks
Stars: ✭ 32 (+52.38%)
Mutual labels:  avro
Aptos
☀️ Avro, Protobuf, Thrift on Swagger
Stars: ✭ 17 (-19.05%)
Mutual labels:  avro
Divolte Collector
Divolte Collector
Stars: ✭ 264 (+1157.14%)
Mutual labels:  avro
Cpp Serializers
Benchmark comparing various data serialization libraries (thrift, protobuf etc.) for C++
Stars: ✭ 533 (+2438.1%)
Mutual labels:  avro
confluent-spark-avro
Spark UDFs to deserialize Avro messages with schemas stored in Schema Registry.
Stars: ✭ 18 (-14.29%)
Mutual labels:  avro
qwery
A SQL-like language for performing ETL transformations.
Stars: ✭ 28 (+33.33%)
Mutual labels:  avro
Iceberg
Iceberg is a table format for large, slow-moving tabular data
Stars: ✭ 393 (+1771.43%)
Mutual labels:  avro
spring-cloud-stream-event-sourcing-testcontainers
Goal: create a Spring Boot application that handles users using Event Sourcing. So, whenever a user is created, updated, or deleted, an event informing this change is sent to Kafka. Also, we will implement another application that listens to those events and saves them in Cassandra. Finally, we will use Testcontainers for integration testing.
Stars: ✭ 16 (-23.81%)
Mutual labels:  avro
Pmacct
pmacct is a small set of multi-purpose passive network monitoring tools [NetFlow IPFIX sFlow libpcap BGP BMP RPKI IGP Streaming Telemetry].
Stars: ✭ 677 (+3123.81%)
Mutual labels:  avro
dotnet-avro
An Avro implementation for .NET
Stars: ✭ 60 (+185.71%)
Mutual labels:  avro
Schema Registry Ui
Web tool for Avro Schema Registry |
Stars: ✭ 358 (+1604.76%)
Mutual labels:  avro
Avsc
Avro for JavaScript ⚡️
Stars: ✭ 930 (+4328.57%)
Mutual labels:  avro
Kafka Storm Starter
Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.
Stars: ✭ 728 (+3366.67%)
Mutual labels:  avro
Devops Python Tools
80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.
Stars: ✭ 406 (+1833.33%)
Mutual labels:  avro

Build Status

Avrocado

Avrocado is a convenience library to handle Avro in golang, built on top of linkedin/goavro. It is split into three parts:

  • Avro marshalling/unmarshalling using structure fields annotations inspired by the JSON standard library.
  • A confluentinc/schema-registry client.
  • A codec registry which handles marshalling/unmarshalling schemas from the schema-registry.

Getting Started

You can start using the library after installing by importing it in your go code. You need to annotate the types you want to marshal with the avro tag. Finally you will have to instantiate a codec with the corresponding Avro schema:

import "github.com/leboncoin/avrocado"

import (
        "fmt"
)

type Someone struct {
        Name string `avro:"name"`
        Age  int32  `avro:"age"`
}

func ExampleCodec() {
        val := Someone{"MyName", 3}
        var decoded Someone

        schema := `{
          "type": "record",
          "name": "Someone",
          "fields": [
            {
              "name": "name",
              "type": "string"
            }, {
              "name": "age",
              "type": "int"
            }
          ]
        }`

        codec, err := NewCodec(schema)
        if err != nil {
                panic(fmt.Sprintf("wrong schema: %s", err))
        }

        avro, err := codec.Marshal(&val)
        if err != nil {
                panic(fmt.Sprintf("unable to serialize to avro: %s", err))
        }

        err = codec.Unmarshal(avro, &decoded)
        if err != nil {
                panic(fmt.Sprintf("unable to deserialize from avro: %s", err))
        }
}

The example can also be found here.

Installing

Just run go get github.com/leboncoin/avrocado.

Examples

See the test files for examples on how to use the library.

Running tests

Just run go test at the root directory of this repository.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].