leboncoin / Avrocado
Licence: mit
Avrocado is a convenience library to handle Avro in Golang
Stars: ✭ 21
Programming Languages
go
31211 projects - #10 most used programming language
Labels
Projects that are alternatives of or similar to Avrocado
Ratatool
A tool for data sampling, data generation, and data diffing
Stars: ✭ 279 (+1228.57%)
Mutual labels: avro
Avro4s
Avro schema generation and serialization / deserialization for Scala
Stars: ✭ 593 (+2723.81%)
Mutual labels: avro
avro-schema-generator
Library for generating avro schema files (.avsc) based on DB tables structure
Stars: ✭ 38 (+80.95%)
Mutual labels: avro
schema-registry-plugin
Gradle plugin to interact with Confluent Schema-Registry.
Stars: ✭ 60 (+185.71%)
Mutual labels: avro
Choetl
ETL Framework for .NET / c# (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)
Stars: ✭ 372 (+1671.43%)
Mutual labels: avro
kafka-compose
🎼 Docker compose files for various kafka stacks
Stars: ✭ 32 (+52.38%)
Mutual labels: avro
Cpp Serializers
Benchmark comparing various data serialization libraries (thrift, protobuf etc.) for C++
Stars: ✭ 533 (+2438.1%)
Mutual labels: avro
confluent-spark-avro
Spark UDFs to deserialize Avro messages with schemas stored in Schema Registry.
Stars: ✭ 18 (-14.29%)
Mutual labels: avro
qwery
A SQL-like language for performing ETL transformations.
Stars: ✭ 28 (+33.33%)
Mutual labels: avro
Iceberg
Iceberg is a table format for large, slow-moving tabular data
Stars: ✭ 393 (+1771.43%)
Mutual labels: avro
spring-cloud-stream-event-sourcing-testcontainers
Goal: create a Spring Boot application that handles users using Event Sourcing. So, whenever a user is created, updated, or deleted, an event informing this change is sent to Kafka. Also, we will implement another application that listens to those events and saves them in Cassandra. Finally, we will use Testcontainers for integration testing.
Stars: ✭ 16 (-23.81%)
Mutual labels: avro
Pmacct
pmacct is a small set of multi-purpose passive network monitoring tools [NetFlow IPFIX sFlow libpcap BGP BMP RPKI IGP Streaming Telemetry].
Stars: ✭ 677 (+3123.81%)
Mutual labels: avro
Kafka Storm Starter
Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.
Stars: ✭ 728 (+3366.67%)
Mutual labels: avro
Devops Python Tools
80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.
Stars: ✭ 406 (+1833.33%)
Mutual labels: avro
Avrocado
Avrocado is a convenience library to handle Avro in golang, built on top of linkedin/goavro. It is split into three parts:
- Avro marshalling/unmarshalling using structure fields annotations inspired by the JSON standard library.
- A confluentinc/schema-registry client.
- A codec registry which handles marshalling/unmarshalling schemas from the schema-registry.
Getting Started
You can start using the library after installing by importing it in your go code. You need to annotate the types you want to marshal with the avro tag. Finally you will have to instantiate a codec with the corresponding Avro schema:
import "github.com/leboncoin/avrocado"
import (
"fmt"
)
type Someone struct {
Name string `avro:"name"`
Age int32 `avro:"age"`
}
func ExampleCodec() {
val := Someone{"MyName", 3}
var decoded Someone
schema := `{
"type": "record",
"name": "Someone",
"fields": [
{
"name": "name",
"type": "string"
}, {
"name": "age",
"type": "int"
}
]
}`
codec, err := NewCodec(schema)
if err != nil {
panic(fmt.Sprintf("wrong schema: %s", err))
}
avro, err := codec.Marshal(&val)
if err != nil {
panic(fmt.Sprintf("unable to serialize to avro: %s", err))
}
err = codec.Unmarshal(avro, &decoded)
if err != nil {
panic(fmt.Sprintf("unable to deserialize from avro: %s", err))
}
}
The example can also be found here.
Installing
Just run go get github.com/leboncoin/avrocado
.
Examples
See the test files for examples on how to use the library.
Running tests
Just run go test
at the root directory of this repository.
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].