GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → confluentinc → Schema Registry

confluentinc / Schema Registry

Licence: other
Confluent Schema Registry for Kafka

Programming Languages

java
68154 projects - #9 most used programming language

Labels

jsonrest-apikafkaprotobufschemajson-schemaavroconfluentschemasschema-registryavro-schemaprotobuf-schema

Projects that are alternatives of or similar to Schema Registry

schema-registry-php-client
A PHP 7.3+ API client for the Confluent Schema Registry REST API based on Guzzle 6 - http://docs.confluent.io/current/schema-registry/docs/index.html
Stars: ✭ 40 (-97.57%)
Mutual labels:  avro, schema-registry, confluent, avro-schema
Kafka Rest
Confluent REST Proxy for Kafka
Stars: ✭ 1,863 (+13.11%)
Mutual labels:  rest-api, kafka, confluent, schema-registry
Schema Registry
A CLI and Go client for Kafka Schema Registry
Stars: ✭ 105 (-93.62%)
Mutual labels:  schema, kafka, avro, confluent
avrora
A convenient Elixir library to work with Avro schemas and Confluent® Schema Registry
Stars: ✭ 59 (-96.42%)
Mutual labels:  avro, schema-registry, confluent, avro-schema
avrow
Avrow is a pure Rust implementation of the avro specification https://avro.apache.org/docs/current/spec.html with Serde support.
Stars: ✭ 27 (-98.36%)
Mutual labels:  schema, avro, avro-schema
srclient
Golang Client for Schema Registry
Stars: ✭ 188 (-88.59%)
Mutual labels:  schema, avro, confluent
Open Bank Mark
A bank simulation application using mainly Clojure, which can be used to end-to-end test and show some graphs.
Stars: ✭ 81 (-95.08%)
Mutual labels:  kafka, avro, confluent
avro ex
An Avro Library that emphasizes testability and ease of use.
Stars: ✭ 47 (-97.15%)
Mutual labels:  schema, avro, avro-schema
schema-registry
📙 json & avro http schema registry backed by Kafka
Stars: ✭ 23 (-98.6%)
Mutual labels:  schema, avro, schema-registry
Kaufmann ex
Kafka backed service library.
Stars: ✭ 86 (-94.78%)
Mutual labels:  schema, kafka, avro
Vue Json Ui Editor
Edit JSON in UI form with JSON Schema and Vue.js
Stars: ✭ 392 (-76.2%)
Mutual labels:  json-schema, json, schema
sbt-avro
Plugin SBT to Generate Scala classes from Apache Avro schemas hosted on a remote Confluent Schema Registry.
Stars: ✭ 15 (-99.09%)
Mutual labels:  schema, avro, schema-registry
avro-serde-php
Avro Serialisation/Deserialisation (SerDe) library for PHP 7.3+ & 8.0 with a Symfony Serializer integration
Stars: ✭ 43 (-97.39%)
Mutual labels:  avro, confluent, avro-schema
avro turf
A library that makes it easier to use the Avro serialization format from Ruby.
Stars: ✭ 130 (-92.11%)
Mutual labels:  schema, avro, schema-registry
Json Schema To Ts
Infer TS types from JSON schemas 📝
Stars: ✭ 261 (-84.15%)
Mutual labels:  json-schema, json, schema
Plank
A tool for generating immutable model objects
Stars: ✭ 449 (-72.74%)
Mutual labels:  json-schema, json, schema
Newtonsoft.json.schema
Json.NET Schema is a powerful, complete and easy to use JSON Schema framework for .NET
Stars: ✭ 167 (-89.86%)
Mutual labels:  json-schema, json, schema
Vue Form Json Schema
Create forms using JSON schema. Bring your components!
Stars: ✭ 253 (-84.64%)
Mutual labels:  json-schema, json, schema
confluent-spark-avro
Spark UDFs to deserialize Avro messages with schemas stored in Schema Registry.
Stars: ✭ 18 (-98.91%)
Mutual labels:  avro, schema-registry, confluent
Pmacct
pmacct is a small set of multi-purpose passive network monitoring tools [NetFlow IPFIX sFlow libpcap BGP BMP RPKI IGP Streaming Telemetry].
Stars: ✭ 677 (-58.89%)
Mutual labels:  json, kafka, avro
View All Similar Projects ➔

Schema Registry

Confluent Schema Registry provides a serving layer for your metadata. It provides a RESTful interface for storing and retrieving your Avro®, JSON Schema, and Protobuf schemas. It stores a versioned history of all schemas based on a specified subject name strategy, provides multiple compatibility settings and allows evolution of schemas according to the configured compatibility settings and expanded support for these schema types. It provides serializers that plug into Apache Kafka® clients that handle schema storage and retrieval for Kafka messages that are sent in any of the supported formats.

This README includes the following sections:

Documentation

Here are a few links to Schema Registry pages in the Confluent Documentation.

Quickstart API Usage examples

The following assumes you have Kafka and an instance of the Schema Registry running using the default settings. These examples, and more, are also available at API Usage examples on docs.confluent.io.

# Register a new version of a schema under the subject "Kafka-key"
$ curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" \
    --data '{"schema": "{\"type\": \"string\"}"}' \
    http://localhost:8081/subjects/Kafka-key/versions
  {"id":1}

# Register a new version of a schema under the subject "Kafka-value"
$ curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" \
    --data '{"schema": "{\"type\": \"string\"}"}' \
     http://localhost:8081/subjects/Kafka-value/versions
  {"id":1}

# List all subjects
$ curl -X GET http://localhost:8081/subjects
  ["Kafka-value","Kafka-key"]

# List all schema versions registered under the subject "Kafka-value"
$ curl -X GET http://localhost:8081/subjects/Kafka-value/versions
  [1]

# Fetch a schema by globally unique id 1
$ curl -X GET http://localhost:8081/schemas/ids/1
  {"schema":"\"string\""}

# Fetch version 1 of the schema registered under subject "Kafka-value"
$ curl -X GET http://localhost:8081/subjects/Kafka-value/versions/1
  {"subject":"Kafka-value","version":1,"id":1,"schema":"\"string\""}

# Fetch the most recently registered schema under subject "Kafka-value"
$ curl -X GET http://localhost:8081/subjects/Kafka-value/versions/latest
  {"subject":"Kafka-value","version":1,"id":1,"schema":"\"string\""}

# Delete version 3 of the schema registered under subject "Kafka-value"
$ curl -X DELETE http://localhost:8081/subjects/Kafka-value/versions/3
  3

# Delete all versions of the schema registered under subject "Kafka-value"
$ curl -X DELETE http://localhost:8081/subjects/Kafka-value
  [1, 2, 3, 4, 5]

# Check whether a schema has been registered under subject "Kafka-key"
$ curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" \
    --data '{"schema": "{\"type\": \"string\"}"}' \
    http://localhost:8081/subjects/Kafka-key
  {"subject":"Kafka-key","version":1,"id":1,"schema":"\"string\""}

# Test compatibility of a schema with the latest schema under subject "Kafka-value"
$ curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" \
    --data '{"schema": "{\"type\": \"string\"}"}' \
    http://localhost:8081/compatibility/subjects/Kafka-value/versions/latest
  {"is_compatible":true}

# Get top level config
$ curl -X GET http://localhost:8081/config
  {"compatibilityLevel":"BACKWARD"}

# Update compatibility requirements globally
$ curl -X PUT -H "Content-Type: application/vnd.schemaregistry.v1+json" \
    --data '{"compatibility": "NONE"}' \
    http://localhost:8081/config
  {"compatibility":"NONE"}

# Update compatibility requirements under the subject "Kafka-value"
$ curl -X PUT -H "Content-Type: application/vnd.schemaregistry.v1+json" \
    --data '{"compatibility": "BACKWARD"}' \
    http://localhost:8081/config/Kafka-value
  {"compatibility":"BACKWARD"}

Installation

You can download prebuilt versions of the schema registry as part of the Confluent Platform. To install from source, follow the instructions in the Development section.

Deployment

The REST interface to schema registry includes a built-in Jetty server. The wrapper scripts bin/schema-registry-start and bin/schema-registry-stop are the recommended method of starting and stopping the service.

Development

To build a development version, you may need a development versions of common and rest-utils. After installing these, you can build the Schema Registry with Maven.

This project uses the Google Java code style to keep code clean and consistent.

To build:

mvn compile

To run the unit and integration tests:

mvn test

To run an instance of Schema Registry against a local Kafka cluster (using the default configuration included with Kafka):

mvn exec:java -pl :kafka-schema-registry -Dexec.args="config/schema-registry.properties"

To create a packaged version, optionally skipping the tests:

mvn package [-DskipTests]

It produces:

  • Schema registry in package-schema-registry/target/kafka-schema-registry-package-$VERSION-package
  • Serde tools for avro/json/protobuf in package-kafka-serde-tools/target/kafka-serde-tools-package-$VERSION-package

Each of the produced contains a directory layout similar to the packaged binary versions.

You can also produce a standalone fat JAR of schema registry using the standalone profile:

mvn package -P standalone [-DskipTests]

This generates package-schema-registry/target/kafka-schema-registry-package-$VERSION-standalone.jar, which includes all the dependencies as well.

OpenAPI Spec

OpenAPI (formerly known as Swagger) specifications are built automatically using swagger-maven-plugin on compile phase.

Contribute

Thanks for helping us to make Schema Registry even better!

License

The project is licensed under the Confluent Community License, except for the client and avro-* libs, which are under the Apache 2.0 license. See LICENSE file in each subfolder for detailed license agreement.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].