All Projects → oslabs-beta → kafkaSaur

oslabs-beta / kafkaSaur

Licence: MIT license
Apache Kafka client for Deno

Programming Languages

typescript
32286 projects
java
68154 projects - #9 most used programming language

Projects that are alternatives of or similar to kafkaSaur

dataStructure
Implement different Data Structures using TypeScript and JavaScript. Deno Third-party Module.
Stars: ✭ 24 (-42.86%)
Mutual labels:  deno, denoland
make-deno-edition
Automatically makes package.json projects (such as npm packages and node.js modules) compatible with Deno.
Stars: ✭ 39 (-7.14%)
Mutual labels:  deno, denoland
logrocket deno api
A functional CRUD-like API with Deno and Postgres
Stars: ✭ 23 (-45.24%)
Mutual labels:  deno, denoland
Fae
A functional module for Deno inspired from Ramda.
Stars: ✭ 44 (+4.76%)
Mutual labels:  deno, denoland
Karafka
Framework for Apache Kafka based Ruby and Rails applications development.
Stars: ✭ 1,223 (+2811.9%)
Mutual labels:  apache-kafka, kafka-client
deno-mongo-api
Example for building REST APIS with deno and MongoDB
Stars: ✭ 17 (-59.52%)
Mutual labels:  deno, denoland
deno-x-ranking
🦕 Deno Third Party Modules Ranking 👑
Stars: ✭ 28 (-33.33%)
Mutual labels:  deno, denoland
ssvm-deno-starter
A template project to run Rust functions in Deno through the Second State WebAssembly engine.
Stars: ✭ 50 (+19.05%)
Mutual labels:  deno, denoland
Kq
Kafka-based Job Queue for Python
Stars: ✭ 530 (+1161.9%)
Mutual labels:  apache-kafka, kafka-client
kafka-net-core
kafka .net core library
Stars: ✭ 23 (-45.24%)
Mutual labels:  apache-kafka, kafka-client
cnpj
🇧🇷 Format, validate and generate CNPJ numbers in Node & Deno
Stars: ✭ 26 (-38.1%)
Mutual labels:  deno, denoland
denoliver
A simple, dependency free static file server for Deno with possibly the worst name ever.
Stars: ✭ 94 (+123.81%)
Mutual labels:  deno, denoland
Apachekafkatutorials
Example Code for Kafka Tutorials @ Learning Journal
Stars: ✭ 155 (+269.05%)
Mutual labels:  apache-kafka, kafka-client
Kafka Ui
Open-Source Web GUI for Apache Kafka Management
Stars: ✭ 230 (+447.62%)
Mutual labels:  apache-kafka, kafka-client
i18next-fs-backend
i18next-fs-backend is a backend layer for i18next using in Node.js and for Deno to load translations from the filesystem.
Stars: ✭ 67 (+59.52%)
Mutual labels:  deno
media types
Deprecated. Use std/media_types instead.
Stars: ✭ 21 (-50%)
Mutual labels:  deno
astrodon
Make Desktop apps with Deno 🦕
Stars: ✭ 826 (+1866.67%)
Mutual labels:  deno
zeno.zsh
zsh fuzzy completion and utility plugin with Deno.
Stars: ✭ 119 (+183.33%)
Mutual labels:  deno
oembed-parser
Extract oEmbed data from given webpage
Stars: ✭ 65 (+54.76%)
Mutual labels:  deno
core
Server side rendering with The Elm Architecture in Deno
Stars: ✭ 16 (-61.9%)
Mutual labels:  deno

Logo

Logo

GitHub stars GitHub issues GitHub last commit

A native Deno client for Apache Kafka

Table of Contents

About the Project

KafkaSaur an Apache Kafka client for deno

Getting Started

https://deno.land/x/[email protected]

Prerequisites

Deno - https://deno.land/manual/getting_started/installation

Apache Kafka - https://kafka.apache.org/

Docker (for examples) -https://www.docker.com/

Usage

To run examples initiate Docker containers included in yaml file:

docker-compose up

Then run example producer/consumer files, in seperate terminals, with the following commands:

deno run --allow-all --unstable examples/example_producer.ts
deno run --allow-all --unstable examples/example_consumer.ts

Your two terminals (one Consuming, and one Producing) will now interact with the Broker and begin consuming and producing respectively.

With the Client imported into your application, you can write the producer/consumer logic like this:

//producer example
import {Kafkasaur} from "https://deno.land/x/kafkasaur/index.ts"

const kafka = new Kafkasaur({
  clientId: 'example-producer',
  brokers: ['localhost:9092']
})

const topic = 'topic-test';

const producer = kafka.producer();

const testmessage = {
  key: 'key',
  value: 'hello there',
  headers: {'correlation-id': `${Date.now()}`}
}

const messages: object[] = [];
messages.push(testmessage)

const sendMessage = () => {
  producer.send({
    topic,
    messages
  })
}

const run = async() => {
  await producer.connect();
  sendMessage();
}

run()
//consumer example
import {Kafkasaur} from "https://deno.land/x/kafkasaur/index.ts"

const kafka = new Kafkasaur({
  clientId: 'example-consumer',
  brokers: ['localhost:9092']
})

const topic = 'topic-test';

const consumer = kafka.consumer({ groupId: 'test-group' })

const run = async () => {
  await consumer.connect()
  await consumer.subscribe({ topic, fromBeginning: true })
  
  await consumer.run({
    eachMessage: async (message: any) => {
      console.log(message.value.toString())
    },
  })
}

run()

To run the instances of your Consumer/Producer be sure to pass the flags

--allow-all

and

--unstable

when issuing your deno run command. This ensures that Deno as the proper configuration to communicate with the Broker, and to log any errors.

Features

  • 🛠 Built with [TypeScript][Deno]

  • 🎬 Producer

  • 🍴 Consumer

  • 🤝 interactive producer with consumer

  • 💂 deno's built in security; No file, network, or environment access, unless explicitly enabled

Want to Contribute?

If you'd like to contribute and help grow the Deno community, just reach out to one of us via LinkedIn or write some code, and make a PR here! We're super excited about getting the conversation started, and working to bring Kafka to Deno!

Developers


Sam Arnold

Wesley Appleget

Adam Blackwell

Stephanie Benitez

Acknowledgements

  • Tommy Brunn - for his guidance and for trailblazing with KafkaJS
  • Ryan Dahl - for building an awesome community with Node.js and then leveling it up even further with Deno
  • Franz Kafka - for making us all remember, we could just be cockroaches.

License

This product is licensed under the MIT License - see the LICENSE.md file for details.

This is an open source product.

This product is accelerated by OS Labs.

Apache Kafka and Kafka are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries. KafkaJS has no affiliation with the Apache Software Foundation.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].