leonlee / Flume Ng Mongodb Sink
Licence: apache-2.0
Flume NG MongoDB source.
Stars: ✭ 69
Programming Languages
java
68154 projects - #9 most used programming language
Labels
Projects that are alternatives of or similar to Flume Ng Mongodb Sink
Spring Boot Extend
在springboot基础上的扩展项目,快速集成Zookeeper、Dubbo、Apollo、Mybatis多数据源
Stars: ✭ 63 (-8.7%)
Mutual labels: mongodb
Laravel Mongodb Passport
A package to get Laravel Passport working with MongoDB
Stars: ✭ 64 (-7.25%)
Mutual labels: mongodb
Mall Learning
mall学习教程,架构、业务、技术要点全方位解析。mall项目(40k+star)是一套电商系统,使用现阶段主流技术实现。涵盖了SpringBoot 2.3.0、MyBatis 3.4.6、Elasticsearch 7.6.2、RabbitMQ 3.7.15、Redis 5.0、MongoDB 4.2.5、Mysql5.7等技术,采用Docker容器化部署。
Stars: ✭ 10,236 (+14734.78%)
Mutual labels: mongodb
Mongoose Geojson Schema
Schema definitions for GeoJSON types for use with Mongoose JS
Stars: ✭ 66 (-4.35%)
Mutual labels: mongodb
Habitica
A habit tracker app which treats your goals like a Role Playing Game.
Stars: ✭ 8,702 (+12511.59%)
Mutual labels: mongodb
Incense
Interactively retrieve data from sacred experiments.
Stars: ✭ 68 (-1.45%)
Mutual labels: mongodb
Flask Restful Authentication
An example for RESTful authentication using nginx, uWSGI, Flask, MongoDB and JSON Web Token(JWT).
Stars: ✭ 63 (-8.7%)
Mutual labels: mongodb
Mongoid auto increment id
Override id field to MySQL like auto increment for Mongoid.
Stars: ✭ 68 (-1.45%)
Mutual labels: mongodb
Todo List Using Flask And Mongodb
Simple implementation of ToDo List using Flask and MongoDB along with Dockerfile and Kubernetes yaml files #Flask #Example #Docker #Kubernetes #k8s
Stars: ✭ 61 (-11.59%)
Mutual labels: mongodb
Shw server
使用SpringCloud和Spring WebFlux开发的学生作业管理系统服务端,前后端分离项目,微服务架构。支持Docker集群化部署,Redis集群化缓存,文件在线预览 压缩包预览 打包上传下载
Stars: ✭ 66 (-4.35%)
Mutual labels: mongodb
Mean Stack Angular6 Crud Example
MEAN Stack Angular 6 CRUD Web Application
Stars: ✭ 69 (+0%)
Mutual labels: mongodb
Event Sourcing Castanha
An Event Sourcing service template with DDD, TDD and SOLID. It has High Cohesion and Loose Coupling, it's a good start for your next Microservice application.
Stars: ✭ 68 (-1.45%)
Mutual labels: mongodb
flume-ng-mongodb-sink
Flume NG MongoDB sink. The source was implemented to populate JSON into MongoDB.
Getting Started
- Clone the repository
- Install latest Maven and build source by 'mvn package'
- Generate classpath by 'mvn dependency:build-classpath'
- Append classpath in $FLUME_HOME/conf/flume-env.sh
- Add the sink definition according to Configuration
Configuration
type: org.riderzen.flume.sink.MongoSink
host: db host [localhost]
port: db port [27017]
username: db username []
password: db password []
model: SINGLE or DYNAMIC [SINGLE], SINGLE means all data will insert into the same collection,
and DYNAMIC means every event will specify collection name by event header 'collection'
db: db name [events]
collection: default collection name, will used in SINGLE model [events]
batch: batch size of insert opertion [100]
autoWrap: indicator of wrap the event body as a JSONObject that has one field [false]
wrapField: using with autoWrap, set the field name of JSONObject [log]
timestampField: date type field that record the creating time of record,
it can be a existing filed name that the sink will convert this filed to date type,
or it's a new filed name that the sink will create it automatically []
the supported date pattern as follows:
"yyyy-MM-dd"
"yyyy-MM-dd HH:mm:ss"
"yyyy-MM-dd HH:mm:ss.SSS"
"yyyy-MM-dd HH:mm:ss Z"
"yyyy-MM-dd HH:mm:ss.SSS Z"
"yyyy-MM-dd'T'HH:mm:ssZ"
"yyyy-MM-dd'T'HH:mm:ss.SSSZ"
"yyyy-MM-dd'T'HH:mm:ssz"
"yyyy-MM-dd'T'HH:mm:ss.SSSz"
authenticationEnabled: true means login by username/password, false means login without authentication [false]
extraFields: use with extra fields to tag the information, for example your ip address, computer name[]
username: required when "authenticationEnabled" is true []
password: required when "authenticationEnabled" is true []
flume.conf sample
agent2.sources = source2
agent2.channels = channel2
agent2.sinks = sink2
agent2.sources.source2.type = org.riderzen.flume.source.MsgPackSource
agent2.sources.source2.bind = localhost
agent2.sources.source2.port = 1985
agent2.sources.source2.channels = channel2
agent2.sinks.sink2.type = org.riderzen.flume.sink.MongoSink
agent2.sinks.sink2.host = localhost
agent2.sinks.sink2.port = 27017
agent2.sinks.sink2.model = SINGLE
agent2.sinks.sink2.collection = events
agent2.sinks.sink2.batch = 100
agent2.sinks.sink2.extraFields.ip = 192.168.1.1
agent2.sinks.sink2.channel = channel2
agent2.channels.channel2.type = memory
agent2.channels.channel2.capacity = 1000000
agent2.channels.channel2.transactionCapacity = 800
agent2.channels.channel2.keep-alive = 3
Event Headers
The sink supports some headers in DYNAMIC model:
'db': db name
'collection' : collection name
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].