All Projects → mustafaturan → monoton

mustafaturan / monoton

Licence: Apache-2.0 License
Highly scalable, single/multi node, sortable, predictable and incremental unique id generator with zero allocation magic on the sequential generation

Programming Languages

go
31211 projects - #10 most used programming language

Projects that are alternatives of or similar to monoton

tsid-creator
A Java library for generating Time Sortable Identifiers (TSID).
Stars: ✭ 16 (-23.81%)
Mutual labels:  snowflake, id-generator
sno
Compact, sortable and fast unique IDs with embedded metadata.
Stars: ✭ 77 (+266.67%)
Mutual labels:  snowflake, id-generator
java-sdk
一些常用的java sdk和工具类(日期工具类,分布式锁,redis缓存,二叉树,反射工具类,线程池,对称/非对称/分段加解密,json序列化,http工具,雪花算法,字符串相似度,集合操作工具,xml解析,重试Retry工具类,Jvm监控等)
Stars: ✭ 26 (+23.81%)
Mutual labels:  snowflake
ulid-creator
A Java library for generating Universally Unique Lexicographically Sortable Identifiers (ULID)
Stars: ✭ 38 (+80.95%)
Mutual labels:  id-generator
ids
高效的分布式id生成器,每个客户端实例tps可达到100万,服务端毫无压力。即使服务端宕机了,id生成依然可用。支持多数据中心,支持id加密。
Stars: ✭ 47 (+123.81%)
Mutual labels:  id-generator
siphash-java
SipHash in Java; zero-allocation and streaming implementations
Stars: ✭ 25 (+19.05%)
Mutual labels:  zero-allocation
astro
Astro allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.
Stars: ✭ 79 (+276.19%)
Mutual labels:  snowflake
dremio-snowflake
Snowflake Connector for Dremio using the ARP SDK.
Stars: ✭ 14 (-33.33%)
Mutual labels:  snowflake
carto-spatial-extension
A set of UDFs and Procedures to extend BigQuery, Snowflake, Redshift and Postgres with Spatial Analytics capabilities
Stars: ✭ 131 (+523.81%)
Mutual labels:  snowflake
go-snowflake
go-snowflake
Stars: ✭ 101 (+380.95%)
Mutual labels:  snowflake
piccolo
Netty4长连接网关
Stars: ✭ 19 (-9.52%)
Mutual labels:  snowflake
distributed-id
基于netty4+twitter-snowFlake分布式Id生成之服务实现
Stars: ✭ 18 (-14.29%)
Mutual labels:  snowflake
snowworker
Website snow! It'll settle on anything that has a .rooftop class.
Stars: ✭ 25 (+19.05%)
Mutual labels:  snowflake
tenjin
📝 A template engine.
Stars: ✭ 15 (-28.57%)
Mutual labels:  zero-allocation
snowflake-starter
A _simple_ starter template for Snowflake Cloud Data Platform
Stars: ✭ 31 (+47.62%)
Mutual labels:  snowflake
simple-ddl-parser
Simple DDL Parser to parse SQL (HQL, TSQL, AWS Redshift, BigQuery, Snowflake and other dialects) ddl files to json/python dict with full information about columns: types, defaults, primary keys, etc. & table properties, types, domains, etc.
Stars: ✭ 76 (+261.9%)
Mutual labels:  snowflake
versatile-data-kit
Versatile Data Kit (VDK) is an open source framework that enables anybody with basic SQL or Python knowledge to create their own data pipelines.
Stars: ✭ 144 (+585.71%)
Mutual labels:  snowflake
sfquickstarts
Follow along with our tutorials to get you up and running with the Snowflake Data Cloud.
Stars: ✭ 83 (+295.24%)
Mutual labels:  snowflake
pre-commit-dbt
🎣 List of `pre-commit` hooks to ensure the quality of your `dbt` projects.
Stars: ✭ 149 (+609.52%)
Mutual labels:  snowflake
SnowFlakeProject
All open source data of the snow flake project.
Stars: ✭ 37 (+76.19%)
Mutual labels:  snowflake

Monoton

Build Status Coverage Status Go Report Card GoDoc

Highly scalable, single/multi node, predictable and incremental unique id generator with zero allocation magic.

Installation

Via go packages: go get github.com/mustafaturan/monoton/v3

API

The method names and arities/args are stable now. No change should be expected on the package for the version 3.x.x except any bug fixes.

Usage

Using with Singleton

Create a new package like below, and then call Next() or NextBytes() method:

package uniqid

// Import packages
import (
	"fmt"
	"github.com/mustafaturan/monoton/v3"
	"github.com/mustafaturan/monoton/v3/sequencer"
)

var m monoton.Monoton

// On init configure the monoton
func init() {
	m = newIDGenerator()
}

func newIDGenerator() monoton.Monoton {
	// Fetch your node id from a config server or generate from MAC/IP address
	node := uint64(1)

	// A unix time value which will be subtracted from the time sequence value.
	// The initialTime value type corresponds to the sequencer type's time
	// representation. If you are using Millisecond sequencer then it must be
	// considered as Millisecond
	// If we want to init the time with 2020-01-01 00:00:00 PST
	initialTime := uint64(1577865600000)

	// Configure monoton with a sequencer and the node
	m, err = monoton.New(sequencer.NewMillisecond(), node, initialTime)
	if err != nil{
		panic(err)
	}

	return m
}

func Generate() string {
	m.Next()
}

func GeneateBytes() [16]byte {
	m.NextBytes()
}

In any other package generate the ids like below:

import (
	"fmt"
	"uniqid" // your local uniqid package from your project
)

func main() {
	for i := 0; i < 100; i++ {
		fmt.Println(uniqid.Generate())
	}
}

Using with Dependency Injection

package main

// Import packages
import (
	"fmt"
	"github.com/mustafaturan/monoton/v3"
	"github.com/mustafaturan/monoton/v3/sequencer"
)

func NewIDGenerator() monoton.Monoton {
	// Fetch your node id from a config server or generate from MAC/IP address
	node := uint64(1)

	// A unix time value which will be subtracted from the time sequence value.
	// The initialTime value type corresponds to the sequencer type's time
	// representation. If you are using Millisecond sequencer then it must be
	// considered as Millisecond
	initialTime := uint64(0)

	// Configure monoton with a sequencer and the node
	m, err := monoton.New(sequencer.NewMillisecond(), node, initialTime)
	if err != nil{
		panic(err)
	}

	return m
}

func main() {
	g := NewIDGenerator()

	for i := 0; i < 100; i++ {
		fmt.Println(g.Next())
	}
}

Features

Time Ordered

The monoton package provides sequences based on the monotonic time which represents the absolute elapsed wall-clock time since some arbitrary, fixed point in the past. It isn't affected by changes in the system time-of-day clock.

Please refer to ADR 01 - Time for details and consequences.

Initial Time

Initial time value opens space for time value by subtracting the given value from the time sequence.

Readable

The monoton package converts all sequences into Base62 format. And Base62 only uses ASCII alpha-numeric chars to represent data which makes it easy to read, predict the order by a human eye.

The total byte size is fixed to 16 bytes for all sequencers. And at least one byte is reserved to nodes.

Please refer to ADR 02 - Encoding for details and consequences.

Multi Node Support

The monoton package can be used on single/multiple nodes without the need for machine coordination. It uses configured node identifier to generate ids by attaching the node identifier to the end of the sequences.

Extendable

The package comes with three pre-configured sequencers and Sequencer interface to allow new sequencers.

Included Sequencers and Byte Orderings

The monoton package currently comes with Nanosecond, Millisecond and Second sequencers. And it uses Millisecond sequencer by default. For each sequencer, the byte orders are as following:

Second:      16 B =>  6 B (seconds)      + 6 B (counter) + 4 B (node)
Millisecond: 16 B =>  8 B (milliseconds) + 4 B (counter) + 4 B (node)
Nanosecond:  16 B => 11 B (nanoseconds)  + 2 B (counter) + 3 B (node)

Please refer to ADR 03 - Byte Sizes for details and consequences.

New Sequencers

The sequencers can be extended for any other time format, sequence format by implementing the monoton/sequencer.Sequencer interface.

Benchmarks

Command:

go test -benchtime 10000000x -benchmem -run=^$ -bench=. github.com/mustafaturan/monoton/v3

Results:

goos: darwin
goarch: amd64
pkg: github.com/mustafaturan/monoton/v3
cpu: Intel(R) Core(TM) i5-6267U CPU @ 2.90GHz
BenchmarkNext-4        	10000000	       102.3 ns/op	       0 B/op	       0 allocs/op
BenchmarkNextBytes-4   	10000000	        97.51 ns/op	       0 B/op	       0 allocs/op
PASS
ok  	github.com/mustafaturan/monoton/v3	2.203s

Contributing

All contributors should follow Contributing Guidelines and ADR docs before creating pull requests.

Credits

Mustafa Turan

License

Apache License 2.0

Copyright (c) 2019 Mustafa Turan

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].