All Projects → souyunku → Snowflake

souyunku / Snowflake

Twitter的分布式自增ID雪花算法snowflake (Java版)

Programming Languages

java
68154 projects - #9 most used programming language

Projects that are alternatives of or similar to Snowflake

Distributedid
基于twitter的雪花算法(SnowFlake)来产生分布式ID,支持SDK、HTTP方式接入
Stars: ✭ 273 (-18.75%)
Mutual labels:  snowflake, twitter
Mastodon Twitter Poster
Crossposter to post statuses between Mastodon and Twitter
Stars: ✭ 317 (-5.65%)
Mutual labels:  twitter
Tweet Tray
Tweet quickly from the desktop without any distractions.
Stars: ✭ 274 (-18.45%)
Mutual labels:  twitter
Twitter Search
Instantly search across your entire Twitter history with a beautiful UI powered by Algolia.
Stars: ✭ 305 (-9.23%)
Mutual labels:  twitter
Twitter Cleanup
🛁 Clean-up inactive accounts and bots from your Twitter
Stars: ✭ 275 (-18.15%)
Mutual labels:  twitter
Surfbird
A Microblogging client built on Electron and Vue
Stars: ✭ 309 (-8.04%)
Mutual labels:  twitter
Twitter Sent Dnn
Deep Neural Network for Sentiment Analysis on Twitter
Stars: ✭ 270 (-19.64%)
Mutual labels:  twitter
The Seo Framework
The SEO Framework WordPress plugin.
Stars: ✭ 329 (-2.08%)
Mutual labels:  twitter
Tweetie
Simple jQuery Twitter feed plugin
Stars: ✭ 314 (-6.55%)
Mutual labels:  twitter
Read Weekly
Think Outside The Box And Monkey Reading / 每周一书
Stars: ✭ 300 (-10.71%)
Mutual labels:  twitter
Socialmanagertools Gui
🤖 👻 Desktop application for Instagram Bot, Twitter Bot and Facebook Bot
Stars: ✭ 293 (-12.8%)
Mutual labels:  twitter
React Static Tweets
Extremely fast static renderer for tweets.
Stars: ✭ 278 (-17.26%)
Mutual labels:  twitter
Hybridauth
Open source social sign on PHP Library. HybridAuth goal is to act as an abstract api between your application and various social apis and identities providers such as Facebook, Twitter and Google.
Stars: ✭ 3,223 (+859.23%)
Mutual labels:  twitter
Rainbowstream
A smart and nice Twitter client on terminal written in Python.
Stars: ✭ 3,344 (+895.24%)
Mutual labels:  twitter
Rsshub
🍰 Everything is RSSible
Stars: ✭ 18,111 (+5290.18%)
Mutual labels:  twitter
Twitter Clone Frontend
Twitter Clone Frontend
Stars: ✭ 271 (-19.35%)
Mutual labels:  twitter
Laravel Social Auto Posting
🌈Laravel social auto posting
Stars: ✭ 306 (-8.93%)
Mutual labels:  twitter
Sockethub
A protocol gateway for the Web.
Stars: ✭ 329 (-2.08%)
Mutual labels:  twitter
Social Media Profiles Regexs
📇 Extract social media profiles and more with regular expressions
Stars: ✭ 324 (-3.57%)
Mutual labels:  twitter
Twitter Api V2 Sample Code
Sample code for the Twitter API early access endpoints (Python, Java, Ruby, and Node.js).
Stars: ✭ 306 (-8.93%)
Mutual labels:  twitter

snowflake的结构如下(每部分用-分开):

概述

分布式系统中,有一些需要使用全局唯一ID的场景,这种时候为了防止ID冲突可以使用36位的UUID,但是UUID有一些缺点,首先他相对比较长,另外UUID一般是无序的。

有些时候我们希望能使用一种简单一些的ID,并且希望ID能够按照时间有序生成。

而twitter的snowflake解决了这种需求,最初Twitter把存储系统从MySQL迁移到Cassandra,因为Cassandra没有顺序ID生成机制,所以开发了这样一套全局唯一ID生成服务。

结构

snowflake的结构如下(每部分用-分开):

0 - 0000000000 0000000000 0000000000 0000000000 0 - 00000 - 00000 - 000000000000

第一位为未使用,接下来的41位为毫秒级时间(41位的长度可以使用69年),然后是5位datacenterId和5位workerId(10位的长度最多支持部署1024个节点) ,最后12位是毫秒内的计数(12位的计数顺序号支持每个节点每毫秒产生4096个ID序号)

一共加起来刚好64位,为一个Long型。(转换成字符串长度为18)

snowflake生成的ID整体上按照时间自增排序,并且整个分布式系统内不会产生ID碰撞(由datacenter和workerId作区分),并且效率较高。据说:snowflake每秒能够产生26万个ID。

本机实测:100万个ID 耗时5秒

/**
 * 描述: Twitter的分布式自增ID雪花算法snowflake (Java版)
 *
 * @author yanpenglei
 * @create 2018-03-13 12:37
 **/
public class SnowFlake {

    /**
     * 起始的时间戳
     */
    private final static long START_STMP = 1480166465631L;

    /**
     * 每一部分占用的位数
     */
    private final static long SEQUENCE_BIT = 12; //序列号占用的位数
    private final static long MACHINE_BIT = 5;   //机器标识占用的位数
    private final static long DATACENTER_BIT = 5;//数据中心占用的位数

    /**
     * 每一部分的最大值
     */
    private final static long MAX_DATACENTER_NUM = -1L ^ (-1L << DATACENTER_BIT);
    private final static long MAX_MACHINE_NUM = -1L ^ (-1L << MACHINE_BIT);
    private final static long MAX_SEQUENCE = -1L ^ (-1L << SEQUENCE_BIT);

    /**
     * 每一部分向左的位移
     */
    private final static long MACHINE_LEFT = SEQUENCE_BIT;
    private final static long DATACENTER_LEFT = SEQUENCE_BIT + MACHINE_BIT;
    private final static long TIMESTMP_LEFT = DATACENTER_LEFT + DATACENTER_BIT;

    private long datacenterId;  //数据中心
    private long machineId;     //机器标识
    private long sequence = 0L; //序列号
    private long lastStmp = -1L;//上一次时间戳

    public SnowFlake(long datacenterId, long machineId) {
        if (datacenterId > MAX_DATACENTER_NUM || datacenterId < 0) {
            throw new IllegalArgumentException("datacenterId can't be greater than MAX_DATACENTER_NUM or less than 0");
        }
        if (machineId > MAX_MACHINE_NUM || machineId < 0) {
            throw new IllegalArgumentException("machineId can't be greater than MAX_MACHINE_NUM or less than 0");
        }
        this.datacenterId = datacenterId;
        this.machineId = machineId;
    }

    /**
     * 产生下一个ID
     *
     * @return
     */
    public synchronized long nextId() {
        long currStmp = getNewstmp();
        if (currStmp < lastStmp) {
            throw new RuntimeException("Clock moved backwards.  Refusing to generate id");
        }

        if (currStmp == lastStmp) {
            //相同毫秒内,序列号自增
            sequence = (sequence + 1) & MAX_SEQUENCE;
            //同一毫秒的序列数已经达到最大
            if (sequence == 0L) {
                currStmp = getNextMill();
            }
        } else {
            //不同毫秒内,序列号置为0
            sequence = 0L;
        }

        lastStmp = currStmp;

        return (currStmp - START_STMP) << TIMESTMP_LEFT //时间戳部分
                | datacenterId << DATACENTER_LEFT       //数据中心部分
                | machineId << MACHINE_LEFT             //机器标识部分
                | sequence;                             //序列号部分
    }

    private long getNextMill() {
        long mill = getNewstmp();
        while (mill <= lastStmp) {
            mill = getNewstmp();
        }
        return mill;
    }

    private long getNewstmp() {
        return System.currentTimeMillis();
    }

    public static void main(String[] args) {
        SnowFlake snowFlake = new SnowFlake(2, 3);

        long start = System.currentTimeMillis();
        for (int i = 0; i < 1000000; i++) {
            System.out.println(snowFlake.nextId());
        }

        System.out.println(System.currentTimeMillis() - start);


    }
}
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].