Distributed globally unique ID generation method

I. snowflake algorithm

twitter's open-source distributed id generation algorithm uses 64 bit long id.

41 bit, time stamp, in milliseconds, 2 ^ 41 - 1 can be converted into 69 years

10 bit, of which 5 are machine rooms and 5 are machine ID S

12 bit, recording different IDs generated in the same millisecond, 2 ^ 12 - 1 = 4096, which can represent 4096 different IDS in the same millisecond

Support tens of thousands of concurrency per second

public class UniqueOrderGenerate {
    // ==============================Fields===========================================
    /** Start time (2018-07-03) */

    private final long twepoch = 1530607760000L;

    /** Number of digits occupied by machine id */
    private final long workerIdBits = 5L;

    /** Number of digits occupied by data id */
    private final long datacenterIdBits = 5L;

    /** The maximum machine id supported, the result is 31 (this shift algorithm can quickly calculate the maximum decimal number that several binary numbers can represent) */
    private final long maxWorkerId = -1L ^ (-1L << workerIdBits);

    /** Maximum data id supported, result is 31 */
    private final long maxDatacenterId = -1L ^ (-1L << datacenterIdBits);

    /** Number of digits of sequence in id */
    private final long sequenceBits = 12L;

    /** Machine ID moved 12 bits to the left */
    private final long workerIdShift = sequenceBits;

    /** Move data id 17 bits to the left (12 + 5) */
    private final long datacenterIdShift = sequenceBits + workerIdBits;

    /** Time cut 22 bits left (5 + 5 + 12) */
    private final long timestampLeftShift = sequenceBits + workerIdBits + datacenterIdBits;

    /** Mask for generating sequence, here is 4095 (0b111111111111=0xfff=4095) */
    private final long sequenceMask = -1L ^ (-1L << sequenceBits);

    /** Working machine ID (0-31) */
    private long workerId;

    /** Data center ID (0-31) */
    private long datacenterId;

    /** Sequence in milliseconds (0-4095) */
    private long sequence = 0L;

    /** Last ID generated by */
    private long lastTimestamp = -1L;

    //==============================Constructors=====================================
    /**
     * Constructor
     * @param workerId Work ID (0-31)
     * @param datacenterId Data center ID (0-31)
     */
    public UniqueOrderGenerate(long workerId, long datacenterId) {
        if (workerId > maxWorkerId || workerId < 0) {
            throw new IllegalArgumentException(String.format("worker Id can't be greater than %d or less than 0", maxWorkerId));
        }
        if (datacenterId > maxDatacenterId || datacenterId < 0) {
            throw new IllegalArgumentException(String.format("datacenter Id can't be greater than %d or less than 0", maxDatacenterId));
        }
        this.workerId = workerId;
        this.datacenterId = datacenterId;
    }

    // ==============================Methods==========================================
    /**
     * Get the next ID (this method is thread safe)
     * @return SnowflakeId
     */
    public synchronized long nextId() {
        long timestamp = timeGen();

        //If the current time is less than the time stamp generated by the last ID, an exception should be thrown when the system clock goes back
        if (timestamp < lastTimestamp) {
            throw new RuntimeException(
                    String.format("Clock moved backwards.  Refusing to generate id for %d milliseconds", lastTimestamp - timestamp));
        }

        //If generated at the same time, sequence in milliseconds
        if (lastTimestamp == timestamp) {
            sequence = (sequence + 1) & sequenceMask;
            //Sequence overflow in MS
            if (sequence == 0) {
                //Block to next MS, get new timestamp
                timestamp = tilNextMillis(lastTimestamp);
            }
        }
        //Timestamp change, sequence reset in MS
        else {
            sequence = 0L;
        }

        //Last ID generated by
        lastTimestamp = timestamp;

        //Shift and combine by or operation to form a 64 bit ID
        return (((timestamp - twepoch) << timestampLeftShift) //
                | (datacenterId << datacenterIdShift) //
                | (workerId << workerIdShift) //
                | sequence);
    }

    /**
     * Blocks to the next millisecond until a new timestamp is obtained
     * @param lastTimestamp Last ID generated by
     * @return Current timestamp
     */
    protected long tilNextMillis(long lastTimestamp) {
        long timestamp = timeGen();
        while (timestamp <= lastTimestamp) {
            timestamp = timeGen();
        }
        return timestamp;
    }

    /**
     * Returns the current time in milliseconds
     * @return Current time (MS)
     */
    protected long timeGen() {
        return System.currentTimeMillis();
    }

    //==============================Test=============================================
    /** test */
    public static void main(String[] args) {
        UniqueOrderGenerate idWorker = new UniqueOrderGenerate(0, 0);
        for (int i = 0; i < 1000; i++) {
            long id = idWorker.nextId();
            //System.out.println(Long.toBinaryString(id));
            System.out.println(id);
        }
    }
}

2. Use data to generate Sequence

Maintain a table in the database

CREATE TABLE `sequence` (
  `ID` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'ID',
  `unique_id` bigint(18) DEFAULT NULL COMMENT 'unique_id',
  `UPDATED_TIME` timestamp NOT NULL COMMENT 'Update time',
  PRIMARY KEY (`ID`)
)

Initialization:

Set step = 5000, min = 0, max = 5000

Intercept time string to the current day, curTime = '20190520'

And step splicing, uniqueId = curTime + step = 201905201000

Main ideas:

Each application server takes the step in advance, which is stored in the JVM memory. nextKey + +, the step is exhausted, and the database is updated.

  1. Create tables in database, save uniqueId, updateTime
  2. Set the database update step, update the database every time the step arrives
  3. Set uniqueId length according to traffic volume, and bigInt can be used
  4. Time plus step splicing as unique id

Pseudo code

Step getUniqueueId(){
	if nextKey > max || curTime == null
		operaDb()
	nextKey++
	seq = curTime + step
}

Operational database operatDb(){
    Query db current uniqueId
    Current uniqueId + step
}

Tags: Database less jvm

Posted on Thu, 07 Nov 2019 11:03:40 -0500 by jonat8