Getting started with Zookeeper -- Zookeeper's Java client API

I have searched some books and blogs about Zookeeper, but many of them mainly talk about its architecture and use, especially the applications in the distributed system, which are really overhyped, but I still can't write code after reading them, which makes me a group of rich theories and completely ignorant of practice. For the use of the Java client API of Zookeeper, it is not involved or explained clearly, or it is very vague. Sure enough, I have to learn Zookeeper's functions through Zookeeper's API, and then think about how to use them? Why? Gradually explore the application of Zookeeper.

1. Environmental preparation

1. Install jdk, which will not be discussed in detail.

2. Install Zookeeper cluster. There are many detailed online tutorials, but it's still easy to post your own configuration file. It can be copied directly. It's just the configuration file of one of the Zookeeper nodes. The other two just change the path of the port, dataDir and dataLogDir.

# Basic time unit of interaction between server and client (ms) 
tickTime=2000   
#zookeeper Cluster contains multiple server, One of them is leader, The rest of the cluster server by follower. initLimit Parameter configuration when initializing a connection, follower and leader Maximum heartbeat time between. This parameter is now set to 10, #Explain that the time limit is 10 times tickTime
initLimit=10  
# This parameter configures the maximum time length of sending message, request and reply between leader and follower. At this time, this parameter is set to 5, indicating that the time limit is 5 times of tickTime 
syncLimit=5
# Basic time unit used in zookeeper, millisecond value
tickTime=2000
#Where to store database snapshots in memory
dataDir=/usr/local/zookeeper-cluster/zookeeper-2181/data
#Directory where logs are stored, if this parameter is not set, Will use and#dataDir is the same setting
dataLogDir=/usr/local/zookeeper-cluster/zookeeper-2181/logs 
#The port used to listen for client connections, or for client requests to connect
clientPort=2181

maxClientCnxns=60
#This section is very important. The key to cluster configuration is server. The number following is the key to identify a Zookeeper. The cluster will use the
server.1=127.0.0.1:2222:2225
server.2=127.0.0.1:3333:3335
server.3=127.0.0.1:4444:4445

#autopurge.snapRetainCount=3

#autopurge.purgeInterval=1

3. Introduce the jar package. It is recommended to use Apache's java client.

		<dependency>
			<groupId>org.apache.zookeeper</groupId>
			<artifactId>zookeeper</artifactId>
			<version>3.5.7</version>
		</dependency>

2.API use

1. First, create the Zookeeper client object. It's very simple, just new directly, but the main thing is to pay attention to the parameters of the construction method. There are many overloaded versions of Zookeeper construction method, so it is necessary to understand the meaning of each parameter.

public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher, long sessionId, byte[] sessionPasswd, boolean canBeReadOnly, HostProvider aHostProvider, ZKClientConfig clientConfig) throws IOException 

(1) String connectString:Zookeeper the IP address and port number of each node of the cluster, separated by commas. for example

String zkNodes = "127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183";

(2)int sessionTimeout: session timeout, in milliseconds. Within the sessionTimeout time, the client and the server will send PING requests directly through the server to maintain the validity of the session, commonly known as "heartbeat detection", and the server will reactivate the corresponding session of the client.

Session refers to the session generated when the Client creates a connection with the Server. After connecting Connected, the session status will be turned on. The Zookeeper Server and the Client adopt the long connection mode (the Client will continuously send heartbeat to the Server) to ensure that the session can always exist without network problems, Server downtime or Client downtime. Therefore, under normal circumstances, the session will always be valid, and all machines on the ZK cluster will save the session information.

If the server does not receive the heartbeat detection request from the client after the session timeout, the server will treat the client as offline and delete the stored session. In ZK, a lot of data and state are bound to the session. Once the session fails, ZK starts to clear the information related to the session, including the temporary nodes created by the session and all registered watchers.

However, another situation is that the client is normal, but the Zookeeper cluster node connected to the current session is down or the heartbeat detection fails for other reasons, that is, it cannot ping. ZK Client will immediately catch this exception, encapsulate it as a ConnectionLoss event, and then start the automatic reconnect mechanism to select a new address in the address list for reconnection. Reconnection has three results:

  • In the session timeout period, if the connection is successful, the client will receive a syncconnected event again, and the connection will be re persisted to the connected state
  • After the session timeout period is exceeded, the reconnection succeeds. The client will receive an expired event and persist the connection to the closed state
  • The client will not receive any event if it is unable to connect again

(3)Watcher watcher: This is a very important feature of Zookeeper. Because the connection between the client and Zookeeper is long, you can register an event listener, that is, the watcher object, on Zookeeper through the client. When an event occurs in Zookeeper, the method in the watcher object will be called back. The role of this watcher object is very important.

(4) Long session id: for the establishment of each session, Zookeeper will assign a globally unique session id to the session automatically, so in general, this id will not be specified by us, and the parameter will not be passed.

(5)boolean canBeReadOnly: whether to provide read-only service (no write service).

(6)HostProvider aHostProvider: randomly provide host for connection. It's useless. The default is OK.

(7)ZKClientConfig clientConfig: connection parameter configuration,

(8)byte[] sessionPasswd: provide the sessionId and password to connect to zookeeper, through which the only client can be determined, so as to provide repeated sessions.

In fact, the really necessary parameters are connectString, sessionTimeout, and watcher. The rest are the default ones. So Zookeeper also provides the overloaded version of these three parameters, which is also the most commonly used construction method.

Note: the establishment of the session between the client and the server of zookeeper is an asynchronous process, that is to say, in the program, our program method returns immediately after handling the client initialization (the program executes the code downward, so in most cases, we haven't really built a usable session, and only when the session declaration cycle is "CONNECTING", can we really establish it BI)

An example code for creating a Zookeeper client and establishing a connection is as follows:

    public static void createZookeeperClient() throws IOException, InterruptedException {
        /*
        Since the establishment and connection of the Zookeeper client object are asynchronous, it is likely that the Zookeeper object has not established a connection,
        The main thread will continue to execute, resulting in execution error reporting. This can be assisted by some thread synchronization classes to ensure that the Zookeeper connection is successfully established before subsequent operations
         */
        final CountDownLatch countDownLatch = new CountDownLatch(1);
        ZooKeeper zooKeeper = new ZooKeeper(zkNodes, 5000, new Watcher() {
            @Override
            public void process(WatchedEvent watchedEvent) {
                //Connection established
                if (watchedEvent.getState().equals(Event.KeeperState.SyncConnected)) {
                    countDownLatch.countDown();
                }
            }
        });
        countDownLatch.await();
        //do something...
    }

2. Use of watcher object: watcher can monitor Zookeeper. This feature comes from the way that Zookeeper client and server use TCP long connection to communicate. When an event occurs in Zookeeper, a callback notification is made through the watcher object. Watcher is an interface, so it is necessary to implement a specific implementation class. There is only one interface method, process(), whose parameter WatchedEvent is the event of Zookeeper server, which contains all the information of the event. When the client registers the watcher with the zk server, it stores the watcher object in the client's WatchManager. When the zk server triggers the watcher event, it will send a notification to the client, and the client thread will retrieve the corresponding watcher object from the watcher manager. To perform the callback logic.

public class WatcherTest1 implements Watcher {
    @Override
    public void process(WatchedEvent watchedEvent) {
        
    }
}

For WatchedEvent, the source code is relatively simple, which only contains three main domain variables, namely EventType eventType, KeeperState keeperState, String path. eventType refers to the event type, keeperState refers to the connection state, and path refers to the data node path where the event occurred.

public class WatchedEvent {
    private final KeeperState keeperState;
    private final EventType eventType;
    private String path;

   ····
}

EventType and KeeperState are enumeration types

        public static enum EventType {
            None(-1),//No such node
            NodeCreated(1),//Node created successfully
            NodeDeleted(2),//Node deleted successfully
            NodeDataChanged(3),//Node data change
            NodeChildrenChanged(4),//Event triggered when a child node is created or deleted
            DataWatchRemoved(5),//Data monitoring removed
            ChildWatchRemoved(6);//Child node monitoring removed

            private final int intValue;
          ....
        }
        public static enum KeeperState {
            
            
            Unknown(-1),//Obsolete since version 3.1.0
            Disconnected(0),//Client and server are disconnected
            NoSyncConnected(1),//Obsolete since version 3.1.0
            SyncConnected(3),//Client and server are connected
            AuthFailed(4),//The status of permission validation failure is usually received at the same time as AuthFailedException
            ConnectedReadOnly(5),//Read only connection
            SaslAuthenticated(6),//Authority verification passed
            Expired(-112),//At this time, the client session fails, and it usually receives a SessionExpiredException at the same time
            Closed(7);//Connection resource close

            ....
        }

Watcher feature

Disposable

Either the server or the client, once a Watcher is triggered, ZooKeeper will remove it from the corresponding storage. Therefore, one thing developers should keep in mind when using Watcher is that it needs to be registered repeatedly. For example, if the client executes getData("/znode1", true), and later changes or deletes / znode1, the client will be notified of the monitoring event of / znode1. If / znode1 is changed again, the client will not send a monitoring event notification if it does not perform a new read to set a new monitoring point.

Client serial execution

The process of client Watcher callback is a process of serial synchronization, which ensures the order for us. At the same time, the developer should pay attention to that the processing logic of one Watcher does not affect the whole client Watcher callback.

Light weight

WatchedEvent is the smallest notification unit of the whole Watcher notification mechanism of ZooKeeper. There are only three parts in this data structure: notification state, event type and node path. That is to say, watcher notification is very simple. It only tells the client that an event has occurred, but not the specific content of the event.

In addition, when the client registers the Watcher with the server, it will not pass the client's real Watcher object to the server, but only marks it with the boolean type attribute in the client's request. At the same time, the server only saves the currently connected ServerCnxn object. Such a lightweight Watcher mechanism design is very cheap in network overhead and server memory overhead.

3. Create node: the Java client of Zookeeper has two implementation methods for data operation (add, delete, modify and query), synchronous and asynchronous. In general, synchronization operation will have return value and throw corresponding exception. The asynchronous operation does not return a value or throw an exception. In addition, based on the synchronous method parameters, two parameters, Callback and context, will be added to the asynchronous method parameters.

Synchronous creation node: full parameter version

    public String create(String path, byte[] data, List<ACL> acl, CreateMode createMode, Stat stat, long ttl)

(1) Path refers to the path of the data node.

(2) data is the byte array form of the string to be stored. (serialization is not supported. If you need to implement serialization, you can use java related serialization framework, such as Hession)

(3) acl refers to the node permission. It is OK to use ids.open ﹣ acl ﹣ unsafe permission uniformly (in general, it is not necessary to pay attention to when the permission is not too high)

(4) CreateMode node type: CreateMode. * provides four types of nodes: PERSISTENT, PERSISTENT and sequential, temporary and sequential

(5) stat node status information: when creating a node, you can manually specify the node status information, but generally you do not need to pass in this parameter. (not required parameter)

(6) ttl: expiration time. If the node does not change within the ttl time, it will be deleted. (not required parameter)

Therefore, the overloaded versions of the commonly used synchronous node creation methods are as follows

public String create(String path, byte[] data, List<ACL> acl, CreateMode createMode)

The sample code is as follows

public static void createNode() throws IOException, KeeperException, InterruptedException {
        final CountDownLatch countDownLatch = new CountDownLatch(1);
        ZooKeeper zooKeeper = new ZooKeeper(zkNodes, 5000, new Watcher() {
            @Override
            public void process(WatchedEvent watchedEvent) {
                //Connection established
                if (watchedEvent.getState().equals(Event.KeeperState.SyncConnected)) {
                    countDownLatch.countDown();
                }
            }
        });
        countDownLatch.await();
        //Node permission settings
        ACL acl = new ACL(ZooDefs.Perms.ALL,ZooDefs.Ids.ANYONE_ID_UNSAFE);
        List<ACL> acls = new ArrayList<ACL>();
        acls.add(acl);
        //Create node
        zooKeeper.create("/demo", "helloworld".getBytes(), acls, CreateMode.PERSISTENT);

        System.out.println("over");
    }

Create nodes asynchronously:

public void create(String path, byte[] data, List<ACL> acl, CreateMode createMode, StringCallback cb, Object ctx) 

In addition to the four required parameters in the create synchronization method, the Create method in asynchronous mode also adds two parameters, callback and context. The processResult method in the callback will be called after the node is created. It has four parameters. The first is the resultCode of type int, which is the server response code. 0 means the call is successful, - 4 means the port connection, - 110 means the specified node exists, - 112 means the session has expired. The second parameter is the path to create the node. The third parameter is context, which is very useful when a StringCallback type object is used as a parameter of multiple create methods. The fourth parameter is the name of the creation node, which is actually the same as the path parameter. The asynchronous method does not throw an exception, but processes all events in the callback StringCallback.

    public interface StringCallback extends AsyncCallback {
        void processResult(int var1, String var2, Object var3, String var4);
    }

Sample code

    public static void createNode() throws IOException, KeeperException, InterruptedException {

        //Create connection object omit


        final CountDownLatch countDownLatch = new CountDownLatch(1);
        //Node permission settings
        ACL acl = new ACL(ZooDefs.Perms.ALL,ZooDefs.Ids.ANYONE_ID_UNSAFE);
        List<ACL> acls = new ArrayList<ACL>();
        acls.add(acl);

        zk.create("/demo", "helloworld".getBytes(), acls, CreateMode.PERSISTENT,new IStringCallBack(), countDownLatch);
        countDownLatch.await();
        System.out.println("over");
    }

    static class IStringCallBack implements AsyncCallback.StringCallback {

        @Override
        public void processResult(int rc, String path, Object ctx, String name) {
            switch (KeeperException.Code.get(rc)) {
                case CONNECTIONLOSS:
                    System.out.println("CONNECTIONLOSS");
                    break;
                case OK:
                    System.out.println("OK - {" + path + ", " + name + ", " + ctx + "}");
                    CountDownLatch countDownLatch = (CountDownLatch) ctx;
                    countDownLatch.countDown();
                    break;
                case NODEEXISTS:
                    System.out.println(path + "exists");
                    break;
                default:
                    System.out.println("DEFAULT");
                    break;
            }
        }

    }

4. Delete node: similar to create node, it is divided into synchronous and asynchronous.

(1) Synchronization

public void delete(String path, int version)

Parameter path, indicating the node path

The parameter version indicates the version number. That is to say, this deletion is for a version of the data. Data modification can succeed only when the value of the version parameter is equal to the dataVersion value in the node status information, otherwise a BadVersion exception will be thrown. This is to prevent the update of lost data. In the API provided by ZooKeeper, all data writes to existing nodes have the version parameter.

(2) Asynchronous

public void delete(String path, int version, VoidCallback cb, Object ctx)

5. Modify node data:

(1) Synchronization

public Stat setData(String path, byte[] data, int version)

(2) Asynchronous

public void setData(String path, byte[] data, int version, StatCallback cb, Object ctx)

Parameter path: indicates the data node path

Parameter data: indicates the data to be set

Parameter version: indicates to modify the data version

6. Get the data of the node:

(1) Synchronization:

public byte[] getData(String path, Watcher watcher, Stat stat)

The return value of the zooKeeper.getData method is the data value stored in the node. It has three parameters. The first parameter is the path of the node, which is used to indicate which node's data to obtain. The third parameter stat is used to store the state information of the node. Before calling the getData method, an empty stat type object will be constructed as a parameter and passed to the getData method. When the getData method call returns, the state information of the node will be filled into the stat object.

private void getDataSync() throws KeeperException, InterruptedException {
    Stat stat = new Stat();
    // The return value of getData is the data value of the node, and the status information of the node will be assigned to the stat object
    byte[] data = zooKeeper.getData("/node_1",true, stat);
    System.out.println(new String(data));
    System.out.println(stat);
}

The second parameter is a watch of bool type, which is more important. When the watch is true, it means that we want to monitor the data changes of this node. When the data of the node changes, we can get the notification pushed to us by zk server. In the process method, there will be code similar to the following:

public void process(WatchedEvent watchedEvent) {
    if (watchedEvent.getState() == Event.KeeperState.SyncConnected) { //Connected to zk server
        if(watchedEvent.getType() == Event.EventType.None && null == watchedEvent.getPath()) {
            createNodeAsync();
        } else if(watchedEvent.getType() == Event.EventType.NodeChildrenChanged) {
            // The list of child nodes of the node changes
        } else if(watchedEvent.getType() == Event.EventType.NodeDataChanged) {
            // The data content of the node changes
        }
    }
}

For the second parameter, we can also use the custom Watcher object, which is another overloaded version of the getData method

public byte[] getData(String path, Watcher watcher, Stat stat)

(2) Asynchronous

public void getData(String path, Watcher watcher, DataCallback cb, Object ctx)
public void getData(String path, boolean watch, DataCallback cb, Object ctx)
private void getDataAsync() {
        zooKeeper.getData("/node", true, new AsyncCallback.DataCallback() {
            public void processResult(int resultCode, String path, Object ctx, byte[] data, Stat stat) {
                System.out.println(resultCode);
                System.out.println(path);
                System.out.println(ctx);
                System.out.println(new String(data));//Data is the data obtained
                System.out.println(stat);
            }
        }, "Asynchronously get node data");
    }

 

 

Published 134 original articles, won praise 4, visited 1890
Private letter follow

Tags: Zookeeper Session Java REST

Posted on Mon, 16 Mar 2020 03:51:54 -0400 by mikevarela