[Java from 0 to architect] Zookeeper foundation - installation, core working mechanism and basic commands

Java from 0 to architect Directory: [Java from 0 to architect] learning record

Zookeeper installation, configuration and operation

2n+1 is recommended for cluster installation

  • For the nodes, leader nodes and follower nodes in the cluster, after the leader node hangs up, you need to select a node in the cluster as the leader node, in which the election strategy is that the minority obeys the majority·

Install Zookeeper: Note: Zookeeper is written in Java and requires a jdk environment

1. Upload installation package to server

2. Extract to the specified directory

tar -zxvf apache-zookeeper-3.6.0-bin.tar.gz  -C /usr/local/

3. Modify the configuration directory where the file is located

zookeeper/conf

# Copy profile
# cp zoo_sample.cfg zoo.cfg
cat zoo_sample.cfg | grep -v "#" | grep -v "^$" > zoo.cfg

# Add the following to the configuration file
dataDir=/usr/local/apache-zookeeper-3.6.0-bin/data
server.1=192.168.52.128:2888:3888 # ip address of M01
server.2=192.168.52.129:2888:3888 # ip address of M02
server.3=192.168.52.130:2888:3888 # ip address of M03

Profile resolution:

tickTime=2000 # Heartbeat time unit (ms): 2s
initLimit=10 # Time limit for cluster startup: 10 * 2000 = 20s
syncLimit=5 # Time limit for synchronizing data: 5 * 2000 = 10s
dataDir=/usr/local/apache-zookeeper-3.6.0-bin/data # Directory of data storage
clientPort=2181 # Client connection port

# The service lists 1, 2 and 3 in the cluster represent the id of the service. The contents in the myid (which must be placed in the dateDir directory) file should always be between 1 and 252
server.1=192.168.52.128:2888:3888 # 2888 data synchronization port | 3888 election port
server.2=192.168.52.129:2888:3888
server.3=192.168.52.130:2888:3888

4. Create the data store directory and create the flag file myid
Create the data directory in / root/apps/zookeeper:

mkdir -p /root/apps/zookeeper/data
cd /root/apps/zookeeper/data
echo 1 > myid

5. Copy the installed configuration to other machines

Note: be sure to modify the myid file
M001, myid is 1
M002, modify myid to 2
M003, modify myid to 3

6. Configure environment variables for Zookeeper

vi /etc/profile
export ZOOKEEPER_HOME=/root/apps/zookeeper
export PATH=$PATH:$ZOOKEEPER_HOME/bin

7. Start service

zkServer.sh start # Start command
zkServer.sh status # View status command
zkServer.sh stop # Stop service command
zkServer.sh restart# Stop service command
jps # Check the java process. QuorumPeerMain is the process of Zookeeper

If you encounter the problem of being unable to connect, it is likely that the firewall has blocked the port:
systemctl stop firewalld # turn off the firewall
systemctl disable firewalld # disable firewall booting

The Zookeeper logs are in logs/zookeeper-root-server-localhost.localdomain.out. If there is a problem, you can check this

Zookeeper's core working mechanism

characteristic

1. Best practice of nodes: a cluster composed of 1 leader and 2 follower s

leader election mechanism: the minority obeys the majority. Only half or more people need to agree to elect leaders, so 2n + 1 node is the most appropriate.
If there are many cluster nodes, the performance of writing data will be affected, because writing data can only be accessed at the Leader node and needs to be synchronized to the slave node.

2. Global data consistency: each server saves a copy of the same data. No matter which server the client is connected to, the data is consistent

3. Distributed read-write, update request forwarding, implemented by the leader

4. The update requests are made in sequence, and the update requests from the same client are executed in sequence according to their sending order

5. Data update is atomic. One data update is either successful (more than half of the nodes are successful) or failed

6. Real time. Within a certain time range, the client can read the latest data (in milliseconds)

The data volume shall not exceed 1M

Data structure, node

1. Hierarchical directory structure, naming in line with conventional file system specifications

2. Each node is called Znode in Zookeeper and has a unique path ID

3. Node Znode can contain data (only a small amount of data can be stored, < 1m) and child nodes

However, nodes of EPHEMERAL type cannot have child nodes, which will be explained later

4. Client applications can set up monitors on nodes

Znode has four types of directory nodes:

  • PERSISTENT - PERSISTENT node
  • PERSISTENT_SEQUENTIAL - persistent sequential node
  • EPHEMERAL - temporary node
  • EPHEMERAL_SEQUENTIAL - temporary node sequence node

Ephemeral: disconnect and delete
Persistent: disconnect without deleting

When creating a Znode, set the sequence ID. a value will be appended to the Znode name. The sequence number is a monotonically increasing counter maintained by the parent node

In the distributed system, the sequence number can be used to sort all events globally, so that the client can infer the sequence of events through the sequence number

Basic operation command

Server startup and monitoring

# Service startup | shutdown | restart | status
zkServer.sh  start | stop | restart | status
#  Check the java process. QuorumPeerMain is the process of Zookeeper
jps
# Monitoring server
netstat -natl

netstat -ntpl

Client connection

Clear the screen by Ctrl + L in the terminal

# If you are connecting to the current local address, you can use zkCli.sh
zkCli.sh -server localhost:2181
# View all executable commands
help

Create node

create [-s] [-e] [-c] [-t ttl] path [data] [acl]
	[-s]: The order node defaults to No
	[-e]: The default value of temporary node is No
	[-c]: Create a container node
	[-t ttl] When creating a persistent node or a persistent sequential node, Specify survival time
	[path]: route | oloo catalogue
	[data]: Set the data stored in the path
# Create persistent node
create /pNod pNode

# Create a temporary node. The temporary node will be automatically deleted after the session is closed
# Child nodes are not allowed for temporary nodes
create -e /eNode eNode

# Create order node
# Sequential node, which allows repeated creation of nodes. By default, a globally unique sequential ID will be added after the node
create /lock
create -s /lock/getId
create -e -s /lock/getId2

# Create container node
# Multiple child nodes are stored under the container node. When the child nodes are deleted, the container node is automatically deleted
# After the child node is deleted, the node znode.container.checkIntervalMs can be deleted directly if there are no child nodes within one minute through the timer
create -c /cnode cnode
create -c /cnode/child01 child01
create -c /cnode/child02 child02
delete /cnode/child01
delete /cnode/child02
get /cnode

View node

ls [-s] [-w] [-R] path
	[-s]: Display statistics
	[-w]: View event information
	[-R]: Show recursive directories 
get [-s] [-w] path
	[-s]: Display statistics
	[-w]: Get event information
get -s /pNod

pNode # Obtained node data
cZxid = 0x600000010 # Transaction Id of the creation node
ctime = Tue Apr 21 16:23:31 CST 2020 # Creation time
mZxid = 0x600000010 # Modified transaction ID
mtime = Tue Apr 21 16:23:31 CST 2020 # Modification time
pZxid = 0x600000010 
cversion = 0 # Current version information
dataVersion = 0 # Data version information
aclVersion = 0 # Permission version information
ephemeralOwner = 0x0 # Session ID of temporary node
dataLength = 5 # Data length
numChildren = 0 # Number of child nodes

Modify node data

set [-s] [-v version] path data
	[-s]: The setting process displays the status information of the node
	[-v]: use CAS Set data, User country song, have access to stat from dataVersion Version found in
set -v 0 /node2 node3
set  /node2 node3

Delete node

delete [-v version] path
	-v: When concurrent, Delete with optimistic lock
deleteall /pNod

Binding event

addWatch [-m mode] path 
# optional mode is one of [PERSISTENT, PERSISTENT_RECURSIVE] - default is PERSISTENT_RECURSIVE

The default mode is to bind all events of the current node and child nodes:

Common events are:
NodeDataChanged: Node data change event
NodeChildrenChanged: Child node number change event
NodeCreated: Node creation time
addWatch -m PERSISTENT /cNode  

If for data acquisition: The general binding is NodeDataChanged
 If you want to get a list of child nodes: The general binding is NodeChildrenChanged

Tags: Java Zookeeper

Posted on Mon, 27 Sep 2021 23:23:48 -0400 by lucianoes