etcd with you

What is etcd?

etcd is a Go Language written for high performance key-value (K-V) storage, service discovery, load balancing, distributed locks, configuration management scenarios under distributed systems, similar to Java zookeeper.Based on Raft protocol, data consistency can be guaranteed.

Official Address

[etcd.io] https://etcd.io

[github.com]https://github.com/etcd-io/etcd

Installation of etcd

There are two ways to install it, either by compiled binary or source code. I'm using a binary installation here, and I haven't tried another source installation.

In addition, the installation here is not a cluster installation, using a single node, only some basic operations.

1. Binary Installation

  1. Click on the address below to select your own system and download different zip packages, such as MacOs download etcd-v3.4.4-darwin-amd64.zip

[https://etcd.io/docs/v3.4.0/integrations/] https://etcd.io/docs/v3.4.0/integrations/

  1. Unzip the downloaded compressed package as follows
 $ ls ~/Applications/etcd
Documentation       README-etcdctl.md   README.md           READMEv2-etcdctl.md etcd     etcdctl
  1. In the above file, etcd is the server; etcdctl is the client, which integrates some common Api s.

2. Source Installation

Because it's written by Go, you can download it directly using go get https://github.com/etcd-io/etcd, but this etcd package is a bit large, if the network is not good, it may not be downloaded, I haven't finished it yet.

So if the download doesn't work, try downloading it in github's zip package, or if it doesn't work, use domestic download acceleration.

I found one online [ https://www.newbe.pro/Mirrors/Mirrors-Etcd/] https://www.newbe.pro/Mirrors/Mirrors-Etcd/, download the package directly from here and put it under your Gopath.

Next, go under the etcd directory and execute the. /build build build.

etcd startup

Either way, to start etcd, find the file etcd and execute it. By default, on port 2379 of your computer, if you want to open it on the public network, specify the following parameters.

$ ~/Applications/etcd/etcd  --advertise-client-urls="https://123.33.44.34:2379"  --listen-client-urls="https://123.33.44.34:2379"

Start locally as follows

$ ~/Applications/etcd/etcd
[WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
2020-03-11 12:55:38.289362 I | etcdmain: etcd Version: 3.4.4
2020-03-11 12:55:38.289461 I | etcdmain: Git SHA: c65a9e2dd
2020-03-11 12:55:38.289468 I | etcdmain: Go Version: go1.12.12
2020-03-11 12:55:38.289476 I | etcdmain: Go OS/Arch: darwin/amd64
2020-03-11 12:55:38.289482 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4
2020-03-11 12:55:38.289492 N | etcdmain: failed to detect default host (default host not supported on darwin_amd64)
2020-03-11 12:55:38.289500 W | etcdmain: no data-dir provided, using default data-dir ./default.etcd
[WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
2020-03-11 12:55:38.292027 I | embed: name = default
2020-03-11 12:55:38.292038 I | embed: data dir = default.etcd
2020-03-11 12:55:38.292045 I | embed: member dir = default.etcd/member
2020-03-11 12:55:38.292050 I | embed: heartbeat = 100ms
2020-03-11 12:55:38.292054 I | embed: election = 1000ms
2020-03-11 12:55:38.292058 I | embed: snapshot count = 100000
2020-03-11 12:55:38.292079 I | embed: advertise client URLs = http://localhost:2379
2020-03-11 12:55:38.407474 I | etcdserver: starting member 8e9e05c52164694d in cluster cdf818194e3a8c32

Okay, if you see the output above, the server will start. If you want to test it, you can use etcdctl to test it.

etcdctl put  "task/task1" "task task"
OK

etcdctl get  "task/task1"
task/task1
task task

Code Operation

Here we use Go to do some simple operations on etcd, go clients can find it on the official website [ https://etcd.io/docs/v3.4.0/integrations/] https://etcd.io/docs/v3.4.0/integrations/

I'm using the clientv3 package here. When this package was installed from the source code above, the downloaded package included both the server and the client code.

Get operation

package main

import (
    "context"
    "fmt"
    "go.etcd.io/etcd/clientv3"
    "time"
)

func main(){
    var (
        config clientv3.Config
        client *clientv3.Client
        kv clientv3.KV
        resp *clientv3.GetResponse
        err error
    )
    config = clientv3.Config{
        Endpoints:[]string{"localhost:2379"},
        DialTimeout:5 * time.Second,
    }

    if client,err = clientv3.New(config); err != nil {
        fmt.Println(err)
        return
    }

    kv = clientv3.NewKV(client)

    if resp,  err = kv.Get(context.TODO(),"cron/jobs/",clientv3.WithPrefix()); err != nil {
        fmt.Println(err)
        return
    }
    for k,v := range resp.Kvs {
        fmt.Println(k,v,string(v.Key), string(v.Value))
    }
}

Renewal operation

package main

import (
    "context"
    "fmt"
    "go.etcd.io/etcd/clientv3"
    "time"
)

func main(){
    var (
        config clientv3.Config
        client *clientv3.Client
        Lease clientv3.Lease
        leaseGrantResp *clientv3.LeaseGrantResponse
        leaseId clientv3.LeaseID
        leaseKeepResp *clientv3.LeaseKeepAliveResponse
        leaseKeepRespChan <-chan *clientv3.LeaseKeepAliveResponse
        getResp *clientv3.GetResponse
        putResp *clientv3.PutResponse

        kv clientv3.KV
        err error
    )
    config = clientv3.Config{
        Endpoints:[]string{"localhost:2379"},
        DialTimeout:5 * time.Second,
    }

    if client,err = clientv3.New(config); err != nil {
        fmt.Println(err)
        return
    }

    Lease = clientv3.NewLease(client)

    if leaseGrantResp, err = Lease.Grant(context.TODO(), 10); err != nil {
        fmt.Println(err)
    }

    leaseId = leaseGrantResp.ID

    ctx, _ := context.WithTimeout(context.TODO(), 10 * time.Second) //Stop renewing for 10 seconds

    if leaseKeepRespChan, err = Lease.KeepAlive(ctx,leaseId); err != nil {
        fmt.Println(err)
    }

    //Start a contract for renewal
    go func(){
        for {
            select {
               case leaseKeepResp  = <-leaseKeepRespChan:
                if leaseKeepResp == nil{
                    fmt.Println("Lease invalidated")
                    goto END
                } else {
                    fmt.Println("Received renewal", leaseKeepResp)
                }
            }
        }
        END:
    }()

    kv = clientv3.NewKV(client)

    if putResp ,err = kv.Put(context.TODO(),"cron/jobs/job3","job3",clientv3.WithLease(leaseId)); err != nil {
        fmt.Println(err)
    } else {
        fmt.Println("Write Successful", putResp.Header.Revision)
    }

    for {
        if getResp, err = kv.Get(context.TODO(),"cron/jobs/job3"); err != nil {
            fmt.Println(err)
        }

        if getResp.Count != 0 {
            fmt.Println(getResp.Kvs)
        } else {
            fmt.Println(" Out of date")
            break
        }

        time.Sleep(2 * time.Second)
    }


}

Op Operation

package main

import (
    "context"
    "fmt"
    "go.etcd.io/etcd/clientv3"
    "time"
)

func main(){
    var (
        config clientv3.Config
        client *clientv3.Client
        opPut clientv3.Op
        opGet clientv3.Op
        opResp  clientv3.OpResponse
        kv clientv3.KV

        err error
    )
    config = clientv3.Config{
        Endpoints:[]string{"localhost:2379"},
        DialTimeout:5 * time.Second,
    }

    if client,err = clientv3.New(config); err != nil {
        fmt.Println(err)
        return
    }

    kv = clientv3.NewKV(client)

    opPut = clientv3.OpPut("cron/jobs/job3","job3333")

    if opResp, err  = kv.Do(context.TODO(),opPut); err != nil {
        fmt.Println(err)
        return
    }

    fmt.Println("Obtain revision", opResp.Put().Header.Revision)


    opGet = clientv3.OpGet("cron/jobs/job3")
    if opResp ,err = kv.Do(context.TODO(),opGet); err != nil {
        fmt.Println(err)
        return
    }

    if len(opResp.Get().Kvs) != 0 {
        fmt.Println("Get value:", string(opResp.Get().Kvs[0].Value))
    }

}

Simple Distributed Lock

package main

import (
    "context"
    "fmt"
    "go.etcd.io/etcd/clientv3"
    "time"
)

//Simple Distributed Lock

func main(){
    var (
        config clientv3.Config
        client *clientv3.Client
        /*opPut clientv3.Op
        opGet clientv3.Op
        opResp  clientv3.OpResponse*/
        kv clientv3.KV
        txn clientv3.Txn
        txnResp *clientv3.TxnResponse

        Lease clientv3.Lease
        leaseGrantResp *clientv3.LeaseGrantResponse
        leaseId clientv3.LeaseID
        leaseKeepResp *clientv3.LeaseKeepAliveResponse
        leaseKeepRespChan <-chan *clientv3.LeaseKeepAliveResponse

        ctx context.Context
        cancelFunc context.CancelFunc
        err error
    )
    config = clientv3.Config{
        Endpoints:[]string{"localhost:2379"},
        DialTimeout:5 * time.Second,
    }

    if client,err = clientv3.New(config); err != nil {
        fmt.Println(err)
        return
    }

    //Lease operation
    Lease = clientv3.NewLease(client)

    if leaseGrantResp, err = Lease.Grant(context.TODO(), 5); err != nil {
        fmt.Println(err)
    }

    leaseId = leaseGrantResp.ID

    ctx, cancelFunc = context.WithCancel(context.TODO())

    defer cancelFunc()
    defer Lease.Revoke(context.TODO(),leaseId)

    if leaseKeepRespChan, err = Lease.KeepAlive(ctx,leaseId); err != nil {
        fmt.Println(err)
    }

    //Start a contract for renewal
    go func(){
        for {
            select {
            case leaseKeepResp  = <-leaseKeepRespChan:
                if leaseKeepResp == nil{
                    fmt.Println("Lease invalidated")
                    goto END
                } else {
                    fmt.Println("Received renewal", leaseKeepResp)
                }
            }
        }
    END:
    }()



    kv = clientv3.NewKV(client)

    //Open Transaction
    txn = kv.Txn(context.TODO())

    txn.If(clientv3.Compare(clientv3.CreateRevision("cron/lock/job3"),"=", 0)).
        Then(clientv3.OpPut("cron/lock/job3","123",clientv3.WithLease(leaseId))).
        Else(clientv3.OpGet("cron/lock/job3"))

    if txnResp,err  = txn.Commit(); err != nil {
        fmt.Println(err)
        return
    }

    if !txnResp.Succeeded {
        fmt.Println("OOH,Lock occupied:", string(txnResp.Responses[0].GetResponseRange().Kvs[0].Value))
        return
    } else {
        fmt.Println("Grabbed")
    }

    fmt.Println("Processing business")
    time.Sleep( 10 * time.Second)



}

The above is to record my basic introduction and some common operations in learning etcd. I hope you like it and look forward to some progress.

Tags: Go github network Java Zookeeper

Posted on Wed, 11 Mar 2020 13:50:39 -0400 by Wozzr