Go language core 36 (go language practice and application 12) -- learning notes

34 | concurrent security dictionary sync.Map (I)

Today we'll talk about a high-level data structure for concurrency security: sync.Map. As we all know, the dictionary type map of Go language is not concurrency safe.

Leading knowledge: the birth history of concurrent security dictionary

In other words, it is not safe for different goroutine codes to read and write the same dictionary at the same time. The dictionary value itself may be confused by these operations, and the related programs may have unpredictable problems.

Before the advent of sync.Map, if we wanted to implement a concurrent secure dictionary, we had to build it ourselves. However, this is actually not a bother. It can be easily done using sync.Mutex or sync.RWMutex, plus the native map.

There are already many libraries on the GitHub website that provide similar data structures. I also provided a relatively complete implementation of concurrent security dictionary in the second edition of Go concurrent programming practice. Its performance is better than similar data structures, because it effectively avoids the dependence on locks to a great extent.

Although there have been many reference implementations, Go language enthusiasts still hope that the Go language official can release a standard concurrent security dictionary.

After many years of suggestions and Tucao, Go language official finally make complaints about Go sync.Map, which was released in 2017, and formally added the sync.Map dictionary type.

This dictionary type provides some common key value access operation methods and ensures the concurrency security of these operations. At the same time, its save, fetch, delete and other operations can basically be completed in a constant time. In other words, their algorithm complexity is O(1) as the map type.

In some cases, using sync. Map can significantly reduce lock contention compared with the scheme using only native map and mutex. Although sync.Map itself also uses locks, it actually avoids using locks as much as possible.

As we all know, using locks means that some concurrent operations must be forcibly serialized. This tends to degrade program performance, especially if the computer has multiple CPU cores.

Therefore, we often say that if you can operate with atoms, do not use locks, but this is very limited. After all, atoms can only support some basic data types.

No matter what scenario we use sync.Map, we should note that it is obviously different from the native map. It is only a member of the Go language standard library, not something at the language level. Because of this, the Go compiler does not perform special type checks on its keys and values.

If you have read the document of sync.Map or actually used it, you will know that the types of keys and values involved in all its methods are interface {}, that is, empty interface, which means that it can be all inclusive. Therefore, we must ensure the correctness of its key type and value type in the program.

Well, here comes the first question. Today's question is: does the concurrent security dictionary require the type of key?

The typical answer to this question is: Yes. The actual type of a key cannot be a function type, dictionary type, or slice type.

Analyze this problem. As we all know, the key type of the native Dictionary of Go language cannot be function type, dictionary type or slice type.

Because the storage medium used in the concurrent security dictionary is the native dictionary, and because the native dictionary key type it uses is also an all inclusive interface {}; Therefore, we must not operate the concurrent security dictionary with any key value whose actual type is function type, dictionary type or slice type.

Because the actual types of these key values can only be determined during program running, the Go language compiler cannot check them at compile time. Incorrect actual types of key values will certainly cause panic.

Therefore, the first thing we should do here is: we must not violate the above rules. We should explicitly check the actual type of key value every time we operate the concurrent security dictionary. This should be the case whether saving, retrieving or deleting.

Of course, it is better to concentrate these operations on the same concurrent security dictionary, and then write the check code uniformly. In addition, encapsulating the concurrent security dictionary in a structure type is often a good choice.

In short, we must ensure that the types of keys are Comparable (or decidable, etc.). If you are really unsure, you can first call the reflect.TypeOf function to get the reflection type value corresponding to a key value (that is, the value of the reflect.Type type type), and then call the Comparable method of this value to get the exact judgment result.

Knowledge expansion

Question 1: how to ensure the type correctness of keys and values in the concurrent security dictionary? (scheme I)

Simply put, you can use type assertion expressions or reflection operations to ensure their type correctness.

In order to further clarify the actual types of key values in the concurrent security dictionary, there are roughly two options.

The first is to let the concurrent security dictionary store only a specific type of key.

For example, specify that the key here can only be int type, or can only be string, or some kind of structure. Once the key type is completely determined, you can use the type assertion expression to check the key type when saving, fetching and deleting.

In general, this inspection is not cumbersome. Moreover, it is more convenient for you to encapsulate the concurrent security dictionary in a structure type. You can ask the Go language compiler to help you with type checking. See the following code:

type IntStrMap struct {
 m sync.Map
}

func (iMap *IntStrMap) Delete(key int) {
 iMap.m.Delete(key)
}

func (iMap *IntStrMap) Load(key int) (value string, ok bool) {
 v, ok := iMap.m.Load(key)
 if v != nil {
  value = v.(string)
 }
 return
}

func (iMap *IntStrMap) LoadOrStore(key int, value string) (actual string, loaded bool) {
 a, loaded := iMap.m.LoadOrStore(key, value)
 actual = a.(string)
 return
}

func (iMap *IntStrMap) Range(f func(key int, value string) bool) {
 f1 := func(key, value interface{}) bool {
  return f(key.(int), value.(string))
 }
 iMap.m.Range(f1)
}

func (iMap *IntStrMap) Store(key int, value string) {
 iMap.m.Store(key, value)
}

As shown above, I wrote a structure type named IntStrMap, which represents a concurrent security dictionary with key type int and value type string. In this structure type, there is only one field m of type sync.Map. Moreover, all methods owned by this type are very similar to those of sync.Map type.

The corresponding method names are exactly the same, and the method signatures are very similar, except that the types of parameters and results related to keys and values are different. In the method signature of IntStrMap type, it is specified that the type of key is int and the type of value is string.

Obviously, these methods don't have to do type checking when they accept keys and values. In addition, when these methods extract keys and values from m, they don't have to worry about their incorrect types, because their correctness was guaranteed by the Go language compiler when they were stored.

To sum up a little. The first scheme is suitable for cases where we can completely determine the specific types of keys and values. In this case, we can use the Go language compiler to do type checking, and use the type assertion expression as an aid, just like IntStrMap.

summary

Today we are talking about the sync.Map type, which is a concurrency safe dictionary. It provides some common key and value access operation methods, and ensures the concurrency security of these operations. At the same time, it also ensures the constant execution time of save, fetch, delete and other operations.

As with native dictionaries, concurrent security dictionaries have requirements for key types. They also cannot be function types, dictionary types, or slice types.

In addition, because the types of keys and values involved in the methods provided by the concurrent security dictionary are interface {}, we often need to check the actual types of keys and values when calling these methods.

There are roughly two schemes here. Today, we mainly mentioned the first scheme, which is to completely determine the types of keys and values during coding, and then use the compiler of Go language to check for us.

In the next article, we will mention another scheme and compare the advantages and disadvantages of the two schemes. In addition, I will continue to explore issues related to concurrent security dictionaries.

package main

import (
	"fmt"
	"sync"
)

// ConcurrentMap represents a self-made simple concurrent security dictionary.
type ConcurrentMap struct {
	m  map[interface{}]interface{}
	mu sync.RWMutex
}

func NewConcurrentMap() *ConcurrentMap {
	return &ConcurrentMap{
		m: make(map[interface{}]interface{}),
	}
}

func (cMap *ConcurrentMap) Delete(key interface{}) {
	cMap.mu.Lock()
	defer cMap.mu.Unlock()
	delete(cMap.m, key)
}

func (cMap *ConcurrentMap) Load(key interface{}) (value interface{}, ok bool) {
	cMap.mu.RLock()
	defer cMap.mu.RUnlock()
	value, ok = cMap.m[key]
	return
}

func (cMap *ConcurrentMap) LoadOrStore(key, value interface{}) (actual interface{}, loaded bool) {
	cMap.mu.Lock()
	defer cMap.mu.Unlock()
	actual, loaded = cMap.m[key]
	if loaded {
		return
	}
	cMap.m[key] = value
	actual = value
	return
}

func (cMap *ConcurrentMap) Range(f func(key, value interface{}) bool) {
	cMap.mu.RLock()
	defer cMap.mu.RUnlock()
	for k, v := range cMap.m {
		if !f(k, v) {
			break
		}
	}
}

func (cMap *ConcurrentMap) Store(key, value interface{}) {
	cMap.mu.Lock()
	defer cMap.mu.Unlock()
	cMap.m[key] = value
}

func main() {
	pairs := []struct {
		k int
		v string
	}{
		{k: 1, v: "a"},
		{k: 2, v: "b"},
		{k: 3, v: "c"},
		{k: 4, v: "d"},
	}

	// Example 1.
	{
		cMap := NewConcurrentMap()
		cMap.Store(pairs[0].k, pairs[0].v)
		cMap.Store(pairs[1].k, pairs[1].v)
		cMap.Store(pairs[2].k, pairs[2].v)
		fmt.Println("[Three pairs have been stored in the ConcurrentMap instance]")

		cMap.Range(func(key, value interface{}) bool {
			fmt.Printf("The result of an iteration in Range: %v, %v\n",
				key, value)
			return true
		})

		k0 := pairs[0].k
		v0, ok := cMap.Load(k0)
		fmt.Printf("The result of Load: %v, %v (key: %v)\n",
			v0, ok, k0)

		k3 := pairs[3].k
		v3, ok := cMap.Load(k3)
		fmt.Printf("The result of Load: %v, %v (key: %v)\n",
			v3, ok, k3)

		k2, v2 := pairs[2].k, pairs[2].v
		actual2, loaded2 := cMap.LoadOrStore(k2, v2)
		fmt.Printf("The result of LoadOrStore: %v, %v (key: %v, value: %v)\n",
			actual2, loaded2, k2, v2)
		v3 = pairs[3].v
		actual3, loaded3 := cMap.LoadOrStore(k3, v3)
		fmt.Printf("The result of LoadOrStore: %v, %v (key: %v, value: %v)\n",
			actual3, loaded3, k3, v3)

		k1 := pairs[1].k
		cMap.Delete(k1)
		fmt.Printf("[The pair with the key of %v has been removed from the ConcurrentMap instance]\n",
			k1)
		v1, ok := cMap.Load(k1)
		fmt.Printf("The result of Load: %v, %v (key: %v)\n",
			v1, ok, k1)
		v1 = pairs[1].v
		actual1, loaded1 := cMap.LoadOrStore(k1, v1)
		fmt.Printf("The result of LoadOrStore: %v, %v (key: %v, value: %v)\n",
			actual1, loaded1, k1, v1)

		cMap.Range(func(key, value interface{}) bool {
			fmt.Printf("The result of an iteration in Range: %v, %v\n",
				key, value)
			return true
		})
	}
	fmt.Println()

	// Example 2.
	{
		var sMap sync.Map
		sMap.Store(pairs[0].k, pairs[0].v)
		sMap.Store(pairs[1].k, pairs[1].v)
		sMap.Store(pairs[2].k, pairs[2].v)
		fmt.Println("[Three pairs have been stored in the sync.Map instance]")

		sMap.Range(func(key, value interface{}) bool {
			fmt.Printf("The result of an iteration in Range: %v, %v\n",
				key, value)
			return true
		})

		k0 := pairs[0].k
		v0, ok := sMap.Load(k0)
		fmt.Printf("The result of Load: %v, %v (key: %v)\n",
			v0, ok, k0)

		k3 := pairs[3].k
		v3, ok := sMap.Load(k3)
		fmt.Printf("The result of Load: %v, %v (key: %v)\n",
			v3, ok, k3)

		k2, v2 := pairs[2].k, pairs[2].v
		actual2, loaded2 := sMap.LoadOrStore(k2, v2)
		fmt.Printf("The result of LoadOrStore: %v, %v (key: %v, value: %v)\n",
			actual2, loaded2, k2, v2)
		v3 = pairs[3].v
		actual3, loaded3 := sMap.LoadOrStore(k3, v3)
		fmt.Printf("The result of LoadOrStore: %v, %v (key: %v, value: %v)\n",
			actual3, loaded3, k3, v3)

		k1 := pairs[1].k
		sMap.Delete(k1)
		fmt.Printf("[The pair with the key of %v has been removed from the sync.Map instance]\n",
			k1)
		v1, ok := sMap.Load(k1)
		fmt.Printf("The result of Load: %v, %v (key: %v)\n",
			v1, ok, k1)
		v1 = pairs[1].v
		actual1, loaded1 := sMap.LoadOrStore(k1, v1)
		fmt.Printf("The result of LoadOrStore: %v, %v (key: %v, value: %v)\n",
			actual1, loaded1, k1, v1)

		sMap.Range(func(key, value interface{}) bool {
			fmt.Printf("The result of an iteration in Range: %v, %v\n",
				key, value)
			return true
		})
	}

}

Note source code

https://github.com/MingsonZheng/go-core-demo

Tags: Go

Posted on Wed, 24 Nov 2021 19:52:13 -0500 by abhi_madhani