go language - Common concurrent mode

  1. Producer consumer model

    The most common example of concurrent programming is the producer / consumer model, which improves the overall processing speed of the program by balancing the working capacity of the production thread and the consumer thread. Simply put, the producer produces some data and puts it in the queue, while the consumer fetches it from the queue. In this way, production and consumption become two asynchronous processes. When there is no data in the queue, the consumer enters the hungry waiting; when the data in the opposite queue is full, the producer faces the problem that the product backlog leads to the CPU deprivation.

//Producer
func Producer(factor int, out chan <- int) {
    for i := 0; ; i++ {
        out <- i * factor
    }
}

//Consumer
func Consumer(in <- chan int) {
    for v := range in {
        fmt.Println(v)
    }
}

func main() {
    ch := make(chan int, 64) //queue
    go Producer(3, ch) //Generate a sequence of multiples of 3
    go Producer(5, ch) //Generate a series of multiples of 5
    go Consumer(ch)    //Consumption generated queue
    
    //Ctrl + C exit
    sig := make(chan os.Signal, 1)
    signal.Notify(sig, syscall.SIGINT, syscall.SIGTERM)
    fmt.Printf("qiut (%v)\n", <-sig)
}

We have opened two Producer production lines to generate sequences of multiples of 3 and 5, respectively. There are no synchronous events between the two producers to refer to. They are concurrent. Therefore, the order of the result sequence output by consumers is uncertain. There is no problem. Producers and consumers can still work together.


2. Publish / subscribe model

The publish subscribe model is usually abbreviated to the pub/sub model. In this model, message producers become publisher s, while message consumers become subscriber s. The relationship between producers and consumers is M:N. In the traditional producer and consumer model, messages are sent to a queue, while the publish / subscribe model publishes messages to a topic.

To do this, we built a publish / subscribe model support package named pubsub:

//Package pubsub implements a simple multi-topic pub-sub library
package pubsub

import (
   "sync"
   "time"
)

type (
   subscriber chan interface{}         //Subscriber as a channel
   topicFunc  func(v interface{}) bool //Subscriber is a filter
)

type Publisher struct {
   m           sync.RWMutex             //Read write lock
   buffer      int                      //Cache size of subscription queue
   timeout     time.Duration            //Publishing timeout
   subscribers map[subscriber]topicFunc //Subscriber information
}

//Build a publisher object to set the publishing timeout and the length of the cache queue
func NewPublisher(publishTimeout time.Duration, buffer int) *Publisher {
   return &Publisher{
      buffer:      buffer,
      timeout:     publishTimeout,
      subscribers: make(map[subscriber]topicFunc),
   }
}

//Add a new subscriber to subscribe to all topics
func (p *Publisher) Subscribe() chan interface{} {
   return p.SubscribeTopic(nil)
}

//Add a new subscriber and subscribe to the filtered topics
func (p *Publisher) SubscribeTopic(topic topicFunc) chan interface{} {
   ch := make(chan interface{}, p.buffer)
   p.m.Lock()
   p.subscribers[ch] = topic
   p.m.Unlock()
   return ch
}

//Exit subscription
func (p *Publisher) Evict(sub chan interface{}) {
   p.m.Lock()
   defer p.m.Unlock()
   delete(p.subscribers, sub)
   close(sub)
}

//Publish a topic
func (p *Publisher) Publish(v interface{}) {
   p.m.RLock()
   defer p.m.Unlock()

   var wg sync.WaitGroup
   for sub, topic := range p.subscribers {
      wg.Add(1)
      go p.sendTopic(sub, topic, v, &wg)
   }
   wg.Wait()
}

//Close the publisher object and all subscriber channels at the same time
func (p *Publisher) Close() {
   p.m.Lock()
   defer p.m.Unlock()

   for sub := range p.subscribers {
      delete(p.subscribers, sub)
      close(sub)
   }
}

//Publish topic, can tolerate certain timeout
func (p *Publisher) sendTopic(
   sub subscriber, topic topicFunc, v interface{}, wg *sync.WaitGroup,
) {
   defer wg.Done()
   if topic != nil && !topic(v) {
      return
   }

   select {
      case sub <- v:
      case <-time.After(p.timeout):
   }
}

In the following example, two subscribers subscribe to all topics and topics with "golang" respectively:

import "path/to/pubsub"

func main() {
    p := pubsub.NewPublisher(100*time.Millisecond, 10)
    defer p.Close()
    
    all := p.Subscribe()
    golang := p.SubscribeTopic(func(v interface{}) bool {
        if s, ok := v.(string); ok {
            return string.Contains(s, "golane")
        }
        return false
    }
})

p.Publish("hello, world!")
p.Publish("hello, golang!")

go func() {
    for msg := range all {
        fmt.Println("all:", msg)
    }
}()

go func() {
    for msg := range golang {
        fmt.Println("golang:", msg)
    }
}()

//Exit after running for a certain time
time.Sleep(3 * time.Second)

In the publish / subscribe model, each message is delivered to multiple subscribers. Publishers usually don't know, and don't care, which subscriber is receiving a topic message. Subscribers and publishers can be added dynamically at runtime, which is a loose coupling relationship between them, which makes the complexity of the system increase with time. In real life, applications such as weather forecast can apply this kind of concurrent mode.

Tags: Go Programming

Posted on Sun, 15 Mar 2020 04:48:38 -0400 by aniket_dj