Play with Golang's channel and implement PubSub mode in 200 lines of code

In the language of Golang, channel is naturally suitable to be the intermediary. Let's implement the tool class EventBus step by step according to the PubSub pattern

First, let's define some basic types and core operations.

//EventID is the unique identifier of the Event type EventID int64 //Event type Event interface { ID() EventID } //EventHandler type EventHandler interface { OnEvent(ctx context.Context, event Event) error CanAutoRetry(err error) bool } // JobStatus holds information related to a job status type JobStatus struct { RunAt time.Time FinishedAt time.Time Err error } //EventBus ... type EventBus struct {} func (eb *EventBus) Subscribe(eventID EventID, handlers ...EventHandler) { } func (eb *EventBus) Unsubscribe(eventID EventID, handlers ...EventHandler) { } func (eb *EventBus) Publish(evt Event) <-chan JobStatus { }

First, consumers need to Subscribe to related topics through Subscribe. The key point is to maintain the subscribed consumers according to the EventID. Naturally, we think of the map. We choose to use handlers map[EventID][]EventHandler for maintenance. Considering the concurrency problem, we also need to add a lock.

//Subscribe ... func (eb *EventBus) Subscribe(eventID EventID, handlers ...EventHandler) { eb.mu.Lock() defer eb.mu.Unlock() eb.handlers[eventID] = append(eb.handlers[eventID], handlers...) }

The implementation here is relatively simple, without considering the problem of repeated subscriptions by one consumer, which is left to the user to deal with. (however, why should the same consumer call subcribe multiple times to subscribe to the same topic? It feels like writing a bug.)

The following is the core Publish function. On the one hand, a channel (preferably with a buffer) must be required to transmit Event data. On the other hand, in order to ensure performance, some resident processes are required to listen to messages and start relevant consumers. The following is the relevant code (logs, error handling, etc. are added in the full version of the code. For the sake of showing the key points, they are omitted for the time being)

func (eb *EventBus) Start() { if eb.started { return } for i := 0; i < eb.eventWorkers; i++ { eb.wg.Add(1) go eb.eventWorker(eb.eventJobQueue) } eb.started = true } func (eb *EventBus) eventWorker(jobQueue <-chan EventJob) { loop: for { select { case job := <-jobQueue: jobStatus := JobStatus{ RunAt: time.Now(), } ctx, cancel := context.WithTimeout(context.Background(), eb.timeout) g, _ := errgroup.WithContext(ctx) for index := range job.handlers { handler := job.handlers[index] g.Go(func() error { return eb.runHandler(ctx, handler, job.event) }) } jobStatus.Err = g.Wait() jobStatus.FinishedAt = time.Now() select { case job.resultChan <- jobStatus: default: } cancel() } } }

After making the above preparations, the following is the real Publish code.

// EventJob ... type EventJob struct { event Event handlers []EventHandler resultChan chan JobStatus } //Publish ... func (eb *EventBus) Publish(evt Event) <-chan JobStatus { eb.mu.RLock() defer eb.mu.RUnlock() if ehs, ok := eb.handlers[evt.ID()]; ok { handlers := make([]EventHandler, len(ehs)) copy(handlers, ehs) //A snapshot of consumers at that time job := EventJob{ event: evt, handlers: handlers, resultChan: make(chan JobStatus, 1), } var jobQueue = eb.eventJobQueue select { case jobQueue <- job: default: } return job.resultChan } else { err := fmt.Errorf("no handlers for event(%d)", evt.ID()) resultChan := make(chan JobStatus, 1) resultChan <- JobStatus{ Err: err, } return resultChan } }

There is no way to get the relevant consumers directly from the handlers according to the ID in the eventWorker. On the one hand, it is to make the eventWorker more general, and on the other hand, it is also to reduce the blocking caused by lock operation.

So far, we have disassembled the core code one by one. For the complete code, please refer to the code in the channelx project event_bus.go

Tool classes without examples are incomplete. An example is provided below.

First define an event, where the id is defined as private, and then force it to be specified in the constructor.

const ExampleEventID channelx.EventID = 1 type ExampleEvent struct { id channelx.EventID } func NewExampleEvent() ExampleEvent { return ExampleEvent } func (evt ExampleEvent) ID() channelx.EventID { return evt.id }

Next is the event handler. According to the actual needs, check whether the received event is a subscribed event in OnEvent and whether the received event structure can be converted to a specific type. After defense programming, you can process the event logic.

type ExampleHandler struct { logger channelx.Logger } func NewExampleHandler(logger channelx.Logger) *ExampleHandler { return &ExampleHandler{ logger: logger, } } func (h ExampleHandler) Logger() channelx.Logger{ return h.logger } func (h ExampleHandler) CanAutoRetry(err error) bool { return false } func (h ExampleHandler) OnEvent(ctx context.Context, event channelx.Event) error { if event.ID() != ExampleEventID { return fmt.Errorf("subscribe wrong event(%d)", event.ID()) } _, ok := event.(ExampleEvent) if !ok { return fmt.Errorf("failed to convert received event to ExampleEvent") } // handle the event here h.Logger().Infof("event handled") return nil }

Finally, the start of EventBus, the subscription and publication of events.

eventBus := channelx.NewEventBus(logger, "test", 4,4,2, time.Second, 5 * time.Second) eventBus.Start() handler := NewExampleHandler(logger) eventBus.Subscribe(ExampleEventID, handler) eventBus.Publish(NewExampleEvent())

I also wrote some articles about the use of channel before,

• How to make the channel of Golang as smooth as the stream of nodejs
• How to use Golang's channel to realize batch processing of messages
• How to play Golang's Channel out of async and await's feel
• The next time you want to write a concurrent processing in Golang, use this template. That's right!

The lightweight util s implemented inside are open source channelx , you are welcome to review. If you have any tools you like, you are welcome to like them or star:)

21 September 2021, 18:44 | Views: 8093