Extend Envoy proxy with Golang - WASM filter

         Envoy is an open source service agent. Envoy is designed for cloud native applications. Envoy has many features, such as connection pool, retry mechanism, TLS management, compression, health check, fault injection, rate limit, authorization, etc. These functions are built-in   http filter   Implemented. Now, we introduce a special filter- WASM filter.

         This article will not explain what WASM is, so we will not introduce WASM too much, but add relevant resource links at the end of the article.

Why use WASM filters

         At Trendyol technology. We use Istio as the service grid. Our team (DevX) is responsible for improving the developer experience by developing applications that meet the common requirements of microservices, such as caching, authorization, rate limiting, cross cluster service discovery, etc.

         Since we are already using Istio, why not take advantage of the scalability of Envoy Proxy?

         Our case demo is to obtain the JWT token used to identify the microservice of the microservice application. When we want to avoid each team writing the same code in different languages, we can create a WASM Filter and inject it into envy proxies to achieve the above functions.

Advantages of WASM filter:

  1. It allows you to write code in any language that supports WASM
  2. Dynamically load code into Envoy
  3. WASM code is isolated from Envoy, so crashes in WASM do not affect Envoy  


         In the Envoy Proxy, there are worker threads dedicated to processing incoming requests. Each worker thread has its own WASM VM(WASM virtual machine). Therefore, if you want to write time-based operation code, it will work separately for each thread.  

         In Envoy Proxy, each worker thread is isolated from each other and has one or more wasm VMS. There is also a concept called WASM Service, which is used for inter thread communication and data sharing (we are not involved in this).

Write WASM with Go   

         We will use tetratelabs/proxy-wasm-go-sdk Write WASM in Go. We still need TinyGo Build our Go code as WASM.

         Our case demo is very simple. We write a code to the JWT API every 15 seconds   Send a request. It extracts the authorization header and sets its value as a global variable, and puts the value in the response header of each request. We also set the "hello from wasm" value to another header named "x-wasm-filter".

         In the OnTick function, we make http calls to the service called cluster in Envoy.   

package main

import (
const tickMilliseconds uint32 = 15000

var authHeader string

func main() {

type vmContext struct {
	// Embed the default VM context here,
	// so that we don't need to reimplement all the methods.

// Override types.DefaultVMContext.
func (*vmContext) NewPluginContext(contextID uint32) types.PluginContext {
	return &pluginContext{contextID: contextID}

type pluginContext struct {
	// Embed the default plugin context here,
	// so that we don't need to reimplement all the methods.
	contextID uint32
	callBack  func(numHeaders, bodySize, numTrailers int)

// Override types.DefaultPluginContext.
func (*pluginContext) NewHttpContext(contextID uint32) types.HttpContext {
	return &httpAuthRandom{contextID: contextID}

type httpAuthRandom struct {
	// Embed the default http context here,
	// so that we don't need to reimplement all the methods.
	contextID uint32

// Override types.DefaultPluginContext.
func (ctx *pluginContext) OnPluginStart(pluginConfigurationSize int) types.OnPluginStartStatus {
	if err := proxywasm.SetTickPeriodMilliSeconds(tickMilliseconds); err != nil {
		proxywasm.LogCriticalf("failed to set tick period: %v", err)
		return types.OnPluginStartStatusFailed
	proxywasm.LogInfof("set tick period milliseconds: %d", tickMilliseconds)
	ctx.callBack = func(numHeaders, bodySize, numTrailers int) {
		respHeaders, _ := proxywasm.GetHttpCallResponseHeaders()
		proxywasm.LogInfof("respHeaders: %v", respHeaders)

		for _, headerPairs := range respHeaders {
			if headerPairs[0] == "authorization" {
				authHeader = headerPairs[1]
	return types.OnPluginStartStatusOK

func (ctx *httpAuthRandom) OnHttpResponseHeaders(int, bool) types.Action {
	proxywasm.AddHttpResponseHeader("x-wasm-filter", "hello from wasm")
	proxywasm.AddHttpResponseHeader("x-auth", authHeader)

	return types.ActionContinue

// Override types.DefaultPluginContext.
func (ctx *pluginContext) OnTick() {
	hs := [][2]string{
		{":method", "GET"}, {":authority", "some_authority"}, {":path", "/auth"}, {"accept", "*/*"},
	if _, err := proxywasm.DispatchHttpCall("my_custom_svc", hs, nil, nil, 5000, ctx.callBack); err != nil {
		proxywasm.LogCriticalf("dispatch httpcall failed: %v", err)

         Let's compile the Go code into   WASM:

        tinygo build -o optimized.wasm -scheduler=none -target=wasi ./main.go

         Now we need to configure the Envoy proxy to use the WASM filter for incoming requests. We will define a routing rule and a WASM filter for our WASM code, and we will also define a cluster representing our services.

# cat /etc/envoy/envoy.yaml
  access_log_path: /tmp/admin_access.log
      protocol: TCP
      port_value: 9901
  - name: listener_0
        protocol: TCP
        port_value: 10000
    - filters:
      - name: envoy.filters.network.http_connection_manager
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
            name: local_route
            - name: local_service
              domains: ["*"]
              - match:
                  prefix: "/"
                  cluster: my_custom_svc
          - name: envoy.filters.http.wasm
              "@type": type.googleapis.com/udpa.type.v1.TypedStruct
              type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
                  name: "my_plugin"
                  root_id: "my_root_id"
                    "@type": "type.googleapis.com/google.protobuf.StringValue"
                    value: |
                    runtime: "envoy.wasm.runtime.v8"
                    vm_id: "my_vm_id"
                        filename: "/etc/envoy/optimized.wasm"
                    configuration: { }
          - name: envoy.filters.http.router
            typed_config: { }
  - name: my_custom_svc
    connect_timeout: 30s
    type: static
    lb_policy: ROUND_ROBIN
      cluster_name: service1
      - lb_endpoints:
        - endpoint:
                port_value: 8080

         I put all the files in the same directory. Now let's use   Docker runs Envoy agent:

        docker run -it  --rm -v "$PWD"/envoy.yaml:/etc/envoy/envoy.yaml -v "$PWD"/optimized.wasm:/etc/envoy/optimized.wasm -p 9901:9901 -p 10000:10000 envoyproxy/envoy:v1.17.0

         As we can see from the log, our WASM filter starts working and updates the JWT API every 15 seconds   Send request.


         Now let's send a request to Envoy Proxy. We configure Envoy to listen for incoming requests from port 10000 and start the container using port mapping. So we can send a request to localhost:10000:


         In the response header, we can see the values of "x-wasm-filter: hello from wasm" and "x-auth".

         Thanks for reading. I hope this article will let you know how and why to use WASM in Envoy Proxy.

reference resources  

1.  https://github.com/mstrYoda/envoy-proxy-wasm-filter-golang

2. https://github.com/tetratelabs/proxy-wasm-go-sdk

3. https://medium.com/trendyol-tech/extending-envoy-proxy-wasm-filter-with-golang-9080017f28ea

Tags: Go istio WebAssembly envoy

Posted on Sun, 28 Nov 2021 01:13:54 -0500 by Shiki