Context propagation over HTTP in Go

https://medium.com/@rakyll/context-propagation-over-http-in-go-d4540996e9b0

Context propagation over HTTP in Go

Go 1.7 introduced a built-in context type a while ago. In systems, context can be used pass request-scoped metadata such as a request ID among different functions, threads and even processes.

Go originally introduced the context package to the standard library to unify the context propagation inside the same process. So the entire library- and framework-space can work against the standard context and we can avoid fragmentation. Before the introduction of the package, each framework was inventing their own context type and no two contexts were compatible with each other. This was resulting in a situation where propagating the current context was becoming hard without writing glue code.

Although introducing a common context propagation mechanism was useful to unify the cases inside the same process, Go context package doesn’t provide any support on the wire. As mentioned above, in networking systems, context should be propagated on the wire among different processes. For example, in a multi-service architecture, a request is going through multiple processes (several microservices, message queues, databases) until a user request is served. Being able to propagate the context among the processes is important for the lower ends of the stack to work with the right context.

If you want to propagate the current context over HTTP, you need to serialize the context yourself. Similarly on the receiving end, you need to parse the incoming request and put the values into the current context. Assume, we want to propagate the request ID in the context.

package request

import "context"

// WithID puts the request ID into the current context.
func WithID(ctx context.Context, id string) context.Context {
 return context.WithValue(ctx, contextIDKey, id)
}

// IDFromContext returns the request ID from the context.
// A zero ID is returned if there are no idenfiers in the
// current context.
func IDFromContext(ctx context.Context) string {
 v := ctx.Value(contextIDKey)
 if v == nil {
  return ""
 }
 return v.(string)
}

type contextIDType struct{}

var contextIDKey = &contextIDType{}

// ...

WithID allows us to read and IDFromContext allows us to put the request ID in a given context. As soon as we want to cross the process boundaries, we need to do manual work to put the context on wire. As well as, parse it from wire to a context on the receiving end.

On HTTP, we can dump the request ID as a header. Most context-aware metadata can be propagate as a header. Some transport layers may not provide headers or headers may not meet the requirements of the propagated data (e.g. due to size limitations and lack of encryption). In such cases, it is up to the implementation to sort out how to propagate the context.

HTTP Propagation

There is no automatic way to put the context into an HTTP request or vice versa. There is also no way to dump the entire context given you cannot iterate context values.

const requestIDHeader = "request-id"

// Transport serializes the request context into request headers.
type Transport struct {
 // Base is the actual round tripper that makes the request.
 // http.DefaultTransport is used if none is set.
 Base http.RoundTripper
}

// RoundTrip converts request context into headers
// and makes the request.
func (t *Transport) RoundTrip(r *http.Request) (*http.Response, error) {
 r = cloneReq(r) // per RoundTrip interface enforces

rid := request.IDFromContext(r.Context())
 if rid != "" {
  r.Header.Add(requestIDHeader, rid)
 }

 base := t.Base
 if base == nil {
  base = http.DefaultTransport
 }
 return base.RoundTrip(r)
}

In Transport above, request ID if exists in the request context, will be propagated as the “request-id” header.

Similarly handlers can parse the incoming request to put the “request-id” into the request context.

// Handler deserializes the request context from request headers.
type Handler struct {
 // Base is the actual handler to call once deserialization
 // of the context is done.
 Base http.Handler
}

func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 rid := r.Header.Get(requestIDHeader)
 if rid != "" {
  r = r.WithContext(request.WithID(r.Context(), rid))
 }
 h.Base.ServeHTTP(w, r)
}

In order to keep propagating the context, make sure you are passing the current context to the outgoing requests from your handlers. The incoming context will be propagated to https://endpoint request.

http.Handle("/", &Handler{
    Base: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        req, _ := http.NewRequest("GET", "https://endpoint", nil)
        // Propagate the incoming context.
        req = req.WithContext(r.Context()) 

        // Make the request.
    }),
})