Middlewares for http clients

[drscre]

go-kit adopts an excellent concept of middlewares for endpoints.

I would suggest to introduce a similar concept for http transport clients.

The problem with current approach is that kit transport http client is using *http.Client, not an interface. So i cannot pass a wrapped http client.

It is possible to wrap http client's transport, since it is a http.RoundTripper interface. But this interface imposes a limitation: it says that round tripper should not modify request params. So i cannot create a round tripper middleware which, for example, adds some request headers.
Also, round tripper will be called for every request in chain of redirects.

Currently similar functionality can be achieved by using ClientBefore() and ClientAfter() options, but it is not very convenient. Sometimes you have to write two or more functions and pass intermediate values via context.

Consider an example of logging external http requests including body params.
You have to extract params before doing the request and replace the body, because after request the body will be already read.
So you will have to use context to pass request params to the logging func:

func CollectRequestParams(ctx context.Context, req *http.Request) context.Context {
   params = extractRequestParams(req)
   return context.WithValue(ctx, "_request_params_", params)
}

func LogRequest(ctx context.Context, resp *http.Response) context.Context {
   params = ctx.Value("_request_params")
   // do logging
}

Also there is no way to log transport error.

If we had some interface for http clients (say, HttpClient), it would be much more convenient to write a single middleware for this:

func LogRequestMiddleware(client HttpClient) HttpClient {
   return func(req *http.Request) (http.Response, error) {
        params = readBody(req)

        resp, err := client.Do(req)

        // do logging, respect er
       return resp, err      
   }
}

Another example can be recording an external segment with Newrelic:

func NewrelicMiddleware(client HttpClient) HttpClient {
  return func(req *http.Request) (http.Response, error) {
    segment := newrelic.StartExternalSegment(txn, request)

    resp, err := client.Do(request)

    if err == nil {
      segment.Response = resp
    }
    segment.End()
    return resp, err
  }
}

I cannot use a round tripper middleware for this, because newrelic.StartExternalSegment() adds some headers to request therefore modifying the request.
And with before & after func i will have to pass started segment via context again.

The similar concept was adopted in twirp framework:
twitchtv/twirp#12

[peterbourgon]

No objection. Please create a PR which defines in package transport/http

type HTTPClient interface {
    Do(*http.Request) (http.Response, error)
}

and accepts this interface in place of *http.Client throughout the package.