Optimally Configuring ASP.NET Core HttpClientFactory

Update (20-08-2018): Steve Gordon kindly suggested a further optimisation to use ConfigureHttpClient. I’ve updated the code below to reflect this.

In this post, I’m going to show how to optimally configure a HttpClient using the new HttpClientFactory API in ASP.NET Core 2.1. If you haven’t already I recommend reading Steve Gordons series of blog posts on the subject since this post builds on that knowledge. You should also read his post about Correlation ID’s as I’m making use of that library in this post. The main aims of the code in this post are to:

  1. Use the HttpClientFactory typed client, I don’t know why the ASP.NET team bothered to provide three ways to register a client, the typed client is the one to use. It provides type safety and removes the need for magic strings.
  2. Enable GZIP decompression of responses for better performance. Interestingly, the HttpClient and ASP.NET Core does not support compression of GZIP requests, only responses. Doing some searching online some time ago suggests that this is an optimisation that is not very common at all, I thought this was pretty unbelievable at the time.
  3. The HttpClient should time out after the server does not respond after a set amount of time.
  4. The HttpClient should retry requests which fail due to transient errors.
  5. The HttpClient should stop performing new requests for a period of time when a consequtive number of requests fail using the circuit breaker pattern. Failing fast in this way helps to protect an API or database that may be under high load and means the client gets a failed response quickly rather than waiting for a timeout.
  6. The Url, timeout, retry and circuit breaker settings should be configurable from the appsettings.json file.
  7. The HttpClient should send a User-Agent HTTP header telling the server the name and version of the calling application. If the server is logging this information, this can be useful for debugging purposes.
  8. The X-Correlation-ID HTTP header from the response should be passed on to the request made using the HttpClient. This would make it easy to correlate a request accross multiple applications.

Usage Example

It doesn’t really matter what the typed client HttpClient looks like, that’s not what we’re talking about but I include it for context.

Here is how we register the typed client above with our dependency injection container. All of the meat lives in these three methods. AddCorrelationId adds a middleware written by Steve Gordon to handle Correlation ID’s. AddPolicies registers a policy registry and the policies themselves (A policy is Polly’s way of specifying how you want to deal with errors e.g. using retries, circuit breaker pattern etc.). Finally, we add the typed HttpClient but with configuration options, so we can configure it’s settings from appsettings.json.

The appsettings.json file below contains the base address for the endpoint we want to connect to, a timeout value of thirty seconds is used if the server is taking too long to respond and policy settings for retries and the circuit breaker.

The retry settings state that after a first failed request, another three attempts will be made (this means you can get up to four requests). There will be an exponentially longer backoff or delay between each request. The first retry request will occur after two seconds, the second after another four seconds and the third occurs after another eight seconds.

The circuit breaker states that it will allow 12 consequitive failed requests before breaking the circuit and throwing CircuitBrokenException for every attempted request. The circuit will be broken for thirty seconds.

Generally, my advice is when allowing a high number of exceptions before breaking, use a longer duration of break. When allowing a lower numer of exceptions before breaking, keep the duration of break small. Another possibility I’ve not tried is to combine these two scenarios, so you have two circuit breakers. The curcuit breaker with the lower limit would kick in first but only break the circuit for a short time, if exceptions are no longer thrown, then things go back to normal quickly. If exceptions continue to be thrown, then the other circuit breaker with a longer duration of break would kick in and the circuit would be broken for a longer period of time.

You can of course play with these numbers, what you set them to will depend on your application.

Configuring Polly Policies

Below is the implementation for AddPollyPolicies. It starts by setting up and reading a configuration section in our appsettings.json file of type PolicyOptions. Then adds the Polly PolicyRegistry which is where Polly stores it’s policies. Finally, we add a retry and circuit breaker policy and configure them using the settings we’ve read from the PolicyOptions.

Notice that each policy is using the HandleTransientHttpError method which tells Polly when to apply the retry and cicuit breakers. One important question is, what is a transient HTTP error according to Polly? Well, looking at the source code in the Polly.Extensions.Http GitHub repository, it looks like they consider any of the below as transient errors:

  1. Any HttpRequestException thrown. This can happen when the server is down.
  2. A response with a status code of 408 Request Timeout.
  3. A response with a status code of 500 or above.

Configuring HttpClient

Finally, we can get down to configuring our HttpClient itself. The AddHttpClient method starts by binding the TClientOptions type to a configuration section in appsettings.json. TClientOptions is a derived type of HttpClientOptions which just contains a base address and timeout value. I’ll come back to CorrelationIdDelegatingHandler and UserAgentDelegatingHandler.

We set the HttpClientHandler to be DefaultHttpClientHandler. This type just enables GZIP and Deflate compression. Brotli support is being added soon, so watch out for that. Finally, we add the retry and circuit breaker policies to the HttpClient.

CorrelationIdDelegatingHandler

When I’m making a HTTP request from an API i.e. it’s an API to API call and I control both sides, I use the X-Correlation-ID HTTP header to trace requests as they move down the stack. The CorrelationIdDelegatingHandler is used to take the correlation ID for the current HTTP request and pass it down to the request made in the API to API call. The implementation is pretty simple, it’s just setting a HTTP header.

The power comes when you are using something like Application Insights, Kibana or Seq for logging. You can now take the correlation ID for a request and see the logs for it from multiple API’s or services. This is really invaluable when you are dealing with a microservices architecture.

UserAgentDelegatingHandler

It’s often useful to know something about the client that is calling your API for logging and debugging purposes. You can use the User-Agent HTTP header for this purpose.

The UserAgentDelegatingHandler just sets the User-Agent HTTP header by taking the API’s assembly name and version attributes. You need to set the Version and Product attributes in your csproj file for this to work. The name and version are then placed along with the current operating system into the User-Agent string.

Now the next time you get an error in your API, you’ll know the client application that caused it (if it’s under your control).

Sample GitHub Project

I realize that was a lot of boilerplate code to write. It was difficult to write this as more than one blog post. To aid in digestion, I’ve created a GitHub sample project with the full working code.

The sample project contains two API’s. One makes a HTTP request to the other. You can pass a query argument to decide whether the callee API will fail or not and try out the retry and circuit breaker logic. Feel free to play with the configuration in appsettings.json and see what options work best for your application.