Handling API errors with Problem JSON

When building RESTful APIs, one question is bound to arise sooner or later: What is the best way to express errors? A good piece of advice is to first look at what the transport spec offers out of the box. In the case of HTTP, the first thing that comes to mind is, you guessed it, status codes. The idea of status codes is simple: just attach a number to every response which indicates whether the API call was successful or not, and if not, what went wrong.

HTTP status codes work fine for the most simple cases but sooner or later we will run into a situation where we hit the limits of what can be expressed with a single number: be it because no existing status code matches what we want to convey or because we want to model two distinct error conditions with the same status code. In any case, we have to make use of the response body to provide more context. The question is: what is the body supposed to look like?

The primitive approach

The naive way to go about it is to just pass an error message to the client alongside the error code.

Let's consider an example of an imaginary endpoint for creating users:

    POST /api/users
  

At least two things can go wrong:

1. The user name is already taken

2. The client sent a malformed payload.

Both error conditions could be expressed with status code 400 ("Bad Request"). With the primitive approach, the response bodies could look like this: Error 1: user name is already taken

    400 BAD REQUEST
{
    "message": "A user named 'Lucy' already exists."
}
  

Error 2: malformed input provided

    400 BAD REQUEST
{
    "message": "Field 'name' is required."
}
  

The issue with these responses is that they are not machine-readable. How is the client (a computer program) supposed to know which error is which when the response code is the same? And how should they react to either of them? A client would probably want to handle both cases differently. But right now, the client has no way to distinguish between both error cases (besides regex matching the message, but let's not go there).

Clearly, we need a more structured way to express errors. This is often the point where teams start reinventing the wheel, for example by defining error enums and whatnot. The result is then the billionth custom solution to the very same problem. Now, before inventing the billion-plus-one-th solution, we should ask ourselves: isn't there a standardized way to handle this? Thankfully, there is: Problem JSON.

What is Problem JSON?

As you might have guessed from its name, Problem JSON is simply a format for expressing errors in JSON. The format is defined in an RFC (RFC-7807) and it is dead simple. Here's an example:

    {
	"type": "https://errors.celonis.com/username-taken", ➊
	"title": "Username already taken.", ➋
	"detail": "A user named 'Lucy' already exists.", ➌
	"context": { "providedUsername": "Lucy" } ➍
}
  

A Problem JSON payload should have at least one property: the "type" field (➊). "type" is a unique identifier for the error at hand. It is typically domain-specific and it can be any type of URI. In this example, the "type" could be a link to a (human-readable) web page with detailed information about this specific error and how to fix it. But it could as well be any other URI, e.g., "urn:problems:username-taken". What matters is that the error type uniquely identifies the error so that the client can make sense of it.

Then, we have a couple of optional fields, "title" (➋) and "detail" (➌), which should be self-explanatory. And lastly, we can provide any kind of free-form fields which are specific to the error type, like the "context" object in the example above (➍).

To signal to the client that the payload is indeed a "Problem", the server should indicate that fact by setting the "Content-Type" HTTP header accordingly:

    Content-Type: application/problem+json
  

This is really all you have to know about Problem JSON. The client can now parse the "type" field and knows exactly which error they're dealing with and react to it accordingly. This is already a big step forward from the example in the beginning but it gets better...

Strongly typed errors across service boundaries

At Celonis, we love strong typing. After all, we use Java and TypeScript as our primary languages. So it comes to no one's surprise that we want our error responses to be strongly typed, too.

Problem JSON allows us to do exactly this. Or more precisely, a very useful little Java library by Zalando, called "Problem", does. In a nutshell, the library provides a way to translate between JSON responses and Java Exceptions. This allows us to throw Problems (like normal Exceptions) on the server and catch those Exceptions again in the client - as if there were no network in-between.

How does that work? Let's walk through it.

1. The client sends a request to the server.

2. The server tries to answer the request but runs into a problem, so a Problem Exception is thrown:

       // Throw the Problem like any ordinary Exception
   throw new UsernameTakenProblem(providedUsername);
     

3. The Exception is turned into its corresponding Problem JSON representation. The Problem library does this for us. We only have to provide mappings between Problem types and their respective Exceptions. The properly serialized JSON response is then returned to the client.

4. The client receives the Problem JSON response which is then translated into the original Exception by the client library. This exception can now be caught, just like any other Exception:

       try {
       apiClient.createUser("Lucy");
       // The API client automatically turns Problem JSON responses into exceptions
   } catch (MalformedInputProblem e) {
       // Here, we can deal with the case that the input is malformed
   } catch (UserNameTakenProblem e) {
       // And here, we can deal with the case that the username is taken
   }
     

There you have it. With Problem JSON (or rather, the tooling around it) we can transmit errors in a well-defined and type-safe fashion across the network.

Conclusion

Problem JSON alleviates a headache that every team faces sooner or later when building an API: how to communicate distinctive errors to API clients in a structured manner. Response codes only get us so far. We need a more elaborate way to discern different error types so that the API client can make an informed decision on how to react. Problem JSON enables exactly this by providing a standardized format for expressing error conditions. Important is the word "standardized": as Problem JSON has its own RFC, there's a rich ecosystem of tooling to support it in all kinds of languages and frameworks. So while the examples above were Java-specific, there's nothing stopping you from doing the same thing with Go or Rust.

What we like in particular is that Problem JSON tooling allows us to transport errors in a type-safe fashion which is a boon not only to developer experience but also the robustness of our microservice architecture.

David Hettler

Senior Software Engineer

David is a Senior Software Engineer in the SaolaDB team. He holds a Master’s degree in Computer Science from the University of Munich. Professionally, he is specialized in Distributed Systems, Data Engineering and Event Driven Architectures. In the limited free time that his two kids grant him he loves to do all kinds of board sports, long distance running and hiking in the alps.