Best practices for REST API design

Today I would like to talk about my experience in designing REST API. I've developed and consumed many APIs over the last few years. Well, during that time, I came across good and bad practices, I wasted a lot of time trying to understand badly designed APIs, I explored many different patterns and I made quite a few mistakes. All things considered, I think I learned a lot. I would now like to share with you the experience I have gained in my journey and the best practices I have discovered and adopted.

But, before starting, in a nutshell, what is a REST API?

What is a REST API?

A REST API (also known as RESTful API) is an application programming interface (API).

REST stands for representational state transfer. It means that when a RESTful API is called, the server will transfer to the client a representation of the status of the requested resource.

Roy Fielding first introduced it in 2000 in his famous dissertation. Like other architectural styles, REST has its principles that must be fulfilled if a service interface is to be designated as RESTful.

A REST API is a contract

Drawings by @tina_pep3

The word API means that it is also the contract between your service and your's clients, whether they are the UI or other services.

Once established, it would be better to fulfill it as best as possible. If you update the contract, it affects all the other stakeholders so you have to make sure that it is clearly defined and doesn't change very often.

Changing the API on a running system without breaking anything or degrading the service can be a nightmare.

In addition, poorly designed APIs can be under-used, misused, and even unused. It can be troublesome and even harmful for both the consumers and the provider.

So design your API carefully and try to ensure that you do it right the first time. That doesn't mean that changes will never be permitted. You can design your API to make it flexible and easily extendable to meet your new requirements. There are also some techniques useful to minimize outages when the changes are necessary.

Contract-First

The contract-first approach is a way of designing and developing APIs starting from a contract.

Using the Open Api specification you can design your API and share it with your consumers and developers.

An OpenAPI document is written in JSON or YAML. Starting from this definition user-friendly HTML documentation can be generated, as well as client and server stubs in various programming languages and frameworks.

You’ll probably know how painful is to work with a badly documented (or even worse, non-documented) API.

For this reason, when is your turn to designing the API, it’s very important to document it well. That way, you’ll make life easier for the developers who want to work with it and for your future self.

It is a simple principle, the faster developers understand your API, the faster they start using it.

Your API documentation must be compiled with precision. It must include all the relevant information needed to consume it, such as the endpoints, the schema for the requests and the responses, the required parameters, the return codes, the authentication method used, and so on.

A big advantage of a contract-first approach is that teams can develop in parallel since coding happens based on the contract.

If cross-team testing is not possible due to different paces of development, stub software can be used to mock the other's behavior.

All the discussion that happens between the provider and consumer teams around the API will be based on the contract. Most of the effort would center around an agreement between teams. If you ensure that the contract is clearly defined, it will be less necessary to change it in the future.

That said, these are the best practices that I have progressively introduced over time, and that I still use when I design my REST APIs.

Let's start!

Accept and respond with JSON

JSON (JavaScript Object Notation) is the most generally popular format to use because, despite its name, it’s language-agnostic, as well as easily readable by both humans and machines.

Prefer JSON over XML because:

JSON is shorter
JSON is quicker to read and write
JSON can use arrays
JSON is easier to parse than XML
JSON in javascript is parsed into a ready-to-use object

To make sure that when your REST API responds with JSON then the clients interpret it correctly, you should set the Content-Type header in the response to application/json. Many server-side frameworks set the response header automatically. Some HTTP clients look at the Content-Type response header and parse the data according to that format.

The only exception to JSON is if we’re trying to send and receive files.

Use HTTP methods consistently

REST is resource-oriented and a resource is represented by a URI. An endpoint is the combination of a method and a URI and can be interpreted as an action on a resource. At a high-level, methods map to CRUD operations

GET /resources                get all the resources  
GET /resources/{id}           get a specific resource  
POST /resources               create a new resource  
PUT  /resources/{id}          update the whole resource  
PATCH /resources/{id}         applies a partial update to the resource
DELETE /resources/{id}        delete a resource

Don't use verbs in URIs that map a CRUD operation. Adding verbs isn’t useful and makes the names unnecessarily long since it doesn’t carry any new information.

Bad

POST /abilities/delete

Good

DELETE /abilities/{id}

Use verb in URI only for actions

This in an exception to the general rule, you can use POST + verb for actions that don't fit into the world of CRUD operations:

POST /connect

Cases

I ♥ kebab!

Adopt a case and be consistent with it, my preferred are:

kebab-case for endpoints

GET /sub-resources

camelCase for JSON fields

startDate

I wrote about the importance to adopt standard cases in: How to name things in code - Case matters.

Dates

Use ISO-8601 format for dates. This representation contains the time zone to avoid any ambiguity. As a general rule, it is best to express dates in UTC, using the special designator "Z", leaving the client the task of formatting the date in the appropriate time zone.

2011-10-05T14:48:00.000Z

Use plural resource nouns for resources

Use plural nouns for resources, it makes it easier for humans to understand the meaning. Let’s go through this example:

Good

GET /ducks  
GET /ducks/{id}

The usage of a plural noun is indicating that this is a collection of different ducks. Now, look at another example:

Bad

GET /duck  
GET /duck/{id}

This example doesn’t clearly show whether there is more than one duck in the system or not. For a human reader, it might be challenging to understand.

Use consistent names across endpoints

As I wrote in another post: each thing should have only one name.

Bad

POST /habitats
GET /all_habitats_list
GET /environments/{id}

Good

POST /habitats
GET /habitats
GET /habitats/{id}

Don't use trailing /

Bad

GET /resources/

Good

GET /resources

Handle trailing slashes gracefully redirecting clients if they use the wrong convention.

GET should never change the resource state on the server

GET should never change the resource state on the server, is defined in this way in the HTTP protocol. It is supposed to be idempotent and cacheable.

If you make the same GET again, nothing should change.

GET and DELETE shouldn't have a body

While it is not explicitly forbidden by the specification to include a body with a DELETE or GET request, most HTTP servers do not support it.

Add Location header to POST that creates resources

If a resource has been created as a result of successfully processing a POST request, the return code should be a 201 (Created) and the response should contain a Location header that provides the URI of the new resource.

POST /ducks  
{ 
  name: "American Wigeon",
}

201 (Created)
Location: /ducks/1

The clients can use this URI to read the newly created resource.

PUT, PATCH, DELETE are idempotent, POST isn't idempotent.

Idempotency means that applying an operation once or applying it multiple times has the same effect. For example, in arithmetic, adding zero to a number is an idempotent operation.

A REST request is idempotent when making multiple identical requests has the same effect as making a single request.

API consumers can make mistakes and there can be duplicate requests coming to the API. You have to make our APIs fault-tolerant, so that duplicate requests do not leave the system unstable.

GET doesn't change the resource state on the server. It is exclusively for retrieving the resource. For that reason GET is idempotent.
PUT and PATCH are used to update the resource state. If you invoke a PUT or a PATCH multiple times, the very first request will update the resource, and the other requests will just overwrite the same resource state again. Therefore, PUT and PATCH are idempotent.
When you invoke multiple DELETE on the same resource, the first request will delete the resource and the others won't find the resource, effectively not changing anything (delete should return 200 OK even if the resource doesn't exist). Therefore, DELETE is idempotent.
Generally (not necessarily) POST is used to create a new resource on the server. When the same POST request is invoked N times, N new resources will be created on the server. Consequently, POST is not idempotent.

REST is stateless

In order for an API to be considered RESTful, it should provide a stateless client-server communication, meaning no client information is stored between get requests and each request is separate and unconnected.

Each request from the client to the server must contain all the information needed to process that request and that information cannot be stored on the server side for any future reference.

The client is responsible for storing and handling the session-related information on its own side.

Advantages:

Easily scalable: as there is no need for any stored information, any instance of the service can handle the client's request.
Decreased complexity: as state synchronization is not needed, it reduces the complexity.
Improved performance: server doesn’t need to keep track of client requests, which increases the performance.
Improved testability: request are not dependent on each other, so the system is easier to test.

Referenced resources

Use <resource>Id to reference other resources in responses.

itemId

Use <resource>Ids (plural) for lists of resource id references.

itemIds

Alternatively, depending on your needs, the whole sub-resource can be included.

{
 ...
 subresource: {
    ...
 }
}

You can also use a nested endpoint for a collection of subresources.

GET /resource/{id}/subresources

Provide an endpoint for searches

Consider providing, in addition to GET /resources, a further endpoint for searches.

POST /resources/search

This endpoint uses POST verb because the search parameters could be a structured object, and this object is passed inside the request body.

Try to make the criteria as flexible as possible. For example, you can use ranges or arrays, and let criteria be optional. The relationship between criteria is logical AND, only resources that match all the criteria will be returned.

Search by authors list and publish date range:

POST /articles/search
{
  authorIds: [1, 2],
  publishDate: { from: "2022-01-01T00:00:00.000Z", to: "2023-01-01T00:00:00.000Z"}
}

200 (OK)
[{
  ...
  authorId:1,
  publishDate: "2022-04-03T12:42:00.000Z"
},
{
  ...
  authorId:2,
  publishDate: "2022-06-05T10:12:00.000Z"
}]

Note that passing a void array is different from omitting the criterion. Adding a void array to the criteria should always return a void result.

POST /articles/search
{
  authorIds: [],
  publishDate: { from: "2022-01-01T00:00:00.000Z", to: "2023-01-01T00:00:00.000Z"}
}

200 (OK)
[]

When the result is void the response code should not be 404 because the search operation was successful even if no resources were returned.

Bad

POST /articles/search
{
  publishDate: { from: "2022-01-01T00:00:00.000Z", to: "2023-01-01T00:00:00.000Z"}
}

404 (Not found)

Good

POST /articles/search
{
  publishDate: { from: "2022-01-01T00:00:00.000Z", to: "2023-01-01T00:00:00.000Z"}
}

200 (OK)
[]

Search - allow sorting and pagination

The databases behind a REST API can get very large. Sometimes, there’s so much data that it shouldn’t be returned all at once. Therefore, you need ways to paginate data so that you only return a few results at a time.

You can design your API by establishing that by default search operations would return only the first N item and provide two special parameters to get a different set of items.

offset - start offset of the first item
limit - max number of items to be returned

Offset and limit can be used in conjunction with the order parameter.

POST /articles/search
{
  publishDate: { from: "2022-01-01T00:00:00.000Z", to: "2023-01-01T00:00:00.000Z"},
  offset: 0,
  limit: 100,
  order: [{field:"authorName", direction:"asc"}, {field:"publishDate", direction:"desc"}]
}

Search - expand referenced resources

To reduce the number of requests and provide a less chatty API you can provide an expand parameter in searches to explicitly expand one or more referenced resources in the response.

POST /articles/search
{
  expand: ["author"]
}

200 (OK)
[{
  ...
  authorId: 1,
  author: {
    name: "Celeste",
    surname: "Cobb",
  },
  publishDate: "2022-04-03T12:42:00.000Z"
}]

Chain expand to include a resource referenced through another resource.

POST /ducks/search
{
  expand: ["species.genus"] 
}

200 (OK)
[
 {
   id: 34,
   name: "Eurasian wigeon",
   ...
   speciesId: 2,
   species: {
     id: 2,
     name: "M. penelope"
     genusId: 12,
     genus: {
       id: 12,
       name: "Mareca"
     }
   }
 }
]

expand <resourceName>Ids to obtain the ids of the requested child resources.

POST /ducks/search
{
  expand: ["imageIds"]
}

200 (OK)
[{
   ...
   imageIds: [12,43]
 }]

expand <resourceName> (plural) to obtain requested child resources.

POST /ducks/search
{
   expand: ["images"]
}

200 (OK)
[{
   ...
   images: [{ id:12, name: "male", url:"duck/34/front.jpg" }, { id:42, name: "female", url:"duck/34/front.jpg" }]
 }]

Search - filter fields

Sometimes a resource has a huge set of fields, but not all of them are relevant and interesting for the client. You can use a fields parameter to limit the response to requested fields.

POST /ducks/search
{
  fields: ["name", "speciesIds"]
}

200 (OK)
[{
   name: "Redhead",
   speciesIds: 78
 }]

Search - Wildcards

Use * wildcard to include all fields.

POST /ducks/search
{
  fields: ["*"]
}

200 (OK)
[{
   name: "Eurasian Wigeon",
   description: "The Eurasian wigeon or European wigeon (Mareca penelope), also known as the widgeon or the wigeon, is one of three species of wigeon in the dabbling duck genus Mareca. It is common and widespread within its Palearctic range."
   speciesId: 2,
   minLength: 42,
   maxLength: 52,
   minWingspan: 17,
   maxWingspan: 20,
   minWeight: 500,
   maxWeight: 1.073
 }]

Use ! wildcard to exclude some fields.

POST /ducks/search
{
  fields: ["*", “!description”]
}

200 (OK)
[{
   name: "Eurasian Wigeon",
   speciesId: 2,
   minLength: 42,
   maxLength: 52,
   minWingspan: 17,
   maxWingspan: 20,
   minWeight: 500,
   maxWeight: 1.073
 }]

Patch lists

The PATCH verb is typically designed to update partially the state of a resource. In the case of a list of resources, you could send a list with only the elements to update or insert. New resources will be added, and existing resources will be updated.

PATCH /ducks 
[
  { "id": 1, "name": "newName1" },
  { "id": 2, "name": "newName2" },
  ...
]

Asynchronous operations

Sometimes a REST API operation might take a considerable amount of time to complete. Instead of letting the client wait until the operation completes we can return an immediate response and process the request asynchronously. The return code for asynchronous responses is 202 Accepted.

POST /heavy-calculation 

202 (Accepted)

Use HTTP status codes consistently

When a client makes a request to an HTTP server, and the server successfully receives the request, the server must notify the client if the request was successfully handled or not.

HTTP accomplishes this with five categories of status codes:

100 level Informational – server acknowledges a request
200 level Success – server completed the request as expected
300 level Redirection – client needs to perform further actions to complete the request
400 level Client error – client sent an invalid request
500 level Server error – server failed to fulfill a valid request due to an error with the server

Response codes for errors

Oh quack!

400 Bad Request – client sent an invalid request
401 Unauthorized – client failed to authenticate with the server
403 Forbidden – client authenticated but does not have permission to access the requested resource
404 Not Found – the requested resource does not exist
412 Precondition Failed – one or more conditions in the request header fields evaluated to false
422 Unprocessable Entity - the server understands the content type of the request entity, and the syntax of the request entity is correct, but it was unable to process the contained instructions.
500 Internal Server Error – a generic error occurred on the server
503 Service Unavailable – the requested service is not available

500 errors signal that some issues or exceptions occurred on the server while handling a request. Generally, this internal error is not useful for the client. Therefore you should handle errors gracefully and respond with status codes that indicate what kind of error occurred wherever possible. For example, if an exception occurs because a requested resource doesn't exist, you should expose this as a 404 rather than a 500 error.

Another reason for catching and handling errors is due to the fact on some servers when an uncaught exception is thrown then the stack trace is included in the response. This is considered a security issue.

Adopt standard problem details for error responses

Sometimes a status code is not enough to show the specifics of the error. When needed, we can use the body of the response to provide the client with additional information. To standardize REST API error handling, the IETF formulated RF 7807, which creates a generalized error-handling schema providing users with relevant information.

This schema is composed of five parts:

type - a URI identifier that categorizes the error
title - a brief, human-readable message about the error
status - the HTTP response code (optional)
detail - a human-readable explanation of the error
instance - a URI that identifies the specific occurrence of the error

See: zalando problems

Headers

API headers are like an extra source of information for each API call you make. Their job is to represent the meta-data associated with an API request and response.

Here are some of the most common API Headers:

Authorization Contains the authentication credentials for HTTP authentication.
Content-Type Tells the client what media type (e.g., application/json, application/javascript, etc.) a response is sent in. This is an important header field that helps the client know how to process the response body correctly.
Cache-Control The cache policy defined by the server for this response, a cached response can be stored by the client and re-used until the time defined by the Cache-Control header
Accept-Language Indicates the natural language and locale that the client prefers.

I find it very useful adding a X-Correlation-ID header. It is a unique identifier value that is attached to requests and messages that allow reference to a particular transaction or event chain. A Correlation ID is generated for each incoming request in every service with an external interface. A service receiving a Correlation should use it on all log messages and propagate it between services.

Protect your API

How do trees access?
They log in.

Most communication between client and server should be private since they send and receive private information. Therefore, using a secure channel such as HTTPS for security is necessary.

Provide an authentication method to allow only authenticated users to access your API.

Adopt a principle of least privilege: a user or service should only have access to the data needed to complete a required task. For example, a normal user shouldn’t be able to access information of another user, and he shouldn't be able to access data of admins. To implement this principle, you could set up a role-based access control.

Maintainability

Suppliers and consumers must be as tolerant as possible to changes in request and response schema. For example, you can provide a default value for new fields and silently ignore unknown fields.

When a change in endpoints is necessary, deprecate obsolete endpoints in favor of new ones for the span of one or more versions before you delete them.

Conclusions

This set of rules is obviously not intended to be exhaustive, it doesn't even claim to be perfect. Those are the ones that I've learned and adopted so far in my daily work and I found that they work well for me. I am constantly learning and I'm always open to suggestions. So please, let me know what you think about it by leaving a comment.

Quack.