Documenting APIs

I've been using Swagger for documenting a REST API, but I've found it has a few shortcomings. I was inspired by this talk to look at other methods for generating API documentation. We've ended up using API Blueprint and I wanted to share some thoughts.

What I look for in a tool for documenting an API

The things that are important to me are that any tool:

1. Allows me to create clear documentation for users
2. Ensures the documentation is kept up to date (for example by generating docs from code or tests, or from the docs themselves being testable)
3. Allows generation of a mock server to make it easy for my consumers to test against

Swagger

Swagger generates a swagger spec of your API from your code. This spec can then be used to produce pretty decent docs, generate mock servers, and more.

The docs generated by Swagger are pretty good, and are a pretty common standard now. There are also a number of tools that read and do useful things with a swagger specification.

The good:

• Swagger specs are (mainly) generated from code, so pretty easy to keep up to date
• The swagger endpoint for viewing documentation is particularly good given it takes almost no work to set up
• There are tools available to generate a mock server from Swagger docs

The bad:

• Swagger doc generation relies heavily on annotations. These annotations are easy to let get out of date. No tests will fail if this happens so there's no way to detect it except someone noticing when trying to use your docs. This is a major downside in my view
• Swagger docs are quite inflexible. For example if you want to document a service with more words to explain context, or organise your docs differently than by route, it isn't possible to do that. Albeit you could write a tool that extracted examples from a swagger specification and put them in asciidoctor files for you to embed in a more readable document...
• Generated mock servers only allow for one response per endpoint, limiting their usefulness in testing

Conclusion on Swagger
It's a really good tool, but many of the shortcomings here are fundamentally because of the approach to generate the spec from code. I still recommend it, but think in the long run other tools will win out.

 

API Blueprint

API Blueprint lets you write API documentation is a set format for a markdown document. This gives you some but not loads of flexibility for the format of the docs.

As with Swagger the spec can then be used for other things, including running tests, generating prettier html docs, and running a mock server.

The good:

• Having made the tests part of the CI pipeline we now always know if the spec is out of date
• Dredd (the testing tool) is fairly flexible as it allows you to run custom hooks before the tests, for example I used these to do an oAuth login required for accessing our API in dev
• As with Swagger some very good tools around the specification, including converting from Swagger spec to API Blueprint spec if you just want to try it out

The bad:

• The next thing on their roadmap is being able to have multiple requests and responses per endpoint. As with Swagger this is much needed as currently it limits the tests you can run, and the usefulness of the mock server
• Docs are a little more flexible than Swagger, but could do with even more flexibility. You tend to need to split it up into separate markdown files if you have large responses

Spring RESTDocs

I do agree with the talk posted above - having tests generate document snippets to include in a larger document for your API seems a great way to produce documentation. Unfortunately I'm not using Spring so that tool is unavailable. I would love to see a testing tool that isn't coupled to a particular framework that can generate documentation snippets. However API Blueprint or Swagger could potentially both generate documentation snippets, so perhaps this advantage is overplayed.

Spring RESTDocs does also give much greater flexibility with testing.

On the downsides I can see it being harder to generate a mock server using this approach, though this isn't a high priority for many people.