Spring REST Docs API specification Integration

This is an extension that adds API specifications as an output format to Spring REST Docs. It currently supports:

• OpenAPI 2.0 in json and yaml
• OpenAPI 3.0.1 in json and yaml
• Postman Collections 2.1.0

We plan to add support for:

• RAML

Please note that this extension was developed for JSON-based APIs. We do not expect this extension to build usable API specification for non-JSON request or response bodies.

Motivation

Spring REST Docs is a great tool to produce documentation for your RESTful services that is accurate and readable.

We especially like its test-driven approach and this is the main reason why we chose it.

It offers support for AsciiDoc and Markdown. This is great for generating simple HTML-based documentation. But both are markup languages and thus it is hard to get any further than statically generated HTML.

API specifications like OpenAPI are a lot more flexible. With e.g. OpenAPI you get a machine-readable description of your API. There is a rich ecosystem around it that contains tools to:

• generate a HTML representation of your API - ReDoc
• generate an interactive API reference - e.g. using services like stoplight.io or readme.io

Also, API specifications like OpenAPI are supported by many REST clients like Postman and Paw. Thus having an API specification for a REST API is a great plus when starting to work with it.

The most common use case to generate an OpenAPI specification is code introspection and adding documentation related annotations to your code. We do not like enriching our production code with this information and clutter it with even more annotations. We agree with Spring REST Docs that the test-driven way to produce accurate API documentation is the way to go. This is why we came up with this project.

• Motivation
• Getting started
  • Project structure
  • Build configuration
    • Gradle
    • Maven
  • Usage with Spring REST Docs
  • Documenting Bean Validation constraints
  • Migrate existing Spring REST Docs tests
    • MockMvc based tests
    • REST Assured based tests
    • WebTestClient based tests
  • Security Definitions in OpenAPI
  • Running the gradle plugin
    • OpenAPI 2.0
    • OpenAPI 3.0.1
    • Postman
  • Gradle plugin configuration
    • Common configuration for all formats
    • Common OpenAPI configuration
    • OpenAPI 2.0
    • OpenAPI 3.0.1
    • Postman
• Generate an HTML-based API reference from OpenAPI
• RAML

Getting started

Project structure

The project consists of the following main components:

• restdocs-api-spec - contains the actual Spring REST Docs extension. This is most importantly the ResourceDocumentation which is the entrypoint to use the extension in your tests. The ResourceSnippet is the snippet used to produce a json file resource.json containing all the details about the documented resource.
• restdocs-api-spec-mockmvc - contains a wrapper for MockMvcRestDocumentation for easier migration to restdocs-api-spec from MockMvc tests that use plain spring-rest-docs-mockmvc.
• restdocs-api-spec-restassured - contains a wrapper for MockMvcRestDocumentation for easier migration to restdocs-api-spec from Rest Assured tests that use plain spring-rest-docs-restassured.
• restdocs-api-spec-gradle-plugin - adds a gradle plugin that aggregates the resource.json files produced by ResourceSnippet into an API specification file for the whole project.

Build configuration

Gradle

1. Add the plugin
   • Using the plugins DSL:

     plugins {
         id 'com.epages.restdocs-api-spec' version '0.11.2'
     }
     

     Examples with Kotlin are also available here
   • OR Using legacy plugin application:
     • 1.1 Use of buildscript requires you to add the https://plugins.gradle.org/m2/ repository.
     • 1.2 add the dependency to restdocs-api-spec-gradle-plugin
     • 1.3 apply restdocs-api-spec-gradle-plugin

     buildscript {
       repositories {
         maven {
           url "https://plugins.gradle.org/m2/" //1.1
         }
       }
       dependencies {
         classpath "com.epages:restdocs-api-spec-gradle-plugin:0.11.2" //1.2
       }
     }
     
     apply plugin: 'com.epages.restdocs-api-spec' //1.3
     
     

2. Add required dependencies to your tests
   • 2.1 add the mavenCentral repository used to resolve the com.epages:restdocs-api-spec module of the project.
   • 2.2 add the actual restdocs-api-spec-mockmvc dependency to the test scope. Use restdocs-api-spec-restassured if you use RestAssured instead of MockMvc.
   • 2.3 add configuration options for restdocs-api-spec-gradle-plugin`. See Gradle plugin configuration

   repositories { //2.1
       mavenCentral()
   }
   
   dependencies {
       //..
       testCompile('com.epages:restdocs-api-spec-mockmvc:0.11.2') //2.2
   }
   
   openapi { //2.3
       host = 'localhost:8080'
       basePath = '/api'
       title = 'My API'
       description = 'My API description'
       tagDescriptionsPropertiesFile = 'src/docs/tag-descriptions.yaml'
       version = '1.0.0'
       format = 'json'
   }
   
   openapi3 {
       server = 'https://localhost:8080'
       title = 'My API'
       description = 'My API description'
       tagDescriptionsPropertiesFile = 'src/docs/tag-descriptions.yaml'
       version = '0.1.0'
       format = 'yaml'
   }
   
   postman {
       title = 'My API'
       version = '0.1.0'
       baseUrl = 'https://localhost:8080'
   }
   

See the build.gradle for the setup used in the sample project.

Maven

The root project does not provide a maven plugin. But you can find a plugin that works with restdocs-api-spec at BerkleyTechnologyServices/restdocs-spec.

Usage with Spring REST Docs

The class ResourceDocumentation contains the entry point for using the ResourceSnippet.

The most basic form does not take any parameters:

mockMvc
  .perform(post("/carts"))
  .andDo(document("carts-create", resource("Create a cart")));

This test will produce the resource.json file in the snippets directory. This file just contains all the information that we can collect about the resource. The format of this file is not specific to an API specification.

{
  "operationId" : "carts-create",
  "summary" : "Create a cart",
  "description" : "Create a cart",
  "privateResource" : false,
  "deprecated" : false,
  "request" : {
    "path" : "/carts",
    "method" : "POST",
    "contentType" : null,
    "headers" : [ ],
    "pathParameters" : [ ],
    "requestParameters" : [ ],
    "requestFields" : [ ],
    "example" : null,
    "securityRequirements" : null
  },
  "response" : {
    "status" : 201,
    "contentType" : "application/hal+json",
    "headers" : [ ],
    "responseFields" : [ ],
    "example" : "{\n  \"total\" : 0,\n  \"products\" : [ ],\n  \"_links\" : {\n    \"self\" : {\n      \"href\" : \"http://localhost:8080/carts/4\"\n    },\n    \"order\" : {\n      \"href\" : \"http://localhost:8080/carts/4/order\"\n    }\n  }\n}"
  }
}

Just like with Spring REST Docs we can also describe request fields, response fields, path variables, parameters, headers, and links. Furthermore you can add a text description and a summary for your resource. The extension also discovers JWT tokens in the Authorization header and will document the required scopes from it. Also basic auth headers are discovered and documented.

The following example uses ResourceSnippetParameters to document response fields, path parameters, and links. We paid close attention to keep the API as similar as possible to what you already know from Spring REST Docs. fieldWithPath and linkWithRel are actually still the static methods you would use in your using Spring REST Docs test.

mockMvc.perform(get("/carts/{id}", cartId)
  .accept(HAL_JSON))
  .andExpect(status().isOk())
  .andDo(document("cart-get",
    resource(ResourceSnippetParameters.builder()
      .description("Get a cart by id")
      .pathParameters(
        parameterWithName("id").description("the cart id"))
      .responseFields(
        fieldWithPath("total").description("Total amount of the cart."),
        fieldWithPath("products").description("The product line item of the cart."),
        subsectionWithPath("products[]._links.product").description("Link to the product."),
        fieldWithPath("products[].quantity").description("The quantity of the line item."),
        subsectionWithPath("products[].product").description("The product the line item relates to."),
        subsectionWithPath("_links").description("Links section."))
      .links(
        linkWithRel("self").ignored(),
        linkWithRel("order").description("Link to order the cart."))
    .build())));

Please see the CartIntegrationTest in the sample application for a detailed example.

⚠️ Use template URIs to refer to path variables in your request

Note how we use the urlTemplate to build the request with RestDocumentationRequestBuilders. This makes the urlTemplate available in the snippet and we can depend on the non-expanded template when generating the OpenAPI file.

mockMvc.perform(get("/carts/{id}", cartId)

Documenting Bean Validation constraints

Similar to the way Spring REST Docs allows to use bean validation constraints to enhance your documentation, you can also use the constraints from your model classes to let restdocs-api-spec enrich the generated JsonSchemas. restdocs-api-spec provides the class com.epages.restdocs.apispec.ConstrainedFields to generate FieldDescriptors that contain information about the constraints on this field.

Currently the following constraints are considered when generating JsonSchema from FieldDescriptors that have been created via com.epages.restdocs.apispec.ConstrainedFields

• NotNull, NotEmpty, and NotBlank annotated fields become required fields in the JsonSchema
• for String fields annotated with NotEmpty, and NotBlank the minLength constraint in JsonSchema is set to 1
• for String fields annotated with Length the minLength and maxLength constraints in JsonSchema are set to the value of the corresponding attribute of the annotation

If you already have your own ConstraintFields implementation you can also add the logic from com.epages.restdocs.apispec.ConstrainedFields to your own class. Here it is important to add the constraints under the key validationConstraints into the attributes map if the FieldDescriptor.

Migrate existing Spring REST Docs tests

MockMvc based tests

For convenience when applying restdocs-api-spec to an existing project that uses Spring REST Docs, we introduced com.epages.restdocs.apispec.MockMvcRestDocumentationWrapper.

In your tests you can just replace calls to MockMvcRestDocumentation.document with the corresponding variant of MockMvcRestDocumentationWrapper.document.

MockMvcRestDocumentationWrapper.document will execute the specified snippets and also add a ResourceSnippet equipped with the input from your snippets.

Here is an example:

resultActions
  .andDo(
    MockMvcRestDocumentationWrapper.document(operationName,
      requestFields(fieldDescriptors().getFieldDescriptors()),
      responseFields(
        fieldWithPath("comment").description("the comment"),
        fieldWithPath("flag").description("the flag"),
        fieldWithPath("count").description("the count"),
        fieldWithPath("id").description("id"),
        fieldWithPath("_links").ignored()
      ),
      links(linkWithRel("self").description("some"))
  )
);

This will do exactly what MockMvcRestDocumentation.document does. Additionally it will add a ResourceSnippet with the descriptors you provided in the RequestFieldsSnippet, ResponseFieldsSnippet, and LinksSnippet.

REST Assured based tests

Also for REST Assured we offer a convenience wrapper similar to MockMvcRestDocumentationWrapper. The usage for REST Assured is also similar to MockMVC, except that com.epages.restdocs.apispec.RestAssuredRestDocumentationWrapper is used instead of com.epages.restdocs.apispec.MockMvcRestDocumentationWrapper.

To use the RestAssuredRestDocumentationWrapper, you have to add a dependency to restdocs-api-spec-restassured to your build.

RestAssured.given(this.spec)
        .filter(RestAssuredRestDocumentationWrapper.document("{method-name}",
                "The API description",
                requestParameters(
                        parameterWithName("param").description("the param")
                ),
                responseFields(
                        fieldWithPath("doc.timestamp").description("Creation timestamp")
                )
        ))
        .when()
        .queryParam("param", "foo")
        .get("/restAssuredExample")
        .then()
        .statusCode(200);

WebTestClient based tests

We also offer a convenience wrapper for WebTestClient which works similar to MockMvcRestDocumentationWrapper. The usage is similar to MockMVC, except that com.epages.restdocs.apispec.WebTestClientRestDocumentationWrapper is used instead of com.epages.restdocs.apispec.MockMvcRestDocumentationWrapper.

To use the WebTestClientRestDocumentationWrapper, you will have to add a dependency to restdocs-api-spec-webtestclient to your build.

webTestClient.get().uri("/sample/{id}?queryParam=something", "1024").exchange()
    .expectStatus().isOk().expectBody()
    .consumeWith(
        WebTestClientRestDocumentationWrapper
            .document("sample",
                RequestDocumentation.pathParameters(
                    parameterWithName("id").description(
                        "description of the path parameter")
                ),
                RequestDocumentation.requestParameters(
                    parameterWithName("queryParam").description(
                        "description of the query parameter")
                ),
                HeaderDocumentation.responseHeaders(
                    headerWithName(HttpHeaders.CONTENT_TYPE)
                        .description(MediaType.APPLICATION_JSON_UTF8_VALUE)
                ),
                responseFields(
                    PayloadDocumentation.fieldWithPath("field1").type(JsonFieldType.STRING)
                        .description("description of field1"),
                    PayloadDocumentation.fieldWithPath("field2").type(JsonFieldType.STRING)
                        .description("description of field2")
                )
            )
    );

Security Definitions in OpenAPI

The project has limited support for describing security requirements of an API. Currently we only support Oauth2 with JWT tokens and HTTP Basic Auth.

restdocs-api-spec inspects the AUTHORIZATION header of a request for a JWT token. Also the a HTTP basic authorization header is discovered and documented. If such a token is found the scopes are extracted and added to the resource.json snippet.

The restdocs-api-spec-gradle-plugin will consider this information if the oauth2SecuritySchemeDefinition configuration option is set (see Gradle plugin configuration). This will result in a top-level securityDefinitions in the OpenAPI definition. Additionally the required scopes will be added in the security section of an operation.

Running the gradle plugin

restdocs-api-spec-gradle-plugin is responsible for picking up the generated resource.json files and aggregate them into an API specification.

OpenAPI 2.0

In order to generate an OpenAPI 2.0 specification we use the openapi task:

./gradlew openapi

OpenAPI 3.0.1

In order to generate an OpenAPI 3.0.1 specification we use the openapi3 task:

./gradlew openapi3

For our sample project this creates a openapi3.yaml file in the output directory (build/api-spec).

Postman

In order to generate a Postman collection we use the postman task:

./gradlew postman

For our sample project this creates a postman-collection.json file in the output directory (build/api-spec).

Gradle plugin configuration

Common configuration for all formats

Common OpenAPI configuration

The restdocs-api-spec-gradle-plugin takes the following configuration options for OpenAPI 2.0 and OpenAPI 3.0.1 - all are optional.

The scopeDescriptionsPropertiesFile is supposed to be a yaml file:

scope-name: A description

OpenAPI 2.0

The restdocs-api-spec-gradle-plugin takes the following configuration options for OpenAPI 2.0 - all are optional.

Example configuration closure:

openapi {
    basePath = "/api"
    host = "api-shop.beyondshop.cloud"
    schemes = ["https"]
    format = "yaml"
    title = 'Beyond REST API'
    version = "1.0.0"
    separatePublicApi = true
    snippetsDirectory="src/docs/asciidoc/generated-snippets/"
    outputDirectory="openapi/"
    oauth2SecuritySchemeDefinition = {
        flows = ['accessCode', 'application']
        tokenUrl = 'https://api-shop.beyondshop.cloud/api/oauth/token'
        authorizationUrl = 'https://api-shop.beyondshop.cloud/api/auth/oauth-ext/authorize'
        scopeDescriptionsPropertiesFile = "src/docs/scope-descriptions.yaml"
    }
}

OpenAPI 3.0.1

The restdocs-api-spec-gradle-plugin takes the following configuration options for OpenAPI 3.0.1 - all are optional.

Example configuration closure:

openapi3 {
    servers = [ { url = "http://some.api" } ]
    title = 'My API title'
    version = '1.0.1'
    format = 'yaml'
    separatePublicApi = true
    outputFileNamePrefix = 'my-api-spec'
    oauth2SecuritySchemeDefinition = {
        flows = ['authorizationCode']
        tokenUrl = 'http://example.com/token'
        authorizationUrl = 'http://example.com/authorize'
        scopeDescriptionsPropertiesFile = "scopeDescriptions.yaml"
    }
}

Example build.gradle.kts configuration closure (by axkb, #112):

configure<com.epages.restdocs.apispec.gradle.OpenApi3Extension> {
    setServer("http://$apiHost:$apiPort")
    title = "Your title"
    description = "Your description"
    version = "0.1.0"
    format = "json"
    tagDescriptionsPropertiesFile = "src/test/resources/tags.yaml"
}

The servers and server property can also contain variables. Is this case the` property can be specified like this:

This configuration follows the same semantics as the 'Servers Object' in the OpenAPI specification

servers = [ {
    url = 'https://{host}/api'
    variables = [
        host: [
            default: 'api-shop.beyondshop.cloud/api',
            description: 'The hostname of your beyond shop',
            enum: ['api-shop', 'oz']
        ]
    ]
} ]

The same structure applies to server. A single server can also be specified using a plain string:

server = 'http://some.api/api'

Postman

The restdocs-api-spec-gradle-plugin takes the following configuration options for Postman collections - all are optional.

Example configuration closure:

postman {
    title = 'Beyond REST API'
    version = '1.0.1'
    baseUrl = 'https://api-shop.beyondshop.cloud/api'
    separatePublicApi = true
    outputFileNamePrefix = 'my-postman-collection'
}

Generate an HTML-based API reference from OpenAPI

We can use redoc to generate an HTML API reference from our OpenAPI specification.

The redoc-cli can be used to bundle (and serve) this API reference:

# Install redoc-cli
npm install -g redoc-cli

# Bundle the documentation into a zero-dependency HTML-file
redoc-cli bundle build/api-spec/openapi.json

# Bundle and serve
redoc-cli serve build/api-spec/openapi.json

RAML

This project supersedes restdocs-raml. So if you are coming from restdocs-raml you might want to switch to restdocs-api-spec.

The API of both projects is fairly similar and it is easy to migrate.

We plan to support RAML in the future. In the meantime you can use one of several ways to convert an OpenAPI specification to RAML. There are converters around that can help you to achieve this conversion.

• oas-raml-converter - an npm project that provides a CLI to convert between OpenAPI and RAML - it also provides an online converter
• api-matic - an online converter capable of converting between many api specifications

In the sample project you find a build configuration that uses the oas-raml-converter-docker docker image and the gradle-docker-plugin to leverage the oas-raml-converter to convert the output of the openapi task to RAML. Using this approach your gradle build can still output a RAML specification.

See openapi2raml.gradle.

./gradlew restdocs-api-spec-sample:openapi
./gradlew -b samples/restdocs-api-spec-sample/openapi2raml.gradle openapi2raml

Maintenance

This section of the README is targeted at project maintainers.

Publish project

The project is published with the help of TravisCI. It's version number is determined by the Git tags (see allegro/axion-release-plugin). The Java dependencies are published to Sonatype with the help of the gradle-nexus/publish-plugin and the Maven Publish Plugin. The Gradle plugin is published to the Gradle plugin portal with the help of the 'plugin-publish' plugin (see docs.gradle.org).

Given that the master branch on the upstream repository is in the state from which you want to create a release, execute the following steps:

1. Create release via the GitHub UI

Use the intended version number as "Tag version", e.g. "0.11.2".

This will automatically trigger a Travis build which publishes the JAR files for this release to Sonatype.

2. Login to Sonatype

Login to Sonatype and navigate to the staging repositories.

3. Close the staging repository

Select the generated staging repository and close it. Check that there are no errors afterwards (e.g. missing signatures or Javadoc JARs).

4. Release the repository

Select the generated staging repository and publish it. Soon after, the release should be available in the "Public Repositories" of ePages.

5. Update documentation

Create a new commit which updates the version numbers in the README file.