Recently I’ve been looking at ways to replace our current REST API documentation build with one that works with Java 8. Our requirements are - generate static html from JAX-RS annotations and JavaDoc. Simple, right? It shouldn’t be much more than finding a maven plugin and using it, right? Wrong.
Here’s how it went …
For a long time we’ve been using
JAX Doclet. It generates JavaDoc html, except that it describes REST endpoints rather than classes, supported verbs rather than class methods, parameter bindings rather than method arguments, and status codes rather than method return types. It’s simple to integrate, and produces decent static html. It has one caveat though - it only works with Java 7. Java 8 brought changes in internal java doclet classes which caused many third party doclets to now maintain separate branches for Java 7, and Java 8. JAX Doclet has been feature complete and dormant for some time now, and there is no Java 8 compatibility on the horizon.
Here we are at a decision point - fork JAX Doclet and tailor it for Java 8, or find a replacement?
Programmers are a lazy bunch, or rather - we like to economize in order to reach our goal faster. Find a replacement then ...
Go to GitHub, and search for ‘jaxrs doclet’. There’s exactly one project with some traction on the topic. This is probably going to work.
The project we find is
Swagger JAXRS Doclet, and it’s been quite some time since it’s seen any activity, but there’s a link to a fork that’s still very active, so we end up using
teamcarma/swagger-jaxrs-doclet.
Good thing is it uses JAX-RS annotations, and JavaDoc descriptions to generate docs. Bad thing is the only output it generates are Swagger 1.2 compatible .json files, and an option to package them together with swagger-ui to get a rich single page web application to explore the API.
Sure, why not, surely you can double-click on index.html and run swagger-ui locally in your browser. It turns out that’s not the case.
Swagger-ui relies on XHR requests, and unbeknownst to me browsers have recently become immensely suspicious of XHR to the point of basically disabling it for local disk access.
That’s a problem for me, as I need static html. There might be magic switches to get browsers to perform XHR against local filesystem, but who wants to deal with that for the sake of browsing documentation ...
What now?
Well, find another maven plugin that takes REST API description in swagger .json format, and creates static HTML from it, of course.
Back on GitHub, searching for ‘swagger maven’, we get many results.
First hit is a very popular project, but immediately it doesn’t sound quite right for what we need -
swagger-maven-plugin. It’s really a replacement for swagger-jaxrs-doclet, except it doesn’t understand JAX RS annotations, or JavaDoc comments, but requires classes to be annotated with Swagger annotations. That’s a deal breaker for me, as I don’t want to pollute our code with non-standard third-party annotations, and we don’t even particularly care about Swagger at this point - it’s just a tool to get to the static html documentation.
But this tool does something neat - it is able to generate not only swagger json files, but static text files of different formats as well - html, asciidoc, markdown … Sigh, if only it could work with JAX RS annotations, and JavaDoc … Would it be difficult to add such functionality? Could we generate Swagger annotations where they are missing, based on JAX RS annotations, and JavaDoc? Do we explore the idea, or look for existing working solution?
Let’s look further …
The next interesting project we find is
swagger2markup-maven-plugin. This one looks exactly as what we need. It takes Swagger .json files, and converts them to asciidoc or markdown - unfortunately not also directly to html. For that we would then need another plugin - asciidoctor-maven-plugin. Not ideal, as we would end up with a bloated pom.xml, and a slow doc generation process, but as long as it does the job ...
Looking further still, we find that there are no more better candidates for what we need. So let’s work with swagger2markup-maven-plugin, and see how far it takes us.
It looks quite easy to use. We just add a few snippets to our pom.xml.
First, let’s make sure all of these extra plugins work with Java 8, since that is the whole reason we’re doing this:
<properties>
<!-- maven-compiler-plugin -->
<maven.compiler.target>1.8</maven.compiler.target>
<maven.compiler.source>1.8</maven.compiler.source>
...
</properties>
Then we add swagger-jaxrs-doclet to javadoc-maven-plugin:
<properties>
...
<version.swagger.doclet>1.0.5</version.swagger.doclet>
...
</properties>
<build>
<plugins>
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<executions>
<execution>
<id>generate-service-docs</id>
<phase>generate-resources</phase>
<configuration>
<doclet>com.carma.swagger.doclet.ServiceDoclet</doclet>
<docletArtifact>
<groupId>com.carma</groupId>
<artifactId>swagger-doclet</artifactId>
<version>${version.swagger.doclet}</version>
</docletArtifact>
<reportOutputDirectory>${project.basedir}/target/apidocs-rest/swagger</reportOutputDirectory>
<useStandardDocletOptions>false</useStandardDocletOptions>
<additionalparam> -skipUiFiles -apiVersion 1 -includeResourcePrefixes org.keycloak.services.resources.admin,org.keycloak.protocol.oidc -docBasePath /apidocs -apiBasePath http://localhost:8080/auth</additionalparam>
</configuration>
<goals>
<goal>javadoc</goal>
</goals>
</execution>
</executions>
</plugin>
...
</plugins>
</build>
This should trigger the doclet during JavaDoc generation, and produce swagger .json files in target/apidocs-rest/swagger directory without also including swagger-ui.
Now that we got the first part done, we move to the next step - configuring swagger2markup-maven-plugin to convert the created swagger .json files to asciidoc document.
This second maven plugin uses its own maven repository:
<repositories>
<repository>
<snapshots>
<enabled>false</enabled>
</snapshots>
<id>central</id>
<name>bintray</name>
<url>http://jcenter.bintray.com</url>
</repository>
</repositories>
<plugins>
…
<plugin>
<groupId>com.redowlanalytics</groupId>
<artifactId>swagger2markup-maven-plugin</artifactId>
<version>0.7.1</version>
<executions>
<execution>
<id>gen-asciidoc</id>
<phase>process-resources</phase>
<goals>
<goal>process-swagger</goal>
</goals>
<configuration>
<inputDirectory>${project.basedir}/target/apidocs-rest/apidocs/swagger</inputDirectory>
<outputDirectory>${project.basedir}/target/apidocs-rest/asciidoc</outputDirectory>
<markupLanguage>asciidoc</markupLanguage>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
That should produce three .adoc files in target/apidocs-rest/asciidoc directory - overview.adoc, paths.adoc, and definitions.adoc. The idea of Swagger2Markup is that these are three distinct document segments that can be combined with additional separately prepared .adocs into a complete and final document. Swagger2Markup itself however will only take it as far as preparing these three document segments for you.
There is one problem though. If you have multiple swagger .json files for input - as a result of multiple REST endpoints, then it looks like the created files will reflect documentation for only the last .json file processed. As if every next .json file processed will generate the three .doc files of the same name, overwriting the previous ones. Lucky for me I’m only dealing with one endpoint, so no need to figure out how to make it work with multiple files.
Next, we have to generate a composite asciidoc, and then perform translation into html. For that we’ll use asciidoctor-maven-plugin.
For starters, we need an asciidoc file that will tie together the three input .adoc files into one.
Let’s call it index.adoc, and let’s keep it very simple:
include::{generated}/overview.adoc[]
include::{generated}/paths.adoc[]
include::{generated}/definitions.adoc[]
We just combine all the three documents into one without any additional content.
We place this file into src/docs/asciidoc directory of our maven project.
Now we configure asciidoctor-maven-plugin:
<plugin>
<groupId>org.asciidoctor</groupId>
<artifactId>asciidoctor-maven-plugin</artifactId>
<version>1.5.2</version>
<executions>
<execution>
<id>generate-docs</id>
<phase>package</phase>
<goals>
<goal>process-asciidoc</goal>
</goals>
<configuration>
<sourceDirectory>${project.basedir}/src/docs/asciidoc</sourceDirectory>
<sourceDocumentName>index.adoc</sourceDocumentName>
<outputDirectory>${project.basedir}/target/apidocs-rest/html</outputDirectory>
<backend>html5</backend>
<attributes>
<!-- List of attributes:
https://github.com/asciidoctor/asciidoctorj/blob/master/asciidoctorj-core/src/main/java/org/asciidoctor/Attributes.java
-->
<generated>${project.basedir}/target/apidocs-rest/asciidoc</generated>
</attributes>
</configuration>
</execution>
</executions>
</plugin>
That will produce a single index.html file in target/apidocs-rest/html directory that is what we were after. Mission accomplished!
Well ... sort of.
There are some problems with this document.
For one, it has a big ‘null’ for its title.
Tracing back the issue, we can see it in overview.adoc already, and is the result of title info missing in service.json file generated by our very first plugin.
Turns out that’s easy to fix.
We create a file apiinfo.json:
{
"title": "Keycloak Admin REST API",
"description": "This is a REST API reference for Keycloak Admin"
}
We place it in src/docs/swagger directory.
Then we add another parameter to <additionalParam> line in the first plugin:
-apiInfoFile ${project.basedir}/src/docs/swagger/apiinfo.json
That should fix the title.
Next problem is that the document is not very readable. We could address that by tweaking the .css.
But that's a styling issue.
More problematic is that the document contains some dead links for generic types like Request, Object, Map which have no model definition in definitions.adoc. We only notice that now when we view the document as html, but the problem originates in swagger .json file already.
Also, some of our operations apparently don’t have the JavaDoc descriptions, and for those the .json file doesn’t contain any description, rather the output falls back to auto-generated string composed of HTTP method and uri. This wasn’t an issue with original JAX Doclet since output was JavaDoc html, and content organisation followed endpoint / uri / method hierarchy, analogous to package / class / method hierarchy - therefore descriptions were optional.
With Swagger2Markup the hierarhy is flat. For a single endpoint all you get is a list of method-uri items where each one’s description serves as its title.
There is one way to fix this - adding the missing JavaDoc descriptions to our code. But then a more readable form would be one without using descriptions as titles - but rather using “{method} {uri}” as a title and putting description in content underneath. Is there a way to do that with Swagger2Markup? The underlying library is certainly capable of it. But can we reach it through maven plugin? Doesn’t look like we can.
If we could do that, we also then wouldn’t have to fill out all the JavaDoc descriptions for our methods - many are self-explanatory, and don’t require a description.
At the end of the day the result we got is not perfect, we’ll have to do more work. Looking over at swagger-maven-plugin (the one that is based on Swagger annotations), it has a very flexible templating support which allows to define how content is generated, by using something called moustache templates. We could skip the whole swagger.json-to-asciidoc-to-html chain of three plugins, and do it all in one - much more elegant. But for our use-case we would first have to enhance that library to support auto-generating the default Swagger annotation values based not only on JAX RS annotations, but also on JavaDoc tags. That’s not a trivial undertaking, especially since here we’re dealing with a standalone maven plugin, not a doclet.
Another option would be to enhance Swagger2Markup library to give us more control over how the documentation content is organised, and make that available through its maven plugin.
Yet another option is to decide that the solution we got thus far is good enough, and not worth investing more time in.
We could also change our mind about swagger annotations. Many projects have gone the Swagger way. In the end we might be better off by simply putting swagger annotations in our code, and use swagger-maven-plugin.
There’s so much other important, and interesting things to do on Keycloak. Let's use this solution for now, and move on to other things. We'll revisit REST API docs generation some other time.
