Auto-numbered Callouts in Asciidoctor

Asciidoctor 1.5.8 comes with a really nice feature which is called autonumber callouts, so you do not have to specify the number of the callout, but just a generic character (.) and then at rendering time, Asciidoctor will set the name correctly.

See the next example shows the number and auto-number feature for callouts:

And the output looks like:

So as you can see the output is exactly the same in both cases, but in the first case the number of the callout is static meanwhile in the second one autonumber feature is used.

Using autonumbering feature is really useful when you've got big blocks of code where you might introduce some callouts into already defined callouts which means a shifting of all of them.

Meanwhile, with the first approach means you need to go to every callout and increase the number manually, in the later one you only need to add a new callout and that's all, you do not need to go manually increasing the callout number.

We keep learning,

Alex.

My body is burning, it starts to shout, Desire is coming, it breaks out loud (Rock You Like Hurricane - Scorpions)

Music: https://www.youtube.com/watch?v=AQRprgbYdTw
Follow me at https://twitter.com/alexsotob

Continuous Documentation with Antora and Travis

Antora is a documentation pipeline that enables docs, product, and engineering teams to create, manage, remix, and publish documentation sites composed in AsciiDoc and sourced from multiple versioned content repositories.

You can see several examples out there from Couchbase documentation to Fedora documentation. And of course, Antora documentation is used to generate Antora documentation. You can see it here.

So basically we have our project with documents in adoc format. Then what we want is regenerating the documentation every time a PR is merged to master.

In our project, we are using Travis-CI as CI server, so I am going to show you how we have done.

First of all, you need to create a .travis.yml file on the root of your project.

First, we define what we want to use. In this case docker and git.

Then in before_install section, we are detecting if we need to regenerate documentation or not.

Basically, we are going to generate documentation in two conditions:

1. If commit message contains the word doc, then docs should be regenerated.
2. If you have modified an adoc file from the documentation folder (or README.adoc) and the branch is master, then the docs should be regenerated.

If any of these conditions are met, then we configure git client with user, email and token to be used for pushing the generated documentation. Notice that this information comes from environment variable defined in Travis console. Also, it is important to note that the documentation should be generated in gh-pages branch (since we are releasing to GitHub pages). For this reason, we are using git worktree which checkouts the gh-pages branch in gh-pages directory.

Then in script section, we are just using Antora docker image to render documentation.

Finally, we just need to enter into gh-pages directory, create a .nojekyll file to avoid Git Hub Pages thinks that this is a Jekyll site, and finally push the changes.

And then for PR merged, the documentation is automatically regenerated and published.

Important: This script is based on one done previously by Bartosz Majsak (@majson) for Asciidoctor. My task has been only adapting it to use Antora.

We keep learning,

Alex.

Y no me importa nada nada (nada), Que rías o que sueñes, que digas o que hagas, Y no me importa nada, Por mucho que me empeñe, estoy jugando y no me importa nada (No me importa nada - Luz  Casal)

Music: https://www.youtube.com/watch?v=YkS-ttHspLs

Follow me at https://twitter.com/alexsotob

Docstract-0.2.0 Released.

Docstract is an small java project which reads a java source file, extracts the comments between /** and */ sequentially from all over the file and stores them inside an output AsciiDoc file. The idea is not new in fact it is from https://github.com/javaee-samples/javaee7-samples/ developed by Aslak Knutsen but instead of using Ruby and inside Awestruct, this project is written in java and it is run from CLI.

Let me show you an example to understand what this application does:

This is a class which contains some comments to be processed.

And the output document looks like:

Note that in comments we are also using include macro for inserting files inside. The rules for inclusion are the next ones:

• if include contains a java file, like first example, the whole class is read and inserted within AsciiDoc source code blocks.
• if include contains a java file but with # symbol, like second one, the right part of # will be used as method name of given class. So for example Project#setId(String, String) will include a method of Project's class called setId and receiving two string parameters.
• if include contains an xml file, the file is inserted "as is" within AsciiDoc source code block.
• if include contains an xml file and between brackets there is an xpath expression, the result of that expression will be inserted.
• any other include is left "as is", so it will be processed by AsciiDoc(tor) processor.

Also note that the include files will be resolved relative from place where the CLI is being run, typically from the root of the project.

You can use callouts inside java and xml code and the processor will render it in the proper way.

In java you can write a callout as a single line comment with callout number between <, > at start.

In xml you can write callouts between +xml+ comments and the callout number being the first word.

You can grab a runnable jar from https://bintray.com/lordofthejars/generic/Docstract/0.2.0/view/files

To render previous class we can call as:

java -jar docstract-<version>.jar --input=src/test/java/com/lordofthejars/asciidoctorfy/MM.java --output=README.adoc

And an AsciiDoc file will be generated from comments blocks.

There is one optional parameter called --baseDir. This parameter is used to set the baseDir in include macros. So for example if you set baseDir to /home/lotj/project, the include macro will be resolved to include::/home/lotj/project/src/test/java/....

You can see an example at community Tomitribe examples with class: https://github.com/tomitribe/community/blob/master/mongodb-example/src/test/java/org/superbiz/nosql/mongodb/GreetingsSessionBeanTest.java and the rendered output: https://github.com/tomitribe/community/blob/master/mongodb-example/README.adoc.

New features will be added when they are required but of course feel free to clone and improve it.

Project Site: https://github.com/lordofthejars/docstract

We Keep Learning,
Alex.

Weiß mit dem Locken umzugehn, Und mich auf's Pfeifen zu verstehn. Drum kann ich froh und lustig sein, Denn alle Vögel sind ja mein. (Die Zauberflöte - Wolfgang Amadeus Mozart)

Music: https://www.youtube.com/watch?v=PRtvVQ1fq8s

DRY with examples on documentation. Include Java Extension for Asciidoctor.

One of the great features that Asciidoctor offers us as a writers of manuals, tutorials and documentation in general is that allows us to follow the DRY pattern. If for example you are writing a tutorial, probably you will have one part of the tutorial which will be the documentation, and another part which will be source code with examples. Asciidoctor allows us to write the documentation and include the examples directly from source code project to document.

For example:

As you can see you can include directly from source code the example and Asciidoctor will render it inside the document. Moreover if you set the language, Asciidoctor will highlight the code too.

But as you can see there is one problem with this approach, you are including the whole file, with imports, javadoc, comments, ... and maybe you only want to show a method of that class. Now with Asciidoctor you have two options:

Using lines attribute:

Which have the problem that modifying original source code also modifies the code embedded inside tutorial.

Using tags:

Which is an invasive method that requires you modify the original source code:

But now we have got another option in case your language is Java. And it is using java-semantic-asciidoctorj extension. This extension allows us to embed parts of the code by its components. That is you can set which method to add, which class, if you want to add imports or not, ... The extension only works with asciidoctorj-1.5.0.preview1 and beyond, and uses the new SPI for extensions, so you only have to add the jar extension on classpath, and the extension is automatically registered.

An example of usage of this extension could be the next one. Let's imagine that we have a class with 5 methods and we want to include only a method called mymethod which receives an String as parameter and it has no return value. Also we want to add the imports, but nothing else.

We could do something like:

Note that now we are not invading our code with comment-tags, nor fixing the position with lines of the method inside our document.

The attributes you can set in include are:

• imports: adds the imports.
• fields: adds the fields of the main class.
• class: adds all the content of the main class.
• enum=<classname>: adds the enum with the given name. The enum should be defined inside the class we are adding.
• annotation=<classname>: same as enum but for annotations.
• class=<classname>: same as enum but for inner classes.
• contructor=<constructorName>(<parameterType1>,<parameterType2>, ....): adds the defined constructor. In this case we must set the constructor name and the type of each parameter. For example: MyClass(String).
• method=<returnType> <methodName>(<parameterType1>, <parameterType2>, ...): same as constructor but for methods, which implies adding the return type. Note that it is not required to add any modifier nor throws classes.

Now we can use our classes within our documentation and by adding the required block.

This version of plugin works with Java 1.7 and before, not with Java 1.8 but it could work in some cases.

The extension is published on bintray, so to install you simply have to add bintray repository and the required dependency:

The project is hosted at: https://github.com/lordofthejars/asciidoctorj-extensions

We keep learning,
Alex.

I feel shouting ya-hoo, And me still feeling hungry, Cowabunga!!, Cookie monster went and ate the new red two. Monster Went and Ate My Red 2 - Elvis Costello & Elmo

Music: https://www.youtube.com/watch?v=KxardpBReQc

Reporting JBehave results in AsciiDoc format.

JBehave is a framework for Behaviour-Driven Development (BDD). BDD is an evolution of test-driven development (TDD) and acceptance-test driven design, and is intended to make these practices more accessible and intuitive to newcomers and experts alike. The core of JBehave from the point of view of end users are user story files, where we define stories that our product should meet to validate that we are aligned with what our client wants. These stories are run every time a change has done in the project, so we can still ensure that nothing has been broken and we are still delivering the right thing. To validate this, a report is generated by JBehave so we can inspect the result of all stories and be sure that all is ok and if there is something wrong, which stories and why they have failed. By default the formats supported by JBehave are TXT, HTML or XML. 

With the advent of markup languages for writing documentation, it has sense to be able to embed this reports inside our documentation system, because the result of a build should be considered as documentation too so everyone involved directly or indirectly on the project can review the real state of the project. One of the trending markup language nowadays is AsciiDoc (and Asciidoctor). It is for this reason that I have developed a JBehave report extension so reports are written in AsciiDoc format and consequently be embedded inside another AsciiDoc document.

Let's see an example of a report generated using AsciiDoc reporter, then the report of story is included inside a master AsciiDoc document and finally rendered using Asciidoctor to HTML.

The first part of the document we can see the story that is being executed (calculator_mult_stories.story). Just below we can see the result of the story in one sentence and with an icon. In this case there is one failure in one step (see below) so an ambulance icon is used. If there would be no errors then a green rocket is used, and if any pending or ignored steps then a yellow lorry is used. After that, description, meta-information and narrative data about story is printed. Then given stories of this story is printed with the result of all of them. In this case one given story has been executed and his scenario has just passed correctly.

Note that you can navigate to the result of a given story by simply clicking on the link provided in Given Stories information section.

And the document continues with:

After user stories the scenarios of current story are printed with their results. In this example as in previous one note that examples are used and are shown as a table. Finally all steps executed with its result are printed. Note that the final step have just failed and the cause of the error is shown.

So as you can see you can embed test result inside another AsciiDoc document and make them look clear for anyone who wants to review the state of the project.

But now let's see how to use AsciiDoc reporter inside JBehave.

Reporter is released on bintray, so the first thing is to add as repository maven bintray.

Then we can add the dependency on our project:

And now we can configure StoryReporterBuilder with adoc format.

And that's all, now when you run JBehave your reports will appear on output directory as HTML or TXT but also  as AsciiDoc.

We keep learning,
Alex.

He's the guy who's the talk of the town, with the restless gun, don't shoot broad out to fool him around (Lo chiamavano Trinità - Annibale E I Cantori Moderni)

Music: https://www.youtube.com/watch?v=w5HkxTLp5jA 

Documentation is a Living System with Asciidoctor

In Agile methodology is very important to iterate over and over again walking small steps so all team (which I also include the customers), can receive quick feedback of how the project is going on. 

Aligned with this requirement, is where Continuous Integration/Continuous Delivery has sense. Every new commit, implies among a lot of other stages compile, execute unit/integration tests, execute code quality or execute acceptance tests. All these kind of tests generate some kind of documentation (usually an HTML file) as a report. To improve the visibility of the project to all team members, it is a good practice to publish these reports to a webserver so all members can access to it and inspect the result of that compilation and the progress of the project.

But with the advent of lightweight markup languages like Markdown or AsciiDoc, the documentation of the projects is (or should be) treated as source code too, so it should be delivered as any other part of the system like for example test reports, so that every member of the team should see exactly the progress of the documentation at the time of packaging. 

In this post we are going to see how to acquire this we are going to use Asciidoctor-Maven-Plugin and Maven-Assembly-Plugin, to create a war file containing your documentation which at the end could be uploaded to a server using for example Cargo-Maven-Plugin or in your Jenkins pipeline.

The first thing to do is create a project with source code, test code and of course documentation. 

Because it is a good idea to treat documentation as code (it would require another post to talk about it), we can put documentation inside src/main/resources, and create a document for each module inside the package of the module. So for example if in src/main/java/com/mycompany/users we have all code developed about user management, you will put documentation about user management in src/main/resources/com/mycompany/users, or if you prefer src/main/docs/com/mycompany/users, but as you can see the documentation should be very close to developer, in fact in same project workspace. 

You can see a full example in https://github.com/lordofthejars/foobank.

And now it is time to render documentation each time the build is done and package it into a war file.

To render the documentation we are going to use Asciidoctor-Maven-Plugin and Assembly-Maven-Plugin.

Let's see how to configure Asciidoctor-Maven-Plugin:

The plugin renders all AsciiDoc files that are at src/main/resources folder and its subfolders. They are rendered in HMTL format, and by default are rendered at target/generated-docs.

Now the we have all our documentation files rendered as HTML, it is time to package them inside a WAR file so we can deploy them into a web server using Assembly-Maven-Plugin.

The name of the output file is foobank because we are using finalBuild tag, and finally Assembly plugin requires a file called assembly.xml to configure which files are included in package and the output directory.

Note that we are adding a suffix to the output file by using id tag, so the final name will be foobank-doc.war. And finally we setup that all files inside target/generated-docs will be packaged.

And now you can deploy this WAR file in your documentation server, so every team on your project will be able to read the latest content generated at the same time of compiling, packaging or deploying your application. Your documentation is treated as code.

You can see a full example in https://github.com/lordofthejars/foobank.

We keep learning,
Alex.

Desolation comes upon the sky, Now I see fire, Inside the mountain, I see fire (I See Fire - Ed Sheeran)

Music: http://www.youtube.com/watch?v=DzD12qo1knM

Asciidoctor Meets JBehave.

User story is one or more sentences in natural language or business language which captures the requirements of what end user wants to be implemented in the product we are developing.

User Stories typically have the form:

"In order to <receive benefit> as a <role>, I want <goal/desire>"

User stories are also documentation for our project, first of all because they express what the product should do, and second one because they will be used by our clients/stakeholders, to validate that we are building the right thing.

JBehave is a framework for Behaviour-Driven Development (BDD). BDD is an evolution of test-driven development (TDD) and acceptance-test driven design, and is intended to make these practices more accessible and intuitive to newcomers and experts alike. The core of JBehave from the point of view of end users are user story files, where we define stories that our product should meet to validate that we are aligned with what our client wants. A user story example in JBehave is:

Note that first sentence is a description, then a narrative section which contains sentences of the form in order to, as a, I want. And finally the definition of scenarios that must meet this story to be considered completed.

So as you can see, story files are project documentation, and as project documentation would be awesome if we could add them inside our documents as documentation and not as simple plain text. And here is where Asciidoctor comes to action. Asciidoctor is a Ruby, Java and Javascript processor for converting AsciiDoc source files and strings into HTML 5, DocBook 4.5 and other formats. Your project may contain different kind of documents written in AsciiDoc like Software Design Specifications (SDS), manuals, or for example a test plan. And it is in test plan where including user stories, could be a really good improvement. And for this reason jbehave-asciidoctorj-extension has been developed.

Let's see an example. Suppose that we are writing a document where we want to test some user stories manually (apart of automatically of course) due to the criticality of the process:

As you can see we have created a simple AsciiDoc document with some information. But the interesting part is include::jbehave::userStories:src/**/*.story[initialLevel=1, manual=true]. Note that we are including all jbehave userstories that are placed on src folder. This extension supports any valid glob expression. And at the end we can set two attributes, the first one is the initial level of the section where user stories will be rendered (remember that the first one is 0), and finally if the steps will be executed manually or not, which has a deep impact on final rendered document as we will see soon.

Our project has two story files:

As you can see the second story has a dependency to the first story, which means that this second story is only valid if first story has passed successfully. Apart from this fact, we can set examples as tables as well.

So let's see an screenshot of previous document rendered:

I think it is important to note two blocks. The first one is the NOTE block. See that GivenStories are rendered as Admonition, and with a link pointing to another story; this is pretty important because clicking there it will move the page where that (precondition) story was defined, so you can also inspect what was that story about. The other interesting part is the Result part. Because we have set user stories as manual, the extension will render three checkboxes so tester can mark if the scenario passes, fails or also if he wants to add some comment.

So now there is no excuse to not adding user stories in your documents in a really fashion way.

You can see the code of this extension at: https://github.com/lordofthejars/asciidoctorj-extensions/tree/master/jbehave-asciidoctorj-extensions

We keep learning,

Alex.

Maman dit Annotate"travailler c'est bien", Bien mieux qu'être mal accompagné, Pas vrai ? (Papaoutai - Stromae)

Music: http://www.youtube.com/watch?v=oiKj0Z_Xnjc

Are Configuration Files Documentation? ... Yes!!! TomEE + Asciidoctor Example

Normally in our projects we generate a considerable amount of documentation. Some documentation is directly a document written as is, for example Software Requirements Specifications (SRS) or Software Design Specifications (SDS), but there are other pieces of our projects that although they are not a document directly, they are consulted by developers/testers/managers, customers ... and treat it as is, for example user stories written by developers and/or QA in Cucumber or JBehave format or  javadoc. And now I ask to myself, are configuration files documentation too? Are people interested in having them in a document and treat them as documentation? And the answer is YES!!!. In this post we are going to see how to transform TomEE's configuration file into a document using an Asciidoctor extension I have developed specially for this occasion.

In TomEE, resources and containers can be configured inside a file called tomee.xml among other places. But what really makes interesting is that a subset of this file can be put inside the project, so each web application is responsible of configuring their own resources, for example DataSources, JMS Queues, ....  And having the ability of putting specific configuration of the server inside the application also opens the doors of creating more than one configuration file per application, and load one or another depending on the environment we are going to deploy the application. For example:

• In developer environment the DataSource could point to an embedded database.
• In QA environment the DataSource could point to a database installed on localhost.
• In Stage environment the same DataSource points to an external IP1.
• In Production environment the same DataSource points to an external IP2.

So as you can see, it would be interesting to have all these information as readable documentation, so everyone involved in the project could review it and for example know exactly which servers are being used in each environment as datasources, or configuration parameters like pool size of stateless manager.

And this is exactly what tomee-asciidoctor-extension does. It allows us to embed Apache TomEE configuration files into an AsciiDoc documentation.

Let's see an example of Apache TomEE configuration file:

This is a typical configuration file where for example you can see some DataSources defined, custom resource or definition of TransactionManager.

Probably your projects will have a document called for example infrastructure (or something similar) where all information related with the software that is being used in the project and  environments used will be described.

A simple example of these kind of documents can be:

Note that besides describing the software that is going to be used for solving our problem, in this case an Apache TomEE, we are using the tomee-asciidoctor-extension to add the Apache TomEE's container and resources configuration file, so when someone wants to inspect the documentation will also see exactly the configuration values that had Apache TomEE at time of generating the document. If you look carefully you only need to add next line to transform a Apache TomEE configuration file into documentation:

include::tomee:/home/alex/git/asciidoctorj-extensions/tomee-asciidoctorj-extensions/src/test/resources/resources.xml[initialLevel=2]

In this case we are including a file called resources.xml. Note that in order to extension detects an Apache TomEE configuration file, we need to use the special keyword tomee: followed by the location of the file. Also note that we are using an attribute called initialLevel. Because we are embedding one document into another one, we need a way to set at which level the document will be rendered, and this is what this attribute is designed for.

Previous document rendered with default style sheet looks like:

See that thanks of Asciidoctor and its extension system, we are able to convert configuration files to documentation.

You can see the code of this extension at: https://github.com/lordofthejars/asciidoctorj-extensions/tree/master/tomee-asciidoctorj-extensions

We keep learning,
Alex.

Siente y baila y goza, Que la vida es una sola, Voy a reír, voy a bailar, Vive, sigue, Siempre pa'lante (Vivir Mi Mida - Marc Anthony)

Music: http://www.youtube.com/watch?v=YXnjy5YlDwk

Enjoy the magic of asciidoctor in java with asciidoctor-java-integration

The asciidoctor-java-integration is the official means of using Asciidoctor to render all your AsciiDoc documentation using Java instead of Ruby.

Installation

Since asciidoctor-java-integration is a standard jar file, the only thing you should do is add library into classpath.

Dependency declaration in Maven

... <dependencies> <dependency> <groupId>org.asciidoctor</groupId> <artifactId>asciidoctor-java-integration</artifactId> <version>${asciidoctor.version}</version> <1> ... </dependency> </dependencies> ...

1. As this library tracks the version of asciidoctor, you can use which every version of asciidoctor you prefer.

Usage

The core interface of asciidoctor-java-integration is Asciidoctor interface. It provides two methods for rendering asciidoc content, render and renderFile. Both of them returns a string with rendered content.

Also a factory method is provided to create an instance of Asciidoctor interface.

Creation of Asciidoctor interface

import static org.asciidoctor.Asciidoctor.Factory.create; import org.asciidoctor.Asciidoctor; ... Asciidoctor asciidoctor = create(); ...

And then we can call render methods depending on our requirements.

Rendering a String

... String rendered = asciidoctor.render("*This* is it.", Collections.EMPTY_MAP); System.out.println(rendered); ...

But also you can render the content of a file.

Rendering a File

... String rendered = asciidoctor.renderFile("target/test-classes/rendersample.asciidoc", Collections.EMPTY_MAP); System.out.println(rendered); ...

Options

Asciidoctor supports different kind of options, like in_place which renders the output inside a file, template_dir used to provide a directory of Tilt-compatible templates to be used instead of the default built-in templates, or for example attributes option where we can set key-value pairs of attributes that will be used within asciidoc document.

The second parameter of render methods are a java.util.Map where we can set all these options.

Example of using in_place Option and backend Attribute

Map<String, Object> attributes = new HashMap<String, Object>(); attributes.put("backend", "docbook"); Map<String, Object> options = new HashMap<String, Object>(); options.put("in_place", true); options.put("attributes", attributes); String render = asciidoctor.renderFile("target/test-classes/rendersample.asciidoc", options);

See that in previous example we have created a Map, where we have put the options and attributes (creating a Map too) required to render input as docbook and generate an output file.

But asciidoctor-java-integration also provides two builder classes to create these maps in a more readable form.

AttributesBuilder is provided for creating a map with required attributes set, and OptionsBuilder can be used for options. Previous example but using these classes looks like:

Example setting attributes and options

import static org.asciidoctor.AttributesBuilder.attributes; import static org.asciidoctor.OptionsBuilder.options; ... Map<String, Object> attributes = attributes().backend("docbook").asMap(); Map<String, Object> options = options().inPlace(true).attributes(attributes).asMap(); String render = asciidoctor.renderFile("target/test-classes/rendersample.asciidoc", options); ...

Utilities

A utility class for searching all asciidoc files present in a root folder and all its subfolders is given. In fact it finds all files that end up with .asc, .asciidoc, .ad or .adoc. This class is AsciidocDirectoryWalker.

Example of finding all asciidoc

DirectoryWalker directoryWalker = new AsciidocDirectoryWalker("target/test-classes/src"); List<File> asciidocFiles = directoryWalker.scan();

We keep learning,
Alex.

Cold, cold heart, Hard done by you, Some things look better, baby, Just passing through (Sacrifice - Elton John)

http://www.youtube.com/watch?v=NrLkTZrPZA4