Reporting JBehave results in AsciiDoc format.

JBehave is a framework for Behaviour-Driven Development (BDD). BDD is an evolution of test-driven development (TDD) and acceptance-test driven design, and is intended to make these practices more accessible and intuitive to newcomers and experts alike. The core of JBehave from the point of view of end users are user story files, where we define stories that our product should meet to validate that we are aligned with what our client wants. These stories are run every time a change has done in the project, so we can still ensure that nothing has been broken and we are still delivering the right thing. To validate this, a report is generated by JBehave so we can inspect the result of all stories and be sure that all is ok and if there is something wrong, which stories and why they have failed. By default the formats supported by JBehave are TXT, HTML or XML. 

With the advent of markup languages for writing documentation, it has sense to be able to embed this reports inside our documentation system, because the result of a build should be considered as documentation too so everyone involved directly or indirectly on the project can review the real state of the project. One of the trending markup language nowadays is AsciiDoc (and Asciidoctor). It is for this reason that I have developed a JBehave report extension so reports are written in AsciiDoc format and consequently be embedded inside another AsciiDoc document.

Let's see an example of a report generated using AsciiDoc reporter, then the report of story is included inside a master AsciiDoc document and finally rendered using Asciidoctor to HTML.

The first part of the document we can see the story that is being executed (calculator_mult_stories.story). Just below we can see the result of the story in one sentence and with an icon. In this case there is one failure in one step (see below) so an ambulance icon is used. If there would be no errors then a green rocket is used, and if any pending or ignored steps then a yellow lorry is used. After that, description, meta-information and narrative data about story is printed. Then given stories of this story is printed with the result of all of them. In this case one given story has been executed and his scenario has just passed correctly.

Note that you can navigate to the result of a given story by simply clicking on the link provided in Given Stories information section.

And the document continues with:

After user stories the scenarios of current story are printed with their results. In this example as in previous one note that examples are used and are shown as a table. Finally all steps executed with its result are printed. Note that the final step have just failed and the cause of the error is shown.

So as you can see you can embed test result inside another AsciiDoc document and make them look clear for anyone who wants to review the state of the project.

But now let's see how to use AsciiDoc reporter inside JBehave.

Reporter is released on bintray, so the first thing is to add as repository maven bintray.

Then we can add the dependency on our project:

And now we can configure StoryReporterBuilder with adoc format.

And that's all, now when you run JBehave your reports will appear on output directory as HTML or TXT but also  as AsciiDoc.

We keep learning,
Alex.

He's the guy who's the talk of the town, with the restless gun, don't shoot broad out to fool him around (Lo chiamavano Trinità - Annibale E I Cantori Moderni)

Music: https://www.youtube.com/watch?v=w5HkxTLp5jA 

Asciidoctor Meets JBehave.

User story is one or more sentences in natural language or business language which captures the requirements of what end user wants to be implemented in the product we are developing.

User Stories typically have the form:

"In order to <receive benefit> as a <role>, I want <goal/desire>"

User stories are also documentation for our project, first of all because they express what the product should do, and second one because they will be used by our clients/stakeholders, to validate that we are building the right thing.

JBehave is a framework for Behaviour-Driven Development (BDD). BDD is an evolution of test-driven development (TDD) and acceptance-test driven design, and is intended to make these practices more accessible and intuitive to newcomers and experts alike. The core of JBehave from the point of view of end users are user story files, where we define stories that our product should meet to validate that we are aligned with what our client wants. A user story example in JBehave is:

Note that first sentence is a description, then a narrative section which contains sentences of the form in order to, as a, I want. And finally the definition of scenarios that must meet this story to be considered completed.

So as you can see, story files are project documentation, and as project documentation would be awesome if we could add them inside our documents as documentation and not as simple plain text. And here is where Asciidoctor comes to action. Asciidoctor is a Ruby, Java and Javascript processor for converting AsciiDoc source files and strings into HTML 5, DocBook 4.5 and other formats. Your project may contain different kind of documents written in AsciiDoc like Software Design Specifications (SDS), manuals, or for example a test plan. And it is in test plan where including user stories, could be a really good improvement. And for this reason jbehave-asciidoctorj-extension has been developed.

Let's see an example. Suppose that we are writing a document where we want to test some user stories manually (apart of automatically of course) due to the criticality of the process:

As you can see we have created a simple AsciiDoc document with some information. But the interesting part is include::jbehave::userStories:src/**/*.story[initialLevel=1, manual=true]. Note that we are including all jbehave userstories that are placed on src folder. This extension supports any valid glob expression. And at the end we can set two attributes, the first one is the initial level of the section where user stories will be rendered (remember that the first one is 0), and finally if the steps will be executed manually or not, which has a deep impact on final rendered document as we will see soon.

Our project has two story files:

As you can see the second story has a dependency to the first story, which means that this second story is only valid if first story has passed successfully. Apart from this fact, we can set examples as tables as well.

So let's see an screenshot of previous document rendered:

I think it is important to note two blocks. The first one is the NOTE block. See that GivenStories are rendered as Admonition, and with a link pointing to another story; this is pretty important because clicking there it will move the page where that (precondition) story was defined, so you can also inspect what was that story about. The other interesting part is the Result part. Because we have set user stories as manual, the extension will render three checkboxes so tester can mark if the scenario passes, fails or also if he wants to add some comment.

So now there is no excuse to not adding user stories in your documents in a really fashion way.

You can see the code of this extension at: https://github.com/lordofthejars/asciidoctorj-extensions/tree/master/jbehave-asciidoctorj-extensions

We keep learning,

Alex.

Maman dit Annotate"travailler c'est bien", Bien mieux qu'être mal accompagné, Pas vrai ? (Papaoutai - Stromae)

Music: http://www.youtube.com/watch?v=oiKj0Z_Xnjc

Are You Locked Up In A World That's Been Planned Out For You? Are You Feeling Like A Social Tool Without A Use? (She - Green Day)

From JBehave site: JBehave is a framework for Behaviour-Driven Development (BDD). JBehave allows developers, QA and non-technical or business participants, to write stories in a plain text file with minimal restrictions about grammar. Then a POJO is created for executing created story. This POJO should have the typical BDD structure Given, When and Then.

From Springsource site: Spring Framework is a Java platform that provides comprehensive infrastructure support for developing Java applications. Spring handles the infrastructure so you can focus on your application.

From Selenium site: Selenium is a suite of tools to automate web app testing across many platforms.


We have three technologies JBehave for acceptance tests, Selenium for web application testing and Spring dealing with infrastructure. In this post I will talk about integrating JBehave with Selenium 2 and Spring.

For this example I have created a very simple web application using Spring MVC. I know that business logic is not accurate to the reality, but it is simple enough to illustrate how to integrate all these technologies together.


I have divided this post in three main subsections:

• Integrating JBehave with Spring. There is no web application here, only business logic.
• Integrating Spring MVC with Selenium 2. Only to show how easy is implementing automated tests with Selenium 2.
• Integrating JBehave with Selenium 2. Web application used in previous example but instead of using only Selenium for automating testing, JBehave instructs to Selenium which steps should execute.

Application that I will use is a simple TraderService, (explained in JBehave page (http://jbehave.org/reference/stable/)). This TraderService generates Stocks, and if a Stock is traded below threshold, its alert status is OFF and if it is traded above, alert status is ON.

In this tutorial I assume that you have a basic idea of JBehave and Spring.

• Integrating JBehave with Spring:

In this case no web GUI is used, we are going to use JBehave for writing acceptance tests of business logic.

Basic classes are:

• TradingService, that defines a method for creating stocks. TradingServiceImpl is the implementation.
• Stock, that contains stock information and logic about its status.
• StockAlertStatus is Stock status Enum.

Acceptance test part:

First of all we should create a story. A story is where stakeholders, developers... should say what they want the application do, and which are the expected results for given parameters. In our case a story has been created for validating that alarm is OFF if trade value is under threshold, and ON otherwise.

The important parts of that file are: Scenario for describing what we are testing, all symbols between <> that are used as variables, and Examples that are values injected to previous "<>" variables. In this story file two examples are provided, so two executions will be produced, one for each row. As final note, Given, When, Then words should be placed at start and are reserved words, also more than one Given, When or Then could be used in each story.

Next step, create a class that transforms a story written in "natural language" to code. We could say that this class is equivalent of creating a junit class; In JBehave these classes are called Steps. Because we are integrating with Spring an annotation called Steps is created. This annotation extends from Component annotation so Spring component-scan can wire up step classes too.

And Steps annotation is used in TradingServiceSteps class.

In TradingServiceSteps is where all magic occurs. This class is responsible of transforming story file to an execution. Let's see:

@Steps because we want Spring creates automatically this bean. TradingService is the business logic we want to test, and is injected using Autowire annotation. And finally one method for each Given, When, Then. Explained quickly, when JBehave finds an @Given, it searches into loaded stories for a phrase starting with Given. After that checks if @Given string value matches the Given definition expressed in story file. If matches then inject the story parameters as method parameters, for example STK1 as string parameter, or 5 as double threshold parameter. Moreover, in this case because we are using Examples in our story file, @Named annotation for each parameter is required. The named parameters allow the parameters to be injected using the table row values with the corresponding header name. Each parameter is converted from String to required parameter type.

We have written stories, and how to execute them (TradingServiceSteps class). JBehave requires another class, that will be responsible of configurating it. Basically you should configure Step classes and story files location, and what kind of reports are generated.

In our case, because we are integrating JBehave with Spring, some information is provided using Spring Injection.

This class is where JBehave is configured and is responsible for running all stories. Let's examine the most important lines:

In line 1 we specify a JUnit runner for running JBehave stories with Spring.
In line 2 we are configuring JBehave with Enum parameter converter, see that StockAlertStatus is an enum, because it is not a primitive parameter, a converter should be provided. JBehave comes with some convertes, but we can implement ours too.
In line 3 the embedder that we will use. This is the standard embedder for JBehave. Embedder represents an entry point to all of JBehave's functionality that is embeddable into other launchers.
And finally with @UsingSpring we are providing two Spring files, one where step classes are defined, and the other one where JBehave is configured.

Configuration file is a standard Spring file injecting required JBehave parameters:

This is a generic configuration file, that I use in all projects. You configure the output, the classloader for Embedder and prefix for parameters

And finally a Spring context file where all step classes are defined. And you know what, thanks of Spring this is as simple as:

No magic, remember that each Step class has an @Steps annotation? Thanks of component-scan, you don't have to define each Step class in @UsingSteps annotation or using tags.

Now run previous class as JUnit, and reports with results are generated.

• Integrating Spring MVC with Selenium 2

Selenium 2 is a suite of tools to automate web app testing across many platforms. In this case WebDriver approach has been used. WebDriver is an interface for automating tests in a programmatic way. Selenium provides several implementations depending on browser where tests are run.

For this example I have created an Spring MVC application, that are composed of two pages, one form where all stock information is provided and a page where status of inserted stock is showed. Of course Spring MVC controller for managing all information is also implemented.

Controller of this small application is:

showForm method is used for showing the form where user will write stock information. submitForm method is called when submit button is pushed, and creates an stock and send to showstatus page the status of stock.

StockForm is simply a class with three attributes (stock, threshold and tradeAt price). No secret here.

Form page is also so simply but I will show it because form information will be used for configuring Selenium:

Page for showing status:

WebDriver is used in JUnit test for automatizing a sequence of events. In this case, the sequence will create an stock below threshold and assert that response page shows that alert status is OFF.

Most important sections of previous JSP are:

JSP taglibs <form:input path=""/> like <form:input path="name"/> in form, and <div id="result">. These fields are important because they are used by Selenium for filling form and asserting showed status.

For example, in Selenium class:

WebElement name = driver.findElement(id("name")) returns a "reference" to <input id="name" type="text"> element and using sendKeys method, you are sending keyboard chars to that component.

WebElement element = driver.findElement(id("result")), returns a "reference" to div element and using
assertThat(element.getText(), is(StockAlertStatus.OFF.name())); getText method, none tag characters of element are returned.

Now running this test is as simple as running TraderIsAlertedSelenium class as a simple JUnit test class. When running this class a browser (Firefox in this case) will be opened, and all programmed interactions will be executed on your screen.

At this point we just have to join both previous parts, and integration between JBehave with Spring and Selenium will be reality.

• Integrating JBehave with Selenium 2 and Spring

JBehave has a module called JBehave-Web, that is used for integrating JBehave with web pages. Base classes are WebDriverProvider and WebDriverPage. Both classes are used by JBehave for abstracting from browser, and also for providing common methods to test webpages. In this example I won't use jbehave-web for two reasons, first because Selenium 2 with WebDriver offers a level of abstraction that is enough for this example, and secondly because WebDriverPage is a class that implements some common funcionalities for testing, but it is abstract, I don't like using extension only for sharing common operations between classes, it is a bad practice (not discussed here), I prefer aggregation. So in this case I have preferred  implementing my class for implementing common functionalities.

In this case abstraction from browser is acquired using WebDriver (Selenium) interface. Moreover all common operations are implemented into this class. The idea of this class is to be used in several projects and for that reason a better design should be desired, but for current example is enough.

Next group of classes are those that use PageUtils object. I have created one class for each page that Selenium should interact with. Acts as a facade to web.

For example class for dealing with page containing form to insert new stock is:

Three operations can be executed in this page, the first one is open the page. Because "insert a new stock" is accessed manually (in this case is the front page), an open method is provided with URL. Also a method for filling stock form and and another for submitting it are provided.

And finally a class that transforms an story written in "natural language" to code (also known as Steps class), this class would be the same used in first example (TradingServiceSteps) but adapted for dealing with web pages (using previous classes).

See that there is no differences between this class and the one created in first example, but using web page interfaces instead of business objects. 

Next modified files are:

Story file:

that has been modified to use web terminology.

Spring file:

that injects into TradingServiceWebSteps required beans.

Configuration file used in first example is the same, and Spring file for configuring JBehave is the same too.

In summary I can definitely say that integrating JBehave with Selenium 2 and Spring is not a difficult task, compared with the benefits that lead us having an automated acceptance test platform. I wish you have found this post useful.

Download Full Code