Testing Spring Data + Spring Boot applications with Arquillian (Part 2)

In previous post, I wrote about how to test Spring Data application using Docker with Arquillian Cube. The test looked like:

This test just starts Redis container, then populate data using restTemplate and post method, then execute the logic under test (testing GET HTTP method) and finally stop the Redis container.

It is good, it works but there are several problems there:

• The first one is that we are using REST API to prepare data set of the test. The problem here is that the test might fail not because a failure on code under test but because of the preparation of the test (insertion of data).
• The second one is that if POST endpoint changes format/location, then you need to remember to change everywhere in the tests where it is used.
• The last one is that each test should leave the environment as found before execution, so the test is isolated from all executions. The problem is that to do it in this approach you need to delete the previous elements inserted by POST. This means to add DELETE HTTP method which might not be always implemented in endpoint, or it might be restricted to some concrete users so need to deal with special authentication things.

To avoid this problem Arquillian Persistence Extension (aka APE) was created. This extensions integrates with DBUnit and Flyway for SQL databases, NoSQLUnit for No SQL databases and Postman collections for REST services so you can populate your backend before testing the real test use case and clean the persistence storage after the test is executed.

Also population data is stored inside a file, so this means that can be reused in all tests and easily changed in case of any schema update.

Let's see example of Part 1 of the post but updating to use APE.

And the file (pings.json) used for populating Redis instance with data looks like:

Notice that in this test you have replaced the POST calls for something that directly inserts into the storage. In this way you avoid any failure that might occurs in the insertion logic (which is not the part under test). Finally after each test method, Redis instance is cleaned so other tests finds Redis clean and into known state. 

Project can be found at https://github.com/arquillian-testing-microservices/pingpongbootredis

We keep learning,
Alex

Y es que no puedo estar así, Las manecillas del reloj, Son el demonio que me tiene hablando solo (Tocado y Hundido - Melendi)

Music: https://www.youtube.com/watch?v=1JwAr4ZxdMk

3 ways of using Docker Containers for Testing in Arquillian

Arquillian Cube is an Arquillian extension that can be used to manager Docker containers from Arquillian.

With this extension you can start a Docker container(s), execute Arquillian tests and after that shutdown the container(s).

The first thing you need to do is add Arquillian Cube dependency. This can be done by using Arquillian Universe approach:

Then you have three ways of defining the containers you want to start.

The first approach is using docker-compose format. You only need to define the docker-compose file required for your tests, and Arquillian Cube automatically reads it, start all containers, execute the tests and finally after that they stop and remove them.

In previous example a docker compose file version 2 is defined (it can be stored in the root of the project, or in src/{main, test}/docker or in src/{main, test}/resources and Arquillian Cube will pick it up automatically), creates the defined network and start the service defined container, executes the given test. and finally stops and removes network and container. The key point here is that this happens automatically, you don't need to do anything manual.

The second approach is using Container Object pattern.  You can think of a Container Object as a mechanism to encapsulate areas (data and actions) related to a container that your test might interact with. In this case no docker-compose is required.

In this case you are using annotations to define how the container should looks like. Also since you are using java objects, you can add methods that encapsulates operations with the container itself, like in this object where the operation of checking if a file has been uploaded has been added in the container object.

Finally in your test you only need to annotate it with @Cube annotation.

Notice that you can even create the definition of the container programmatically:

In this case a Dockerfile file is created programmatically within the Container Object and used for building and starting the container.

The third way is using Container Object DSL. This approach avoids you from creating a Container Object class and use annotations to define it. It can be created using a DSL provided for this purpose:

In this case the approach is very similar to the previous one, but you are using a DSL to define the container.

You've got three ways, the first one is the standard one following docker-compose conventions, the other ones can be used for defining reusable pieces for your tests.

You can read more about Arquillian Cube at http://arquillian.org/arquillian-cube/

We keep learning,
Alex

And did you think this fool could never win, Well look at me, i'm coming back again, I got a taste of love in a simple way, And if you need to know while i'm still standing you just fade away (I'm still Standing - Elton John)

Music: https://www.youtube.com/watch?v=ZHwVBirqD2s

Arquillian Chameleon for the sake of simplicity

When using Arquillian, one of the things you need to do is defining under which container you want to execute all your tests.

And this is done by adding a dependency in the classpath for the adapter and depending on the mode used (embedded, managed or remote) having to download the application server manually. For example this happens when Wildfly is used in embedded or managed mode.

An example of a pom.xml using Wildfly could be:

Notice that in previous script, you need to define the Arquillian adapter, in this case the managed one, and use maven-dependency-plugin to download Wildfly distribution file used by Arquillian.

This approach is good and it works, but it has three drawbacks:

1. You need to repeat all these lines in every build script you want to use Arquillian and Wildfly.
2. In case you need to use another application server in another project, you need to know which adapter artifact is required and if it is necessary to download  the artifacts or not. For example in case of Jetty embedded it is not necessary to download any distribution, you only need set the embedded dependency.
3. If you want to test your code against several application servers you have the problem number 2 plus start dealing with profiles.

But all these problems can be fixed using Arquillian Chameleon. Arquillian Chameleon is a generic container which reads from arquillian.xml which container, which version and which mode you want to use in your tests, and he will take care of adding required adapter into classpath, download any required distribution and configure the protocol (this is something that as a user you should not touch).

How to use Arquillian Chameleon is pretty easy. Do whatever you would do normally such as adding Arquillian bom and add Chameleon Container instead of any application-server specific artifact:

Then create in src/test/resources the Arquillian configuration file called arquillian.xml with next configuration:

Notice that now you only need to use a friendly property called chameleonTarget to define which container, version and mode you want to use. In previous example Wildfly 9.0.0.Final with managed adapter.

When running any test with this configuration, Chameleon will check if Wildfly 9.0.0.Final distribution is downloaded, and if not download it, then will add to classpath the managed adapter for Wildfly 9.0.0 and finally execute the test as any other Arquillian test.

What's happening if you want to use Payara instead of Wildfly? You only need to change chameleonTarget property to payara:4.1.1.163:managed, to for example run tests against Payara 4.1.1 in managed mode.

TIP: You can set this property using a Java system property (-Darq.container.chameleon.chameleonTarget = payara:4.1.1.163:managed)

Currently next containers are supported by Chameleon:

• JBoss EAP 6.x, 7.x
• WildFly 10.x, 9.x, 8.x
• JBoss AS 7.x
• GlassFish 3.1.2, 4.x
• Payara 4.x

We keep learning,
Alex.

I can see you, Your brown skin shining in the sun, I see you walking real slow(The boys of summer - The Ataris)

Music: https://www.youtube.com/watch?v=Qt6Lkgs0kiU

Container Object pattern. A new pattern for your tests.

If you search for a description of what Page Object is, you’ll find that The Page Object Pattern gives us a common sense way to model content in a reusable and maintainable way.

And also points that: Within your web app’s UI there are areas that your tests interact with. A Page Object simply models these as objects within the test code.
This reduces the amount of duplicated code and means that if the UI changes, the fix need only be applied in one place.

As you can see, Page Object applies to UI elements. We (the Arquillian community) has coined a new pattern following Page Object pattern logic called Container Object pattern.
You can think about Container Object as areas of a container (for now Docker container) that your test might interact with. For example some of these areas could be:

• To get the host IP where container is running.
• The bounded port for a given exposed port.
• Any parameter configured inside the configuration file (Dockerfile) like a user or password to access to the service which the container exposes.
• Definition of the containers.

A Container Object might contain an aggregation of more than one Container Object inside it. This effectively builds a relation ship (link) between containers.

An example of configuration parameters might be for example, in case of running a MySQL database in a container, it could be the user and password to access to database. 

Notice that nothing prevents you to generate the correct URL for accessing to the service from the test, or execute commands against container like retrieving an internal file.

And of course as Page Object does, Container Object gives you a way to build a model content that can be reused for several projects.

Before looking at how this pattern is implemented in Arquillian Cube, let’s go thorough an example:

Suppose all of your applications need to send a file to an FTP server. To write an integration/component test you might need a FTP server to send the file and check that the file was correctly sent.
One way to do this is using Docker to start a FTP server just before executing the test, then execute the test using this Docker container for FTP server, before stopping the container check that the file is there, and finally stop the container.

So all these operations that involves the FTP server and container could be joined inside a Container Object. This container object might contain information of:

• Which image is used
• IP and bounded port of host where this FTP server is running
• Username and password to access to the FTP server
• Methods for asserting the existence of a file

Then from the point of view of test, it only communicate with this object instead of directly hard coding all information inside the test.
Again as in Page Object, any change on the container only affects the Container Object and not the test itself.

Now let’s see how Arquillian Cube implements Container Object pattern with a very simple example:

Arquillian Cube and Container Object

Let’s see a simple example on how you can implement a Container Object in Cube. Suppose you want to create a container object that encapsulates a ping pong server running inside Docker.
The Container Object will be like a simple POJO with special annotations:

In previous example you must pay attention at next lines:

1. @Cube annotation configures Container Object.
2. A Container Object can be enriched with Arquillian enrichers.
3. Bounded port is injected for given exposed port.
4. Container Object hides how to connect to PingPong server.

@Cube annotation is used to configure this Container Object. Initially you set that the started container will be named pingpong and the port binding information for the container instance, in this case 5000→8080/tcp.
Notice that this can be an array to set more than one port binding definition.

Next annotation is @CubeDockerFile which configure how Container is created. In this case using a Dockerfile located at default classpath location. The default location is the package+classname, so for example in previous case, Dockerfile should be placed at org/superbiz/containerobject/PingPongContainer directory.
Of course you can set any other class path location by passing as value of the annotation. CubeDockerFile annotation sets the location where the Dockerfile is found and not the file itself.
Also this location should be reachable from ClassLoader, so it means it should be loaded from classpath in order to find it.

Any Cube can be enriched with any client side enricher, in this case with @HostIp enricher, but it could be enriched with DockerClient using @ArquillianResource as well.

Finally the @HostPort is used to translate the exposed port to bound port.
So in this example port value will be 5000. You are going to learn briefly why this annotation is important.

And then you can start using this container object in your test:

The most important thing here is that you need to set Container Object as a field of the class and annotate with @Cube.

It is very important to annotate the field with Cube, so before Arquillian runs the test, it can detect that it needs to start a new Cube (Docker container), create the Container Object and inject it in the test.

Notice that this annotation is exactly the same as used when you defined the Container Object.
And it is in this way because you can override any property of the Container Object from the test side. This is why @HostPort annotation is important, since port can be changed from the test definition, you need to find a way to inject the correct port inside the container object.

In this post I have introduced Container Object pattern and how can be used in Arquillian Cube. But this is only an small taste, you can read more about Arquillian Cube and Container Object integration at https://github.com/arquillian/arquillian-cube#arquillian-cube-and-container-object

Also a running examples can be found at https://github.com/arquillian/arquillian-cube/tree/master/docker/ftest-docker-containerobject

We keep learning,
Alex.

It's time to see what I can do, To test the limits and break through, No right, no wrong, no rules for me, I'm free! (Let It Go - Idina Menzel) 

Music: https://www.youtube.com/watch?v=moSFlvxnbgk

Java EE, Gradle and Integration Tests

In the last years Apache Maven has become the de-facto build tool for Java and Java EE projects. But from two years back Gradle is gaining more and more users. Following my previous post (http://www.lordofthejars.com/2015/10/gradle-and-java-ee.html), In this post you are going to see how to use Gradle for writing integration tests for Java EE using Arquillian.

Gradle is a build automation tool like Ant or Maven but introducing a Groovy-based DSL language instead of XML. So as you might expect the build file is a Groovy file. You can read in my previous post (http://www.lordofthejars.com/2015/10/gradle-and-java-ee.html) how to install Gradle.

To write integration tests for Java EE, the de-facto tool is Arquillan. If you want to know what Arquillian is, you can get a Getting Start Guide in (http://arquillian.org/guides/getting_started/) or in book Arquillian In Action.

To start using Arquillian, you need to add Arquillian dependencies, which comes in form of BOM. Gradle does not support BOM artefacts out of the box, but you can use dependency-management-plugin Gradle plugin to have support to define BOMs.

Moreover Gradle offers the possibility to add more test source sets apart from the default one which as in Maven is src/test/java and src/test/resources. The idea is that you can define a new test source set where you are going to put all integration tests. With this approach each kind of tests are clearly separated into different source sets. You can write Groovy code in Gradle script to achieve this or you can just use gradle-testsets-plugin which it is the easiest way to proceed.

So to register both plugins (dependency and testsets) you need to add next elements in build.gradle script file:

buildscript {

    repositories {

        jcenter()

    }

    dependencies {

        classpath "io.spring.gradle:dependency-management-plugin:0.5.3.RELEASE"

        classpath 'org.unbroken-dome.gradle-plugins:gradle-testsets-plugin:1.2.0'

    }

}

apply plugin: "io.spring.dependency-management"

apply plugin: 'org.unbroken-dome.test-sets'

Now it is time to add Arquillian dependencies. You need to add the Arquillian BOM, and two dependencies, one that sets that we are going to use Arquillian with JUnit, and another one that sets Apache TomEE application server as target for deploying the application during test runs.

build.gradle with Arquillian, TomEE and Java EE dependency might look like:

dependencyManagement {

    imports {

        mavenBom 'org.arquillian:arquillian-universe:1.0.0.Alpha1'

    }

}

dependencies {

    testCompile group: 'org.arquillian.universe', name: 'arquillian-junit', ext: 'pom'

    testCompile group: 'org.apache.openejb', name: 'arquillian-tomee-embedded', version:'1.7.2'

    testCompile group: 'junit', name: 'junit', version:'4.12'

    providedCompile group: 'org.apache.openejb',name: 'javaee-api', version:'6.0-6'

}

Finally you can configure the new integration test folder as source set by adding next section:

testSets {

    integrationTests

}

Where integrationTest is the name of the test set. testSets automatically creates and configures next elements:

• src/integrationTests/java and src/integrationTests/resources as valid source set folders.
• A dependency configuration named integrationTestsCompile which extends from testCompile, and another one called integrationTestRuntime which extends from testRuntime.
• A Test task named integrationTests which runs the tests in the set.
• A Jar task named integrationTestsJar which packages the tests. 

Notice that you can change the integrationTests to any other value like intTests and Gradle would configure previous elements automatically to the value set it inside testSets, such as src/intTests/java or for example the test task would be called intTests.

Next step is creating the integration tests using Arquillian inside integrationTests test set. For example an Arquillian test for validating that you can POST a color in a REST API and it is returned when GET method is called, would look like:

You can now run integration tests by simply executing gradlew integrationTests

You'll notice that if you run gradlew build, the integration test task is not run. This happens because task is not registered within the default build lifecycle. If you want to add integrationTests task to be executed automatically during build you need to add next lines:

check.dependsOn integrationTest

integrationTest.mustRunAfter test

Ensure that integration tests are run before the check task and that the check task fails the build if there are failing integration tests and also ensures that unit tests are run before integration tests. This guarantees that unit tests are run even if integration tests fails.

So now when you run gradlew build, the integration tests are going to be executed as well.

And finally, what's happen if you are running JaCoCo plugin for code coverage? You will get two JaCoCo files, one for the unit test executions and another one for the integrationTests execution. But probably you want to see an aggregated code coverage report of both runs into one file, so you can inspect the code coverage degree of the application after the execution of all kind of tests. To achieve it you only need to add next task:

task jacocoRootTestReport(type: JacocoReport) {

    sourceSets sourceSets.main

    executionData files([

            "$buildDir/jacoco/test.exec",

            "$buildDir/jacoco/integrationTests.exec"

    ])

    reports {

        xml.enabled false

        csv.enabled false

    }    

}

In this case you are creating a task which aggregates the coverage results of test.exec file (which comes from unit tests) and integrationTests.exec which comes from integration tests.

And to generate the reports you need to explicitly call the jacocoRootTestReport task when you run Gradle. 

So it is so simple to write a Gradle script for running Java EE tests and more important the final script file looks very compact and readable without being tight to any static convention at all.

We keep  learning,
Alex.

There must be more to life than this, There must be more to life than this, How do we cope in a world without love (There Must Be More To Life Than This - Freddie Mercury - Michael Jackson)

Music: https://www.youtube.com/watch?v=Oj6wgyPbsYA 

Code: https://github.com/lordofthejars/cd_javaee

Arquillian Cube: Write Tests Once, Run Them Everywhere

Arquillian Cube is an Arquillian extension that can be used to manage Docker containers from Arquillian. Basically it starts all Docker containers required for your tests, deploys the application (or micro-application) which can be Java based or not, runs the tests and finally stops all of them.

Thanks of Arquillian Cube you can run your integration tests from your local IDE in similar situation as in production environment since in both cases everything is running inside Docker.

But you can go one step forward and you can instruct Arquillian Cube to not start Docker container instances locally (or inside your local boot2docker) but start them in external locations such as your preproduction infrastructure.

Thanks of Digital Ocean that has provided us a free account with some money, we can show you in next screencast how by simply changing one attribute (which could be automated with maven-resources-plugin or just using system properties), we are running the same test against local Docker instance or remotely to Digital Ocean infrastructure.

You can read more about Arquillian and Arquillian Cube in book Arquillian In Action (www.manning.com/sotobueno).

We keep learning,
Alex.

You’re a shooting star I see, A vision of ecstasy, When you hold me, I’m alive, We’re like diamonds in the sky (Diamonds - Rihanna)

Music: https://www.youtube.com/watch?v=lWA2pjMjpBs

Arquillian Cube. Let's zap ALL these bugs, even the infrastructure ones.

Docker is becoming the de-facto project for deploying applications inside lightweight software containers in isolation. Because they are really lightweight they are perfect not only to use in production, but to be used inside developer/qa/CI machine. So the natural step is start writing tests of your software that runs against these containers.

In fact if you are running Docker on production, you can start writing infrastructure tests and run them as a process of your release chain (even debugging in developer machine) before touching production.

That's pretty cool but you need a way to automate all these steps from the point of view of developer. It should be amazing if we could startup Docker containers, deploy there the project and execute the tests from a JUnit and as easy as a simple JUnit. And this is what Arquillian Cube does.

Arquillian Cube is an Arquillian extension that can be used to manager Docker containers from Arquillian.

Extension is named Cube for two reasons:

• Because Docker is like a cube
• Because Borg starship is named cube and well because we are moving tests close to production we can say that "any resistance is futile, bugs will be assimilated".

With this extension you can start a Docker container with a server installed, deploy the required deployable file within it and execute Arquillian tests.

The key point here is that if Docker is used as deployable platform in production, your tests are executed in a the same container as it will be in production, so your tests are even more real than before.

But also let you start a container with every required service like database, mail server, … and instead of stubbing or using fake objects your tests can use real servers.

Let's see a really simple example which shows you how powerful and easy to use Arquillian Cube is. Keep in mind that this extension can be used along with other Arquillian extensions like Arquillian Drone/Graphene to run functional tests.

Arquillian Cube relies on docker-java API.

To use Arquillian Cube you need a Docker daemon running on a computer (it can be local or not), but probably it will be at local.

By default Docker server is using UNIX sockets for communication with the Docker client, however docker-java client uses TCP/IP to connect to the Docker server, so you will need to make sure that your Docker server is listening on TCP port. To allow Docker server to use TCP add the following line to /etc/default/docker:

DOCKER_OPTS="-H tcp://127.0.0.1:2375 -H unix:///var/run/docker.sock"

After restarting the Docker daemon you need to make sure that Docker is up and listening on TCP.

$ docker -H tcp://127.0.0.1:2375 version

Client version: 0.8.0
Go version (client): go1.2
Git commit (client): cc3a8c8
Server version: 1.2.0
Git commit (server): fa7b24f
Go version (server): go1.3.1

If you cannot see the client and server versions then means that something is wrong in Docker installation.

After having a Docker server installed we can start using Arquillian Cube.

In this case we are going to use a Docker image with an Apache Tomcat and we are going to test a Servlet.

And the test:

Notice that the test looks like any other Arquillian test.

Next step is add Arquillian dependencies as described in http://arquillian.org/guides/getting_started and add arquillian-cube-docker dependency:

Because we are using Tomcat and because it is being executed in a remote host (in fact this is true because Tomcat is running inside Docker which is external to Arquillian), we need to add Tomcat remote adapter.

And finally arquillian.xml is configured:

(1) Arquillian Cube extension is registered.
(2) Docker server version is required.
(3) Docker server URI is required. In case you are using a remote Docker host or Boot2Docker here you need to set the remote host ip, but in this case Docker server is on same machine.
(4) A Docker container contains a lot of parameters that can be configured. To avoid having to create one XML property for each one, a YAML content can be embedded directly as property.
(5) Configuration of Tomcat remote adapter. Cube will start the Docker container when it is ran in the same context as an Arquillian container with the same name.
(6) Host can be localhost because there is a port forwarding between container and Docker server.
(7) Port is exposed as well.
(8) User and password are required to deploy the war file to remote Tomcat.

And that’s all. Now you can run your test and you will see how tutum/tomcat:7.0 image is downloaded and started. Then test is executed and finally the docker container is stopped.

This has been a simple example, but you can do a lot of more things like creating images from Dockerfile, orchestrate several docker images or enrich the test to programmatically manipulate containers.

For more information you can read https://github.com/arquillian/arquillian-cube and of course any feedback will be more than welcomed.

We keep learning,

Alex.

Come on now, who do you, Who do you, who do you, who do you think you are?, Ha ha ha, bless your soul, You really think you're in control? (Crazy - Gnarls Barkley)

Music: https://www.youtube.com/watch?v=bd2B6SjMh_w 

Aliens have invaded Undertow

What is Undertow?

Undertow is a flexible performant web server written in java, providing both blocking and non-blocking API’s based on NIO.

Undertow has a composition based architecture that allows you to build a web server by combining small single purpose handlers. The gives you the flexibility to choose between a full Java EE servlet 3.1 container, or a low level non-blocking handler, to anything in between.

Undertow is sponsored by JBoss and is the default web server in the Wildfly Application Server.

Writing tests for Undertow

Arquillian-Container-Undertow as every Arquillian Container Extension  it take cares of you of starting, stopping and deploying the application. Also provides an Shrinkwrap resolver for creating the Undertow deployment file.

To simplify the development of tests on Undertow we have created two Shrinkwrap resolvers:

• Embedded Servlet Deployment
• Non-blocking handler

Maven Artifacts

Embedded Servlet Deployment

When you want to deploy a servlet/s inside Undertow, you must create a DeploymentInfo class and provide all the required information. 

For this reason Arquillian-Container-Undertow provides a Shrinkwrap resolver named UndertowWebArchive.

And then we can write our test:

Non-blocking handler

But with Undertow you can also write non-blocking handlers. For creating a non-blocking handler in Undertow you simply must create a class that implements HttpHandler interface and register it. For this reason Arquillian-Container-Undertow provides a Shrinkwrap resolver named UndertowHttpHandlerArchive.

And then we can write our test:

Configuration

You can configure the bind address and port of Undertow. By default Undertow is opened at localhost:8080.

But you can also set the bind address and the listening port.

<1> If port is set to -1, a random port between 1024 and 49151 is used.

So now you can write tests for Undertow container within the context of Arquillian.

We keep learning,
Alex.

Oh django! After the showers is the sun. Will be shining... (Django - Luis Bacalov & Rocky Roberts)

Music: https://www.youtube.com/watch?v=UX3h22aABIc

NoSQLUnit 0.7.7 Released

NoSQLUnit is a JUnit extension to make writing unit and integration tests of systems that use NoSQL backend easier. Visit official page for more information.

In 0.7.7 release, next changes has been added:

• Allow for array types in GraphML (https://github.com/lordofthejars/nosql-unit/issues/82)
• Enhancing manual index support in GraphML (https://github.com/lordofthejars/nosql-unit/pull/81)
• Bug Issue #79 (https://github.com/lordofthejars/nosql-unit/issues/79)
• Updated Cassandra-Unit to 1.1.2.1 and Cassandra to 1.2.0 (https://github.com/lordofthejars/nosql-unit/pull/76)

I would like to say thank you to javahelp, ochrons, Jdourd, JordiAranda and anton-kostyliev for their help.

We keep learning,
Alex.

I belong with you, You belong with me, You're my sweetheart (Ho Hey - The Lumineers)

Music: http://www.youtube.com/watch?v=zvCBSSwgtg4

NoSQLUnit 0.7.6 Released

NoSQLUnit is a JUnit extension to make writing unit and integration tests of systems that use NoSQL backend easier. Visit official page for more information.

In 0.7.6 release, next changes has been implemented:

• FongoDB version has been updated to 1.0.7. Issue #73 (https://github.com/lordofthejars/nosql-unit/issues/73)
• Update logging dependencies. Issues #71 and #69. Thanks sslavic.
• Update pom configuration. Issue #70. (https://github.com/lordofthejars/nosql-unit/pull/70). Thanks sslavic.
• Now you can add options (like options, geospatial stuff, ...)  in MongoDB indexes. Issue #67. (https://github.com/lordofthejars/nosql-unit/issues/67). See an example for creating unique indexes here.
• Improved the way for testing Redis replication. Issue #62 (https://github.com/lordofthejars/nosql-unit/issues/62). See an example of how to define replication scenario here.
• All Redis tests has been passed using Redis 2.6.x. Redis 2.4.x is deprecated and will be removed  its support at then end of 2013.  

We keep learning,
Alex.

De dia vivire pensando en tu sonrisa, De noche las estrellas me acompañaran, Seras como un luz que alumbra en mi destino, Me voy pero te juro que mañana volvere (Un Beso Y Una Flor - Niño Bravo)

Music: http://www.youtube.com/watch?v=Q3UqusEWGeo

Testing Spring Data Neo4j Applications with NoSQLUnit

Spring Data Neo4j

Spring Data Neo4j is the project within Spring Data project which provides an extension to the Spring programming model for writing applications that uses Neo4j as graph database.

To write tests using NoSQLUnit for Spring Data Neo4j applications, you do need nothing special apart from considering that Spring Data Neo4j uses a special property called type in graph nodes and relationships which stores the fully qualified classname of that entity.

Apart from type property at node/relationship level, we also need to create one index for nodes and one index for relationships. In case of nodes, types index name is required, meanwhile rel_types is required for relationships. In both cases we must set key value to className and value to full qualified classname.

Type mapping IndexingNodeTypeRepresentationStrategy and IndexingRelationshipTypeRepresentationStrategy are used as default type mapping implementation, but you can also use SubReferenceNodeTypeRepresentationStrategy which stores entity types in a tree in the graph representing the type and interface hierarchy, or you can customize even more by implementing NodeTypeRepresentationStrategy interface.

Hands on Work

Application

Starfleet has asked us to develop an application for storing all starfleet members, with their relationship with other starfleet members, and the ship where they serve.

The best way to implement this requirement is using Neo4j database as backend system. Moreover Spring Data Neo4j is used at persistence layer.

This application is modelized into two Java classes, one for members and another one for starships. Note that for this example there are no properties in relationships, so only nodes are modelized.

Member class

@NodeEntity public class Member { private static final String COMMANDS = "COMMANDS"; @GraphId Long nodeId; private String name; private Starship assignedStarship; public Member() { super(); } public Member(String name) { this.name = name; } @Fetch @RelatedTo(type=COMMANDS, direction=Direction.OUTGOING) private Set<Member> commands; public void command(Member member) { this.commands.add(member); } public Set<Member> commands() { return this.commands; } public Starship getAssignedStarship() { return assignedStarship; } public String getName() { return name; } public void assignedIn(Starship starship) { this.assignedStarship = starship; } //Equals and Hash methods }

Starship class

@NodeEntity public class Starship { private static final String ASSIGNED = "assignedStarship"; @GraphId Long nodeId; private String starship; public Starship() { super(); } public Starship(String starship) { this.starship = starship; } @RelatedTo(type = ASSIGNED, direction=Direction.INCOMING) private Set<Member> crew; public String getStarship() { return starship; } public void setStarship(String starship) { this.starship = starship; } //Equals and Hash methods }

Apart from model classes, we also need two repositories for implementing CRUD operations, and spring context xml file. Spring Data Neo4j uses Spring Data Commons infrastructure allowing us to create interface based compositions of repositories, providing default implementations for certain operations.

MemberRepository class

public interface MemberRepository extends GraphRepository<Member>, RelationshipOperationsRepository<Member> { Member findByName(String name); }

See that apart from operations provided by GrapRepository interface like save, findAll, findById, … we are defining one query method too called findByName. Spring Data Neo4j repositories (and most of Spring Data projects) provide a mechanism to define queries using the known Ruby on Rails approach for defining finder queries.

StarshipRepository class

public interface StarshipRepository extends GraphRepository<Starship>, RelationshipOperationsRepository<Starship> { }

application-context file

<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context" xmlns:neo4j="http://www.springframework.org/schema/data/neo4j" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd http://www.springframework.org/schema/data/neo4j http://www.springframework.org/schema/data/neo4j/spring-neo4j.xsd"> <context:component-scan base-package="com.lordofthejars.nosqlunit.springdata.neo4j"/> <context:annotation-config/> <neo4j:repositories base-package="com.lordofthejars.nosqlunit.springdata.repository"/> </beans>

Testing

Unit Testing

As it has been told previously, for writing datasets for Spring Data Neo4j, we don’t have to do anything special beyond creating node and relationship properties correctly and defining the required indexes. Let’s see the dataset used to test the findByName method by seeding Neo4j database.

star-trek-TNG-dataset.xml file

<?xml version="1.0" ?> <graphml xmlns="http://graphml.graphdrawing.org/xmlns" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://graphml.graphdrawing.org/xmlns http://graphml.graphdrawing.org/xmlns/1.0/graphml.xsd"> <key id="name" for="node" attr.name="name" attr.type="string"></key> <key id="__type__" for="node" attr.name="__type__" attr.type="string"></key> <key id="starship" for="node" attr.name="starship" attr.type="string"></key> <graph id="G" edgedefault="directed"> <node id="3"> <data key="__type__">com.lordofthejars.nosqlunit.springdata.neo4j.Member</data> <data key="name">Jean-Luc Picard</data> <index name="__types__" key="className">com.lordofthejars.nosqlunit.springdata.neo4j.Member</index> </node> <node id="1"> <data key="__type__">com.lordofthejars.nosqlunit.springdata.neo4j.Member</data> <data key="name">William Riker</data> <index name="__types__" key="className">com.lordofthejars.nosqlunit.springdata.neo4j.Member</index> </node> <node id="4"> <data key="__type__">com.lordofthejars.nosqlunit.springdata.neo4j.Starship</data> <data key="starship">NCC-1701-E</data> <index name="__types__" key="className">com.lordofthejars.nosqlunit.springdata.neo4j.Starship</index> </node> <edge id="11" source="3" target="4" label="assignedStarship"></edge> <edge id="12" source="1" target="4" label="assignedStarship"></edge> <edge id="13" source="3" target="1" label="COMMANDS"></edge> </graph> </graphml>

See that each node has at least one type property with full qualified classname and an index with name types, key className and full qualified classname as value.

Next step is configuring application context for unit tests.

application-context-embedded-neo4j.xml

<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context" xmlns:neo4j="http://www.springframework.org/schema/data/neo4j" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd http://www.springframework.org/schema/data/neo4j http://www.springframework.org/schema/data/neo4j/spring-neo4j.xsd"> <import resource="classpath:com/lordofthejars/nosqlunit/springdata/neo4j/application-context.xml"/> <neo4j:config storeDirectory="target/config-test"/> </beans>

Notice that we are using Neo4j namespace for instantiating an embedded Neo4j database.

And now we can write the JUnit test case:

WhenInformationAboutAMemberIsRequired

@RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration("application-context-embedded-neo4j.xml") public class WhenInformationAboutAMemberIsRequired { @Autowired private MemberRepository memberRepository; @Autowired private StarshipRepository starshipRepository; @Autowired private ApplicationContext applicationContext; @Rule public Neo4jRule neo4jRule = newNeo4jRule() .defaultSpringGraphDatabaseServiceNeo4j(); @Test @UsingDataSet(locations = "star-trek-TNG-dataset.xml", loadStrategy = LoadStrategyEnum.CLEAN_INSERT) public void information_about_starship_where_serves_and_members_under_his_service_should_be_retrieved() { Member jeanLuc = memberRepository.findByName("Jean-Luc Picard"); assertThat(jeanLuc, is(createMember("Jean-Luc Picard"))); assertThat(jeanLuc.commands(), containsInAnyOrder(createMember("William Riker"))); Starship starship = starshipRepository.findOne(jeanLuc.getAssignedStarship().nodeId); assertThat(starship, is(createStarship("NCC-1701-E"))); } private Object createStarship(String starship) { return new Starship(starship); } private static Member createMember(String memberName) { return new Member(memberName); } }

There are some important points in the previous class to take under consideration:

1. Recall that we need to use Spring ApplicationContext object to retrieve embedded Neo4j instance defined into Spring application context.
2. Since lifecycle of database is managed by Spring Data container, there is no need to define any NoSQLUnit lifecycle manager.

Integration Test

Unit tests are usually run against embedded in-memory instances, but in production environment you may require access to external Neo4j servers by using Rest connection, or in case of Spring Data Neo4j instantiating SpringRestGraphDatabase class. You need to write tests to validate that your application still works when you integrate your code with a remote server, and this tests are typically known as integration tests.

To write integration tests for our application is as easy as defining an application context with SpringRestGraphDatabase and allow NoSQLUnit to control the lifecycle of Neo4j database.

.application-context-managed-neo4j.xml

<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context" xmlns:neo4j="http://www.springframework.org/schema/data/neo4j" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd http://www.springframework.org/schema/data/neo4j http://www.springframework.org/schema/data/neo4j/spring-neo4j.xsd"> <import resource="classpath:com/lordofthejars/nosqlunit/springdata/neo4j/application-context.xml"/> <bean id="graphDatabaseService" class="org.springframework.data.neo4j.rest.SpringRestGraphDatabase"> <constructor-arg index="0" value="http://localhost:7474/db/data"></constructor-arg> </bean> <neo4j:config graphDatabaseService="graphDatabaseService"/> </beans>

Note how instead of registering an embedded instance, we are configuring SpringRestGraphDatabase class to connect to localhost server. And let’s implement an integration test which verifies that all starships can be retrieved from Neo4j server.

WhenInformationAboutAMemberIsRequired

@RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration("application-context-managed-neo4j.xml") public class WhenInformationAboutStarshipsAreRequired { @ClassRule public static ManagedNeoServer managedNeoServer = newManagedNeo4jServerRule() .neo4jPath( "/Users/alexsotobueno/Applications/neo4j-community-1.7.2") .build(); @Autowired private StarshipRepository starshipRepository; @Autowired private ApplicationContext applicationContext; @Rule public Neo4jRule neo4jRule = newNeo4jRule() .defaultSpringGraphDatabaseServiceNeo4j(); @Test @UsingDataSet(locations = "star-trek-TNG-dataset.xml", loadStrategy = LoadStrategyEnum.CLEAN_INSERT) public void information_about_starship_where_serves_and_members_under_his_service_should_be_retrieved() { EndResult<Starship> allStarship = starshipRepository.findAll(); assertThat(allStarship, containsInAnyOrder(createStarship("NCC-1701-E"))); } private Object createStarship(String starship) { return new Starship(starship); } }

Because defaultSpringGraphDatabaseServiceNeo4j method returns a GraphDatabaseService instance defined into application context, in our case it will return the defined SpringRestGraphDatabase instance.

Conclusions

There is not much difference between writing tests for none Spring Data Neo4j applications than the ones they use it. Only keep in mind to define correctly the type property and create required indexes.

Also see that from the point of view of NoSQLUnit there is no difference between writing unit or integration tests, apart of lifecycle management of the database engine.

Download Code

We keep learning,
Alex.

Gonna rise up, Burning black holes in dark memories, Gonna rise up, Turning mistakes into gold (Rise - Eddie Vedder)

Music: http://www.youtube.com/watch?v=uuVf4xiRsJQ