domingo, abril 28, 2013

Git Bisect in Action



Today it is time for a screencast where I show an amazing git command, called git bisect. Git bisect command command uses git to find in which commit a bug was introduced. After that, I am sure you will wonder where it's been you whole career. Thanks Jason for showing me the way.

You can watch it in HD to see clearly each command.



We keep learning,
Alex.

Siempre quise ir a L.A. Dejar un día esta ciudad. Cruzar el mar en tu compañía.  (Cadillac Solitario - Loquillo)
Music: http://www.youtube.com/watch?v=kg0Jlpgzel4

lunes, abril 22, 2013

NoSQLUnit 0.7.6 Released


NoSQLUnit is a JUnit extension to make writing unit and integration tests of systems that use NoSQL backend easier. Visit official page for more information.

In 0.7.6 release, next changes has been implemented:

We keep learning,
Alex.

De dia vivire pensando en tu sonrisa, De noche las estrellas me acompañaran, Seras como un luz que alumbra en mi destino, Me voy pero te juro que mañana volvere (Un Beso Y Una Flor - Niño Bravo)

Music: http://www.youtube.com/watch?v=Q3UqusEWGeo

lunes, abril 15, 2013

Enjoy the magic of asciidoctor in java with asciidoctor-java-integration


The asciidoctor-java-integration is the official means of using Asciidoctor to render all your AsciiDoc documentation using Java instead of Ruby.

Installation

Since asciidoctor-java-integration is a standard jar file, the only thing you should do is add library into classpath.

Dependency declaration in Maven

...
<dependencies>
  <dependency>
    <groupId>org.asciidoctor</groupId>
    <artifactId>asciidoctor-java-integration</artifactId>
    <version>${asciidoctor.version}</version>                   <1>
    ...
  </dependency>
</dependencies>
...
  1. As this library tracks the version of asciidoctor, you can use which every version of asciidoctor you prefer.

Usage

The core interface of asciidoctor-java-integration is Asciidoctor interface. It provides two methods for rendering asciidoc content, render and renderFile. Both of them returns a string with rendered content.

Also a factory method is provided to create an instance of Asciidoctor interface.

Creation of Asciidoctor interface

import static org.asciidoctor.Asciidoctor.Factory.create;
import org.asciidoctor.Asciidoctor;
...
Asciidoctor asciidoctor = create();
...

And then we can call render methods depending on our requirements.

Rendering a String

...
String rendered = asciidoctor.render("*This* is it.", Collections.EMPTY_MAP);
System.out.println(rendered);
...

But also you can render the content of a file.

Rendering a File

...
String rendered = asciidoctor.renderFile("target/test-classes/rendersample.asciidoc", Collections.EMPTY_MAP);
System.out.println(rendered);
...

Options


Asciidoctor supports different kind of options, like in_place which renders the output inside a file, template_dir used to provide a directory of Tilt-compatible templates to be used instead of the default built-in templates, or for example attributes option where we can set key-value pairs of attributes that will be used within asciidoc document.

The second parameter of render methods are a java.util.Map where we can set all these options.

Example of using in_place Option and backend Attribute

Map<String, Object> attributes = new HashMap<String, Object>();
attributes.put("backend", "docbook");

Map<String, Object> options = new HashMap<String, Object>();
options.put("in_place", true);
options.put("attributes", attributes);

String render = asciidoctor.renderFile("target/test-classes/rendersample.asciidoc", options);

See that in previous example we have created a Map, where we have put the options and attributes (creating a Map too) required to render input as docbook and generate an output file.

But asciidoctor-java-integration also provides two builder classes to create these maps in a more readable form.

AttributesBuilder is provided for creating a map with required attributes set, and OptionsBuilder can be used for options. Previous example but using these classes looks like:

Example setting attributes and options

import static org.asciidoctor.AttributesBuilder.attributes;
import static org.asciidoctor.OptionsBuilder.options;

...

Map<String, Object> attributes = attributes().backend("docbook").asMap();
Map<String, Object> options = options().inPlace(true).attributes(attributes).asMap();

String render = asciidoctor.renderFile("target/test-classes/rendersample.asciidoc", options);

...

Utilities


A utility class for searching all asciidoc files present in a root folder and all its subfolders is given. In fact it finds all files that end up with .asc, .asciidoc, .ad or .adoc. This class is AsciidocDirectoryWalker.

Example of finding all asciidoc

DirectoryWalker directoryWalker = new AsciidocDirectoryWalker("target/test-classes/src");
List<File> asciidocFiles = directoryWalker.scan();

We keep learning,
Alex.
Cold, cold heart, Hard done by you, Some things look better, baby, Just passing through (Sacrifice - Elton John)

http://www.youtube.com/watch?v=NrLkTZrPZA4

lunes, abril 01, 2013

Installing TomEE from Puppet



Apache TomEE is an all-Apache stack aimed at Java EE 6 Web Profile certification where Tomcat is top dog. It is the conjunction of Tomcat + Java EE

Puppet is a tool designed to manage the configuration of our systems declaratively. We only have to describe system resources and their state. This description is stored in the core-files of Puppet, which are called Puppet manifests.

In current post we are going to see how to define TomEE as Puppet resource, so it can be installed automatically in all computers that are managed by Puppet. I am using Ubuntu 12.04, but of course this could be adapted to your system.


Because TomEE is written in Java, we obviously need to ensure that a JDK is present in our system, for our example we are going to use OpenJDK 1.6. The way to install a package to our system is using Package resource, which uses underlying package manager to find, download and install it.

Let's create an init.pp manifest file and the first thing we are going to do is create an exec task which updates the package manager with a list of last packages available.


Then we can define a class that is in charge of installing the OpenJDK. A class in Puppet can be understood as a collection of resources that will be seen by Puppet as a unit.


It is pretty intuitive I guess, first of all we are ensuring that openjdk-6-jdk package is already present in system. If it is not installed, then Exec["update-package-list"] resource is executed, and finally the package manager installs OpenJDK into the system. 

After executing this part we can run java -version or javac -version without any problem, OpenJDK is there.

Next step is installing TomEE. Probably TomEE package is not in your distribution package repository in software package format (for example in case of Debian .deb). For this reason we need a different approach to the one followed by OpenJDK. We are going to download tar.gz file from TomEE site and uncompress it to an installation directory.


As with OpenJDK we are creating a class called tomee. First we are creating the directory where TomEE will be installed (tar command requires that destination directory should be already created). 

Then we are downloading TomEE from Apache site using wget command, to finally uncompressing it to already created directory.

Note the use of ->. Puppet does not guarantee the order of execution of defined resources, so you cannot infer "a priori" if TomEE will be downloaded first and then installed or vice versa. To define an order we are using -> operator which sets a priority between resources.

Now we have got Apache TomEE installed on computer, but obviously is not started and stopped automatically, you must execute /opt/tomee-1.5.1/bin/startup.sh to have TomEE available. Let's change this by using service resource. As its name suggests it registers as service an installed server. So inside tomee class define next service resource:


Keep an eye in two points of previous declaration, the first one is the provider/start/stop declarations, provider is set to init, which means we want to use the standard init-style service management, but for example you could use launchd (case of Mac OS X), upstart, windows (for Windows machines), ... see [http://docs.puppetlabs.com/references/latest/type.html#service] for more information. And because TomEE is not implemented as a Linux service by default (basically because has not been installed from a native package like .deb), we need to specify which command should be executed to start and stop TomEE

The second point is that service resource requires to have TomEE unpacked and OpenJDK installed, for this reason require attribute contains two declarations.

See full script here.

Final notes about example:
  • In Puppet, creates attribute inside exec task, is used to know if the resource should be executed or not (case that the file exists, exec will not be executed). In our case we are downloading TomEE in tmp directory. Most OS removes periodically this directory so it is a bad location for downloading it, but for this tutorial it works perfectly because I could re-execute the script every time as a new execution.
  • For simplifications we have add all content inside a single file, in your enterprise I suggest creating a TomEE module so you could share across all your projects.
  • TomEE version should be set as a variable/parameter/hiera so same class can be reused when new version of TomEE is released. 

We keep learning,
Alex.

Ven al desfile de sirenas, de cuerpos bien cebados, sube al séptimo cielo (Ellos Las Prefieren Gordas - Orquesta Mondragon)

Music:  http://www.youtube.com/watch?v=_Z_LTvQcw38

martes, marzo 19, 2013

Testing Spring Data Neo4j Applications with NoSQLUnit

Spring Data Neo4j


Spring Data Neo4j is the project within Spring Data project which provides an extension to the Spring programming model for writing applications that uses Neo4j as graph database.
To write tests using NoSQLUnit for Spring Data Neo4j applications, you do need nothing special apart from considering that Spring Data Neo4j uses a special property called type in graph nodes and relationships which stores the fully qualified classname of that entity.

Apart from type property at node/relationship level, we also need to create one index for nodes and one index for relationships. In case of nodes, types index name is required, meanwhile rel_types is required for relationships. In both cases we must set key value to className and value to full qualified classname.

Note Type mapping
IndexingNodeTypeRepresentationStrategy and IndexingRelationshipTypeRepresentationStrategy are used as default type mapping implementation, but you can also use SubReferenceNodeTypeRepresentationStrategy which stores entity types in a tree in the graph representing the type and interface hierarchy, or you can customize even more by implementing NodeTypeRepresentationStrategy interface.

Hands on Work

Application

Starfleet has asked us to develop an application for storing all starfleet members, with their relationship with other starfleet members, and the ship where they serve.

The best way to implement this requirement is using Neo4j database as backend system. Moreover Spring Data Neo4j is used at persistence layer.

This application is modelized into two Java classes, one for members and another one for starships. Note that for this example there are no properties in relationships, so only nodes are modelized.

Member class

@NodeEntity
public class Member {

        private static final String COMMANDS = "COMMANDS";

        @GraphId Long nodeId;

        private String name;

        private Starship assignedStarship;

        public Member() {
                super();
        }

        public Member(String name) {
                this.name = name;
        }

        @Fetch @RelatedTo(type=COMMANDS, direction=Direction.OUTGOING)
        private Set<Member> commands;

        public void command(Member member) {
                this.commands.add(member);
        }

        public Set<Member> commands() {
                return this.commands;
        }

        public Starship getAssignedStarship() {
                return assignedStarship;
        }

        public String getName() {
                return name;
        }

        public void assignedIn(Starship starship) {
                this.assignedStarship = starship;
        }

        //Equals and Hash methods
}
Starship class

@NodeEntity
public class Starship {

        private static final String ASSIGNED = "assignedStarship";

        @GraphId Long nodeId;

        private String starship;

        public Starship() {
                super();
        }

        public Starship(String starship) {
                this.starship = starship;
        }

        @RelatedTo(type = ASSIGNED, direction=Direction.INCOMING)
        private Set<Member> crew;

        public String getStarship() {
                return starship;
        }

        public void setStarship(String starship) {
                this.starship = starship;
        }

        //Equals and Hash methods
}

Apart from model classes, we also need two repositories for implementing CRUD operations, and spring context xml file. Spring Data Neo4j uses Spring Data Commons infrastructure allowing us to create interface based compositions of repositories, providing default implementations for certain operations.

MemberRepository class

public interface MemberRepository extends GraphRepository<Member>,
                RelationshipOperationsRepository<Member> {

        Member findByName(String name);

}

See that apart from operations provided by GrapRepository interface like save, findAll, findById, … we are defining one query method too called findByName. Spring Data Neo4j repositories (and most of Spring Data projects) provide a mechanism to define queries using the known Ruby on Rails approach for defining finder queries.

StarshipRepository class

public interface StarshipRepository extends GraphRepository<Starship>,
                RelationshipOperationsRepository<Starship> {
}
application-context file

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:neo4j="http://www.springframework.org/schema/data/neo4j"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
           http://www.springframework.org/schema/beans/spring-beans-3.1.xsd
           http://www.springframework.org/schema/context
           http://www.springframework.org/schema/context/spring-context-3.1.xsd
           http://www.springframework.org/schema/data/neo4j
           http://www.springframework.org/schema/data/neo4j/spring-neo4j.xsd">

     <context:component-scan base-package="com.lordofthejars.nosqlunit.springdata.neo4j"/>
     <context:annotation-config/>

     <neo4j:repositories base-package="com.lordofthejars.nosqlunit.springdata.repository"/>

</beans>

Testing

Unit Testing

As it has been told previously, for writing datasets for Spring Data Neo4j, we don’t have to do anything special beyond creating node and relationship properties correctly and defining the required indexes. Let’s see the dataset used to test the findByName method by seeding Neo4j database.

star-trek-TNG-dataset.xml file

<?xml version="1.0" ?>
<graphml xmlns="http://graphml.graphdrawing.org/xmlns"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://graphml.graphdrawing.org/xmlns http://graphml.graphdrawing.org/xmlns/1.0/graphml.xsd">
     <key id="name" for="node" attr.name="name" attr.type="string"></key>
     <key id="__type__" for="node" attr.name="__type__" attr.type="string"></key>
     <key id="starship" for="node" attr.name="starship" attr.type="string"></key>
     <graph id="G" edgedefault="directed">

       <node id="3">
        <data key="__type__">com.lordofthejars.nosqlunit.springdata.neo4j.Member</data>
        <data key="name">Jean-Luc Picard</data>
        <index name="__types__" key="className">com.lordofthejars.nosqlunit.springdata.neo4j.Member</index>
      </node>

      <node id="1">
        <data key="__type__">com.lordofthejars.nosqlunit.springdata.neo4j.Member</data>
        <data key="name">William Riker</data>
        <index name="__types__" key="className">com.lordofthejars.nosqlunit.springdata.neo4j.Member</index>
      </node>

      <node id="4">
        <data key="__type__">com.lordofthejars.nosqlunit.springdata.neo4j.Starship</data>
        <data key="starship">NCC-1701-E</data>
        <index name="__types__" key="className">com.lordofthejars.nosqlunit.springdata.neo4j.Starship</index>
      </node>

      <edge id="11" source="3" target="4" label="assignedStarship"></edge>
      <edge id="12" source="1" target="4" label="assignedStarship"></edge>
      <edge id="13" source="3" target="1" label="COMMANDS"></edge>

    </graph>
</graphml>

See that each node has at least one type property with full qualified classname and an index with name types, key className and full qualified classname as value.
Next step is configuring application context for unit tests.

application-context-embedded-neo4j.xml

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:neo4j="http://www.springframework.org/schema/data/neo4j"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
           http://www.springframework.org/schema/beans/spring-beans-3.1.xsd
           http://www.springframework.org/schema/context
           http://www.springframework.org/schema/context/spring-context-3.1.xsd
           http://www.springframework.org/schema/data/neo4j
           http://www.springframework.org/schema/data/neo4j/spring-neo4j.xsd">


        <import resource="classpath:com/lordofthejars/nosqlunit/springdata/neo4j/application-context.xml"/>
        <neo4j:config storeDirectory="target/config-test"/>

</beans>
Notice that we are using Neo4j namespace for instantiating an embedded Neo4j database.
And now we can write the JUnit test case:

WhenInformationAboutAMemberIsRequired

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration("application-context-embedded-neo4j.xml")
public class WhenInformationAboutAMemberIsRequired {

        @Autowired
        private MemberRepository memberRepository;
        @Autowired
        private StarshipRepository starshipRepository;

        @Autowired
        private ApplicationContext applicationContext;

        @Rule
        public Neo4jRule neo4jRule = newNeo4jRule()
                        .defaultSpringGraphDatabaseServiceNeo4j();

        @Test
        @UsingDataSet(locations = "star-trek-TNG-dataset.xml", loadStrategy = LoadStrategyEnum.CLEAN_INSERT)
        public void information_about_starship_where_serves_and_members_under_his_service_should_be_retrieved()  {

                Member jeanLuc = memberRepository.findByName("Jean-Luc Picard");

                assertThat(jeanLuc, is(createMember("Jean-Luc Picard")));
                assertThat(jeanLuc.commands(), containsInAnyOrder(createMember("William Riker")));

                Starship starship = starshipRepository.findOne(jeanLuc.getAssignedStarship().nodeId);
                assertThat(starship, is(createStarship("NCC-1701-E")));
        }

        private Object createStarship(String starship) {
                return new Starship(starship);
        }

        private static Member createMember(String memberName) {
                return new Member(memberName);
        }
}

There are some important points in the previous class to take under consideration:
  1. Recall that we need to use Spring ApplicationContext object to retrieve embedded Neo4j instance defined into Spring application context.
  2. Since lifecycle of database is managed by Spring Data container, there is no need to define any NoSQLUnit lifecycle manager.

Integration Test

Unit tests are usually run against embedded in-memory instances, but in production environment you may require access to external Neo4j servers by using Rest connection, or in case of Spring Data Neo4j instantiating SpringRestGraphDatabase class. You need to write tests to validate that your application still works when you integrate your code with a remote server, and this tests are typically known as integration tests.
To write integration tests for our application is as easy as defining an application context with SpringRestGraphDatabase and allow NoSQLUnit to control the lifecycle of Neo4j database.

.application-context-managed-neo4j.xml

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:neo4j="http://www.springframework.org/schema/data/neo4j"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
           http://www.springframework.org/schema/beans/spring-beans-3.1.xsd
           http://www.springframework.org/schema/context
           http://www.springframework.org/schema/context/spring-context-3.1.xsd
           http://www.springframework.org/schema/data/neo4j
           http://www.springframework.org/schema/data/neo4j/spring-neo4j.xsd">


        <import resource="classpath:com/lordofthejars/nosqlunit/springdata/neo4j/application-context.xml"/>

        <bean id="graphDatabaseService" class="org.springframework.data.neo4j.rest.SpringRestGraphDatabase">
                <constructor-arg index="0" value="http://localhost:7474/db/data"></constructor-arg>
        </bean>
        <neo4j:config graphDatabaseService="graphDatabaseService"/>

</beans>

Note how instead of registering an embedded instance, we are configuring SpringRestGraphDatabase class to connect to localhost server. And let’s implement an integration test which verifies that all starships can be retrieved from Neo4j server.

WhenInformationAboutAMemberIsRequired

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration("application-context-managed-neo4j.xml")
public class WhenInformationAboutStarshipsAreRequired {

        @ClassRule
        public static ManagedNeoServer managedNeoServer = newManagedNeo4jServerRule()
                        .neo4jPath(
                                        "/Users/alexsotobueno/Applications/neo4j-community-1.7.2")
                        .build();

        @Autowired
        private StarshipRepository starshipRepository;

        @Autowired
        private ApplicationContext applicationContext;

        @Rule
        public Neo4jRule neo4jRule = newNeo4jRule()
                        .defaultSpringGraphDatabaseServiceNeo4j();

        @Test
        @UsingDataSet(locations = "star-trek-TNG-dataset.xml", loadStrategy = LoadStrategyEnum.CLEAN_INSERT)
        public void information_about_starship_where_serves_and_members_under_his_service_should_be_retrieved() {

                EndResult<Starship> allStarship = starshipRepository.findAll();

                assertThat(allStarship, containsInAnyOrder(createStarship("NCC-1701-E")));

        }

        private Object createStarship(String starship) {
                return new Starship(starship);
        }

}

Because defaultSpringGraphDatabaseServiceNeo4j method returns a GraphDatabaseService instance defined into application context, in our case it will return the defined SpringRestGraphDatabase instance.

Conclusions

There is not much difference between writing tests for none Spring Data Neo4j applications than the ones they use it. Only keep in mind to define correctly the type property and create required indexes.

Also see that from the point of view of NoSQLUnit there is no difference between writing unit or integration tests, apart of lifecycle management of the database engine.

Download Code

We keep learning,
Alex.
Gonna rise up, Burning black holes in dark memories, Gonna rise up, Turning mistakes into gold (Rise - Eddie Vedder)

Music: http://www.youtube.com/watch?v=uuVf4xiRsJQ

miércoles, marzo 06, 2013

NoSQLUnit 0.7.5 Released



NoSQLUnit is a JUnit extension to make writing unit and integration tests of systems that use NoSQL backend easier. Visit official page for more information.

In 0.7.5 release, next changes has been added:
We keep learning,
Alex.
Cuando la pena cae sobre mi, el mundo deja de existir, miro hacia atrás, y busco entre mis recuerdos (Entre Mis Recuerdos - Luz Casal)

Music: http://www.youtube.com/watch?v=zUnOWCURe2M

lunes, febrero 25, 2013

Code Quality stage using Jenkins


In Continuous Delivery each build is potentially shippable. This fact implies among a lot of other things, to assign a none snapshot version to your components as fast as possible so you can refer them through all the process.

Usually automated software delivery process consist of several stages like Commit stage, Code Quality, Acceptance Tests, Manual Test, Deployment, ... But let's focusing on second stage related to code quality. Note that in my previous post (http://www.lordofthejars.com/2013/02/conditional-buildstep-jenkins-plugin.html) there are some concepts that are being used here.

Second stage in continuous delivery is the code quality. This step is very important because is where we are running static code analysis for detecting possible defects (mostly possible NPE), code conventions or unnecessary object creation. Some of projects that are typically used are Checkstyle, PMD or FindBugs among others. In this case we are going to see how to use Checkstyle, but of course it is very similar in any other tool.

So the first thing to do is configure Checkstyle into our build tool (in this case Maven). Because we only want to run the static analysis in second stage of our pipeline we are going to register the Checkstyle Maven plugin into a metrics profile. Keep in mind that all plugins run for code analysis should be added into that profile.


Now that we have our pom configured with Checkstyle, we can configure Jenkins to run Code Quality stage after the first stage (explained in my previous post).

In this case we are going to use Trigger Parameterized Build plugin to execute code quality job from commit stage.

Because code of current build version has been pushed into a release branch (see my previous post) during commit stage, we need to set branch name as parameter for the code quality Jenkins job, so code can be downloaded and then run the static analysis.

In build job of our first stage, we add a Post-build Action of type Trigger parameterized build on other projects. First we open the Configure menu of first build job of pipeline and we configure it so next build job of the pipeline (helloworld-code-quality) is executed only if current job is stable. Also we define the RELEASE_BRANCH_NAME parameter with branch name.



Then let's create a new build job that will be in charge of running static code analysis, we are going to name it helloworld-code-quality.

And we configure the new build job. First of all check the option "This build is parameterized", and add a String parameter and set the name RELEASE_BRANCH_NAME. After that we can use RELEASE_BRANCH_NAME parameter in current job. So at Source Code Management section we add the repository URL and in Branches to build we set origin/${RELEASE_BRANCH_NAME}.

Then at Build section we add a Maven build step, which executes Checkstyle goal: checkstyle:checkstyle -P metrics.

And finally to have a better visibility of the result, we can install Checkstyle Jenkins plugin and publish the report. After plugin is installed, we can add a new Post-build Actions with name "Publish Checkstyle analysis result". In our case report is located at **/target/checkstyle-result.xml.



And that's all for current stage, next stage is the responsible of executing the acceptance tests, but this would be in another post.

So in summary we have learned how after code is compiled and some tests are executed (in first stage of pipeline), the Code Quality stage is run into Jenkins using Checkstyle Maven plugin.

We keep learning,
Alex
En algun lugar de un gran pais, Olvidaron construir, Un hogar donde no queme el sol, Y al nacer no haya que morir… (En Algún Lugar - Dunncan Dhu)
Music: http://www.youtube.com/watch?v=Myn7ghLQltI