jueves, diciembre 20, 2012

NoSQLUnit 0.7.1 Released


NoSQLUnit is a JUnit extension to make writing unit and integration tests of systems that use NoSQL backend easier. Visit official page for more information.

In 0.7.1 release:
  • One new NoSQL system is supported and is CouchDB
  • JUnit version has been upgraded to 4.11. Now using @Inject does not require to pass this reference.
  • Business objects do not contain any dependency to JUnit classes. This is the first step to integration to Arquillian Framework.
  • Now we can test sharding and master/slave replication of Redis servers.
  • Bug fixing.
In next version of NoSQLUnit support for Infinispan, adding class capabilities for testing sharding and master/slave replication in MongoDB and bug fixes will be provided.

We keep learning,
Alex.

Como una sonrisa, eres tú, eres tú.
Así, así, eres tú. (Eres Tú - Mocedades)



jueves, diciembre 13, 2012

Metrics, A new way to monitorize your application


When you are running long term applications like web applications, it is good to know some statistics about them, like number of requests served, request durations, or the number active requests. But also some more generic information like the state of your internal collections, how many times some portion of code is being executed, or health checks like database availability, or any kind of connection to an external system.

All this kind of instrumentalization can be achieved by using native JMX or using a modular project like Metrics. Metrics provides a powerful way to measure the behaviour of your critical components and reporting them to a variety of systems like, JConsole, System Console, Ganglia, Graphite, CSV, or making them available through a web server.

To install Metrics, we only have to add metrics dependency. In this example we are going to use Maven.

<dependencies>
    <dependency>
        <groupId>com.yammer.metrics</groupId>
        <artifactId>metrics-core</artifactId>
        <version>2.2.0</version>
    </dependency>
</dependencies>

Now it is time to add some metrics to our code. In Metrics we can use 6 types of metrics:
  • Gauges: an instantaneous measurement of a discrete value. 
  • Counters: a value that can be incremented and decremented. Can be used in queues to monitorize the remaining number of pending jobs.
  • Meters: measure the rate of events over time. You can specify the rate unit, the scope of events or  event type.
  • Histograms: measure the statistical distribution of values in a stream of data.
  • Timers: measure the amount of time it takes to execute a piece of code and the distribution of its duration.
  • Healthy checks: as his name suggests, it centralize our service's healthy checks of external systems.
So let's write a really simple application (in fact it is a console application) which sends queries to Google Search system. We will measure the number of petitions, the number of characters sent to Google, the last word searched, and a timer for measuring the rate of sending a request and receiving a response.

The main class where Measures will be applied is called MetricsApplication and is the responsible of connecting to Google and sending the entered word.

The first thing we can see is the counter instance. This counter will count the number of characters that are sent to Google in the whole life of the applications (meanwhile you don't stop it).

The next property is a meter that measures the rate of sending queries over time.

Then we have got a timer that rates the sendQueryToGoogle method callings and its distribution over time.

And finally a LinkedList for storing all queries sent. This instance will be used to return the last query executed, and is used in gauge for returning the last inserted element.

Notice that in each measure we are setting a class which will be used as folder in jconsole. Moreover a label is provided to be used as name inside folder.

Let's see a screenshot of jconsole with previous configuration and an execution of three searches:


By default all metrics are visible via JMX. But of course we can report measurements to console, http server, Ganglia or Graphite.  

Also note that in this example we are mixing business code and metrics code. If you are planning to use  Metrics in your production code I suggest you to put metrics logic into AOP whenever possible.

We have learned an easy way to monitorize our applications without using JMX directly. Also keep in mind that Metrics comes with some built-in metrics for instrumenting HttpClient, JDBI, Jetty, Jersey, Log4j, Logback or Web Applications.

We Keep Learning,
Alex.

But I know that one and one is two, And if this one could be with you, What a wonderful world this would be. (Wonderful World - Sam Cooke)

jueves, diciembre 06, 2012

Ada Is Here


Yesterday my daughter was born in Barcelona. We named Ada in honor of Augusta Ada King, who is considered the first person that wrote the first computer program. Both are Sagittarius, Ada was born on December 10 and my daughter on December 5, almost the same day.

Hope to see her with Octocat Onesie quickly and pushing code :D.

We Keep Learning,
Alex.

Im pickin up good vibrations, Shes giving me excitations, Im pickin up good vibrations (Good Vibrations - The Beach Boys)
Music: http://www.youtube.com/watch?v=B0yoiBYbT2I

martes, diciembre 04, 2012

Writing Acceptance Tests for Openshift + MongoDb Applications



Acceptance testing are used to determine if the requirements of a specification are met. It should be run in an environment as similar as possible of the production one. So if your application is deployed into Openshift you will require a parallel account to the one used in production for running the tests. In this post we are going to write an acceptance test for an application deployed into Openshift which uses MongoDb as database backend.

The application deployed is a very very simple library which returns all the books available for lending. This application uses MongoDb for storing all information related to books.

So let's start describing the goal, feature, user story and acceptance criteria for previous application.

Goal: Expanding lecture to most people.
Feature: Display available books.
User Story: Browse Catalog -> In order to find books I would like to borrow, As a User, I want to be able to browse through all books.
Acceptance Criteria: Should see all available books.

Scenario:
Given I want to borrow a book
When I am at catalog page
Then I should see available books information: The Lord Of The Jars - 1299 - LOTRCoverUrl , The Hobbit - 293 - HobbitCoverUrl

Notice that this is a very simple application, so the acceptance criteria is simple too.

For this example, we need two test frameworks, the first one for writing and running acceptance tests, and the other one for managing the NoSQL backend. In this post we are going to use Thucydides for ATDD and NoSQLUnit for dealing with MongoDb.

The application is already deployed in Openshift, and you can take a look at https://books-lordofthejars.rhcloud.com/GetAllBooks



Thucydides is a tool designed to make writing automated acceptance and regression tests easier. 

Thucydides uses WebDriver API to access HTML page elements. But also helps you to organise your tests and user stories by using a concrete programming model, create reports of executed tests, and finally it also measures functional cover. 

To write acceptance tests with Thucydides next steps should be followed. 
  • First of all choose a user story of one of your features. 
  • Then implement the PageObject class. PageObject is a pattern which models web application's user interface elements as objects, so tests can interact with them programmatically.  Note that in this case we are coding "how" we are accessing to html page.
  • Next step is implementing steps library. This class will contain all steps that are required to execute an action. For example creating a new book requires to open addnewbook page, insert new data, and click to submit button. In this case we are coding "what" we need to implement the acceptance criteria.
  • And finally coding the chosen user story following defined Acceptance Criteria and using previous step classes.








NoSQLUnit is a JUnit extension that aims us to manage lifecycle of required NoSQL engine, help us to maintain database into known state and standarize the way we write tests for NoSQL applications. NoSQLUnit is composed by two groups of JUnit rules,  and two annotations. In current case, we don't need to manage lifecycle of NoSQL engine, because it is managed by external entity (Openshift).

So let's getting down on work:

First thing we are going to do is create a feature class which contains no test code; it is used as a way of representing the structure of requirements.

Note that each implemented feature should be contained within a class annotated with @Feature annotation. Every method of featured class represents a user story.

Next step is creating the PageObject class. Remember that PageObject pattern models web application's user interface as object. So let's see the html file to inspect what elements must be mapped.

The most important thing here is that table tag has an id named listBooks which will be used in PageObject class to get a reference to its parameters and data. Let's write the page object:

Using @DefaultUrl we are setting which URL is being mapped, with @FindBy we map the web element with id listBooks, and finally getBooksTable() method which returns the content of generated html table.

The next thing to do is implementing the steps class; in this simple case we only need two steps, the first one that opens the GetAllBooks page, and the other one which asserts that table contains the expected elements.

And finally class for validating the acceptance criteria:

There are some things that should be considered in previous class:
  • @Story should receive a class defined with @Feature annotation, so Thucydides can create correctly the report.
  • We use MongoDbRule to establish a connection to remote MongoDb instance. Note that we can use localhost address because of port forwarding Openshift capability so although localhost is used, we are really managing remote MongoDb instance.
  • Using @Steps Thucydides will create an instance of previous step library.
  • And finally @UsingDataSet annotation to populate data into MongoDb database before running the test.


Note that NoSQLUnit maintains the database into known state by cleaning database before each test execution and populating it with known data defined into a json file.

Also keep in mind that this example is very simple so only and small subset of capabilities of Thucydides and NoSQLUnit has been shown. Keep watching both sites: http://thucydides.info and https://github.com/lordofthejars/nosql-unit

We keep learning,
Alex.
Love Is A Burning Thing, And It Makes A Fiery Ring, Bound By Wild Desire, I Fell Into A Ring Of Fire (Ring of Fire - Johnny Cash)

lunes, noviembre 19, 2012

Improving Readability of Your Conditional Expressions with Bool

Introduction

Bool is a project that uses Hamcrest matchers to provide a clean way to write conditionals and making them readable even for your clients.
Note that apart from cleaning your code, it forces you to maintain the level of abstraction of methods without having to populate your classes with a list of methods that are composed by one line returning a boolean value like next example:
...
if(areResultsAvailable(messages)) {
...
}

private boolean areResultsAvailable(List<String> messages) {
    return messages.size()>0;
}

From Hamcrest documentation we can read that:
Hamcrest it is not a testing library: it just happens that matchers are very useful for testing.
Hamcrest is used in many projects, JUnit for asserting tests (most of us have use it in this way), or Mockito. But as previous cite said,Hamcrest can be also used outside testing scope, Lambdaj is an example, and Bool is another one.

Installation

To use Bool you only have to add it to classpath.
<dependency>
    <groupId>com.lordofthejars</groupId>
    <artifactId>bool</artifactId>
    <version>0.9.0</version>
</dependency>

In Action

All required methods are provided as static methods in the class Bool:
import static com.lordofthejars.bool.Bool.*;
Let's explore a simple example to see the differences between using an if as usually and with Bool project:
String name = "Alex";

if(isAlex(name) {
...
}

private boolean isAlex(String name) {
    return "Alex".equals(name);
}
you can use Hamcrest directly in condition without using Bool, but see that readability is not improved:
import static org.hamcrest.CoreMatchers.equalTo;

if(equalTo("Alex").matches(name)) {
...
}
and finally using Bool:
if(the(name, is(equalTo("Alex")))) {
...
}
Note that the important word here is the, which receives the element to compare and a matcher (similar to assertThat method). See in previous example how condition has been improved so much.
Another place where Bool improves readability is with not operator (!).
If you want to return if name is not Alex, you should add a new method with ! operator which does not help us too much in improving the code legibility.
if(isNotAlex(name) {
...
}

private boolean isAlex(String name) {
    return "Alex".equals(name);
}

private boolean isNotAlex(String name) {
    return !isAlex(name)
}
But with Bool:
if(the(name, is(not(equalTo("Alex"))))) {
...
}
Bool is also useful when you are dealing with collections. Because you can use any matcher from Hamcrest project you can implement conditions that would require some work if you didn't use Bool, in a few seconds; see next example:
List ages = Arrays.asList(21, 25, 30);

if(areAllPersonsAdult(ages)) {
...
}

private boolean areAllPersonsAdult(List<Integer> ages) {

    for(int age:ages) {
        if(age < 18) {
            return false;
        }
    }

return true;

}
To something like:
List ages = Arrays.asList(21, 25, 30);

if(the(ages, is(everyItem(greaterThan(18))))) {
..
}
But also with arrays are improved so much, when you want to compare two arrays you cannot do by using equals method directly but using Arrays.equals method. But Hamcrest matchers deals with this problem, hiding this details from developers so we can compare arrays safely.
byte[] name = "alex".getBytes();

if(the(name, is(equalTo("Alex".getBytes())))) {
...
}
And of course you can use all matchers you can imagine like containsemptycontainsInAnyOrder, ...
Let's complicate things a bit more.
Normally, our conditions contain more than one clause. For example, the name should be "Alex" or "Ada"

For covering this case, be and is keyword is also available. is keyword provides a better readability to your conditions, but can conflict with Hamcrest "is" method. So be is also provided. You can use any of them.
if(the(name, be(equalTo("Alex")).or(be(equalTo("Ada"))))) {
...
}
and is also valid, but remember that the is method is from Bool class not the Hamcrest one.
if(the(name, is(equalTo("Alex")).or(is(equalTo("Ada"))))) {
...
}
be matcher also contains and operation too.
But sometimes rules are more complex and imply more than one variable. For solving this cases, we must use twice the the keyword. So for example if name should be "Alex" and surname "Soto":
String name = "Alex";String surname = "Soto";

if(the(name, be(equalTo("Alex")).and(the(surname, equalTo("Soto"))))) {...}
Note that be keyword is only mandatory when we want to concatenate conditions.
As final notes, you can even use matchers defined in Lambdaj project, and a pretty example could be to compare a property of a class:
private class Person {

        private String name;

        public Person(String name) {
            this.name = name;
        }

        public void setName(String name) {
            this.name = name;
        }

        public String getName() {
            return name;
        }
}

if(the(person, having(on(Person.class).getName(), is(equalTo("Alex"))))) {
...
}
And also you can use Bool in unit tests by using shouldBe method. For example a valid assertion could be:
assertThat(the(name, shouldBe(equalTo("Alex")).and(shouldBe(startsWith("A")))), is(true));

Performance

It seems reasonable that using Bool performance should be worst than using native conditions. But I have to say that I have been pleasantly surprised by the performance of Bool and Hamcrest matchers.
I have run 5000 times some examples provided in documentation. For example one single condition, one condition negated, two conditions over the same attribute with and keyword, two attributes being compared with one condition, and finally one collection comparison. And the results are the next ones:
Performance
Note that using simple conditions the time is the same, and is after we start creating more complex condition expressions that the performance is a bit reduced.
You can find the performance test in com.lordofthejars.bool.performance.PerformanceTests

Final Notes

Please keep in mind that the border between writing clean and readable code and something unintelligible is very thin. You can start creating a complex chaining of calls, train wreck, so no one can understand the real meaning of the condition. 

One good practice to avoid this case is splitting complex chaining calls into multiple variables and then the final calls are executed inside the if (or inside a method which extracts all this logic). But anyway if you found creating a really complex condition, think about the correctness of this logic before coding it, because maybe you are providing to a condition a heavy responsibility.
Also note that you can use Bool in all of your conditions, but where you really take the full power is in  conditions that represents important rules for your business.

Stay In Touch

And any suggestion, improve, or bug don't hesitate to open an issue.

E il Tacchino GluGluGlu, E il Gallo CoroCoco, E la Gallina Coo, E il Pulcino Pio (Il Pilcino Pio - I Blogger)
Music: http://www.youtube.com/watch?v=YNwjkEdBpOk

lunes, noviembre 05, 2012

Meet Me at Devoxx 2012



Next week starts the Devoxx. I will be there as speaker and as listener. You can meet me on two presentations (a Tools in Action and a BOF ones).




In Tools In Action I will introduce you NoSQLUnit, a JUnit extension for writing tests for NoSQL applications.

This will be on Monday 12th at 16:45.


Don't miss it if you are planning to use NoSQL systems as backend in your applications.




In the BOF, Bartosz Majsak, John Ferguson Smart, Paul Bakker, Dan Allen, Aslak Knutsen, Sarah White, Mircea Markus, Lukáš Fryč, David Blevins and Me, are going to talk about killing bugs.

This will be on Monday 12th at 21:00.


Don't miss it for anything else in the world.


Hope to see all of you there.
Alex.
And through it all she offers me protection, A lot of love and affection, Whether I'm right or wrong (Angel - Robbie Williams)



jueves, noviembre 01, 2012

NoSQLUnit 0.6.0 Released




NoSQLUnit is a JUnit extension to make writing unit and integration tests of systems that use NoSQL backend easier. Visit official page for more information.

In 0.6.0 release, one new NoSQL system is supported and is HBase.

Apache HBase is an open-source, distributed, versioned, column-oriented store.

As all databases supported by NoSQLUnit, two set of rules are provided to write HBase tests:

First set of JUnit Rules are those responsible of managing database lifecycle; basically starting and stopping HBase instance.
  • Embedded: com.lordofthejars.nosqlunit.hbase.EmbeddedHBase
  • Managed: com.lordofthejars.nosqlunit.hbase.ManagedHBase
Depending on the kind of tests you are implementing (unit test, integration test, deployment tests, …) you will require an embedded, managed or remote approach. Note that for now I haven't implemented an In-Memory approach because there is no in-memory HBase  instance, but embedded strategy for unit tests will be the better one. 

Second set of rules are those responsible of maintaining database into known state;
  • NoSQLUnit Management: com.lordofthejars.nosqlunit.hbase.HBaseRule
And finally default dataset file format in HBase is json. Dataset in HBase is the same used by Cassandra-Unit but not all fields are supported. Only fields available in TSV HBase application can be set into dataset.

We will use a very simple example used in HBase tutorial as an example of how to write unit tests for systems that uses HBase database as backend.

First of all, dataset used to maintain HBase into known state:


and finally the test case:


Stay in touch with the project and of course I am opened to any ideas that you think that could make NoSQLUnit better.
We de ze zu bu, We de sooo a ru, Un va-a pesh a lay, Un vi-I bee (Now We Are Free - Lisa Gerrard)
Music: http://www.youtube.com/watch?v=ObGYFInWrU0

Donate If You Can and Find Post Useful