miércoles, diciembre 14, 2011

Elle ne me quitte pas d'un pas, fidèle comme une ombre. Elle m'a suivi ça et là, aux quatres coins du monde. Non, je ne suis jamais seul avec ma solitude (Ma Solitude - Georges Moustaki)

In current post I have uploaded my Devoxx presentation. This year I was at Devoxx as speaker. My presentation was about how to speed up HTML 5 applications, and Javascript and CSS in general, using Aggregation and Minification

Also you can visit two entries of my blog where I talk about same theme.
Links of technologies I talked about:

Finally I want to say thank you to all people who came to watch me, and of course Devoxx folks, for organising such amazing event.

I wish you have discovered a great way to speed up your web applications.

Music: http://www.youtube.com/watch?v=qSZKO5K2eTE

viernes, diciembre 09, 2011

Far away, long ago, glowing dim as an Ember, Things my heart use to know, things it yearns to remember (Once upon a December - Anastasia)

You never develop code without version control, why do you develop your database without it? Flyway  is database-independent library for tracking, managing and applying database changes.

Personally I find that using a database migration tool like Flyway is "a must", because covers two scenarios of our software life-cycle:

  • Multiple developers developing an application with continuous integration.
  • Multiple clients each one with different versions of production code.

Let's start with first point. If your project is big enough there will be more than one developer working on it, each one developing a new feature. Each feature may require a database update (adding a new table, a new constraint, ...), so developer creates a .sql file with all required changes.

After each developer finishes its work, these changes are merged into main branch and integration/acceptance tests are executed on test machine. And the problem is obvious which process updates testing database? And how? QA department executes sql files manually? Or we develop a program that executes these updates automatically? And in what order must be executed? Also same problem arises in production environment.

Second point is only applicable if your application is distributed across multiple clients. And at this point the problem is further accentuated because each client may have different software versions. Hence when an update is required by our client (for example because a bug), you should know which database version was installed and what changes must be applied to get expected database.

Don't worry Flyway comes to rescue you, and will help to fix all previous questions. Let me start explaining some features of Flyway that in my opinion make it a good tool.

  • Automatic migration: Flyway will update from any version to the latest version of schema. Flyway can be executed as Command-line (can be used with non JVM environments), Ant script, Maven script (to update integration/acceptance test environments) or within application (when application is starting up).
  • Convention over configuration: Flyway comes with default configuration so no configuration is required to start using.
  • Plain SQL scripts or Java classes: To execute updates, you can use plain SQL files or Java classes for advanced migrations.
  • Highly reliable: safe for cluster environments.
  • Schema clean: Flyway can clean existing schema, so empty installation is produced. 

Conventions to be followed if they are not explicitly modified are:

  • Plain SQL files go to db/migration directory inside src/main/resources structure.
  • Java classes go to db.migration package.
  • Files (SQL and Java) must follow next name convention: V<version>[__description]. Where each version number is separated by dots (.) or underscore (_) and if description is provided, two underscores must proceed. A valid example is V_1_1_0__Update.sql

So let's see Flyway in action. In this application I am going to focus only on how to use Flyway, I am not going to create any DAO, DTO or Controller class, only database migration part.

Imagine we are going to develop a small application using Spring Framework that will allow us registering authors and which books they have written.

First version will contain two tables, Author and Book related with one to many relationship.

First step is registering Flyway into Spring Application Context. Flyway class is the main class and requires a javax.sql.DataSource instance. migrate method is responsible to start migration process.

See that there is no secret. Only be careful because if your project uses JPA or ORM frameworks for persistence, you should configure them to avoid auto creation of tables, because now Flyway is responsible of managing database structure. And because of that, creation of SessionFactory (in case of Hibernate) or EntityManagerFactoryBean( in case of JPA),  should depends on Flyway bean.

Flyway is configured. Each time you start application, it will review if configured datasource requires an update or not.

And now let's write first version of SQL migration. Create db/migration directory into src/main/resources and create a file called V1__Initial_version.sql with next content:

This script creates Author and Book tables with their respective attributes.

And if you run next JUnit both tables are created into database.

Take a look at your console and next log message has appeared:

10:33:49,512  INFO glecode.flyway.core.migration.DbMigrator: 119 - Current schema version: null
10:33:49,516  INFO glecode.flyway.core.migration.DbMigrator: 206 - Migrating to version 1
10:33:49,577 INFO glecode.flyway.core.migration.DbMigrator: 188 - Successfully applied 1 migration (execution time 00:00.085s).

And if you open your database:

Note that Flyway has created a table to annotate all updates that have been executed (SCHEMA_VERSION) and last insert is a "Flyway insert" marking which is the current version.

Then your first version of application is distributed across the world.

And you can start to develop version 1.1.0 of application. For next release, Address table must be added with a relationship to Author.

As done before, create a new SQL file V1_1_0__AddressTable.sql into db/migration folder.

And run next unit test:

your database will be upgraded to version 1.1.0. Also take a look at log messages and database:

11:27:30,149  INFO glecode.flyway.core.migration.DbMigrator: 119 - Current schema version: 1
11:27:30,152  INFO glecode.flyway.core.migration.DbMigrator: 206 - Migrating to version 1.1.0
11:27:30,191 INFO glecode.flyway.core.migration.DbMigrator: 188 - Successfully applied 1 migration (execution time 00:00.053s).

New table is created, and a new entry into SCHEMA_VERSION table is inserted marking that current database version is 1.1.0.

When your 1.1.0 application is distributed to your clients, Flyway will be the responsible of updating their databases without losing data.

Previously I have mentioned that Flyway also supports Java classes for advanced migrations. Let's see how.

Imagine that in your next release, authors can upload their personal photo, and you decide to store as  blob attribute into Author table. The problem resides on already created authors because you should set some data into this attribute. Your marketing department decides that authors inserted prior to this version will contain a photo of Spock,

So now you must alter Author table and moreover update a field with a photo. You can see clearly that for this update you will need something more than a simple SQL file, because you will need to add a new property and updating them with chunk of bytes. This problem could be accomplished using only one Java class but for showing a particularity of Flyway, problem will be treated with one SQL and one Java object.

First of all new SQL script adding a new binary field is created. This new feature will be implemented on version 2.0.0, so script file is named V2_0_0__AddAvatar.sql.

Next step is developing a Java Migration class. Create a new package db.migration on src/main/java. Notice that this class cannot be named V2_0_0_AddAvatar.java because Flyway will try to execute two different migrations with same version, and obviously Flyway will detect a conflict.

To avoid this conflict you can follow many different strategies, but in this case we are going to add a letter as version suffix, so class will be named V2_0_0_A__AddAvatar.java instead of V2_0_0__AddAvatar.java.

Before run previous unit test, open testdb.script file and add next line just under SET SCHEMA PUBLIC command.


And running unit test, next lines are logged:

20:21:18,032  INFO glecode.flyway.core.migration.DbMigrator: 119 - Current schema version: 1.1.0
20:21:18,035  INFO glecode.flyway.core.migration.DbMigrator: 206 - Migrating to version 2.0.0
20:21:18,088  INFO glecode.flyway.core.migration.DbMigrator: 206 - Migrating to version 2.0.0.A
20:21:18,114 INFO glecode.flyway.core.migration.DbMigrator: 190 - Successfully applied 2 migrations (execution time 00:00.094s).

And if you open updated database, next lines are added:

See how all previous authors have avatar column with data.

Note that now you have not to worry about database migrations, your application is packaged and delivered to all your clients regardless of the version they had installed; Flyway will execute only required migration files depending on installed version.

If you are not using Spring, you can update your database using Flyway-Maven-Plugin. Next piece of pom shows you how to execute migration during test-compile phase. By default plugin is executed during pre-integration-test phase.

Thanks of Maven plugin, we can configure our continuous integration system so all environments (test, production,...) would be updated during deployment of application.

I wish Flyway will help you make better life as developer.

Music: http://www.youtube.com/watch?v=oyUBdLm3s9U

jueves, diciembre 01, 2011

All my scientists are working on a deadline, So my psychologist is working day and nighttime, They say they know what's best for me, But they don't know what they're doing (Atomic Garden - Bad Religion)

Maven archetypes are project templates that allow users create project structure with a simple Maven command. In my company we are using archetypes because provide a way to standardize projects structure. All our projects are built using the same directory structure, all of us use the same version of common libraries like JUnit, Hamcrest, Spring Framework, Mockito, or in case of web applications bundling them with company's approved CSS and Javascript libraries. Also PMD, checkstyle or findbugs coding rules can be stored in distributed archetype.

If each time you start a new project you are of those who copy files from existing projects to the new one, apply DRY principle and create a Maven archetype from existing project.

First thing to do is create your template project with all files to be bundled into archetype. In this example, simple Spring MVC project will be transformed  to be a Maven archetype.

After template project is created and all desired files are added, you should have a directory layout like:

My personal advice is that if you are thinking about distributing this archetype with community (not only for your company), remove all IDE specific files.

Now you have your project created and ready to be packaged as archetype. Execute next command on root of your project.
mvn archetype:create-from-project
And Maven console output should be:

And now your archetype is created in target/generated-sources/archetype directory with next hierarchy:

Now project is inside archetype-resources directory. This directory contains all files that will be added in generated project.

At first sight, not much differences between original project and "template" project, it seems that only three files has been added archetype-metadata.xml, archetype.properties and goal.txt, but shortly you will see that original project content has been modified too.

Before continuing see that in project exists two poms, one pom that is in root directory, that will be called archetype pom, because it contains all archetype configuration, and another one into archetype-resources, called template pom, because it will be the pom used in generated project.

Next step is isolate archetype project into separate folder, so can be dealt as alone project.
mv target/generated-sources/archetype ../spring-mvc-archetype

Following step is adding a name to generated archetype, so open archetype pom and change  <name> tag value to your archetype name, for example spring-mvc-archetype, and if you want  artifactId and groupId too.

After this modification, open archetype-resources' pom, and see how <artifactId> or <groupId> values are surrounded with ${artifactId} or ${groupId}. When you are creating a new archetype, by default Maven will ask you to enter four parameters, groupId, artifactId, version and package. Entered values will be used to fill placeholders.

With default four parameters should be enough, but imagine you want that user provides more information, for example, war name. To get this, open archetype-metadata.xml file (src/main/resources/META-INF/maven) and add one required property, using <requiredProperties> tag.

In previous file we are adding a new required property named warName. And last thing to do is update
archetype.properties located on test/resources/projects/basic with default value of new property.

And that's all, if you open any Java class or any Xml file, you will see that has been modified with ${package} variable. This information is filled when you generate the project.

Now you can install archetype into your local catalog and start generating standardized  projects.
mvn clean install
And your artifact is ready to be used. Try next command or if you have installed m2Eclipse plugin open Eclipse and try your new archetype:
mvn archetype:generate -DarchetypeCatalog=local
A list of all installed archetypes is shown. Choose previously created and fill up all required properties, and your new project is built and configured. You can start coding with same libraries that your workmates use and same style rules.

In this post a simple example has been provided, but think about all kind of elements that you copy and paste from one project to another like SCM connection, surefire plugin configuration, release plugin tag name, to cite a few, and how you can integrate them into your archetype.

I wish you have found this post interesting.

Music: http://www.youtube.com/watch?v=AhzhiQA6-Aw&ob=av3e