Nasha nasha krovatka delala shik - shik, Ya tvo pianino , a ty moi nastroishchik, My tak letali chto ne zametili tvoyu matʹ, Ahaa..I ona skazala chto ya prosto blaz (Mama Lyuba - Serebro))

When we develop any application, after we finish it or when we end up any of its module, we start optimization process. Most applications contain database access, and if you are using an ORM, probably you will use hibernate. Optimizing hibernate persistence layer, requires to be prepared to read, understand and evaluate sql.

If we take an overview on hibernate configuration, two properties, hibernate.format_sql and hibernate.use_sql_comments, should be enabled to print performed sql code through console.

This is a good start but it seems that we need more information to make an accurate diagnosis of performance like connection events, returned data on queries, or parameters binding (hibernate shows parameters values with question mark ?) . Hence, we need another way to inspect generated sql. Log4jdbc is a jdbc driver that can log sql/jdbc calls. In fact log4jdbc is an implementation of proxy pattern which will automatically load popular jdbc drivers (Oracle, Derby, MySql, PostgreSql, H2, Hsqldb, ...), intercept calls, log information, and then send data to "spied" driver.

In log4jdbc, there are 5 loggers that can be used depending on data to monitor:

• jdbc.sqlonly: logs executed sql with binding arguments replaced with bound data.
• jdbc.sqltiming: logs how long a sql took to execute.
• jdbc.audit: logs all jdbc calls except for ResultSets.
• jdbc.resultset: same as jdbc.audit plus ResultsSets.
• jdbc.connection: logs open and close connection events.

In this post we are going to see how to configure log4jdbc-remix, a fork of log4jdbc, which apart from inheriting log4jdbc capabilities, also let us:

• jdbc.resultsettable: log results set in table format.
• configure it as datasource.
• available in maven repository (log4jdbc is not present on maven repositories).

For this example we are going to use the project created by JPA Spring Template which contains two entities Order and Item associated with one-to-many relationship, and one test that executes some database operations.

First thing to do is add log4jdb-remix and slf4j-log4j12 dependencies to project:

Next thing to do is configure active loggers.  Depending on the data we are interested to monitor, we activate the required loggers. As an example let's configure log4j.xml so result set is printed in table format and also time taken to execute each query is shown.

After configuring loggers, run test and inspect the output.


Output is printed in a fashion format, queries contains bind parameters (not a question mark (?)), and process time is also informed.

Notice that logging more or less information is simply a matter of configuring a log. Moreover depending on log level, more or less information will be provided in each case. If logger is configured in DEBUG class name and line number (if available) at which the sql was executed will be included. In INFO will include only sql, and finally ERROR which show stacktraces if any SQLException occurs.

Optimizing hibernate applications can imply touching many parts of an application (JVM configuration, database engine, network, …) but one very important aspect to take care is the number of queries that are sent to RDBMS (for example N+1 problem), and the amount of data that is retrieved from database (Projection problem), and log4jdbc-remix perfectly fits to help in this purpose.

As final note, log4jdbc(-remix) is a jdbc logger, so it is not necessary to use only in hibernate applications, can be used with any framework that uses a datasource.

I wish this library would help you.

Keep Learning,
Alex

Download Code
Music: http://www.youtube.com/watch?v=h9HRHOXfRBI

Qui dit crise te dis monde dit famine dit tiers- monde, Qui dit fatigue dit réveille encore sourd de la veille, Alors on sort pour oublier tous les problèmes, Alors on danse... (Alors on Danse - Stromae)

Let's introduce another hibernate performance tip. Do you remember the model of previous hibernate post? We had a starship and officer related with a one to many association.

Now we have next requirement:

We shall get all officers assigned to a starship by alphabetical order.

To solve this requirement we can:

1. implementing an HQL query with order by clause.
2. using sort approach.
3. using order approach.

The first solution is good in terms of performance, but implies more work as a developers because we should write a query finding all officers of given starship ordered by name and then create a finder method in DAO layer (in case you are using DAO pattern).

Let's explore the second solution,  we could use SortedSet class as association, and make Officer implements Comparable, so Officer has natural order. This solution implies less work than the first one, but requires using @Sort hibernate annotation on association definition. So let's going to modify previous model to meet our new requirement. Note that there is no equivalent annotation in JPA specification.

First we are going to implement Comparable interface in Officer class.

We are ordering officer by name by simply comparing name field. Next step is annotating association with @Sort.

Notice that now officers association is implemented using SortedSet instead of a List.   Furthermore we are adding @Sort annotation to relationship, stating that officers should be natural ordered. Before finishing this post we will insist more in @Sort topic, but for now it is sufficient.

And finally a method that gets all officers of given starship ordered by name, printing them in log file.

All officers are sorted by their names, but let's examine which queries are sent to RDBMS.

First query is resulting of calling find method on EntityManager instance finding starship.

Because one to many relationships are lazy by default when we call getOfficers method and we access first time to SortedSet, second query is executed to retrieve all officers. See that no order by clause is present on query, but looking carefully on output, officers are retrieved in alphabetical order.

So who is sorting officer entities? The explanation is on @Sort annotation. In hibernate a sorted collection is sorted in memory being Java the responsible of sorting data using compareTo method.

Obviously this method is not the best performance-way to sort a collection of elements. It is likely that we'll need a hybrid solution between using SQL clause and using annotation instead of writing a query.

And this leads us to explain the third possibility, using ordering approach.

@OrderBy annotation, available as hibernate annotation and JPA annotation, let us specifies how to order a collection by adding “order by" clause to generated SQL.

Keep in mind that using javax.persistence.OrderBy allows us to specify the order of the collection via object properties, meanwhile org.hibernate.annotations.OrderBy order a collection appending directly the fragment of SQL (not HQL) to order by clause.

Now Officer class should not be touched, we don't need to implement compareTo method nor a java.util.Comparator. We only need to annotate officers field with @OrderBy annotation. Since in this case we are ordering by simple attribute, JPA annotation is used to maintain fully compatibility to other “JPA ready” ORM engines. By default ascendent order is assumed.

And if we rerun get all officers method, next queries are sent:

Both queries are still executed but note that now select query contains order by clause too.

With this solution you are saving process time allowing RDBMS sorting data in a fast-way, rather than ordering data in Java once received.

Furthermore OrderBy annotation does not force you to use SortedSet or SortedMap collection. You can use any collection like HashMap, HashSet, or even a Bag, because hibernate will use internally a LinkedHashMap, LinkedHashSet or ArrayList respectively.

In this example we have seen the importance of choosing correctly an order strategy. Whenever possible you should try to take advantage of capabilities of RDBMS, so your first option should be using OrderBy annotaion (hibernate or JPA), instead of Sort. But sometimes OrderBy clause will not be enough. In this case, I recommend you using Sort annotation with custom type (using java.util.Comparator class), instead of relaying on natural order to avoid touching model classes.

I wish this post helped you to understand differences between "sort" and "order" in hibernate.

Keep learning.

Music: http://www.youtube.com/watch?v=VHoT4N43jK8&ob=av3n

Moi je pense à l'enfant, Entouré de soldats, Moi je pense à l'enfant, Qui demande pourquoi (Non Non Rien N'a Changé - Les Poppys)

After 8 years developing server and embedded applications using Hibernate as ORM, squeezing my brain seeking solutions to improve Hibernate performance, reading blogs and attending conferences, I decided to share this knowledge acquired during these years with you.

This is the first post of many more posts to come:

Last year I went to Devoxx as speaker but also I attended Patrycja Wegrzynowicz conference about Hibernate Anti-Patterns. In that presentation Patrycja shows us an anti-pattern that shocks me because it proved to expect the unexpected.

We are going to see the effect it has when Hibernate detects a dirty collection and should re-create it.

Let's start with the model we are going to use, only two classes related with one-to-many association:

In previous classes, we should pay attention in three important points:

• we are annotating at property level instead of field level.
• @OneToMany and @ManyToOne uses default options (apart from cascade definition)
• officers getter on Starship class returns an immutable list. 

To test model configuration, we are going to create a test which creates and persists one Starship and seven Officers, and in different Transaction and EntityManager finds created Starship.

Now that we have created this test, we can run it and we are going to observe Hibernate console output.

See the number of queries executed during first commit (persisting objects) and during commit of second transaction (finding a Starship). In total and ignoring sequence generator, we can count 22 inserts, 2 selects and 1 delete, not bad when we are only creating 8 objects and 1 find by primary key.

At this point let's examine why these SQL queries are executed:

First eight inserts are unavoidable; they are required by inserting data into database.

Next seven inserts are required because we have annotated getOfficers property without mappedBy attribute. If we look closely at Hibernate documentation, it points us that “Without describing any physical mapping, a unidirectional one to many with join table is used.”

Next group of queries are even stranger, the first select statement is to find Starship by id, but what are these deletes and inserts of data that we have already created?

During commit Hibernate validates whether collection properties are dirty by comparing object references. When a collection is marked as dirty, Hibernate needs to re-create whole collection, even containing the same objects. In our case when we are getting officers we are returning a different collection instance, concretely an unmodifiable list, so Hibernate considers officers collection as dirty.

Because a join table is used, Starship_Officer table should be re-created, deleting previous inserted tuples and inserting the new ones (although they have the same values).

Let's try to fix this problem. We start by mapping a bidirectional one-to-many association, with many-to-one side as owning side.

And now we rerun the same test again and we inspect the output again.

Although we have reduced the number of SQL statements, from 25 to 10, we still have an unnecessary query, the ones just in commit section of second transaction. Why if officers are lazy by default (JPA specification), and we are not getting officers in transaction, Hibernate executes a select on Officers table?  By the same reason as previously configuration, returned collection has different Java identifier, so Hibernate marks it as newly instantiated collection, but now obviously join table operations are no longer required. We have reduced the number of queries but we still have a performance problem. It is likely that we'll need some other solution, and the solution is not the most obvious one, we are not going to return collection objects returned by Hibernate, we might expand on this later, but we are going to change annotations location.

What we are going to do is to change mapping location from property approach to use field mapping. Simply we are going to move all annotations to class attributes rather than on getters.

And finally we are going to run the test again, and see what's happen:

Why using property mapping Hibernate runs queries during commit and using field mapping are not executed? When a Transaction is committed, Hibernate execute a flush to  synchronize the underlying persistent store with persistable state held in memory. When property mapping is used, Hibernate calls getter/setter methods to synchronize data, and in case of getOfficers method, it returns a dirty collection (because of unmodifiableList call). On the other side when we are using field mapping, Hibernate gets directly the field, so collection is not considered dirty and no re-creation is required.

But we have not finished yet, I suppose you are wondering why we have not removed Collections.unmodifiableList from getter, returning Hibernate collection? Yes I agree with you that we finished quickly, and change would look like @OneToMany(cascade={CascadeType.ALL}) public List<Officer> getOfficers() {officers;} but returning original collection ends up with an encapsulation problem, in fact we are broken encapsulation!. We could add to mutable list anything we like; we could apply uncontrolled changes to the internal state of an object.

Using an  unmodifiableList is an approach to use to avoid breaking encapsulation, but of course we could have used different accessors for public access and hibernate access, and not calling  Collections.unmodifiableList method.

Considering what we have seen today, I suggest you to use always field annotations instead of property mapping, we are going to save from a plenty of surprises.

Hope you have found this post useful.

Screencast of example shown here:



Download code
Music: http://www.youtube.com/watch?v=H14VIsnr6aA

Giuro per sempre a te, Di viver, morire per te, Se tu sarai con me lo so, Dea Roma, vincerò!

One of the common problems of people that start using Hibernate is performance, if you don't have much experience in Hibernate you will find how quickly your application becomes slow. If you enable sql traces, you would see how many queries are sent to database that can be avoided with little Hibernate knowledge. In current post I am going to explain how to use Hibernate Query Cache to avoid amount of traffic between your application and database.

Hibernate offers two caching levels:

• The first level cache is the session cache. Objects are cached within the current session and they are only alive until the session is closed.
• The second level cache exists as long as the session factory is alive. Keep in mind that in case of Hibernate, second level cache is not a tree of objects; object instances are not cached, instead it stores attribute values.

After this brief introduction (so brief I know) about Hibernate cache, let's see what is Query Cache and how is interrelated with second level cache.

Query Cache is responsible for caching the combination of query and values provided as parameters as key, and list of identifiers of objects returned by query execution as values. Note that using Query Cache requires a second level cache too because when query result is get from cache (that is a list of identifiers), Hibernate will load objects using cached identifiers from second level.

To sum up, and as a conceptual schema, given next query: "from Country where population > :number", after first execution, Hibernate caches would contain next fictional values (note that number parameter is set to 1000):

L2 Cache
[
id:1, {name='Spain', population=1000, ....}
id:2, {name='Germany', population=2000,...}
....
]
QueryCache
[{from Country where population > :number, 1000}, {id:2}]

So before start using Query Cache, we need to configure cache of second level.
First of all you must decide what cache provider you are going to use. For this example Ehcache is chosen, but refer to Hibernate documentation for complete list of all supported providers.

To configure second level cache, set next Hibernate properties:

hibernate.cache.provider_class = org.hibernate.cache.EhCacheProvider
hibernate.cache.use_structured_entries = true
hibernate.cache.use_second_level_cache = true

And if you are using annotation approach, annotate cachable entities with:

@Cacheable
@Cache(usage = CacheConcurrencyStrategy.NONSTRICT_READ_WRITE)

See that in this case cache concurrency strategy is NONSTRICT_READ_WRITE, but depending on cache provider, other strategies can be followed like TRANSACTIONAL, READ_ONLY, ... take a look at cache section of Hibernate documentation to chose the one that fits better with your requirements.

And finally add Ehcache dependencies:

<dependency>
<groupId>net.sf.ehcache</groupId>
<artifactId>ehcache-core</artifactId>
<version>2.5.0</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-ehcache</artifactId>
<version>3.6.0.Final</version>
</dependency>

Now second level cache is configured, but not query cache; anyway we are not far from our goal.

Set hibernate.cache.use_query_cache property to true.

And for each cachable query, we must call setCachable method during query creation:

List<Country> list = session.createQuery("from Country where population > 1000").setCacheable(true).list();

To make example more practical I have uploaded a full query cache example with Spring Framework. To see clearly that query cache works I have used one public database hosted in ensembl.org. The Ensembl project produces genome databases for vertebrates and other eukaryotic species, and makes this information freely available online. In this example query to dna table is cached.

First of all Hibernate configuration:

It is a simple Hibernate configuration, using properties previously explained to configure second level cache.

Entity class is an entity that represents a sequence of DNA.

To try query cache, we are going to implement one test where same query is executed multiple times.

We can see that we are returning first fifty dna sequences, and if you execute it, you will see that elapsed time between creation of query and commiting transaction is printed. As you can suppose only first iteration takes about 5 seconds to get all data, but the other ones only milliseconds.

The foreach line just before query iteration will print object identifier through console. If you look carefully none of these identifiers will be repeated during all execution. This fact just goes to show you that Hibernate cache does not save objects but properties values, and the object itself is created each time.

Last note, remember that Hibernate does not cache associations by default.

Now after writing a query, think if it will contain static data and if it will be executed often. If it is the case, query cache is your friend to make Hibernate applications run faster.


Download Code

Music: http://www.youtube.com/watch?v=fw1VJSU92mw

Deep Inside, You Cry Cry Cry, Don’t Let Your Hopes, Die Die Die

JPA2 is present in major ORM vendors (EclipseLink, Hibernate, OpenJPA). Main features are present like multiple levels of embedded objects, ordered lists, support for validation, new criteria query API.

Now it time to talk (with an example) about new criteria query API. For this example I will use Hibernate and its Hibernate Metamodel Generator.

New Criteria API allows criteria queries to be constructed in a strongly-typed manner, using metamodel objects to provide type safety. As is mentioned in Hibernate site: "For developers it is important that the task of the metamodel generation can be automated".

And that's is what Hibernate Metamodel Generator does. This metamodel generator is an annotation processor that is run each time an annotated @Entity bean is compiled, and generates its metamodel class.

Let's see an example:

A class like:

@Entity

public class Pizza {

@Id

@GeneratedValue

private Integer id;

public Integer getId() {return id;}

public void setId(Integer id) {this.id = id;}

@Column

private String name;

public String getName() {return name;}

public void setName(String name) {this.name = name;}

@OneToMany(cascade= CascadeType.PERSIST, mappedBy="pizza")

private Set ingredients = new HashSet();

public Set getIngredients() {return ingredients;}

public void setIngredients(Set ingredients) {this.ingredients = ingredients;}

//Constructor, Transient Methods, ...

}

generated metamodel will look:

@StaticMetamodel(Pizza.class)

public abstract class Pizza_ {

public static volatile SetAttribute ingredients;

public static volatile SingularAttribute id;

public static volatile SingularAttribute name;

} 

Easy to understand, each persistent attribute has its type and which entity belongs. For example field "name" belongs to Pizza and is a "String", and thats so important because precisely that will allow Criteria API to be typesafely.

Ok, now we know that new Criteria API is typesafely and this is acquired with metamodel programming. How about starting from the start?

First of all is configuring Eclipse for compiling using Annotation Processor (in Hibernate site is explained for using in other IDEs or Maven).

Only configure annotations processor output directory and which annotation processor is required, in our case hibernate-jpamodelgen and hibernate-jpa-2.0.

Next step is create our entity class:

@Entity

public class Pizza {

@Id

@GeneratedValue

private Integer id;

public Integer getId() {return id;}

public void setId(Integer id) {this.id = id;}

@Column

private String name;

public String getName() {return name;}

public void setName(String name) {this.name = name;}

@OneToMany(cascade= CascadeType.PERSIST, mappedBy="pizza")

private Set ingredients = new HashSet();

public Set getIngredients() {return ingredients;}

public void setIngredients(Set ingredients) {this.ingredients = ingredients;}

public Pizza(String name) {

super();

this.name = name;

}

protected Pizza() {

}

}

Nothing different here. Same is applied for Ingredient entity (not showed here).

But wait go to target/metamodel, refresh the directory and what you see there? The metamodel classes. Ok if you have arrived successful here, then no problem you have done the complicated thing. Now let's write our first new Criteria query.

CriteriaBuilder criteriaBuilder = this.entityManager.getCriteriaBuilder();

CriteriaQuery pizzaQuery = criteriaBuilder.createQuery(Pizza.class);

SetJoin ingredientsNode = pizzaQuery.from(Pizza.class).join(Pizza_.ingredients);

pizzaQuery.distinct(true).where(criteriaBuilder.equal(ingredientsNode.get(Ingredient_.name), ingredient));

TypedQuery typedQuery = this.entityManager.createQuery(pizzaQuery);

return typedQuery.getResultList();

This method returns all pizzas that are produced with given ingredient. If you analyze line per line, first of all is created the query about pizza, then is created the join relationship ingredientsNode and "woop" metamodel class starts the game, we are making a join with Pizza_.ingredients, this is the representation of ingredients class. Also we are telling to filter ingredients where the name is the given one criteriaBuilder.equal(ingredientsNode.get(Ingredient_.name), ingredient), and you see that Ingredient_.name is used. This information is crucial because we are telling to Criteria API that the field is the one defined as Ingredient.name. If name attribute is deleted, metamodel class is modified too and our code does not compile.

Take a look the same criteria query but in Hibernate Criteria:

Criteria criteria = session.createCriteria(Pizza.class);

criteria.createAlias("ingredients", "i")

.add(Restrictions.eqProperty("id", "i.pizza"))

.add(Restrictions.eq("i.name", ingredient));

return (List)criteria.list();

For me it is easy to understand first one than second one, it is like more natural, and does not required any cast.

In Hibernate site more examples of new Criteria API can be found. I encourage you to start using JPA2, not only for new Criteria API but for using all new features.