Generate Spring Boot migrations from Hibernate entities

Agile methodologies have database structures evolve on a weekly basis for most web applications. Database migrations make it easy for developers to manage these changes, both for local development but also for databases running in production with real data. The idea is to provide scripts that describe the queries to run on the database to make it reach its wanted new structure.

In the Java, Kotlin and other JVM based ecosystems, Liquibase is a popular solution which allows writing migrations in SQL, but also in XML, YAML or JSON which are then transformed into SQL queries understandable by most database flavors.

However, when wanting to change the database's structures, developers usually find themselves doing in two steps which are redundant:

• Creating/Updating/Deleting entities which are managed through the Hibernate ORM
• Writing by hand similar changes but in a different syntax in a new Liquibase changelog files
• Registering the migration file in the master Liquibase changelog

In this article, we explore how we managed to automate the second and third step to increase our productivity and enforce coherency between the database structure and the ORM entities.

Determining the wanted behavior looking at best in class players

What we want to achieve is to have database migrations be generated by comparing the current state of the database with its wanted state by looking at the entities which are currently present in the code. Many libraries and frameworks include an out of the box system to make it possible. Here is some for them:

• Django (Python) includes this feature in their CLI
• Symfony (PHP) provides a bundle to provide this feature
• Alembic provides this feature for SQLAlchemy powered Python applications

What we seek is thus a single one line command, where the developer simply has to provide a description label for the migration which will trigger 

• the creation in src/main/resources/db/changelog/changes of a file named including the current timestamp as well as the developer provided name. For instance, a migration file could be named 20190801093305-user-creation.yaml 
• the filling of this file with all changes required to make the database go from its current state to the wanted state
• the registering of this newly created file in the main liquibase changelog, which is in my case in src/main/resources/db/changelog/db.changelog-master.yaml

Warnings and assumptions before getting started

The following tutorial makes several assumptions due to stability issues I found using the few libraries required to make it work, namely:

• using the exact versions of the librariries listed in the article : I found that using previous or next versions are not working correctly at the moment
• using the YAML format for migrations, as I found using different formats came with errors and badly formatted files
• we will be using a simple in file h2 database to simplify the setup

Creating the entity

We will assume we want to create a new entity which is to be persisted in the database. Here is the Hibernate entity, created in Kotlin

We thus want to create a new table with two columns in the database

Installing dependencies

We provided the relevant dependencies to include in the pom.xml. In our case, we are using spring-boot-starter-data-jpa, spring-boot-starter-jdbc and h2 for the ORM and the database connexion, and liquibase-core for handling migrations.

We also use the liquibase-maven-plugin in the pom.xml plugins section to provide the maven goal which will generate the migration file. Three things are notable:

• we inject a Spring property to provide the diffChangeLogFile which is path of the file which will contain the migration to create, in order for the user to be able to provide the name of the file
• we list several dependencies for the plugin, which are also listed in the main pom dependencies : the plugin will not run in the same context and will need to access Spring/Hibernate related beans
• we stress once again that all versions listed have to be used as increasing or decreasing versions will likely cause the whole process not to work at this moment

Here is finally how we provide the ability to inject the diffChangeLog property in the pom.xml file

Configuring Liquibase and Liquibase-Maven-Plugin

We here describe the directory structure we use for configuring Liquibase and holding the changelogs. Make sure all the db/changelog/changes directory is created before going further.

In the Spring application.properties file, we simply specify we use an H2 in file database and we make sure Hibernate will not use the entities to create or update the database on its own. Indeed, Liquibase will be performing that job.

In the liquibase.properties properties file which we specified in the liquibase maven plugin propertyFile field, we :

• provide the path to the master Liquibase changelog file
• also provide the database url, driver and default h2 credentials
• also provide the referenceUrl which contains the package in which your Hibernate entities are located (com.exemple.app.domain in our case). Make sure to change the package as well as any reference to the H2 dialect if you use a different database

Finally, the master Liquibase changelog file should contain a single line and new migration files references will be appended to it with each generation (keep an empty line at the end of the file)

Wrap it all up in a Makefile

We want to define a single command to:

• prompt the user for the migration label
• generate the migration file path from a current timestamp and the label
• append the reference to the migration file in the master changelog

Makefile is a good tool for such a purpose

 

It's all ready! You can now test your new system using

make makeMigration MIGRATION_LABEL="user-creation"

Conclusion

We were able to quickly create a simple make command which generates migration on the fly from database entities. Make sure to always check the generated migrations as nothing beats an expert developer eye for:

• catching mistakes
• improving the generated changes
• adding indexes or uniques for performance matters

Always use liquibase for running the generated migration up and down locally, before pushing it to production.

The whole exemple can be found in this Github repository