Thursday, December 9, 2010

Using Liquibase for managing Database Migrations in grails


Here are some notes I recently sent to a developer about our usage LiquiBase for database migrations. I was just about to send it to a second person and decided it was time to publish the information so more people could get some use out of it as well. We use Liquibase to manage database migrations for Grails applications using two plugins for Grails: Liquibase and LiquibaseRunner. Note however that LiquiBase is NOT specific to Grails – you can use it on any java project that needs to manage DB changes.

Changelog structure:
We typically set up a root changelog that then references other changelog files. We create an “initial” changelog file plus one per release. It works well for keeping the changelog files relatively small and also keeps things into nice manageable chunks. It looks something like this (where all the files live in the grails-app/migrations directory):

changelog.xml:
<?xml version="1.0" encoding="UTF-8"?>


<databaseChangeLog
  xmlns="http://www.liquibase.org/xml/ns/dbchangelog/1.9"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog/1.9
         http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-1.9.xsd">


    <include file="000-initial-changelog.xml" relativeToChangelogFile="true"/>
    <include file="001-changelog-r1.xml" relativeToChangelogFile="true"/>
    <include file="002-changelog-r2.xml" relativeToChangelogFile="true"/>


</databaseChangeLog>

The 000, 001, 002 naming convention isn’t required, it’s a leftover artifact from using autobase but we also like it in that it keeps the files in order if you’re looking at them in a file system. The “relativeToChangelogFile” works for us because we use the liquibase-runner plugin which copies all the migrations to the classes/migrations dir in _Events.groovy (at package end).
Typically we develop new applications using the HSQL database up until the point at which we deploy the app as a WAR file on a tomcat instance. Once we get there, we start an initial changelog. The sooner you start one the better, but we have also had success retro-fitting a changelog on to apps that didn’t start out with one. It requires a couple manual steps, but as you know the Liquibase plugin has some handy scripts for helping out with that (generate-changelog, I’m looking at you).

Types – use generic types 
When generating the changelog, Liquibase uses whatever types it finds in the DB – that is, it doesn’t convert things to generic types. So usually I go through a generated changelog and switch types to the generic ones. For example, when using MySql, I switch “BIT” to “BOOLEAN” and “TINYCLOB” to “CLOB”, etc. The list of generic types Liquibase supports can be found on the “Column” page in the manual: http://www.liquibase.org/manual/column This is only required if you want your changeset to be able to work against multiple different database engines.

MySql Innodb
The generated changelog doesn’t appear to capture the type of table/database in MySql. So even if you generate a changelog from a DB that was created with a dialect of org.hibernate.dialect.MySQLInnoDBDialect it doesn’t reflect that. We learned this the hard way when we deployed an app to a MySql database that was configured as MyISAM by default and all our tables turned out to be MyISAM instead of InnoDB. Now, we always add a “modifySql” tag to the end of every “createTable”, like this:

<changeSet author="mjhugo (generated)" id="1288615205809-4">
    <createTable tableName="security_role">
        <column autoIncrement="true" name="id" type="BIGINT">
            <constraints nullable="false" primaryKey="true"/>
        </column>
        <column name="version" type="BIGINT">
            <constraints nullable="false"/>
        </column>
        <column name="authority" type="VARCHAR(255)">
            <constraints nullable="false"/>
        </column>
    </createTable>
    <modifySql dbms="mysql">
        <append value=" engine innodb"/>
    </modifySql>
</changeSet>

Running migrations automatically when the application starts
We use the liquibase-runner plugin on most of our projects to automatically run all the migrations when the WAR file starts up. It’s also useful in development because once a changeset is checked in, every developer database is automatically up to date with the domain classes. An interesting side effect of this is that once a change set has been checked in, we consider it to be written in stone – we don’t necessarily know where that changeset has already been run. So if we make a mistake in a changeset, we create a new changeset to fix the mistake, rather than editing the changeset that had a mistake in the first place.

No comments:

Post a Comment