IBM DB2 Docker container startup speedup

If you develop cloud-native applications using IBM’s DB2 database you will probably want to use DB2 Docker image for your developing, testing and CI/CD needs.

When working on a set of applications using shared DB2 database hosted at our client’s infrastructure we prepared a customized DB2 Docker image as explained on the IBM’s DB2 Docker Hub page.

It boils down to preparing a set of scripts and copying them to /var/custom directory in docker container so they can be used to create & populate DB2 instance after the main DB2 setup has completed. Those scripts are executed on each container start.

During our development, we noticed that it takes quite a while for this container to start. After we added new tables and new data to this DB2 instance startup time of container started to grow.

When we prepared automated tests using the Testcontainers project, the startup time of our database container become the slowest point of our setup.

Initial setup was done using Maven build configuration defined in pom.xml defined as exec-maven-plugin plugin:

<plugin>
	<groupId>org.codehaus.mojo</groupId>
	<artifactId>exec-maven-plugin</artifactId>
	<version>1.6.0</version>
	<executions>
		<execution>
			<id>docker-build</id>
			<phase>install</phase>
			<goals>
				<goal>exec</goal>
			</goals>
			<configuration>
				<executable>docker</executable>
				<workingDirectory>${project.basedir}</workingDirectory>
				<arguments>
					<argument>build</argument>
					<argument>-t</argument>
					<argument>${project.groupId}/${project.artifactId}:${project.version}</argument>
					<argument>.</argument>
				</arguments>
			</configuration>
		</execution>
	</executions>
</plugin>

It just runs docker build with a Dockerfile in our project. Dockerfile was quite simple:

FROM ibmcom/db2:11.5.4.0
COPY db2-setup-scrips/ /var/custom/
RUN chmod a+x /var/custom/*

After some debugging, we realized that it takes a lot of time for db2 statement create database to create an empty database (noticeable time period, not dependant on the number of tables and data) and some more time to execute all create, alter, update & insert` statements (this time period was growing when we started to add more tables and data to our database).

At some point we realized it would be much faster if we create this Docker image as a 3 step process:

• create a DB2 Docker “initial” image with all database initialization scripts prepared in same way we did initialy with minor changes
  • make this image appropriate to single task
  • add script to create a backup of the whole database instance after it is properly setup
• run this image with temporary container to create a backup in a project’s target directory which is a Docker volume mapped location
• create a DB2 Docker “final” image where all database initialization scripts are replaced by single script which restores database from a backup copied in this image

Our Dockerfile.initial now looks something like this:

FROM ibmcom/db2:11.5.4.0
COPY db2-setup-initial/ /var/custom/
RUN chmod a+x /var/custom/*

# Don't try to execute "ddl" directory in folder with scripts
# Remove last part of script with infinite loop tailing db2diag.log
RUN sed -i -n 's|`ls /var/custom`|`ls -p /var/custom \| grep -Ev "/$"`|; s/while/#while/; 1,/#while/ p' /var/db2_setup/lib/setup_db2_instance.sh

This “hacky” last part of this Dockerfile is required to stop container after database backup is created and to avoid error when trying to run a directory with DDL scripts (/var/custom/ddlin our case) when listing files inside /var/custom directory.

Our Dockerfile.final is more simple:

FROM ibmcom/db2:11.5.4.0
COPY db2-setup-restore/ /var/custom/
RUN chmod a+x /var/custom/*

COPY ./target/db2bak/ /tmp/db2bak/
RUN chmod a+r /tmp/db2bak/*

Our Maven setup became a bit more complex, with multiple steps required to make everything work as expected. The first step is making sure the root user from our privileged docker container (required) don’t break our build by the creation of directory, we won’t be able to delete later. Steps after that are ones describe above:

• create the initial image
• run the initial image to create a backup of our database
• create the final image which recreates the database from backup instead by creating a new instance and populating it by a bunch of scripts

<!--
    Without this "target" directory will be created by
    root user from Docker and mvn clean would fail afterwards.
-->
<plugin>
    <artifactId>maven-antrun-plugin</artifactId>
    <version>1.8</version>
    <executions>
        <execution>
            <phase>initialize</phase>
            <goals>
                <goal>run</goal>
            </goals>
            <configuration>
                <target>
                    <mkdir dir="${project.build.directory}/db2bak" />
                </target>
            </configuration>
        </execution>
    </executions>
</plugin>
<plugin>
	<groupId>org.codehaus.mojo</groupId>
	<artifactId>exec-maven-plugin</artifactId>
	<version>1.6.0</version>
	<executions>
		<execution>
			<id>docker-build-initial</id>
			<phase>install</phase>
			<goals>
				<goal>exec</goal>
			</goals>
			<configuration>
				<executable>docker</executable>
				<workingDirectory>${project.basedir}</workingDirectory>
				<arguments>
					<argument>build</argument>
					<argument>-t</argument>
					<argument>${project.groupId}/${project.artifactId}-initial:${project.version}</argument>
					<argument>-f</argument>
					<argument>Dockerfile.initial</argument>
					<argument>.</argument>
				</arguments>
			</configuration>
		</execution>
		<execution>
			<id>docker-init-db-and-create-backup</id>
			<phase>install</phase>
			<goals>
				<goal>exec</goal>
			</goals>
			<configuration>
				<executable>docker</executable>
				<workingDirectory>${project.basedir}</workingDirectory>
				<arguments>
					<argument>run</argument>
					<argument>--rm</argument>
					<argument>-v</argument>
					<argument>${project.build.directory}/db2bak:/tmp/db2bak</argument>
					<argument>--privileged</argument>
					<argument>${project.groupId}/${project.artifactId}-initial:${project.version}</argument>
				</arguments>
			</configuration>
		</execution>
		<execution>
			<id>docker-build-final</id>
			<phase>install</phase>
			<goals>
				<goal>exec</goal>
			</goals>
			<configuration>
				<executable>docker</executable>
				<workingDirectory>${project.basedir}</workingDirectory>
				<arguments>
					<argument>build</argument>
					<argument>-t</argument>
					<argument>${project.groupId}/${project.artifactId}:${project.version}</argument>
					<argument>-f</argument>
					<argument>Dockerfile.final</argument>
					<argument>.</argument>
				</arguments>
			</configuration>
		</execution>
	</executions>
</plugin>

On my machine this changed time to startup our database container from approx. 80 seconds to approx. 30 seconds.

If you want to speed up your DB2 container startup, even if you don’t have many tables and much data in your database instance, creating the database from a backup instead of creating a new blank db2 instance will make quite a difference. Just replacing create database with restore database cuts time to start a container to almost half.

The cover image is created by Kaboompics .com from Pexels.