Subscribe to our 0800-DEVOPS Newsletter

    Get in touch

    Contact us for all inquiries regarding services and general information






      Get in touch

      Contact us for all inquiries regarding services and general information






        Use the form below to apply for course





          Get in touch

          Contact us for all inquiries regarding services and general information






          Blog

          IBM DB2 Docker container startup speedup

          03.03.2021

          If you develop cloud-native applications using IBM’s DB2 database you will probably want to use DB2 Docker image for your developing, testing and CI/CD needs.

          When working on a set of applications using shared DB2 database hosted at our client’s infrastructure we prepared a customized DB2 Docker image as explained on the IBM’s DB2 Docker Hub page.

          Initial setup

          It boils down to preparing a set of scripts and copying them to /var/custom directory in docker container so they can be used to create & populate DB2 instance after the main DB2 setup has completed. Those scripts are executed on each container start.

          During our development, we noticed that it takes quite a while for this container to start. After we added new tables and new data to this DB2 instance startup time of container started to grow.

          When we prepared automated tests using the Testcontainers project, the startup time of our database container become the slowest point of our setup.

          Initial setup was done using Maven build configuration defined in pom.xml defined as exec-maven-plugin plugin:

          <plugin>
          	<groupId>org.codehaus.mojo</groupId>
          	<artifactId>exec-maven-plugin</artifactId>
          	<version>1.6.0</version>
          	<executions>
          		<execution>
          			<id>docker-build</id>
          			<phase>install</phase>
          			<goals>
          				<goal>exec</goal>
          			</goals>
          			<configuration>
          				<executable>docker</executable>
          				<workingDirectory>${project.basedir}</workingDirectory>
          				<arguments>
          					<argument>build</argument>
          					<argument>-t</argument>
          					<argument>${project.groupId}/${project.artifactId}:${project.version}</argument>
          					<argument>.</argument>
          				</arguments>
          			</configuration>
          		</execution>
          	</executions>
          </plugin>
          

           

          It just runs docker build with a Dockerfile in our project. Dockerfile was quite simple:

          FROM ibmcom/db2:11.5.4.0
          COPY db2-setup-scrips/ /var/custom/
          RUN chmod a+x /var/custom/*
          

           

          Optimized setup

          After some debugging, we realized that it takes a lot of time for db2 statement create database to create an empty database (noticeable time period, not dependant on the number of tables and data) and some more time to execute all create, alter, update & insert` statements (this time period was growing when we started to add more tables and data to our database).

          At some point we realized it would be much faster if we create this Docker image as a 3 step process:

          • create a DB2 Docker “initial” image with all database initialization scripts prepared in same way we did initialy with minor changes
            • make this image appropriate to single task
            • add script to create a backup of the whole database instance after it is properly setup
          • run this image with temporary container to create a backup in a project’s target directory which is a Docker volume mapped location
          • create a DB2 Docker “final” image where all database initialization scripts are replaced by single script which restores database from a backup copied in this image

          Our Dockerfile.initial now looks something like this:

          
          FROM ibmcom/db2:11.5.4.0
          COPY db2-setup-initial/ /var/custom/
          RUN chmod a+x /var/custom/*
          
          # Don't try to execute "ddl" directory in folder with scripts
          # Remove last part of script with infinite loop tailing db2diag.log
          RUN sed -i -n 's|`ls /var/custom`|`ls -p /var/custom \| grep -Ev "/$"`|; s/while/#while/; 1,/#while/ p' /var/db2_setup/lib/setup_db2_instance.sh
          

           

          This “hacky” last part of this Dockerfile is required to stop container after database backup is created and to avoid error when trying to run a directory with DDL scripts (/var/custom/ddlin our case) when listing files inside /var/custom directory.

          Our Dockerfile.final is more simple:

          
          FROM ibmcom/db2:11.5.4.0
          COPY db2-setup-restore/ /var/custom/
          RUN chmod a+x /var/custom/*
          
          COPY ./target/db2bak/ /tmp/db2bak/
          RUN chmod a+r /tmp/db2bak/*
          

           

          Our Maven setup became a bit more complex, with multiple steps required to make everything work as expected. The first step is making sure the root user from our privileged docker container (required) don’t break our build by the creation of directory, we won’t be able to delete later. Steps after that are ones describe above:

          • create the initial image
          • run the initial image to create a backup of our database
          • create the final image which recreates the database from backup instead by creating a new instance and populating it by a bunch of scripts
          <!--
              Without this "target" directory will be created by
              root user from Docker and mvn clean would fail afterwards.
          -->
          <plugin>
              <artifactId>maven-antrun-plugin</artifactId>
              <version>1.8</version>
              <executions>
                  <execution>
                      <phase>initialize</phase>
                      <goals>
                          <goal>run</goal>
                      </goals>
                      <configuration>
                          <target>
                              <mkdir dir="${project.build.directory}/db2bak" />
                          </target>
                      </configuration>
                  </execution>
              </executions>
          </plugin>
          <plugin>
          	<groupId>org.codehaus.mojo</groupId>
          	<artifactId>exec-maven-plugin</artifactId>
          	<version>1.6.0</version>
          	<executions>
          		<execution>
          			<id>docker-build-initial</id>
          			<phase>install</phase>
          			<goals>
          				<goal>exec</goal>
          			</goals>
          			<configuration>
          				<executable>docker</executable>
          				<workingDirectory>${project.basedir}</workingDirectory>
          				<arguments>
          					<argument>build</argument>
          					<argument>-t</argument>
          					<argument>${project.groupId}/${project.artifactId}-initial:${project.version}</argument>
          					<argument>-f</argument>
          					<argument>Dockerfile.initial</argument>
          					<argument>.</argument>
          				</arguments>
          			</configuration>
          		</execution>
          		<execution>
          			<id>docker-init-db-and-create-backup</id>
          			<phase>install</phase>
          			<goals>
          				<goal>exec</goal>
          			</goals>
          			<configuration>
          				<executable>docker</executable>
          				<workingDirectory>${project.basedir}</workingDirectory>
          				<arguments>
          					<argument>run</argument>
          					<argument>--rm</argument>
          					<argument>-v</argument>
          					<argument>${project.build.directory}/db2bak:/tmp/db2bak</argument>
          					<argument>--privileged</argument>
          					<argument>${project.groupId}/${project.artifactId}-initial:${project.version}</argument>
          				</arguments>
          			</configuration>
          		</execution>
          		<execution>
          			<id>docker-build-final</id>
          			<phase>install</phase>
          			<goals>
          				<goal>exec</goal>
          			</goals>
          			<configuration>
          				<executable>docker</executable>
          				<workingDirectory>${project.basedir}</workingDirectory>
          				<arguments>
          					<argument>build</argument>
          					<argument>-t</argument>
          					<argument>${project.groupId}/${project.artifactId}:${project.version}</argument>
          					<argument>-f</argument>
          					<argument>Dockerfile.final</argument>
          					<argument>.</argument>
          				</arguments>
          			</configuration>
          		</execution>
          	</executions>
          </plugin>
          

           

          Conclusion

          On my machine this changed time to startup our database container from approx. 80 seconds to approx. 30 seconds.

          If you want to speed up your DB2 container startup, even if you don’t have many tables and much data in your database instance, creating the database from a backup instead of creating a new blank db2 instance will make quite a difference. Just replacing create database with restore database cuts time to start a container to almost half.

          The cover image is created by Kaboompics .com from Pexels.

          Vedran Vidović is a proactive and experienced software developer. Since 2001st in the IT industry, he was working in development, analysis, design, and architecture, applying different technologies on many projects from various business domains. He has broad experience in J2EE technologies on projects of various complexity. Since 2009th he gathered extensive experience in the enterprise integration projects, using mostly IBM DataPower Gateway and IBM API Connect.

          Using Datefudge to fake Docker Date & Time for testing

          icon

          How to force our service to behave as it was called on any specific date and time – not only the current time of test execution?

          Read the article

          CONTACT

          Get in touch

          Contact us