Moving a database container with Docker Swarm and Flocker

Please note: because this demo uses Powerstrip, which is only meant for prototyping Docker extensions, we do not recommend this configuration for anything approaching production usage. When Docker extensions become official, Flocker will support them. Until then, this is just a proof-of-concept.


Today we are going to demonstrate migrating a database container and its data across hosts using only the Docker client as the trigger. Update: see Ben Firshman demoing this live on stage at Container Camp!

Imagine we are running a database container with data being saved in a Docker volume and now we want to upgrade the hardware for the host the database container is running on. Clearly, we need a way of moving the container and the data as a single atomic unit.

Docker Swarm is capable of scheduling the container to be run on a particular host (using constraints) – we are going to demonstrate how to combine Flocker with Swarm and migrate the data alongside the container by combining the following tools:

Swarm + Flocker + Weave + Powerstrip + Powerstrip-flocker + Powerstrip-weave

Overview

Here is an overview of the components that are used to make this example work:

overview

Scenario

We have 2 simple services in our stack – a HTTP API and a Database API, also exposed over HTTP. We have used Flocker to handle our volumes and Weave to handle our networking.

We quickly discover that the node with the spinning disk which hosts our database container is too slow. We need an SSD drive and this means migrating our database container (along with its data) to another host.

Ideally – we want to run our database container on a second machine and for the data to just move with the container. Because we are using Powerstrip-flocker together with Swarm – we can stop the container on the first node, start it up on the second node and our data will have followed the database container – using only the Docker client!

Before

Here is a diagram of the setup before we have moved the database container:

Layout before

  • Node 1 (disk)
    • HTTP server
    • Database server (10.255.0.10 + /flocker/data)
  • Node 2 (SSD)

After

Here is a diagram of the setup after we have moved the database container (and its data + IP address):

Layout after

  • Node 1 (disk)
    • HTTP server
  • Node 2 (SSD)
    • Database server (10.255.0.10 + /flocker/data)

Try it yourself! Requirements for the demo

First you need to install: VirtualBox + Vagrant

Start

To run the demo virtual machines:

$ git clone https://github.com/binocarlos/powerstrip-swarm-demo $ cd powerstrip-swarm-demo $ vagrant up

Run

We have included a script that will run through each of the commands in the demo automatically:

$ vagrant ssh master master$ sudo bash /vagrant/run.sh demo

Manual example

If we run each step of the example manually, we can see clearly what is happening on each step.

Step 1

First we SSH onto the master and export DOCKER_HOST:

$ vagrant ssh master
master$ export DOCKER_HOST=localhost:2375

Step 2

Then we start the HTTP server on node1:

master$ docker run -d \
  --name demo-server \
  -e constraint:storage==disk \
  -e WEAVE_CIDR=10.255.0.11/24 \
  -e API_IP=10.255.0.10 \
  -p 8080:80 \
  binocarlos/multi-http-demo-server:latest

Step 3

Then we start the DB server on node1:

master$ docker run -d \
  --hostname disk \
  --name demo-api \
  -e constraint:storage==disk \
  -e WEAVE_CIDR=10.255.0.10/24 \
  -v /flocker/data1:/tmp \
  binocarlos/multi-http-demo-api:latest

Step 4

Then we hit the web service a few times to increase the number:

master$ curl -L http://172.16.255.251:8080
master$ curl -L http://172.16.255.251:8080
master$ curl -L http://172.16.255.251:8080
master$ curl -L http://172.16.255.251:8080
master$ curl -L http://172.16.255.251:8080

Step 5

Then we kill the database container:

master$ docker rm -f demo-api

Step 6

Now we start the DB server on node2:

master$ docker run -d \
  --hostname ssd \
  --name demo-api \
  -e constraint:storage==ssd \
  -e WEAVE_CIDR=10.255.0.10/24 \
  -v /flocker/data1:/tmp \
  binocarlos/multi-http-demo-api:latest

Step 7

Then we hit the web service a few times to ensure that it:

  • has kept the state after having migrated
  • can still communicate with the DB server
master$ curl -L http://172.16.255.251:8080
master$ curl -L http://172.16.255.251:8080
master$ curl -L http://172.16.255.251:8080
master$ curl -L http://172.16.255.251:8080
master$ curl -L http://172.16.255.251:8080

Step 8

Now we close the 2 containers

master$ docker rm -f demo-api
master$ docker rm -f demo-server

Info

You can see the state of the swarm by doing this on the master:

$ vagrant ssh master
master$ DOCKER_HOST=localhost:2375 docker ps -a

This displays the containers used for Powerstrip, Flocker and weave

You can see the state of the weave network by doing this on node1 or node2:

$ vagrant ssh node1
node1$ sudo bash /vagrant/install.sh weave status

Conclusion

We have moved both an IP address and a data volume across hosts using nothing more than the Docker client!

What do you think? Join the discussion over on HackerNews.

Like what you read?

Signup for a free FlockerHub account.
Sign Up

Get all the ClusterHQ News

Stay up to date with our newsletter. No spam. Ever.