Now that Docker plugins and the Flocker volume driver are a reality, we want to share a guide on how to use Mesosphere’s Marathon framework and the Flocker plugin for Docker together to migrate a MySQL volume from a spinning disk constrained node to one with faster solid state drives.
Performance: your product is sitting at #1 on the front page of hackernews AND reddit and your customer facing features are suffering due: to the low IOPS, (Input/Output Operations Per Second) performance of your Marathon-managed MySQL volume. Another few hours of this and your site is kaputt.
This is your moment to shine! When Marathon reschedules the container to another machine, the Flocker plugin for Docker will automatically orchestrate the migration of the data volume alongside it. In our example we will be migrating a MySQL volume from a spinning disk-constrained machine to one with solid state drives for faster database performance.
A brief overview of the architecture
The master node is running the following components:
- Mesos-master: the master node for the mesos cluster
- Marathon: a mesos framework that runs long running processes
- Zookeeper: a distributed key/value store
- Flocker-control-service: the control service for the Flocker cluster.
The slave nodes run:
- Mesos-slave: the slave process that communicates with the mesos-master
- Docker engine w/ Flocker volume plugin - This plugin will communicate to the Flocker control service the current application state and the changes to the current state of the container.
- Flocker-agent: the flocker slave process that communicates with the flocker-control-service
Lets get started.
1. Clone the demo repo and stand up Vagrant
node1 is setup with spinning disk attribute and node2 with ssd disk constraint. Then navigate to http://172.16.79.250:8080/ - you will see an empty Marathon interface with no application group deployed. Let’s change that.
2. Deploy an application group via Marathon API
To deploy the Marathon application group, we’ll use the built-in Rest API and pass in our application group manifest located at app/application-group-spinning-disk.json
Take note of the following application parameters in the configuration:
- the docker
volume-driveris set to
- path our volume is mounted at
- a Marathon cluster disk constraint of
spinningdisks. We will be migrating our data from the old volume on node1 onto for performant ssd drives on node2.
The volume being established in this instance is a copy-on-write ZFS volume that Flocker will eventually migrate in the next few steps.
Use the following command to send a POST to the
POST /V2/groups endpoint with our configuration “application-group-spinning-disk.json”
Then navigate to http://172.16.79.250:8090/ and you will see the familiar PHPMyAdmin Interface.
Login to the MySQL admin panel with a default
let’s proceed to add some databases, tables, and insert some data we don’t want to lose in the migration.
3. Migrate the MySQL data volume to SSD drives
First we will remove the application from the node with the spinning disk.
Then redeploy the application group with a new application constraint from
["disk", "CLUSTER", "spinning"] to
["disk", "CLUSTER", "ssd"].
This does not destroy the mysql volume (
mesosdemo:/var/lib/mysql) that we have established.
To do this, we will pass in a different application group manifest located at app/application-group-ssd-disk.json.
Now navigate to our new node at http://172.16.79.251:8090/ in a browser, login once more, and your data is there.
Flocker treats the container AND data volume as a single, atomic unit and will move both together. Voila!
Your data volume will always be there following your application in tandem with changes in your Marathon configuration.
Fork this repo: https://github.com/ClusterHQ/marathon-flocker-plugin-demo
We’d love to hear your feedback on these or any features or ideas have might have.