Docker extensions: the new Docker plugins model

meeting at dockercon amsterdam

On Friday at DockerCon in Amsterdam I had the privilege to spend nearly four hours in discussions with Solomon Hykes and a small, focused group of other Docker ecosystem partners about Docker extensions (a.k.a. Docker plugins) in general and networking extensions in particular.

Along with Solomon, the folks in the room included Weave (Alexis Richardson and Matthias Radestock), Rancher (Darren Shepherd), Tutum (Fernando Mayo), ClusterHQ (Itamar Turner-Trauring and myself), Adrian Colyer, plus folks from VMware and SocketPlane.

The problem which Docker currently faces is that by moving to become a platform it is being seen to threaten its own ecosystem. The proposed solution is that Docker ships its own additions to Docker as late-bound, composable, optional extensions and enables other vendors to do likewise. Docker calls this “batteries included but removable”. This was my conclusion from the meeting.

I came away feeling optimistic about Docker extensions, and by extension (no pun intended), a future of healthy innovation in the Docker ecosystem. Here’s why.

The changing nature of Docker

Since Docker’s recent $40m fundraising, there has been a change of tone from describing Docker as a component to describing Docker as a platform for building applications. This has understandably caused concern in the Docker ecosystem, especially amongst folks who have started building businesses filling the gaps around Docker. Quite reasonably, the ecosystem’s fear is: what if Docker tries to do everything, and in doing so, cannibalizes its own ecosystem?

More specifically, the question that ecosystem vendors have been asking is:

How can we innovate in the Docker ecosystem, with a focus on providing specialized services for things like networking (e.g. Weave), orchestration (e.g. Cloudsoft), or persistent storage (e.g. ClusterHQ), without wrapping Docker?

By “wrapping Docker” I mean “building tools on top of Docker with tooling which competes with other vendors for the user interface”. Avoiding this is good:

  • for Docker, who want to continue to own the developer experience — and provide a portable developer interface,
  • for users, because it means they’ll be able to compose multiple solutions, i.e. use more than one extension at a time,
  • for the ecosystem vendors, because they won’t need to compete with each other about whose tool the user interacts with.

But because Docker has no viable extensions system yet, everyone is being forced to wrap the Docker API, and the vendors feel more threatened by Docker.

This state of affairs is nicely summed up from a user’s perspective by Adrian Colyer:

A thriving ecosystem is better for everyone, including Docker

So Docker have recently been faced with a dilemma: limit themselves to just providing a container runtime, and allow the ecosystem to monetize around them, or build their own versions of the pieces which can be monetized, and risk alienating the community which has got them to where they are today.

This is a tricky situation for a company which has just raised money at a $400m valuation and has the world’s attention. A thriving ecosystem is essential to win trust from users, but without building out multi-container, multi-host solutions themselves, Docker’s chance to monetize would be limited.

Docker’s great compromise — as proposed to the ecosystem at DockerCon last week — is the mantra: “batteries included but removable”.

In other words, Docker want us to believe that while they’ll build tools (like Swarm for clustering, or a default networking implementation) it will be possible for users to swap out these tools and replace them with more sophisticated alternatives from a hopefully thriving community of “battery vendors”.

How Docker extensions will work

Given our meeting on Friday, I am cautiously optimistic that Docker will ship an extensions system which will really be useful to vendors, and which will allow us all to stop wrapping the Docker API. We agreed that the criteria for such an extensions system is as follows:

  • Late-bound. Must allow extensions to be loaded after Docker is compiled (we want to be able to plug into a Docker which has been shipped as part of, let’s say, Red Hat Enterprise Linux). Given that Go has no dynamically loadable libraries, this means extensions must be out-of-process.
  • Composable. Must allow multiple unrelated extensions to be loaded at the same time (so that a networking extension vendor can coexist peacefully with a storage extension vendor).
  • Optional. It must be possible to start Docker without using one of the built-in default extensions, and use some other vendor’s instead.

Give these attributes, my challenge to Docker is this: implement your next product launch as an optional late-bound Docker extension — one which extends Docker without wrapping it. In other words, prove that you mean “batteries included but removable” by using your own public extension points.

This is a timely opportunity for Docker to continue enabling people to agree on a portable Docker developer experience, while retaining the goodwill of the ecosystem and enabling choice for implementors (“ops people”) of Docker deployments.

If this happens, there will be a bright future for the Docker community, and I remain excited to be a part of it.

What do you think of the proposed model for Docker extensions? Join the discussion over on Hacker News.

ClusterHQ is the team behind the innovative open-source project for running databases in containers, Flocker. We are venture backed, growing fast, and on a mission to be The Container Data People. Learn more or get involved at GitHub.

About Flocker

Flocker is an open-source container data volume manager for your Dockerized application. It gives ops teams the tools they need to run containerized stateful services like databases in production.

Like what you read?

Signup for a free VolumeHub account.
Sign Up

Get all the ClusterHQ News

Stay up to date with our newsletter. No spam. Ever.