The new Docker 1.12 release has integrated Swarm into the engine and comes with a bunch of new features which completely change the way containers are orchestrated. You can check more details here although these are some of the most interesting new features:
- Replication and load balancing
- Fault tolerance
- Service discovery
- Integrated KV-store for Swarm
In this hands on post, we will see these new features in action and compare them with equivalent Kubernetes ones. Kubernetes is an alternative container orchestration engine developed by Google. It is quite powerful, it has been providing many of the new Swarm features for some time now and it is more oriented to the cloud. However it turns out to be a bit complex to setup. This post only covers a local setup, a cloud deployment will be covered in an upcoming post.
In order to compare these technologies we will orchestrate dockercoins, a multi-container application developed by Docker engineer Jérôme Petazzoni which is composed of 5 different microservices. All code and commands are in this github repo although you can just copy-paste the following code snippets.
Requirements
We will create two different clusters, one for Swarm and another for Kubernetes. It is recommended to work with only one tool at a time because both setups create up to four virtual machines which are quite resource consuming. In Windows many of the following requirements are provided by docker-toolbox.
- Both: virtualbox, docker 1.12, docker-compose 1.8+, git and bash
- Swarm: docker-machine 0.8+
- Kubernetes: vagrant
Cluster setup
We’ll start by creating a cluster with several machines. For Swarm we will use docker-machine 0.8.1 to setup up 3 machines with Docker installed while we rely on a pre-configured coreos-kubernetes cluster for kubernertes example. Kubernetes cluster is composed of 4 machines, one for etcd, another is the controller and the last two are workers. Kubernetes is configured with flannel multihost networking driver and etcd KV store.
Cluster state and connection
Here we connect to the cluster and check its state. We also configure the environment for the following commands. In Swarm everything is done by docker-machine so we only need to connect the client to Swarm master node. In Kubernetes, connection configuration is created by vagrant so we only need to specify Kubernetes client to use that configuration. Kubernetes cluster may take a while to setup so wait till kubectl reports three available nodes (c1, w1 and w2).
Registry deployment
It is common in any development or testing environment to setup a local private registry. In order to be able to use it with Docker we can either secure it with TLS or access it through localhost interface. In this example we will use localhost interface so we will deploy a registry in the cluster and configure it for listening in every node in the same port.
Build dockercoins
Building an application is done directly with Docker commands. In Kubernetes cluster, Docker daemon is not exposed so we build the application locally and redirect registry connections to Kubernetes registry.
Swarm vs kunernetes abstractions
Docker Swarm orchestrates containers around “services” which function like a proxy for containers behind it providing load balancing and service discovery through the service name. Further they also replicate and keep the desired number of container replicas in order to provide fault tolerance. Thus in order to create a Swarm service we can specify the name and the ports where this service should be reached, the container image and the number of desired number of replicas.
docker service create --name redis -p 6379 --image redis --replicas 2
In Kubernetes these duties are separated in two high level entities: services and deployments. A service provides service discovery and load balancing in a similar way than in Docker however replication control is done through deployments. These entities don’t control directly Docker containers but another Kubernetes abstraction called “pod” which represents a set of containers which share the same network stack. However for the purpose of this post, you can treat a pod as a container.
# Create a deployment of redis with two replicas
kubectl run redis --image redis --replicas 2
# Create a service called redis from redis deployment
kubectl expose deployment redis --port 6379 --name=redis
Deploy dockercoins
In Swarm we create a service per microservice while in Kubernetes we create a deployment per microservice and a service for those which are accessed by the other ones. Here services are automatically created with the --expose option in kubectl run.
Check application and containers
The following commands allow to check the application state. In Kubernetes this should be done through its abstractions.
Scale
It is really easy to scale a service in both technologies. New replicas are automatically added to the service which load balances traffic to them. In Docker Swarm this is done with a native Linux IPVS load balancer while in Kubernetes a proxy container performs this duty.
Update
Here we create dockercoins_worker version 2 and update worker container image. Both technologies gracefully replace old replicas for new ones always keeping some of them running in order to avoid downtime.
Fault tolerance
Kill a non master node (it is not replicated here) and check how containers are rescheduled. In this example, Kubernetes is configured to set a container as “failed” after it has failed for 30 seconds so we should wait at least 30 seconds to see containers being re-scheduled.
Clean and destroy cluster
Conclusion
This straightforward example showed Swarm is really simple whereas Kubernetes has more abstractions and relies in many independent services (we automatically setup Kubernetes with vagrant but it configures more than 5 different services). This is the main reason why Swarm is loved by developers. Previous Swarm versions were very simple and worked out of the box with all Docker client commands but lacked many features like service discovery, replication management or load balancing which made Kubernetes a better choice for cluster orchestration. With Docker 1.12 release these gaps are narrowed down and it is closer to Kubernetes in orchestration features. Whether to use Swarm or Kubernetes still depends on many factors but it is clear that Swarm is a better choice for moving from a local/development environment to a cluster. Any other scenario like a production or a cloud one requires many other features which are not covered in this initial post but they will do it in a future one.