Creating a Kubernetes cluster is not trivial. We have to make many choices, and we can easily get lost in the myriad of options. The number of permutations is getting close to infinite and, yet, our clusters need to be configured consistently. Experience from the first attempt to set up a cluster can easily convert into a nightmare that will haunt you for the rest of your life.
Unlike Docker Swarm that packs almost everything into a single binary, Kubernetes clusters require quite a few separate components running across the nodes. Setting them up can be very easy, or it can become a challenge. It all depends on the choices we make initially. One of the first things we need to do is choose a tool that we'll use to create a Kubernetes cluster.
If we'd decide to install a Docker Swarm cluster, all we'd need to do is to install Docker engine on all the servers, and execute docker swarm init or docker swarm join command on each of the nodes. That's it. Docker packs everything into a single binary. Docker Swarm setup process is as simple as it can get. The same cannot be said for Kubernetes. Unlike Swarm that is highly opinionated, Kubernetes provides much higher freedom of choice. It is designed around extensibility. We need to choose among many different components. Some of them are maintained by the core Kubernetes project, while others are provided by third-parties. Extensibility is probably one of the main reasons behind Kubernetes' rapid growth. Almost every software vendor today is either building components for Kubernetes or providing a service that sits on top of it.
Besides the intelligent design and the fact that it solves problems related to distributed, scalable, fault-tolerant, and highly available systems, Kubernetes' power comes from adoption and support from a myriad of individuals and companies. You can use that power, as long as you understand that it comes with responsibilities. It's up to you, dear reader, to choose how will your Kubernetes cluster look like, and which components it'll host. You can decide to build it from scratch, or you can use one of the hosted solutions like Google Cloud Platform (GCE) Kubernetes Engine. There is a third option though. We can choose to use one of the installation tools. Most of them are highly opinionated with a limited amount of arguments we can use to tweak the outcome.
You might be thinking that creating a cluster from scratch using kubeadm cannot be that hard. You'd be right if running Kubernetes is all we need. But, it isn't. We need to make it fault tolerant and highly available. It needs to stand the test of time. Constructing a robust solution would require a combination of Kubernetes core and third-party components, AWS know-how, and quite a lot of custom scripts that would tie the two together. We won't go down that road. At least, not now.
We'll use Kubernetes Operations (kops) to create a cluster. It is somewhere in the middle between do-it-yourself-from-scratch and hosted solutions (for example, GCE). It's an excellent fit for both newbies and veterans. You'll learn which components are required for running a Kubernetes cluster. You'll be able to make some choices. And, yet, we won't go down the rabbit hole of setting up the cluster from scratch. Believe me, that hole is very deep, and it might take us a very long time to get out of it.
Typically, this would be a great place to explain the most significant components of a Kubernetes cluster. Heck, you were probably wondering why we didn't do that early on when we began the journey. Still, we'll postpone the discussion for a while longer. I believe it'll be better to create a cluster first and discuss the components through live examples. I feel that it's easier to understand something we can see and touch, instead of keeping it purely on the theoretical level.
All in all, we'll create a cluster first, and discuss its components later.
Since I already mentioned that we'll use kops to create a cluster, we'll start with a very brief introduction to the project behind it.
If you visit Kubernetes Operations (kops) (https://github.com/kubernetes/kops) project, the first sentence you'll read is that it is "the easiest way to get a production-grade Kubernetes cluster up and running." In my humble opinion, that sentence is accurate only if we exclude Google Kubernetes Engine (GKE). Today (February 2018), other hosting vendors did not yet release their Kubernetes-as-a-service solutions. Amazon's Elastic Container Service for Kubernetes (EKS) (https://aws.amazon.com/eks/) is still not open to the public. Azure Container Service (AKS) (https://azure.microsoft.com/en-us/services/kubernetes-service/) is also a new addition that still has a few pain points. By the time you read this, all major hosting providers might have their solutions. Still, I prefer kops since it provides almost the same level of simplicity without taking away the control of the process. It allows us to tweak the cluster more than we would be permitted with hosted solutions. It is entirely open source, it can be stored in version control, and it is not designed to lock you into a vendor.
If your hosting vendor is AWS, kops is, in my opinion, the best way to create a Kubernetes cluster. Whether that's true for GCE, is open for debate since GKE works great. We can expect kops to be extended in the future to other vendors. For example, at the time of this writing, VMWare is in alpha and should be stable soon. Azure and Digital Ocean support are being added as I write this.
We'll use kops to create a Kubernetes cluster in AWS. This is the part of the story that might get you disappointed. You might have chosen to run Kubernetes somewhere else. Don't be depressed. Almost all Kubernetes clusters follow the same principles even though the method of setting them up might differ. The principles are what truly matters, and I'm confident that, once you set it up successfully on AWS, you'll be able to transfer that knowledge anywhere else.
The reason for choosing AWS lies in its adoption. It is the hosting vendor with, by far, the biggest user-base. If I'd have to place a blind bet on your choice, it would be AWS solely because that is statistically the most likely choice. I could not explore all the options in a single chapter. If I am to go through all hosting vendors and different projects that might help with the installation, we'd need to dedicate a whole book to that. Instead, I invite you to explore the subject further once you're finished with installing Kubernetes in AWS with kops. As an alternative, ping me on slack.devops20toolkit.com or send me an email to
[email protected] and I'll give you a hand. If I receive enough messages, I might even dedicate a whole book to Kubernetes installations.
I went astray from kops...
Kops lets us create a production-grade Kubernetes cluster. That means that we can use it not only to create a cluster, but also to upgrade it (without downtime), update it, or destroy it if we don't need it anymore. A cluster cannot be called "production grade" unless it is highly available and fault tolerant. We should be able to execute it entirely from the command line if we'd like it to be automated. Those and quite a few other things are what kops provides, and what makes it great.
Kops follows the same philosophy as Kubernetes. We create a set of JSON or YAML objects which are sent to controllers that create a cluster.
We'll discuss what kops can and cannot do in more detail soon. For now, we'll jump into the hands-on part of this chapter and ensure that all the prerequisites for the installation are set.
We'll continue using the specifications from the vfarcic/k8s-specs repository, so the first thing we'll do is to go inside the directory where we cloned it, and pull the latest version.
All the commands from this chapter are available in the 14-aws.sh (https://gist.github.com/vfarcic/04af9efcd1c972e8199fc014b030b134) Gist.
cd k8s-specs git pull
I will assume that you already have an AWS account. If that's not the case, please head over to Amazon Web Services (https://aws.amazon.com/) and sign-up.
If you are already proficient with AWS, you might want to skim through the tex...