Mastering KVM Virtualization

Vedran Dakic, Humble Devassy Chirammal, Prasad Mukhedkar, Anil Vettathu

• Publisher: Packt Publishing
• Year: 2020
• ISBN:
  9781838822972
• Edition: 2
• Topic: Computer Science
• Subtopic: Systems Architecture
• Index: Computer Science

Learn how to configure, automate, orchestrate, troubleshoot, and monitor KVM-based environments capable of scaling to private and hybrid cloud modelsKey Features• Gain expert insights into Linux virtualization and the KVM ecosystem with this comprehensive guide• Learn to use various Linux tools such as QEMU, oVirt, libvirt, Cloud-Init, and Cloudbase-Init• Scale, monitor, and troubleshoot your VMs on various platforms, including OpenStack and AWSBook DescriptionKernel-based Virtual Machine (KVM) enables you to virtualize your data center by transforming your Linux operating system into a powerful hypervisor that allows you to manage multiple operating systems with minimal fuss. With this book, you'll gain insights into configuring, troubleshooting, and fixing bugs in KVM virtualization and related software.This second edition of Mastering KVM Virtualization is updated to cover the latest developments in the core KVM components - libvirt and QEMU. Starting with the basics of Linux virtualization, you'll explore VM lifecycle management and migration techniques. You'll then learn how to use SPICE and VNC protocols while creating VMs and discover best practices for using snapshots. As you progress, you'll integrate third-party tools with Ansible for automation and orchestration. You'll also learn to scale out and monitor your environments, and will cover oVirt, OpenStack, Eucalyptus, AWS, and ELK stack. Throughout the book, you'll find out more about tools such as Cloud-Init and Cloudbase-Init. Finally, you'll be taken through the performance tuning and troubleshooting guidelines for KVM-based virtual machines and a hypervisor.By the end of this book, you'll be well-versed with KVM virtualization and the tools and technologies needed to build and manage diverse virtualization environments.What you will learn• Implement KVM virtualization using libvirt and oVirt• Delve into KVM storage and network• Understand snapshots, templates, and live migration features• Get to grips with managing, scaling, and optimizing the KVM ecosystem• Discover how to tune and optimize KVM virtualization hosts• Adopt best practices for KVM platform troubleshootingWho this book is forIf you are a systems administrator, DevOps practitioner, or developer with Linux experience looking to sharpen your open-source virtualization skills, this virtualization book is for you. Prior understanding of the Linux command line and virtualization is required before getting started with this book.

Section 1: KVM Virtualization Basics

Part 1 provides you with an insight into the prevailing technologies in Linux virtualization and its advantages over other virtualization solutions. We will discuss the important data structures and the internal implementation of libvirt, QEMU, and KVM.

This part of the book comprises the following chapters:

• Chapter 1, Understanding Linux Virtualization
• Chapter 2, KVM as a Virtualization Solution

Chapter 1: Understanding Linux Virtualization

Virtualization is the technology that started a big technology shift toward IT consolidation, which provides more efficient use of resources and the cloud as a more integrated, automated, and orchestrated version of virtualization with a focus on not only virtual machines but also additional services. There are a total of 16 chapters in this book, all of which have been lined up to cover all the important aspects of Kernel-based Virtual Machine (KVM) virtualization. We will start with basic KVM topics such as the history of virtualization concepts and Linux virtualization and then move on and look at advanced topics in KVM such as automation, orchestration, virtual networking, storage, and troubleshooting. This chapter will provide you with an insight into the prevailing technologies in Linux virtualization and their advantages over others.

In this chapter, we will cover the following topics:

• Linux virtualization and its basic concepts
• Types of virtualization
• Hypervisor/VMM
• Open source virtualization projects
• What Linux virtualization offers you in the cloud

Linux virtualization and how it all started

Virtualization is a concept that creates virtualized resources and maps them to physical resources. This process can be done using specific hardware functionality (partitioning, via some kind of partition controller) or software functionality (hypervisor). So, as an example, if you have a physical PC-based server with 16 cores running a hypervisor, you can easily create one or more virtual machines with two cores each and start them up. Limits regarding how many virtual machines you can start is something that's vendor-based. For example, if you're running Red Hat Enterprise Virtualization v4.x (a KVM-based bare-metal hypervisor), you can use up to 768 logical CPU cores or threads (you can read more information about this at https://access.redhat.com/articles/906543). In any case, hypervisor is going to be the go-to guy that's going to try to manage that as efficiently as possible so that all of the virtual machine workloads get as much time on the CPU as possible.

I vividly remember writing my first article about virtualization in 2004. AMD just came out with its first consumer 64-bit CPUs in 2003 (Athlon 64, Opteron) and it just threw me for a loop a bit. Intel was still a bit hesitant to introduce a 64-bit CPU – a lack of a 64-bit Microsoft Windows OS might have had something to do with that as well. Linux was already out with 64-bit support, but it was a dawn of many new things to come to the PC-based market. Virtualization as such wasn't something revolutionary as an idea since other companies already had non-x86 products that could do virtualization for decades (for example, IBM CP-40 and its S/360-40, from 1967). But it sure was a new idea for a PC market, which was in a weird phase with many things happening at the same time. Switching to 64-bit CPUs with multi-core CPUs appearing on the market, then switching from DDR1 to DDR2, and then from PCI/ISA/AGP to PCI Express, as you might imagine, was a challenging time.

Specifically, I remember thinking about the possibilities – how cool it would be to run an OS, and then another couple of OSes on top of that. Working in the publishing industry, you might imagine how many advantages that would offer to anyone's workflow, and I remember really getting excited about it.

15 or so years of development later, we now have a competitive market in terms of virtualization solutions – Red Hat with KVM, Microsoft with Hyper-V, VMware with ESXi, Oracle with Oracle VM, and Google and other key players duking it out for users and market dominance. This led to the development of various cloud solutions such as EC2, AWS, Office 365, Azure, vCloud Director, and vRealize Automation for various types of cloud services. All in all, it was a very productive 15 years for IT, wouldn't you say?

But, going back to October 2003, with all of the changes that were happening in the IT industry, there was one that was really important for this book and virtualization for Linux in general: the introduction of the first open source Hypervisor for x86 architecture, called Xen. It supports various CPU architectures (Itanium, x86, x86_64, and ARM), and it can run various OSes – Windows, Linux, Solaris, and some flavors of BSD – and it's still alive and kicking as a virtualization solution of choice for some vendors, such as Citrix (XenServer) and Oracle (Oracle VM). We'll get into more technical details about Xen a little bit later in this chapter.

The biggest corporate player in the open source market, Red Hat, included Xen virtualization in initial releases of its Red Hat Enterprise Linux 5, which was released in 2007. But Xen and Red Hat weren't exactly a match made in heaven and although Red Hat shipped Xen with its Red Hat Enterprise Linux 5 distribution, Red Hat switched to KVM in Red Hat Enterprise Linux 6 in 2010, which was – at the time – a very risky move. Actually, the whole process of migrating from Xen to KVM began in the previous version, with 5.3/5.4 releases, both of which came out in 2009. To put things into context, KVM was a pretty young project back then, just a couple of years old. But there were more than a few valid reasons why that happened, varying from Xen is not in the mainline kernel, KVM is, to political reasons (Red Hat wanted more influence over Xen development, and that influence was fading with time).

Technically speaking, KVM uses a different, modular approach that transforms Linux kernels into fully functional hypervisors for supported CPU architectures. When we say supported CPU architectures, we're talking about the basic requirement for KVM virtualization – CPUs need to support hardware virtualization extensions, known as AMD-V or Intel VT. To make things a bit easier, let's just say that you're really going to have to try very hard to find a modern CPU that doesn't support these extensions. For example, if you're using an Intel CPU on your server or desktop PC, the first CPUs that supported hardware virtualization extensions date all the way back to 2006 (Xeon LV) and 2008 (Core i7 920). Again, we'll get into more technical details about KVM and provide a comparison between KVM and Xen a little bit later in this chapter and in the next.

Types of virtualization

There are various types of virtualization solutions, all of which are aimed at different use cases and are dependent on the fact that we're virtualizing a different piece of the hardware or software stack, that is, what you're virtualizing. It's also worth noting that there are different types of virtualization in terms of how you're virtualizing – by partitioning, full virtualization, paravirtualization, hybrid virtualization, or container-based virtualization.

So, let's first cover the five different types of virtualization in today's IT based on what you're virtualizing:

• Desktop virtualization (Virtual Desktop Infrastructuree (VDI)): This is used by a lot of enterprise companies and offers huge advantages for a lot of scenarios because of the fact that users aren't dependent on a specific device that they're using to access their desktop system. They can connect from a mobile phone, tablet, or a computer, and they can usually connect to their virtualized desktop from anywhere as if they're sitting at their workplace and using a hardware computer. Benefits include easier, centralized management and monitoring, much more simplified update workflows (you can update the base image for hundreds of virtual machines in a VDI solution and re-link that to hundreds of virtual machines during maintenance hours), simplified deployment processes (no more physical installations on desktops, workstations, or laptops, as well as the possibility of centralized application management), and easier management of compliance and security-related options.
• Server virtualization: This is used by a vast majority of IT companies today. It offers good consolidation of server virtual machines versus physical servers, while offering many other operational advantages over regular, physical ...

Table of contents

• Mastering KVM Virtualization Second Edition
• Why subscribe?
• Preface
• Section 1: KVM Virtualization Basics
• Chapter 1: Understanding Linux Virtualization
• Chapter 2: KVM as a Virtualization Solution
• Section 2: libvirt and ovirt for Virtual Machine Management
• Chapter 3: Installing KVM Hypervisor, libvirt, and oVirt
• Chapter 4: Libvirt Networking
• Chapter 5: Libvirt Storage
• Chapter 6: Virtual Display Devices and Protocols
• Chapter 7: Virtual Machines: Installation, Configuration, and Life Cycle Management
• Chapter 8: Creating and Modifying VM Disks, Templates, and Snapshots
• Section 3: Automation, Customization, and Orchestration for KVM VMs
• Chapter 9: Customizing a Virtual Machine with cloud-init
• Chapter 10: Automated Windows Guest Deployment and Customization
• Chapter 11: Ansible and Scripting for Orchestration and Automation
• Section 4: Scalability, Monitoring, Performance Tuning, and Troubleshooting
• Chapter 12: Scaling Out KVM with OpenStack
• Chapter 13: Scaling out KVM with AWS
• Chapter 14: Monitoring the KVM Virtualization Platform
• Chapter 15: Performance Tuning and Optimization for KVM VMs
• Chapter 16: Troubleshooting Guidelines for the KVM Platform
• Other Books You May Enjoy