Infrastructure Monitoring with Amazon CloudWatch

Ewere Diagboya

• Publisher: Packt Publishing
• Year: 2021
• ISBN:
  9781800564350
• Edition: 1
• Topic: Computer Science
• Subtopic: Systems Architecture
• Index: Computer Science

Explore real-world examples of issues with systems and find ways to resolve them using Amazon CloudWatch as a monitoring serviceKey Features• Become well-versed with monitoring fundamentals such as understanding the building blocks and architecture of networking• Learn how to ensure your applications never face downtime• Get hands-on with observing serverless applications and servicesBook DescriptionCloudWatch is Amazon's monitoring and observability service, designed to help those in the IT industry who are interested in optimizing resource utilization, visualizing operational health, and eventually increasing infrastructure performance. This book helps IT administrators, DevOps engineers, network engineers, and solutions architects to make optimum use of this cloud service for effective infrastructure productivity. You'll start with a brief introduction to monitoring and Amazon CloudWatch and its core functionalities. Next, you'll get to grips with CloudWatch features and their usability. Once the book has helped you develop your foundational knowledge of CloudWatch, you'll be able to build your practical skills in monitoring and alerting various Amazon Web Services, such as EC2, EBS, RDS, ECS, EKS, DynamoDB, AWS Lambda, and ELB, with the help of real-world use cases. As you progress, you'll also learn how to use CloudWatch to detect anomalous behavior, set alarms, visualize logs and metrics, define automated actions, and rapidly troubleshoot issues. Finally, the book will take you through monitoring AWS billing and costs. By the end of this book, you'll be capable of making decisions that enhance your infrastructure performance and maintain it at its peak.What you will learn• Understand the meaning and importance of monitoring• Explore the components of a basic monitoring system• Understand the functions of CloudWatch Logs, metrics, and dashboards• Discover how to collect different types of metrics from EC2• Configure Amazon EventBridge to integrate with different AWS services• Get up to speed with the fundamentals of observability and the AWS services used for observability• Find out about the role Infrastructure As Code (IaC) plays in monitoring• Gain insights into how billing works using different CloudWatch featuresWho this book is forThis book is for developers, DevOps engineers, site reliability engineers, or any IT individual with hands-on intermediate-level experience in networking, cloud computing, and infrastructure management. A beginner-level understanding of AWS and application monitoring will also be helpful to grasp the concepts covered in the book more effectively.

Section 1: Introduction to Monitoring and Amazon CloudWatch

This part is focused on introducing the concepts of monitoring, logging, metrics, and alerts in a monitoring system. It will also explain the importance of monitoring infrastructure and applications and the advantages of using CloudWatch as a managed service for collecting metrics and alerting.

The following chapters are included in this section:

• Chapter 1, Introduction to Monitoring
• Chapter 2, CloudWatch Events and Alarms
• Chapter 3, CloudWatch Logs, Metrics, and Dashboards

Chapter 1: Introduction to Monitoring

Monitoring is a broad topic that covers different human endeavors. Ignorance of monitoring ideals and concepts can adversely affect how to handle and manage engineering and computer systems effectively. Systems are usually not 100% efficient, and there are times they break down or do not work optimally as intended. The only way to understand and predict a breakdown is by monitoring the system. When a system is monitored, its pattern of behavior can be better understood, and this can help to predict a failure before it eventually happens. A proper maintenance process based on what has been monitored can be used to minimize failure of the system.

To help start the journey into monitoring, we will begin with understanding what monitoring is and the building blocks of every monitoring setup and infrastructure. We will explore the techniques used to monitor any infrastructure and for which scenario both of them are designed, and the relationship that exists between different monitoring components. Then, I will explain the importance of monitoring using real-life scenarios to help to emphasize and better your understanding of each importance mentioned. To crown it all, I will explain how the AWS Well-Architected framework portrays monitoring as a very important aspect of your AWS workload, using the principles of the pillar to galvanize what we have already talked about in terms of importance and how it makes the architecture of any Cloud workload complete. The purpose of this chapter is to help you understand what monitoring is, provide a little historical background of monitoring, explain the different ways software applications can be monitored, and shed light on the importance of monitoring and software applications.

In this chapter, we are going to cover the following topics:

• Introducing monitoring
• Discovering the types of monitoring
• Understanding the components of monitoring
• Getting to know Amazon CloudWatch
• Introducing the relationship between Amazon CloudWatch and Well-Architected

Technical requirements

To be able to engage in the technical section of this chapter, it is required that you already have an AWS account. If you do not have one, you can quickly sign up for the free tier.

Check out the following link to see how to sign up for an AWS account:

https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/

Introducing monitoring

Man has always found a way to take note of everything. In ancient times, man invented a way to create letters and characters. A combination of letters and characters made a word and then a sentence and then paragraphs. This information was stored in scrolls. Man also observed and monitored his environment and continued to document findings and draw insights based on this collected information. In some cases, this information might be in a raw form with too many details that might not be relevant or might have been processed into another form that removed irrelevant information, to allow for better understanding and insight.

This means the data was collected as historic data after an activity occurred. This could be a memorable coronation ceremony, a grand wedding occasion, or even a festival or a period of war or hunger and starvation. Whatever that activity is in time, it is documented for various purposes. One of the purposes is to look at the way things were done in the past and look for ways it can either be stopped or made better. There is a saying that goes as follows:

"If you cannot measure it, you cannot improve it."

– Lord Kelvin (1824-1907)

So, being able to make records of events is not only valuable in helping to draw insights but can also spur the next line of action based on the insight that has been drawn from the data.

Borrowing from this understanding of how man has been recording, documenting, and making records, we can list two major reasons for monitoring data —to draw insights from the data collected and to act based on the insights received. This can be taken into consideration with a system that we build too. For every system man has developed, from the time of the pyramids of Egypt, where massive engineering was needed to draw, architect, and build the pyramids and other iconic structures, documentation of historic works has been very essential. It helped the engineers in those days to understand the flaws in earlier designs and structures, to figure out ways the new structures could be designed, and to eventually fix the flaws that were identified. It is usually a continuous process to keep evaluating what was done before to get better and better with time using these past experiences and results. Documented information is also very helpful when the new project to be embarked on is bigger than the earlier one. This gives foundational knowledge and understanding of what can be done for a new and bigger project due to the historical metrics that have been acquired.

Applying new methods go beyond just the data that has been collected—there is also the culture and mindset of understanding that change is constant and always being positioned to learn from earlier implementations. Building new systems should be about applying what has been learned and building something better and, in some cases, improving the existing system based on close observation:

Figure 1.1 – A basic monitoring flow

What we have been explaining so far is monitoring. Monitoring is the act or process of collecting, analyzing, and drawing insights from data that has been collected from the system. In software systems and infrastructure, this includes analyzing and drawing insights from the data that has been collected from systems performing specific tasks or multiple tasks. Every system or application is made up of a series of activities, which we also call events. Systems in this context can mean mechanical systems (cars, industrial machines, or trucks), electrical systems (home appliances, transformers, industrial electronics machines, or mobile phones), or computer systems (laptops, desktops, or web or mobile applications).

Algorithms are the bedrock of how complex systems are built, a step-by-step approach to solving a problem. When a complex system is built, each of these step-by-step processes that have been built in to solve a specific problem or set of problems can be called an event.

Consider the following example of the process of making a bottle of beer:

1. Malted barley or sorghum is put in huge tanks and blended.
2. Yeast is added to the mixture to allow the fermentation process to occur to generate alcohol.
3. After fermentation, sugar is added to the mixture to sweeten it.
4. The beer is stored in huge drums.
5. An old bottle undergoes a mechanical process that washes and disinfects the bottle.
6. The washed bottle is taken through a conveyor belt to be filled up with beer.
7. After being filled up, the bottle is corked under high pressure with CO2.
8. The bottle is then inserted into a crate with other bottles.

In this algorithm of preparing beer, there are various stages; each stage has various touchpoints, and each of these touchpoints is a potential for failure. The failure can be within a process itself or...

Table of contents

• Infrastructure Monitoring with Amazon CloudWatch
• Contributors
• About the author
• About the reviewer
• Preface
• Section 1: Introduction to Monitoring and Amazon CloudWatch
• Chapter 1: Introduction to Monitoring
• Chapter 2: CloudWatch Events and Alarms
• Chapter 3: CloudWatch Logs, Metrics, and Dashboards
• Section 2: AWS Services and Amazon CloudWatch
• Chapter 4: Monitoring AWS Compute Services
• Chapter 5: Setting Up Container Insights on Amazon CloudWatch
• Chapter 6: Performance Insights for Database Services
• Chapter 7: Monitoring Serverless Applications
• Chapter 8: Using CloudWatch for Maintaining Highly Available Big Data Services
• Chapter 9: Monitoring Storage Services with Amazon CloudWatch
• Chapter 10: Monitoring Network Services
• Chapter 11: Best Practices and Conclusion
• Assessments
• Other Books You May Enjoy