Security Operations in Practice

Mike Sheward

• Publisher: BCS, The Chartered Institute for IT
• Year: 2020
• ISBN:
  9781780175089
• Topic: Computer Science
• Subtopic: Cyber Security
• Index: Computer Science

Security operations departments are growing in importance and recognition; they are responsible for the secure day to day running of an organisation's network, endpoint, application, identity and physical security controls. This book walks you through how to establish and grow a strong security operations team, including hiring the right people, knowing when to build a tool rather than buy, and crafting procedures that allow the team to detect and respond to a wide variety of security threats.

1 INTRODUCTION

Much of what we read and hear about the relationship between information security professionals and the end users of the systems that we’re working to protect is often overly adversarial. We’re not at war with our users. Far from it. We’re there to serve them. We’re there to make it so difficult to violate a security policy, by providing the right tools and techniques, that no one can be bothered to do it another, less secure way. That’s what security operations in practice is all about. Throughout this book we’ll be discussing how to build an effective security operations team that can serve, protect and enable end users and the organisations they’re a part of.

The idea of service and building a service-focused culture within a security operations team will be core themes within this book. This is deliberate, because I believe a security operations team that serves will be a successful security operations team. At this point in my career I’ve overseen the build out of security operations teams at a couple of different software-as-a-service (SaaS) organisations, and I hope to lean on the various experiences and lessons learned along the way to better prepare the reader who may be about to embark upon the same journey. It’s not always a smooth journey, but it’s one that is worth taking.

Perhaps one of my earliest memories in the security operations space was a meeting to introduce a project my team and I wanted to run to tidy up user accounts and permission levels that had been assigned within a finance application, in a somewhat haphazard fashion, over the course of several years. We wanted to align the application with our new organisation-wide identity and access management (IAM) programme.

I went into the meeting expecting little resistance. After all, the application contained sensitive information that could do the company significant harm if it was exposed. It was a complex application ‘managed’ by a couple of long tenured folks from the corporate IT organisation, who had knowledge of its innards. I needed support from the senior IT executive for the project, as I needed to borrow a chunk of her people’s time to get the project done. I felt confident this wouldn’t be an issue, since it would make the application more secure, easier to manage and in the long term, reduce the overall burden on those two specific people.

The response I got was surprisingly negative, and I was told the needed resources wouldn’t be forthcoming. When I defaulted, as so many of us in this industry do, to ‘but…but…security!’ the response came as follows: ‘Security is great, but we’re not in business to be secure.’

This took me back and although it was a frustrating response it was 100 per cent correct. To this day, it has defined my thinking on the role of the corporate security team. Businesses aren’t in business to be secure, but it sure helps them if they are. Understanding that security may not always play the leading part but has a role to play in the overall production is tremendously important. If you understand and appreciate this, often you can define and gradually increase the scope of that role. If you get offended, upset or confrontational by this notion, then emotion can take over and security can be seen as sulking off into the background.

WHAT IS SECURITY OPERATIONS?

If Hollywood movies or even just TV adverts produced by the major public cloud providers are to believed, security operations involves a group of people sitting in a room surrounded by monitors, making split second defensive decisions based on what those monitors are telling them, billions of times a day. Of course, real-life security operations, while it may involve such a room, frequently known as a security operations centre (SOC), is markedly less movie-worthy. In the movies, they always show you the SOC, but they never show you the meetings or processes you had to go through to get all the purchase orders, tickets and other documentation in order, to actually build and maintain the SOC.

Security operations is where the words and aspirations committed to information security policies are put into practice and a person, or people, begin the information security equivalent of painting the Forth Bridge – the never ending, always evolving tasks of detecting, responding, patching, analysing and understanding.

The understanding portion is key. Understanding and differentiating between the cause and effect of an event helps a security operations team determine severity, which in turn drives how the team decides to respond, and the response is ultimately where value is realised.

Processors of events and incidents

The majority of the time, security operations teams are dealing with security events rather than incidents. There is a crucial difference between these two terms, which is worth clarifying before we go any further. Various information security standards and publications have slightly different takes on the two terms, but many have followed the lead from the National Institute of Standards and Technology (NIST) Special Publication 800-61, Computer Security Incident Handling Guide (Cichonski et al., 2012).

The NIST definition of an event is ‘any observable occurrence in a system or network’, which covers a broad range of activities, both legitimate and otherwise. A user logging into an application that they are entitled to with valid credentials is an observable event. Likewise, a file being copied between two machines on the network is an observable event.

An incident, however, is something altogether more severe. According to the NIST definition, a security incident is ‘the act of violating an explicit or implied security policy’. A person attempting to gain access to a system they are not authorised to use would be an example of an incident. Attempting to steal data from a web application would be another.

Now, this is all pretty foundational stuff, but it’s important to get this distinction right. Security operations teams typically receive data pertaining to events, and they set about figuring out which of those events constitute an incident. This is typically done by correlating a group of events to figure out which ones might be related to something that warrants further investigation and response. The challenge here is that there are often quite a lot of events received by the operations team. We’re talking thousands, tens of thousands or even millions of events per day in some cases. So how on earth are the security operations team supposed to deal with that level of ‘noise’ and figure out where to concentrate their efforts? Well, we’ve just arrived at a core challenge faced by security operations teams and one that, depending on how they choose to respond, can have a real impact on their effectiveness within an organisation.

There is no simple answer as to how you deal with this problem. There is no one size fits all approach. There are plenty of vendors who claim they will solve it for you with their software, but the truth is, while they might get you some of the way there, no single vendor makes software that can reliably detect every incident in your specific environment. We’ll be walking through the various strategies for effectively extracting the signal of an incident from the noise of routine events throughout this book. For now, I’ll mention one strategy that is doomed to fail: attempting to offload the event noise onto other teams within an organisation, or to put it another way, your customers.

It’s tempting for a security operations team that is drowning in events to attempt to offload some of them directly to other teams in the hopes that they’ll be able to stem the tide. Security operations teams should be service-focused, and most service organisations aren’t there to increase the workload for their customers, they’re supposed to lessen it. When it comes to ‘noise’, security operations should be a filter, not an amplifier. A great way to lose credibility and trust with your customers is to bombard them with useless, irrelevant or unactionable data. In the early days of a security operations team, this situation is more likely to occur, as a combination of new eyes, new tools and new insight into how various systems are running leads to an increased amount of paranoia and greater number of false positives.

I’ve been in a situation where a new security operations team had a rough start to life for the exact reason I’ve just described. The team was created in the shadow of a well-established incident response team, who were also doing a lot of event triage work. The aim of the security operations team was to reduce the workload on the incident responders, therefore allowing them to concentrate on more in-depth investigative work. At the same time the team was being created, a security incident and event management (SIEM) suite was being deployed and other security tools were starting to come online. The result was a lot of new people and a lot of new tools.

This was a big organisation and the event count ran extremely high. The security operations team, determined to prove their value early, set about alerting various contacts and teams to every event they were able to get their hands on. It soon became unmanageable and the security operations team ran the risk of losing credibility, as the defined escalation paths outside the security department started to refuse to respond to every escalation.

To fix this, I made a change to the way the new team would escalate. Rather than escalating directly, we leaned on the incident response team, who temporarily acted as a second tier. Using their prior knowledge of the environment, they proved to be an effective filter before escalations made their way outside the security team. Over time, both sides learned from each other how to recognise what was worthy of escalation and what was not. The temporary second tier was removed and the company started to realise the benefit of their investment in security operations.

When building a security operations team, always remember to take raw, unfiltered events as an input and produce actionable, detailed reporting and instruction as an output. This will result in a more secure organisation, which is ultimately what we’re working towards.

Built to fit

Security operations teams should understand the business they are supporting, and build security to fit that business. Teams that attempt to operate things the other way around will always run into problems that limit their effectiveness, cause frustration and ultimately lead to the development of a less secure environment. This was true 10 years ago, is true today and, despite the explosion in awareness of cybersecurity issues, will probably still be true in another 10 years. It can be disheartening at first, but I promise you the reward comes later, when the security operations team is seen as a trusted entity that enables the business. You will be invited into more projects to get ahead of what is coming. You will receive more reports from end users and customers about potential incidents and problems if they trust you. Developers will be more willing to work with you to address important security problems if they know you don’t pull the alarm cord continuously.

When an enterprise purchases a large, complex piece of software, the vendor often provides professional services to get the thing set up and rolled out properly. The same thinking should be applied to a security operations team, except, obviously, it’s a group of people rather than a piece of software. To get the most out of the team there has to be some time set aside for orientation and to fit them into the day-to-day operating rhythm of the business. Considering the question ‘what do we want from security operations?’ will go a long way towards defining what exactly security operations does for your organisation.

If your organisation is a SaaS provider, chances are you’ll want security operations monitoring the software you provide to your users via the internet, keeping their accounts and data secure as they do so. If your organisation is running a power plant, you’ll probably want security operations focused mainly on keeping the internal networks and computers that keep it ticking over free of malware.

Security operations is a form of risk treatment in this regard. You’ve identified a risk to your business, and one of the ways to lessen that risk is to employ a team to be on the lookout for signs that it may be about to materialise. Therefore, it’s important to remember that the scope of a security operations team not only differs between organisations, but can change depending on the direction the business takes. For this reason, you always hope that a member of the security leadership team is present when those business decisions are made, which (thankfully) is increasingly the case. The best security operations teams are those who are seen not only as a force for good inside an organisation, but also as an enabler of the business, and perhaps even a differentiator in the sales cycle.

As general awareness around information security has increased in recent years, so too has the role that the security team plays in the sales cycle. This is especially true when considering business-to-business (B2B) deals, involving the handling or hosting of sensitive information. Once relegated to a darkened room and kept out of public gaze, these days it’s not uncommon to find a security professional called up to join a sales call. This is fantastic news, for all parties, as exemplified by this next story.

I was asked to join a meeting with a particularly cloud-averse prospect (while working for a cloud-based software company). At this meeting the customer’s IT compliance officer started to list their concerns with our product. An incredibly specific risk case was cited; one that, to this day, I believe was invented because the officer wanted to kill the deal and not have to do any more paperwork.

While the heads of the product, sales and development teams started to talk about how they could solve the particular issue raised by the compliance officer, I opened my laptop and went to the SIEM. I was able to create a new event condition, based on correlating values from two different logs that came from different parts of our application, in about 3 minutes. Meanwhile, back in the conversation, new features were being proposed that would take weeks to implement.

‘Hey, so I just created a new event type to detect this occurring – obviously I’d like to test it, but I’m fairly sure this’ll alert my team if that ever pops up’, I said, somewhat smugly. Well, extremely smugly, actually.

And with that, the number of prospect meetings I was invited to increased dramatically. More importantly, the number of alerts for events that pertained to things our customers actually worried about increased, which is of course what it’s all about.

In, or on?

While talking about what security operations is, we’d be remiss if we didn’t discuss what security operations shouldn’t be. Although securit...

Table of contents

• Front Cover
• Half-Title Page
• BCS, THE CHARTERED INSTITUTE FOR IT
• Title Page
• Copyright Page
• Contents
• List of figures and tables
• About the author
• Foreword
• Acknowledgements
• Abbreviations
• Glossary
• Useful websites
• Preface
• 1. Introduction
• 2. Establishing a Security Operations Team
• Part I: Blue Teams
• Part II: Red Teams
• Endnotes
• Index
• Back Cover