Symphony of Self-Destruction: Strengthening Security With a Red Team
According to Rafal Los, Security Strategist for HP Software, policy is a poor substitute preparation. Organizations across the globe have implemented technical defenses, authored policies, developed IR (incident response) plans and business continuity plans, and yet they operate daily with a false sense of security—because while some aspects of information security look great on paper, when it comes to actual attack scenarios, most organizations don’t have a clue where they stand.
The key to getting good at something is to practice. In a recent interview, Los uses an analogy of a Formula 1 team, where mechanics do nothing but train on how to best change tires, day in and day out. They have the process down to a science, and can get a car stopped and fitted with a new set in a matter of seconds.
“So what does changing tires have to do with security?” Los asked rhetorically. “Really it’s everything, it’s all about practice, and I think that’s where we fall down. We have a lot of practice that we’re missing” in the security industry.
Not ‘If’ But ‘When’
Finish this sentence: “Practice makes _______.”
If you said “perfect,” and are currently working in information security, then you’re wrong. In the security world, Los said, practice makes “good enough.” Nobody can claim to have a perfect security program.
In business, when it comes to security, it isn’t a case of if your organization will be attacked, but when. This is why organizations have security policies, processes, procedures, controls, and most importantly, technology. Combined, all of this exists solely to protect the business. And in order to prevent attacks, to address that notion of when, organizations map out a security plan and prepare.
Yet this preparation is where the gaps emerge. It starts with the target of the organizations’ preparations—and most of the time, they’re preparing for auditors over criminals, because at the end of the day a business exists to make money. To make money a business needs to meet certain regulatory requirements (i.e. compliance), so they prepare and practice for that first. That helps stick them at the level of “good enough.”
Granted, organizations prepare and practice against some types of attacks that aren’t driven by compliance, but more often than not, those attacks are staged. These staged attacks are often sold as pentesting (penetration testing), and there is a good deal of value in having an organization pentested, as long as it is done right.
“The way most organizations sell [pentesting] and implement it, I mean the bar is so low for pentesting right now, it makes my mind hurt. Any idiot can pentest right now. Anybody that can fire up a Nessus console can pentest,” Los said.
Certain procedures drastically reduce the value of any such testing. These include applying limits to the pentester, such as telling them they can only focus on a select block of IP addresses, or that certain applications or servers are off limits due to them being in production. These staged attacks sometimes exclude all C-Level executives, even though the security manager knows they can be the easiest way in; or they only happen during certain off-peak hours of a certain day of the week, dramatically reducing their value. If this happens, all an organization is going to receive at the end of the day is a false sense of assurance.
Most times, pentesting is used to test defenses against synthetic risks. This is when the company is looking to protect itself from the threat-of-the-day, depending on what the headlines are reporting.
While this is valuable, it isn’t something that should determine the entire organization’s security stance. Low quality attackers like the script kiddies who use automated attacks to target databases, they’re easy to deal with—policy and technology, such as IPS signatures and endpoint defenses, can be configured to blunt that threat.
But high-quality attackers are a different story.
“It’s Very Real and It Sucks”
“[High quality attackers] are paid only if they make it through your defenses,” Los said. “I got a news flash for you kiddo: they’re going to make it through your defenses. It’s not a question of if, because if they can’t through by penetrating your website, they’ll try your partners, they’ll try your vendors. Worst case scenario, they’ll get hired in your call center and steal data that way. How do I know? I’ve watched it happen, it’s very real and it sucks.”
Most organizations won’t prepare for the high impact attacks, because the odds of them happening are low. Most security plans are designed around risk, and why most risk-based security plans focus on what’s most important to the organization. This is where the false sense of security comes in, because the organization tests their risk-based protection plan by using unrealistic methods. So when the attack comes, and the attacker uses unexpected methods, someone is in for a very bad day.
Validation is an absolute requirement when it comes to a security program. The organization’s defenses, responses, and technology, must all be validated. And true validation comes from being attacked realistically. This is where the notion of a Red Team comes in to play.
A Red Team breaks things. They validate an organization’s defenses and response plans by hitting hard and fast, with no restrictions at all. Most organizations do not have the skill internally to form an effective Red Team, so they outsource it to a neutral third party.
There are plenty of solid pentesting organizations to pick from (and you, dear reader, should check them out), including Trustwave, Lares Consulting, Rapid7, HP’s Shadow Labs, eEye Digital Security, Offensive Security, Dell’s SecureWorks, and IBM’s Security Services.
So how can an organization hire a legitimate Red Team to validate their defenses, while avoiding disastrous outages and business-ruining issues during the Red Team’s attack? By starting small, and taking baby steps.
“Random destruction within your organization causes resume-generating events, otherwise known as you’re fired,” Los joked, explaining his thoughts about the value of Red Teams.
As the security manager for an organization, you’re not going to hire a Red Team to go completely bonkers on your organization starting tomorrow, he added, because you’d probably get a lot of things broken, a bunch of people freaking out, and your probable dismissal. And assuming you still have a job at the end of the Red Team’s assessment, you’re likely left with a massive report full of all the things that are broken and vulnerable, without a clue about how to start fixing them. When launching a Red Team initiative, start small.
There are four phases to that, Los said.
The first is figuring out the organization’s security posture as it stands on paper. This means determining company’s defenses and how prepared the company is against both low quality and high quality attackers.
Once that is done, the other part to the first phase is to assess what could break. If various systems or applications were to break in the middle of a security incident, how bad could things get? The answers here come from artifacts from the past, such as previous pentesting reports, compliance reports, etc.
The second phase is defining all the things that are in-scope. This is where the Red Team is given the objectives list, which will include vendors, partners, as well as anything and everything about the organization itself. Again, the goal is a realistic validation, so the Red Team should be given the green light to attack the same way the low quality and high quality attackers would.
Given that you’re starting small, when you go in-scope, start with standalone app. “You start with the smallest parts,” Los explained. “An application we think is standalone, we think is non-critical; we think it doesn’t do anything, and if it got hacked we’re sure this thing is going to be ok. And then we Red Team that app and we find out that it’s actually connected to the same piece of physical hardware, in the same SQL database as our ERP system. Because when somebody went to SELECT * FROM, it accidently crashed everything, and that entire database cluster went down.”
After that small start, an organization can move to testing an application cluster, then the most important applications, followed by sub-business units, then entire business units, until everything is in-scope. That process could take years.
The Red Team initiative should be known to as few people as possible. It does the company no good if everyone in the security and IT department is aware of it before it starts. However, some people do need to be in the loop on things, including the organization’s legal officer, as well as risk officer (assuming those positions exist).
An example of why this important made headlines recently, when the CIO of Tulsa, Ok., was placed on administrative leave after being accused of overreacting to alerts that were triggered during a pentest. The CIO issued breach notifications to the public over what was later determined to be a false alarm. The city had hired a pentesting firm to do spot checks, and they used methods that were unfamiliar to the security teams, who were unaware that pentesting was taking place.
Once all that is done, the third phase is ready to start. This is the action phase, where reconnaissance, target acquisition, the process of splitting high-value targets out from the low-value ones, additional reconnaissance, additional target evaluations, and finally exfiltration are performed.
During the third phase, it is important that the Red Team keep highly detailed notes, which most of the aforementioned professional Red Teams will do. But if the Red Team is internal, notes are a necessity in order to do accomplish any sort of post-mortem examination and mitigation once the attack is finished, the key piece to phase four of the initiative.
It should be expected that something will break during phase three, and the objective during phase four is to determine what broke and why. Often, this is where a company discovers systems that were unintentionally interconnected. This phase is also where resiliency failures will come to light. The idea of asking whether the attack succeeded is also valid, but it’s the wrong question to ask. It’s better to ask how successful the attack was, because attacks rarely fail.
Measure the attack’s success by looking at time (How long did it take the Red Team to get what they were after?), damage (How extensive was the attack itself? What broke?), and criticality (How critical was the attacked asset?). The next step is where good notes come in handy, because they will be used to follow the attack path and determine where security failed. From there, the organization will need to make the necessary changes.
These types of changes are typically optimizing what’s already on the network. Rarely will an organization need to go and purchase new technologies. More often than not, the organization will end up tuning the existing hardware, such as updating IPS/IDS signatures, or adding “I WAS HACKED” to the first row of a database in order to detect SQL Injections more quickly.
Once phase four is finished, the process needs to be repeated, by moving up a level and targeting something larger, or by testing another standalone application.
Technology evolves. Security is the same way. Leveraging a Red Team to constantly assess and validate an organization’s security posture isn’t easy, nor is it quick, but can help develop a solid security program. At the same time, Red Team assessments should be treated as a process and evolution, not a one-time event. If they’re not, then it only proves the “good enough” state of readiness after hours of practice, and assures a false sense of security.
“If you’re doing sound strategy based on the way people actually attack networks, then you’ll actually have a defensible security posture…it’s safer and more sane to fail at your own hands, rather than your enemy’s,” Los concluded.