What goes into validating a cyber team? Fifth Domain was provided exclusive access to U.S. Cyber Command's premier annual training exercise, Cyber Flag, in which 12 teams were used as the capstone toward reaching full operational capability.
The entire cyber mission force reached initial operational capability in October 2016. Prior to that, this exercise was partly used to validate teams for IOC.
The full spectrum of cyber teams participated in training — from defensive to offensive. These teams squared up against a live and free-thinking opposing force that operated in the same cyber terrain as blue teams, or friendlies.
The teams are evaluated by an assessment group that certifies teams meet or check off certain events during the exercise. The assessment team lead, who, like many, spoke to Fifth Domain during the exercise on the condition of anonymity, said these assessment teams are made up of no fewer than five individuals who grade the exams given to cyber teams.
Within that construct, a team controller sits within what is called the white cell, or the brain of the exercise, understands the team's objectives and controls the pace of the exam, the team lead said. No fewer than three individuals are embedded with participants to observe their actions and input them into a database. A training analyst works with the opposing force, or OPFOR, to create a visualization to show the blue team how it performed.
This visualization — which is a subset taken from a tool in a control room that shows all network traffic and activity in blue (friendly), gray (cyber no man's land) and red (adversarial) space — allows assessors a literal playback of how the blue force reacted during a particular event to determine if it was the correct course of action.
Participants and OPFOR are playing on a synthetic network or range that has to be developed prior to the exercise. One range lead told Fifth Domain during a walk-through at the exercise that they build all the networks — to include virtual networks for members playing in remote locations — and vulnerabilities, meaning they purposefully built in vulnerabilities for the OPFOR to exploit. The network was only patched up to December 2016, affording the OPFOR all the vulnerabilities that were relevant and unpatched at that time. Moreover, the range can also take down security measures if they want to simulate bad business practices or bad systems administrators.
Teams are assessed against certain criteria. And while being tested to the point of failure makes it an unfair fight by design, teams do debrief with the OPFOR every evening to provide limited insight for the next day's exercises. The OPFOR and exercise leads are "bending over backwards" for the participants to ensure the right training is occurring, a blue team lead said.
This means teams can request to run through particular scenarios again if they feel they need additional practice.
For example, the OPFOR team lead pointed out that perhaps a blue team never experienced or didn't have much training on a data exfiltration operation, so they'll run such a mission. If the team didn't catch it, they'll rerun that again in coordination with all involved and maybe make it easier the next time, if necessary, or provide a little bit of extra intelligence.
It could be the team's skill sets of dealing with data exfiltration are spot on, Fifth Domain was told, but maybe the intel lead wasn't paying attention. There are multiple factors that could be blamed for a failed data exfiltration response, and the OPFOR in conjunction with exercise planners and assessors must pinpoint where the failure occurred, try to correct it and see how they perform again.
"If we don't hear chatter over the course of the day, we'll generally encourage that: Take a break and do like a mini catch-up so everyone's on the same page to give us a pulse of how things are going," a blue team lead said. "At that point, it's: Take a deep breath; it's OK; what else do you want to focus on today? Some of the guys have been up front to say: 'I want to dig into this tomorrow.' OK, we will work with the mission owner to get you that time to do what you want to do."
Going off script
While there is a script, so to speak, to which the the exercise tries to follow, there are specific things everyone must learn and experience, according to a blue team lead.
However, the script isn't structured like: "A," then "B," then "C," then "D," another blue team lead said. "A" might happen and the team must react. "So they're like: Well, 'B' doesn't really follow, but we have 'A' sub one and hopefully we'll get them back on track to what we want to see from them."
There also are specific injects available to assessors and the OPFOR that can be used to force the blue teams to perform a certain action. "I didn't really see this out of you, but I want to see this out of you, so how do I make that happen?" the team lead said.
"I don't feel like I'm being attacked by OPFOR. I feel like I'm being presented with potential scenarios I might really face," another blue team lead told Fifth Domain. "I don't feel like I'm being personally attacked by anybody, I feel like I'm very much in a scenario where I'm being presented with as much realism as possible. OPFOR isn't there to attack you — they're providing opportunity for things to react to."
While an extremely tactically focused exercise, teams might be asked in a scenario to provide mission assurance to the mission owner, who needs to make sure team members can communicate with other assets, be it a naval vessel or otherwise.
"There's one thing or two things that are absolutely critical for the mission to be successful. The rest of it is just a mitigation effort by the teams," an exercise leader said. "We have to make sure this stuff is working, then do damage control on the rest of it. That's the problem set they're given."
The operational tempo for the exercise stresses participants to the point where they are left with nothing but the ability to rely on their basic skills, a participant said. If there's one super star that might carry things, this situation kind of highlights that, Fifth Domain was told. The exercise is compressing what might occur in a month's time into six days.
Officials provided less details regarding offensive team activities and validation efforts. Their activity is tailored differently because their play is somewhat unique, one exercise leader said. "There's one team, and that aspect of it is it's highly tailored to them, but they're being assessed against their objectives and mission sets. That's an aspect of the exercise," an official explained.
"What's unique about how they're doing it in this exercise is they're doing it together," another lead said. Protection teams, support teams — which provide analysis and intelligence — and mission teams — which conduct offensive ops — are all working against the same adversary in conjunction.
"They're feeding one another, which is hard to do," an official said. "But the value they get out of it is hard to beat."
Mark Pomerleau is a reporter for C4ISRNET, covering information warfare and cyberspace.
