FORT PICKETT, Virginia – Soldiers, Marines and special operators have spent the past few weeks on live fire ranges and rushing through shoot house drills in small teams here all while seeing likely what is the immersive, interconnected future of close combat and potentially a way to finally merge technological advances to make the grunt as combat capable as the best trained fighter pilots.
The Integrated Visual Augmentation System, or IVAS, has gotten some amount of hype as being an advanced set of goggles, one day a sunglasses-sized device that will provide next-level night and thermal vision while also adding in layers of other actions such as navigation and targeting.
Developers are hoping for much more.
The IVAS, like many military programs now, aims to be an interchangeable platform. That means that an array of other devices, software programs and capabilities would flow through the system, tailored through preferences by individual troops.
The core of what’s happening with the device relies on mixed reality. Essentially, software providing visual symbols in the user’s field of view. More augmented reality than virtual reality. The user still sees the real world but can add and enhance what they see in their view.
While that might mean cool graphics and 360-degree experiences for video gamers, educational programmers and researchers, it can mean life or death for soldiers or Marines.
“No other piece of equipment has had this kind of impact since the introduction of night vision,” said Undersecretary of the Army Ryan McCarthy, a former Ranger. “This takes night vision to the PhD level.”
McCarthy and Army Vice Chief of Staff Gen. James McConville told reporters that while full production and fielding decisions will have to follow prototyping and testing, they expect to have a usable device in the troops’ hands within the next 18 to 24 months.
First they’ll have to integrate all of the capabilities, sensors and options already in the Army’s research and development entities. Along the way there will need to be cyber security elements built in and ruggedization trials.
“This has to be used in the woods, in tough conditions,” McConville said. “We want the system to operate and thrive in combat conditions.”
The enabling aspects hold a lot of promise for the dismounted ground troops.
“This allows us to do really the type of fighting which is the way the Army fights,” McConville said. “Coordinating the fires in effect with the infantry on the ground. If you know where you’re at and you know where the enemy is and you have key communications, you’ll get the overmatch.”
Military Times saw a demonstration of the prototype technology, supported by Microsoft’s HoloLens project. At the recent event, three users were permitted to answer questions on their feedback using the IVAS.
The physical piece of the system will include goggles wirelessly connected to a small computer and a camera on their primary weapon. Those devices will communicate wirelessly within the squad on either radio frequency or wireless internet. In some scenarios they will be linked to cloud computing and in others to local network or data storage.
During the demonstration, developers walked users through early variations they’re working on with soldiers and Marines from immersive navigational modes that show the user’s location as well as the real-time location of squad or teammates in blue. Each member can also drop digital symbols on a shared map, marking enemy locations or other items.
At the same time, the map view gives a constant heading for compass readings. Troops can overlay point markers, phase lines and locations of friendly units.
And when they remove the map from their vision, the markers remain, glowing atop the real world in front of them.
The same lens technology allows for advanced, white phosphorous screen viewing – no more night vision green glow – and a variety of thermal options, from white to black to outline modes.
During the demonstration a developer walked behind a set of plants to show how the thermal view would immediately highlight a concealed person through body heat signature.
Views were clear enough to identify clothing types, weapons and other items at close range, much like advanced Forward Looking InfraRed cameras used in many vehicle and aircraft viewfinders to see through fog and in conditions too dark for standard night vision.
Army Staff Sgt. Nicholas Schneider, with the 82nd Airborne, wants to see the device ready for use in real-world missions but noted that it can be used for types of mission prep that are not possible now.
For example, troop-leading procedures that involve rehearsals for a real-time mission.
“We might not know what a compound looks like but we can predict, take Google images and build something off of that,” he said. “To take that and implement it into the IVAS is a huge boost to our rehearsal.”
Schneider and Marine Cpl. Anthony Olcott, with 2nd Battalion, 2nd Marines, both explained that it’s common for him and his soldiers to use multiple devices, from navigational tools to night vision goggles to smartphones to achieve some of the same capabilities.
“Thermals, compass, protractor, After Action Review boards, all in one,” Olcott said.
Olcott also commented on features that would allow troops to design the types of rooms and reconfigure them for different scenarios.
Right now, engineer tape, grass drills or empty barracks rooms are sometimes all that’s available to train fire team and squad tactics.
“You can only clear out the laundry room so many times,” Olcott said.
Schneider said one feature of the early prototype was helpful that he could see friendly teammates on the other sides of walls while in the shoot house, increasing his situational awareness and avoiding simulated fratricide.
They’ve been able to give feedback already.
USER FEEDBACK
Developers told reporters that they’re working to use military grid coordinates in updated versions of displays. Olcott and Schneider said they’ve been asking for things they’re not even sure whether developers can deliver yet.
For instance, some of the enemy avatar movements are not exactly realistic. They’re too “video game like,” the pair said.
“We’re asking for more patterns of life from enemy avatars. Can they move around the house, can they do this or that?” Schneider said.
The IVAS is being built to also be used during daylight conditions. It’s a wear all the time device, officials said.
And wear all the time means for both training and real-world missions.
Officials from the Close Combat Lethality Task Force to two of the Army’s Cross Functional Teams, on soldier lethality and synthetic training environments, see the IVAS as the way to get at the age-old problem of making infantry soldier drills more realistic, giving them variety and vastly more repetitions than are possible on today’s current live-fire ranges.
The STE piece is fundamental to the training aspects of the device. The CFT-STE team is working on developing “one-world terrain,” a software that would allow commanders to draw upon existing 3D mapping data to set up scenarios virtually from any place in the real world.
Then, anyone from a fire team leader to a company commander to start and later even battalion and brigade commanders, could set up realistic training scenarios with virtual enemies that mimic patterns, tactics and behaviors of real opponents.
Couple that with accurate terrain and troops could repeat scenarios at home station, on ship while floating on a Marine Expeditionary Unit, at an overseas base as a mission rehearsal to troubleshoot their tasks during the days or hours before they step off.
This falls closely in line with former Defense Secretary Jim Mattis’ initiative to have the close combat troops fight “25 bloodless” battles before they first taste combat, much like fighter pilots spend hours upon hours in simulators before ever taking off in a cockpit for even a semi-standard training flight.
ONE DEVICE, MULTIPLE FUNCTIONS
IVAS is drawing on existing and developing Army programs to bring a host of functions and capabilities into one device.
The Android Tactical Assault Kit is a program that allows soldiers to use current military radios to swap information back and forth, from mapping items mentioned earlier to short text messages. Its main function is to provide situational awareness for troops.
It’s the backbone for the chat, location marking, mapping both from the cloud and stored formats, navigation and altitude data.
Nett Warrior links soldiers together and can provide full motion video, even connecting dismounted soldiers to ground vehicles. It also accesses Blue Force Tracker, a location software in use with ground vehicles since at least the early 2000s.
Another feature ties in another program, the Family of Weapons Sights-Individual, Rapid Target Acquisition capability. Demonstrated to media in 2017 and again since then, the RTA pipes live video imagery from a rifle-mounted camera into the troop’s field of view in a night vision device.
Combined with the thermal capabilities, RTA lets troops shoot around corners, from concealed locations and through fog and other obscurants without even shouldering the weapon.
Users can select whether to view only what’s in front of them, only what their weapon is pointing at or use a picture-in-picture mode to see both views.
That capability is also being incorporated into IVAS.
All of these functions have immediate tangible effects for users. But what might be even more transformative, if successful, is what’s happening on the other end of the devices.
INSTANT REPLAY
With a combination of cameras, sensors and sophisticated artificial intelligence, developers plan to build ways to monitor eye movement, pupil dilation, breathing and heart rates, muzzle position and reaction times when moving and firing under stress.
Videos being installed or already in place at shoot houses and larger training centers along with the immediate Point of View camera on the device allow for NFL-like instant replay of basic team and squad drills.
That lets the team leader and his higher commanders better assess squad performance, develop metrics to help define successful missions and tasks for units that for too long have been overlooked in the big data push.
The behavioral measures and biometrics they hope to capture could help identify weak spots in individuals and AI could, over time, gather enough data to advise on the best squad formation, weapons needed and group loadout for a given mission.
Somewhere in there, commanders will have to develop plans and protocols to avoid burdening the soldiers with too much information, also known as “cognitive overload.”
“We want to have the capability to govern what kind of information they get,” McConville said. “How much information does the private, squad leader, platoon leader need? It’s different.”
And there always needs to be a backup, in case the system is degraded or fails.
BACKUP PLAN
“The other thing we’ve got to build into this thing is the PACE plan – Primary, Alternate, Contingency and Emergency,” McConville said. “You’ve have to have different levels.”
That means if one feature isn’t working then a backup of some kind is available. Just like with GPS navigation, troops must still know analog land navigation.
All of the potential lies in the future. But it’s a not too distant future.
TEAM IVAS was formed less than a year ago with both CFT and PEO personnel. Tasked in May 2018, they went through assessments of private industry tech, demonstrations and an award by November.
In early March, 50 headsets were delivered by Microsoft and they began the first capability set, Mark Stephens, Program Executive Office-Soldier’s director of acquisitions. told Military Times in an email.
The navigational, night vision and thermal capabilities are being adjusted through troop feedback and the state they’ve reached so far took about six weeks, officials said.
That was the first of four capability sets they’re working, Stephens said.
The first set used some of the commercial HoloLens with some of the defense applications, integrating the thermal sensor with the device and demonstrating initial STE work, which was having soldiers conduct a room entry and clearing a shoot house with the device.
Those 50 headsets incorporated thermal, fired digital bullets to hit mixed reality targets and include the initial TAK software.
WHAT’S NEXT
This fall the next set will integrate more thermal and low-light sensors, add in more features of the TAK software, add more battle drills for STE and further develop Rapid Targeting Acquisition in the IVAS.
In early 2020 through the summer they’ll incorporate the Heads Up Display that’s been developed by the Army, testing the thermal and low light sensors, assess Cyber Electromagnetic Activity and scale up the battle drills to larger teams.
All of that in the hopes to meet an ambitious timeline of a production prototype by November 2020.
Todd South has written about crime, courts, government and the military for multiple publications since 2004 and was named a 2014 Pulitzer finalist for a co-written project on witness intimidation. Todd is a Marine veteran of the Iraq War.
