But take that to the battlefield, and it's a different story: There's no Siri, no Google maps, and you're not looking for a 7-Eleven. However, imagine a similar premise, with enemy targets displayed not just on your smartphone screen, but on your helmet-mounted night vision goggles.
That's the goal of multiple ongoing projects at the Army Communications-Electronics Research, Development and Engineering Center that focus on human-computer interaction, or HCI. And the department can't develop the programs fast enough — the demand for HCI in the combat theater only continues to grow.
"Expeditionary forces pose new challenges from a capability perspective. If forces are going to be more distributed, the commander will still need to be able to perform uninterrupted mission command across home station, en route or other deployed conditions. The capabilities we've developed need to be expeditionary also," said Dan Lenhardt, one of the engineers in CERDEC's Command, Power and Integration Directorate. "We're trying to enable seamless, uninterrupted mission command, no matter where a commander is…we're also working on decision support capabilities, things like automation that can help out in high-tempo environments."
Interested in learning more about how C2/Comms technology is being implemented at a tactical level? Col James Jenkins, Chief Information Commanding Officer, Marine Aircraft Group 29, U.S. Marine Corps, will participate in a panel discussion on how new satellite capabilities, on-the-move, and expeditionary communications are revolutionizing tactical communications. Register today for the C4ISR & Networks Conference on May 26th at the Renaissance Arlington.
Much of that effort relies on moving away from the traditional keyboard, mouse, display (KMD) paradigm, instead using modes of communication like automatic speech recognition, gestures, eye-tracking and more high-tech ways of conducting command and control. That's at the center of what CERDEC officials hope to do with HCI.
"As the force moves away from having a fixed command post, and it needs to perform all the functions on the move and even dismounted, we need to re-look at all the technologies we use to interact with the software," said Dan Yaeger, an engineer in the Command, Power and Integration Directorate. "There are a lot of limitations in that KMD paradigm; it's impractical for those applications to require use of the soldier's hands to operate. So what we're trying to do is look at interaction technologies that allow the soldier and the commander to interact with the software — potentially hands-free — and while they're on the move. The real advantage for the commander is with these advanced HCI technologies we can actually provide a more intuitive interface that decreases the amount of interaction time, which increases the time they have to actually analyze and make decisions based on the data they're receiving."
Cue the wearable technologies such as heads-up display and augmented reality, or situational awareness on display — technologies that allow the soldiers access to data "while keeping their heads up and hands on the weapon, keeping their situational awareness and combat effectiveness at the same time," Yaeger said.
The insatiable demand for hands-free
The current HUNTR research and development prototype configured for night vision.
Photo Credit: CERDEC
Across the Army, a huge number of systems being fielded have requirements for either voice or another HCI component. It's not cheap to provide that today, and it means programs take on risk and require HCI developer expertise, according to Zach Deering, another of CERDEC's Command, Power and Integration engineers. A better solution would be something that could be used at more of an enterprise level — like a simplified software development toolkit that allows users at the installation level to provide advanced HCI for any software application on the systems. The result is reduced costs and risk, and a simplified learning curve for developers who haven't worked with advanced HCI concepts before.
That's just one of the projects in the works. There are also efforts like SMASH, or single multimodal for Android service for HCI, which serves as a small-footprint service providing HCI on any system without the need for a network or cloud. Instead, CERDEC provides the service and the software development kit to all of the Defense Department.
"What we've been focusing on for now is an application-defined and user-customizable voice command service. [So far] we've developed voice and we're working on other modalities," Deering said.
"Based on soldier feedback we got, the soldiers want to be able to customize the commands they can say as compared to us limiting it to a set list. So we created the HCI User Linguistic Customizer, or HULC. It allows the soldiers to go in themselves, change what they have to say for a voice command, and it really reduces their cognitive load of having to remember what they can say and allows them to use advanced HCI concepts without having to remember what we told them," he said.
HULC currently provides grammar-based voice command service, but the engineering team is looking to add capabilities like issuing commands through gestures via arm bands or a similar hardware solution.
"And the way SMASH was set up, we can drop in these additional capabilities without breaking the existing installs on the systems," Deering added.
Achieving the vision
HUNTR prototype in the day vision configuration.
Photo Credit: CERDEC
The work on heads-up display technology started as a collaboration project with the Defense Advanced Research Projects Agency, which originally developed the Heads Up Navigation, Tracking and Reporting (HUNTR) system to visualize data. The program continues to grow as DoD engineers work together to create a comprehensive, integrated heads-up display system through which soldiers can view critical data literally before their eyes — not by looking down at a handheld or vehicle-mounted device.
The heads-up display itself is built on existing infrastructure and hardware that supports the use of helmet-mounted night vision goggles. HUNTR is designed for use at night, but also for soldiers to swap out the night goggles for a daytime version as well.
By integrating the HUNTR hardware and SMASH software, "information is projected in a geo-registered world as you look at it, day or night. So we have the helmet-mounted displays both for night vision goggles and also for daytime, and this allows you to project that information from mission command over the real world," said David Randall, associate director of CERDEC's Night Vision and Electronic Sensors directorate. "And hopefully, if we get our accuracies correct, you would be able to locate an icon to show it on the real world where you're looking at the object itself. So that's the visual interface for the information that comes down from the mission command or the individual team leaders and squad leaders."
Back in the theater, "it's all about visualization and seeing the information," Randall said. "We've loaded all the software onto the handheld — Samsung 5 or Nett Warrior — and have integrated the SMASH software…and we're now operating our system using their voice recognition and displaying that information in the heads-up display."
The technology isn't quite ready for show time, though. Once CERDEC finishes their end of the development process, the capability is handed off to the program of record. In this case, the mission command component of the Army's Program Executive Office for Command Control Communications-Tactical.
That's expected to happen this year, said Cyndi Carpenter with the Command, Power and Integration Directorate. At that point, that organization would take over with the requisite testing, evaluation, integration, certification and accreditation needed before the technology hits the battlefield.
Then, ideally, it's ready to go: No training necessary.
"Intuitive by definition is something you're able to do without thought. It's reflexive. So we try to make a lot of this stuff as similar as possible to what's being used in the commercial world…while still providing a capability at a higher level necessary for operations," Yaeger said. After all, "today's users, the next-generation commanders who are younger now, they grew up with touch-screen technology."
