In developed nations, there is a rich trove of data that the intelligence community can and does mine.
Valuable information can be pulled from media reports, public financial information and social media posts. Websites track user activity, and smartphones are constantly gobbling up information about their users, from geolocations to search histories and more. By using artificial intelligence tools, analysts are able to make sense of this torrent of publicly available data and turn it into usable open-source intelligence, known as OSINT.
But not every part of the world produces that vast torrent of data.
“That exists in a very small handful of places throughout the world where that doesn’t exist—basically every developing economy,” said Ben Leo, chief executive of FRAYM, a geospatial data and analytics company. “[In developing economies] you are not able to get a comprehensive and representative picture of what the population looks like through the same types of techniques that are being used in the U.S.”
The U.S. military and intelligence community are increasingly interested in leveraging OSINT for predictive analysis—after all, properly collected and processed OSINT can help warn regional commanders of upcoming political protests, political violence, extremist attacks or other kinds of security related events could take place, said Leo. Notably, the Army awarded BAE Systems a $437 million task order for open source intelligence support in October.
Of course, in order to create usable and reliable OSINT, companies like FRAYM will need to create data rich analysis in data poor areas.
“What we do is we gobble up the very high quality, underutilized datasets that are out there. We bring in additional public datasets and we bring them all together using our AI/ML algorithms to produce this hyper local data at scale,” said Leo.
The company takes geotagged household data and feeds that into its machine learning algorithm, and from there it can then produce data down to a 1 km x 1 km grid level across dozens of characteristics, such as religion, ethnicity, language, age, education access, electricity, media consumption and more.
FRAYM has provided its services to the U.S. government in the past, but company officials declined to name any agencies they were currently working with or would like to work with.
In the past, he explained, there were really only two ways to make predictions in data poor areas. First, analysts could monitor events through social media. While that can help commanders understand the situation on the ground, it has very limited predictive power.
“Previously, you’ve been stuck in two worlds,” said Leo. “You’ve either been stuck in a world where ‘I’m going to monitor social media and try to apply natural language processing or other tools that will aggregate and make sense of that data so that I can try to identify a tipping point.’ When is the chatter getting to a certain point where it feels like something important is happening in a particular city? It’s too late at that point. That might be helpful for basic situational awareness, but that is not nearly as helpful or powerful for a practitioner or combatant commander than getting way left of ‘boom.’”
“When is the chatter getting to a certain point where it feels like something important is happening in a particular city? It’s too late at that point. That might be helpful for basic situational awareness, but that is not nearly as helpful or powerful for a practitioner or combatant commander than getting way left of ‘boom,’” he explained.
The other method was to basically plot out how and where events have unfolded in the past and try to find correlations that can predict future events. That, too, is very limited.
But with access to proprietary data the U.S. government does not have, Leo says his company has been able to create a unique source of OSINT for data starved areas.
“This is the first time that this kind of very comprehensive and rich data has been brought to bear in developing markets. Basically, anywhere where there is not a data identity-centric ecosystem, FRAYM brings a tremendous set of powerful solutions to the marketplace,” said Leo.
Nathan Strout covers space, unmanned and intelligence systems for C4ISRNET.
