For a long time, open source intelligence (OSINT) was primarily composed of insights from foreign news sources. It was supplemental public information that analysts could layer on top of classified intelligence to gain a full operational picture. Many in the intelligence community viewed it as a “nice to have” rather than a mission-critical data source — something to confirm rather than to foundationally inform.
That era is over.
It’s over for two reasons. First, the types and scope of open source intelligence have exploded in the digital age. From public legal records to social media platforms to the dark web, OSINT now encompasses every online channel that bad actors are using to communicate and mobilize.
Second, technology has evolved to address the two major obstacles to transforming OSINT into mission-critical decisions: Speed and scale. Historically, before automated technology was developed, the exponentially growing amount of OSINT data overwhelmed analysis tools and made it challenging to deliver insights fast enough to stay a step ahead of threats. Artificial intelligence (AI) has dramatically increased the volume of data that can be analyzed in real-time. Today’s AI-enabled platforms empower intelligence analysts to leverage OSINT as the foundation that can help uncover hidden threats, corroborate classified reporting, and pinpoint the targets that warrant resource-intensive, traditional intelligence gathering.
Evolution of OSINT
OSINT is derived from publicly and commercially available information (PAI and CAI, respectively). It now includes elements such as public records (people/business insights), crowdsourced and hyperlocal events, domain and IP information, and blockchain and crypto activities along with interactions happening on the dark web. It has evolved into the foundational information resource that feeds into the other traditional intel disciplines — for example, crowdsourced, on-the-scene reporting via apps to supplement traditional human intelligence (HUMINT) or space-based, open source imagery (including even open source, space-based radar and RF collection!) to supplement traditional imagery intelligence and geospatial intelligence.
Entire intelligence disciplines like targeting, which in the past relied almost exclusively on sensitive data only available to governments (e.g. call detail records or email traffic obtained via subpoenas) can now be directly supported by comparable but open source data, including relationship network analysis of social and dark web data or location insights.
As the types of sources and applications have increased, the volume of OSINT data has grown exponentially, requiring new analytical approaches to deliver actionable insights.
Fortunately, appropriate AI tools can reshape the paradigm and increase OSINT’s utility. While its sheer volume has meant that only a small percentage of OSINT been tapped for intelligence purposes, AI now delivers the ability to unlock the full potential of this data, maximizing the insights derived from OSINT analytics. In essence, AI turns the challenge of large data volumes into an advantage and ultimately creates a symbiotic relationship with OSINT.
Scalability
AI, machine learning and automation tools are able to scan massive amounts of data and conversations, proficiently identifying connections and risks that warrant further investigation. To help ensure protection from threats, analyzing OSINT demands a 24/7 schedule, and AI helps keep pace on assessing and identifying risks in near real-time. These approaches greatly reduce the time analysts spend gathering and processing data, instead focusing on extracting relevant insights. AI solutions can be used to observe patterns in data at a fast rate and reach more sources than traditional human-driven searches. They are even highly successful at detecting machine-generated threats (i.e., AI defeating other AI that can be disguised).
Accuracy
AI limits the number of errors made when conducting analysis, typically by providing more comprehensive and consistent processing. AI and machine-learning do not replace traditional analysis by intelligence experts, but they empower human analysts to prioritize their time and focus on the most critical information to form assessments. Technology is now able to deliver a foundation of OSINT insights for national security and intelligence leaders to use to inform the strategic allocation of mission resources.
Mission Effectiveness
AI-powered OSINT can also be leveraged to assess any situation in real time, making connections about where threats and opportunities exist. Benefits include continuous screening – particularly for identifying insider risks within an organization or supply chains – that automatically identifies potential red flags. Compared to the hours of labor previously spent on periodic re-inspections, screening can now be completed in a matter of seconds and in a routine, precise and timely process that can alert decision-makers immediately.
With AI, OSINT has never been more valuable or mission critical. It is time to fully embrace this readily available resource now that technology has advanced to the point where OSINT can be analyzed in near real time. AI isn’t going to replace intelligence analysts, but it can deliver a strong foundation of insight to help focus their time and attention on the most pressing national security threats.
McDaniel Wicker is the vice president of strategy at Babel Street, and Patrick Butler is the senior vice president of strategy.
