When it comes to finding a better way of conducting business or developing a cutting-edge technology, the advanced research arm of the Intelligence Community is never content. IARPA, the Intelligence Advanced Research Projects Activity, constantly works to identify new and improved methods in everything from machine learning to analyzing text, speech, imagery and video.
IARPA’s approach to this mission embraces broad participation from the research community and beyond — nearly anyone can take part. Those who do rise to the challenge stand to walk away with cold, hard cash in their pocket, courtesy of the U.S. Intelligence Community.
Founded in 2007, IARPA exists to fund research at the cutting edge of science, technology and engineering. This research has a singular goal —generating revolutionary capabilities for the U.S. Intelligence Community.
IARPA invests exclusively in high-risk, high-payoff research programs that have the potential to provide the nation with an overwhelming intelligence advantage.
According to IARPA Deputy Director Stacey Dixon, the organization was conceived to serve a gap in the Intelligence Community’s R&D capabilities. Before IARPA, most science and technology research was done by individual IC members, which proved to be problematic for the kind of longer term, higher risk programs that IARPA embraces.
“Because they’re operational agencies, they always had a hard time putting aside funding for the longer term, riskier research because their needs were so pressing for what was happening today or what was happening in six months,” Dixon said.
Welcoming a crowd
IARPA’s research programs, through which academic and industry teams compete with and against each other to solve a well-defined set of technical problems, each have a clearly defined end-goal, typically three to five years out, as well as intermediate milestones to measure progress towards that goal. To assess their success, teams are regularly scored on a shared set of metrics and milestones. This approach helps ensure IARPA is agile in its ability to anticipate and deliver, does not institutionalize programs and that fresh ideas and perspectives are always coming in.
“To achieve our goals, we’re unusually outwardly facing for a part of the IC,” said Jason Matheny, IARPA’s director. “We’ve funded research at over 500 organizations in academia and industry, as well as some work in national labs, and most of our work is unclassified because it’s sufficiently early in the development of a technology or it’s a research area where we don’t face an intelligent adversary.”
“No matter who you are, most of the smartest people work for somebody else. I think that’s especially true in domains like science and engineering where expertise is widely distributed.”
- Jason Matheny, IARPA’s director
One way IARPA fosters innovation is through crowdsourcing.
“We believe in something called Joy’s Law, named after the founder of Sun Microsystems,” Matheny said. “The law is, ‘No matter who you are, most of the smartest people work for somebody else.’ I think that’s especially true in domains like science and engineering where expertise is widely distributed.
“There are extraordinary scientists and engineers in academia and industry who we work with, most of whom don’t hold clearances,” Matheny continued. “You want to be able to find some way of leveraging the creativity and ingenuity of those scientists and engineers, so that their intellect can be applied to some of our hardest problems.”
Dixon expanded on Matheny’s sentiment, stating that IARPA purposely structures research programs so that, at a minimum, aspects of them can be run at an unclassified level, which invites broader participation.
“The best ideas and the smartest folks are not necessarily in the IC, so how can you leverage the information and the people that are outside the IC to benefit our intelligence mission,” she said.
One of the ways IARPA does this is through forecasting tournaments. Another is prize-driven challenges. While those concepts may not fit the public percpetion of the Intelligence Community, they are in fact part of the IC culture and Matheny said both initiatives continue to yield positive results.
“You could be a hobbyist or you could be a computer scientist working in your pajamas in your basement, and if you’re smart and you’re ingenious, you can figure out a solution to a problem that a large contractor or university might not be able to solve,” he said. “We’ve found increasingly, that some of the most innovative solutions are coming from the places that you’d least expect.”
Throughout all of these programs, IARPA partners closely with other IC components to understand and anticipate their evolving needs, searches for innovative concepts and aims to bring them to fruition. Agility is the key in the IC and IARPA constantly seeks new innovative ideas and perspectives.
“The best ideas and the smartest folks are not necessarily in the IC, so how can you leverage the information and the people that are outside the IC to benefit our intelligence mission.”
- Stacey Dixon, IARPA’s deputy director
Accepting failure
“We currently support all 17 elements of the IC and look at hard problems and challenges that are enduring for them that can only be solved with science and technology,” Dixon said. “Also, interestingly enough, the things that we deliver are often capabilities that many different partners need and can take advantage of. While we sometimes start a program focused on one particular partner, by the time we are ready to deliver a capability or deliver it to them to further refine, it’s something that many different partners can benefit from.”
At IARPA, status quo is always questioned and only the best ideas are acted upon. Program managers are encouraged to take risks and accept failure in the pursuit of their innovative research ideas, provided that they do not sacrifice technical or programmatic integrity, and that they fully document their research results.
“We won’t invest in a research effort unless we’re confident that the question motivating the research program is sufficiently well formed and focused, and that we’ll be pursuing good research,” Matheny said. “Now it may not be successful, and in fact I’d say if the odds are better than half that it’ll be successful, then we probably shouldn’t be funding it because our mission is to go after the hardest problems. By the time we launch a program we know that there’s enough reason to think that the problem is scientifically tractable, and that we can measure success or failure.”
One of the challenges IARPA is currently tackling involves analyzing video. Recording video has never been easier, thanks to modern technology. Thus, figuring out how to navigate through and manage all of this video has become a priority for the Intelligence Community.
Aladdin Video program
The Aladdin Video program — Searching for Events-of-Interest in Massive Unconstrained Video — developed tools that can search through large volumes of video, without keywords, using only the content of the videos. This is important because it enables intelligence analysts to cope with the large volume of video images that pour into their offices each day from unmanned aerial vehicles, on-the-ground surveillance and other sources in danger zones.
“Aladdin is going after a very hard problem,” Matheny said. “How do you automatically analyze the enormous volume of video that exists in the world?”
In contrast to most methods used to find videos commercially, Aladdin doesn’t depend on a text tag that other users have associated with a particular video. Rather, the program is designed to sift through massive quantities of video and produce accurate, actionable analysis without relying on human intervention.
“If you are searching online for a video, you’re going to use a text-based query that will look for the text tags users have associated with a particular video. Searching by tags is great if you’re trying to find a video from a TV show or something else that’s popular,” Matheny said. “It’s not so good if you’re trying to find posted video of a martyrdom or an IED ‘how to’ video clip, because the people posting those videos don’t tag them. They don’t want us to find them. They post the video and send out the URL directly to their colleagues. We need to find a way of identifying those videos automatically, based on the content of the videos and that’s what Aladdin does.”
All of this leads to a perfectly reasonable question: how does a computer identify what is happening in a video?
“It’s extraordinary that it’s able to build queries from an example set of other videos that are similar,” Matheny explained. “Then it goes out and finds videos like that. If you want to build a query to find martyrdom videos, you can provide enough examples for it to go out and search for those videos automatically.”
Finder program
While today’s technology makes it easier than ever to include the location where an image or video was captured, not everyone is eager to do so. For this imagery, intelligence analysts work hard to deduce as much as they can using reference data from many sources, including overhead and ground-based images, digital elevation data, existing well-understood image collections, surface geology, geography and cultural information. Such image/video geolocation is an extremely time-consuming and labor-intensive activity that often meets with limited success.
Another IARPA initiative, called Finder, is designed to help IC members locate non-geotagged photographs and/or video, specifically with an eye towards improving search capabilities and speeding up the process.
“The goal of the Finder program is to develop tools to help analysts locate where in the world images or video were taken,” said Jill Crisman, IARPA’s original program manager for both Finder and Aladdin.
“If the odds are better than half that it’ll be successful, then we probably shouldn’t be funding it because our mission is to go after the hardest problems.”
- Jason Matheny, IARPA’s director
To do this, the system requires a model of the world. The model includes a wide variety of reference data, such as satellite imagery, maps and photographs publicly available on the internet.
“There are so many pictures uploaded all over the world and while some of the larger commercial organizations can tell you in a split second, this is picture of the Eiffel Tower, we are really interested in places where photos aren’t taken nearly as frequently near famous landmarks,” Dixon said. “Being able to identify where those pictures are taken, especially if the picture was captured as part of something that we are interested in from an intelligence perspective, will be extremely important.”
Janus program
A third and final example is Janus, which leads the world in facial recognition under real-world conditions.
“Janus is a program that’s focused on automated facial recognition using non-optimal images,” said Matheny. “Most facial recognition systems right now that are used to identify terrorist suspects or criminal suspects rely on images that are from a front pose, relatively high resolution, with no objects that block the face. Unfortunately, in many cases we don’t have such images.”
The program is designed to recognize people in photos, even if their faces are in profile, out-of-focus or poorly illuminated. This technology is extremely helpful during an event like the investigation of the 2013 Bombing Marathon bombing, where tens of thousands of videos and images needed to be analyzed during a time-sensitive investigation.
Moving forward
IARPA has been a decade-long experiment in figuring out the best way to develop an advanced research organization for national intelligence. Having developed and practiced many of the processes needed to successfully run multiple research programs, IARPA has successfully moved beyond “start-up mode,” expanded its research portfolio and focused on more and more difficult intelligence problems.
IARPA had a record year in 2016, including:
- 46 technical workshops with over 2,700 attendees
- 12 new multi-year research programs covering diverse technical fields, including biosecurity, chemical sensors, underwater autonomy, biometrics, video analysis and machine translation
- 2 new public challenges offering cash prizes to innovators across the country
- Over 250 peer-reviewed publications from IARPA-funded research
- 22 transition agreements to transfer IARPA-funded technologies to other government agencies
“Our research portfolio has become more diverse and we have a greater diversity of researchers who work on our programs,” Matheny said. “In the last few years we’ve also expanded our work in fields like biology, chemistry and neuroscience that are emerging as key areas for intelligence for the future.”
While it’s hard to predict what the future holds for IARPA, make no mistake about — they’re working on a better way to figure that out too.
