Since 2015, the direction of the intelligence community has shifted. This is down to three primary drivers: flexibility of the enemy, speed of war, and the evolution of the analyst. In this article I will examine each one and explain why analysts will shape the future of the intelligence industry.
What is my background in Intelligence software?
In mid 2005, I applied and was accepted into the US Air Force Intelligence Officer program and immediately scheduled to attend the Air Force Intelligence school at Goodfellow Air Force Base, San Angelo, Texas. I had previously served 19 years in various enlisted and officer roles and was intrigued by the mystique of the Intelligence Community. The seven-month school was the third longest school in the Air Force preceded by the Fighter Pilot and Meteorologist schools. Ten months past and I found myself in the middle of Operation Iraqi Freedom at the Perfume Palace in Baghdad, Iraq where I supported the J2 and Task Force 134, Detainee Operations. Tensions were high with the ongoing trial of Saddam Hussein and daily attacks from Al-Qaeda in Iraq.
As a junior Captain, I led a team of 19 intelligence analysts and was responsible for supporting TF 134’s mission of overseeing the detainees and their ultimate release to the Iraqi Central Criminal Courts. It was my first deployment and the team’s primary mission focused on analysing thousands of Theatre Interrogation Reports (TIRs) where we determined which detainees were ongoing threats to the Coalition Forces. The technology we used ranged from IBM i2’s Analyst’s Notebook, Esri ArcGIS, CIDNE, Query Tree and Pathfinder. Analysts spent their days searching for persons of interest and determining if the detainee was a continued threat against coalition forces or was just in the wrong place at the wrong time. US Forces had been in Iraq since 2003 and the effect of war was starting to be felt.
Intelligence became addictive and I ended up supporting seven classified missions in Iraq and Afghanistan from 2006 to 2013; in 2006/07 and 2013 I was a Senior Captain and Major, and then from 2008 to 2012 a contractor, where I actually did the work. As a contractor, I captured the unique perspective as an intelligence analyst that many military officers simply don’t witness.
In the war zone, if the technology didn’t exist, we created it. I have a degree in IT and would often write my own software code. The war fighter didn’t have time to go back to industry, scope the solution and wait. The enemy is at the gate and you do whatever it takes to complete the mission. Much of my perspectives on intelligence software were born out of necessity during the threat of war.
Learn More: Are SMBs Prepared for the Coronavirus’s Economic OutcomesOpens a new window
The changing nature of intelligence software
Since 2015, the direction of the intelligence community shifted. Three primary drivers drove the change and included the flexibility of the enemy, speed of war, and the evolution of the analyst.
First, when you combat a non-state actor like Al-Qaeda or the Taliban you find the enemy has no clearly defined command of control. The flexibility of the enemy proved they do what they want, when they want, where they want and how they want. They are attacking you from avenues that you just don’t see coming and are controlling the tempo of war, which in turn demanded we change how intelligence is used. The enemy is not a bunch of misfits – they are highly educated, motivated and driven by a common cause. We witnessed tactics we had no way of predicting.
Second, the speed of war was now measured in real time for the whole world to see. The volumes of available data are becoming insane, in that sense you must be careful what you wish for. Levels of intelligence, surveillance and reconnaissance data collection have become the highest in the history of warfare. Historically, if you discovered a vehicle and discovered homemade explosives were hidden under a tarp, you would systematically analyse it from top to bottom. Perhaps you would conclude that there were 200 pounds of homemade explosives in it, and a report would be filed. But there were many checks and balances that were conducted alongside that report to ensure accuracy. Now, if you find the vehicle, the soldier reports it via computers to HQ, and the report is read by the leadership immediately – no checks or balances. It goes straight to the top. But perhaps a day later the vehicle is re-analysed, and we realise it’s not 200 pounds but 2,000 pounds. Suddenly you have to re-report and re-educate leadership. Leadership is trying to make rapid decisions, however the accuracy of the information they’re working with is constantly changing. Technology is designed to make an analysts’ job easier but sometimes it introduces more challenges.
Learn More: Why Enterprises Are Shifting to Event-driven ArchitecturesOpens a new window
Where is the market currently?
Historically, vendors controlled the data and have offered solutions which required a tremendous amount of customization – slowing software deployments and concerning leadership. We thought more data would help us do our jobs better. It just created more data. The technology failed to create value and solve real world issues. Asking an analyst to synthesize over 1 billion points in a few hours proved pointless. We failed to develop technology that scaled understanding and insight while we generated more data. We had to change our intelligence approach and get back to basics.
Finally, the enemy has changed yet so have analysts. They’re coming out of college with no preconceived notions. They haven’t grown up on television. They grew up on the Internet. For them, everything is instantaneous. They are technology savvy and understand what technology can do to improve their lives. If the technology doesn’t work, they throw it out. They are no longer content with historically sound solutions or the status quo. The result is having a big impact on how intelligence software is developed.
Integrating the new wave of analysts and data scientists
Data scientists are academics and often lack real-world experience. When I say real-world, I mean the threat of war. Data science analysis is not life or death, it’s based on probabilities – i.e. the enemy is probably going to attack based on the evidence of the machine learning we’ve done. They are removing human judgement from of the loop and trying to make it more scientific. However, it is critical you are able to synthesis what you know and break it down to key elements so that your users will understand you.
Consider the smart phone. Steve Jobs didn’t create the Internet, email or the telephone. He added them all into one appliance and simplified the UI – making it easy to access. I think data scientists struggle with that concept. You need to bridge the brilliance of data scientists and the ingenuity, creativity and insight of analysts. They have different backgrounds. Analysts are geopolitical and built from the soft sciences whereas data scientists originate from the hard sciences. In short, we have soft science blended with hard science. It works when you keep analysts central to the process and create tiger teams of analysts, data scientist and programmers. Predominantly, this is the way the intelligence community is moving.
What about the future for Intelligence software?
In the next five to ten years, next generation technology will be web based with machine learning playing a bigger role. The human will remain the primary driver, but machine learning will act as a confirmation, helping me if I’m looking in the wrong place or providing suggestions as to where I should look. I believe we’ll see an in improvement in Natural Language Processing and Natural Language Generation, helping report creation and dissemination. We will also synthesise data better, which is where solutions like SirenOpens a new window come into their own.
We are going to see more demand for plug and play from the next generation of analysts. Analysts don’t have time to wait and are impatient. It’s a me generation. Picture 19 to 24-year-olds. They will not tolerate long deployment cycles. They have no loyalty and their demands for effective technology are far more aggressive. When I say plug and play, I mean immediacy. If they need a capability, they should be able to load it without getting IT involved. They will want to build their own visualisations and have an easy user interface.
Future solutions will be priced differently – more of a pay as you use model. It will be expected that technologies will easily integrate. I think you will also see companies working closer together and in earlier stages of their product lifecycles.
In summary, the technology that is going to win will need to be fast, easy to use and flexible. Due to the market consolidating capabilities into single ecosystems, the need to adopt new ways of doing analysis coupled with emerging technologies like Siren will be key.
Let us know if you liked this article on LinkedInOpens a new window , TwitterOpens a new window , or FacebookOpens a new window . We would love to hear from you!