In the Big Data Era the fight against terrorism can take huge benefits from innovative data analytics techniques, like data mining and prescriptive analytics. Social network, telecommunication, surveillance cameras videos, criminal records, etc. are giant information sources. Everybody is connected to each other.
An ex-Israeli security chief says Big Data and Data Analytics have been widely used by the Israeli military and intelligence agencies to track down enemies of the Israeli state, including several senior Hamas leaders killed during the Israeli incursion into Gaza Strip last summer. He stressed that the flood of unstructured data in the form of video, images, text and speeches has been utilized to the Israeli military to track down and kill enemies [link].
Today the most diffused data mining platform in counter-terrorism intelligence is Palantir. This platform is designed for large-scale quantitative investigation. Palantir integrates across multiple sources of data, builds graph models and provides some advanced analysis tools.
Watch Palantir in action to tracking terrorist during investigations on 2009 Jakarta bombings:
The key of success in terrorism prevention is the analysis of associations between people. For example, suppose that John comes from Country X and he is associated with James who has a criminal record and a large percentage of people from Country X have performed some form of terrorist attacks in last years. Because of the associations between John and Country X, as well as between John and James, and James and criminal records, one may need to conclude that John has to be under observation. Associations between people create a network.
Once we have a huge amount of data about people we can apply data mining tools to find automatically the most relevant associations and to build the network. With some advanced techniques, like link analysis, we can fully analyze the network and find suspicious people or actions that may require anti-terrorism proceedings. The ultimate step is creating a predictive model which gives real-time alerts to anti-terrorism agencies when some particular and dangerous associations are identified. This dynamic predictive model can be built by modern machine learning techniques, like support vector-machine (SVM) and graph classification, or by more classical techniques , like neural network classification or decision trees.
This techniques are very powerful, but there are several challenges that can limit their applicability or effectiveness:
- Privacy: gathering information about people, conduction surveillance activities and examining say e-mail messages and phone conversations are all threats to privacy and civil liberties.
- Big data: a huge amount of relevant data it’s not structured and requires very complex techniques to extract information.
- Data-federation: every Country has different and independent intelligence agencies. How can federate the data between them?
- Real-time: the predictive model must give alerts as soon as possible, with strict timing constraints.
- Efficiency: False positives could be disastrous for various individuals. False negatives could increase terrorist activities. The challenge is to find the “needle in the haystack.”