Data Mining to prevent terrorism

In the Big Data Era the fight against terrorism can take huge benefits from innovative data analytics techniques, like data mining and prescriptive analytics. Social network, telecommunication, surveillance cameras videos, criminal records, etc. are giant information sources. Everybody is connected to each other.

An ex-Israeli security chief says Big Data and Data Analytics have been widely used by the Israeli military and intelligence agencies to track down enemies of the Israeli state, including several senior Hamas leaders killed during the Israeli incursion into Gaza Strip last summer. He stressed that the flood of unstructured data in the form of video, images, text and speeches has been utilized to the Israeli military to track down and kill enemies [link].

Today the most diffused data mining platform in counter-terrorism intelligence is Palantir. This platform is designed for large-scale quantitative investigation. Palantir integrates across multiple sources of data, builds graph models and provides some advanced analysis tools.

Watch Palantir in action to tracking terrorist during investigations on 2009 Jakarta bombings:

The key of success in terrorism prevention is the analysis of associations between people. For example, suppose that John comes from Country X and he is associated with James who has a criminal record and a large percentage of people from Country X have performed some form of terrorist attacks in last years. Because of the associations between John and Country X, as well as between John and James, and James and criminal records, one may need to conclude that John has to be under observation. Associations between people create a network.


Once we have a huge amount of data about people we can apply data mining tools to find automatically the most relevant associations and to build the network. With some advanced techniques, like link analysis, we can fully analyze the network and find suspicious people or actions that may require anti-terrorism proceedings. The ultimate step is creating a predictive model which gives real-time alerts to anti-terrorism agencies when some particular and dangerous associations are identified. This dynamic predictive model can be built by modern machine learning techniques, like support vector-machine (SVM) and graph classification, or by more classical techniques , like neural network classification or decision trees.

This techniques are very powerful, but there are several challenges that can limit their applicability or effectiveness:

  • Privacy: gathering information about people, conduction surveillance activities and examining say e-mail messages and phone conversations are all threats to privacy and civil liberties.
  • Big data: a huge amount of relevant data it’s not structured and requires very complex techniques to extract information.
  • Data-federation: every Country has different and independent intelligence agencies. How can federate the data between them?
  • Real-time: the predictive model must give alerts as soon as possible, with strict timing constraints.
  • Efficiency: False positives could be disastrous for various individuals. False negatives could increase terrorist activities. The challenge is to find the “needle in the haystack.”


  • Andy Reply

    Let’s hope that the states with access to this tech don’t turn into despotic regimes, but historical record is not encouraging in this regard.

  • Dragon Lance Reply

    I believe IBM is more big data that your reference, and i2 has more market share. Real time analytics and cognitive capabilities.
    Please your references.

  • Brian Reply

    It is interesting that people are talking about Big Data “Brother” as a tool. The psychological profile is an important factor in the process. If you want to hunt a Terrorist, you must understand how they think. You also have to look at history. The Germans learned about propaganda from sales executives in the United States. Terrorists use the same techniques, because we have educated many of them in our western universities. But the true leaders don’t uses technology to communicate. They use relationships. Generally I would say men in their late 40’s that are not digital natives. Today you can catch the small fish with technology but it is hard to catch the big catfish.
    There are 5 techniques that must be employed to defeat the enemy.CFLTS

Leave a Reply

Your email address will not be published. Required fields are marked *