“Big data analysis is how big problems are diagnosed and solved,” he told Computer Weekly at the Microsoft Cybercrime Center, the DCU’s world headquarters in Redmond, Washington.
The facility, unveiled in November 2013, was set up as a centre of excellence to advance the global fight against cyber crime and enable the DCU to work more effectively with partners to fight online crime.
Cloud-scale criminal activity demands cloud-scale partnerships and systems, said Hurd, because criminals are “pickpocketing whole countries with almost no cost and getting away with it".
Big data analytics a shared responsibility
But he emphasised the importance of transparency around the types of data used for this kind of analysis through open public-private partnerships.
“The thing about big data as a positive force in this equation is that often you cannot hide global-scale crime, no matter how hard you try, because it is so monumental. The volume and velocity of this data requires us to work together faster because criminals can change their tactics and move their operations amazingly rapidly," said Hurd.
“It is only by shared big data analytics that we can begin to solve this problem,” he said, underlining that positive value of big data, which is often associated with risk for legitimate business.
More on big data analytics
- How to slash the cost of big data analytics
- Big Data analytics: Adoption and employment trends
- Big Data and Content Analytics: measuring the ROI
- Big data analytics applications pull business value from info vaults
- Are SSDs necessary to perform analytics on big data efficiently?
- Big data, bad analytics
- Taming the big data analytics tiger
- Big data analytics make Lean methodologies leaner still
Big data identifies software theft
An example of a capability that this approach delivers that has not been available in the past is identifying stolen product activation key aggregators by being able to detect testing activity.
“Through analysis and visualisation of big data, we are now able to see spikes in activity that points to criminals testing large numbers of product keys they have stolen,” said Hurd.
This is an example of a transparent process used by Microsoft, he said. When activation codes are entered, that calls out to the software producer and presents a serial number to turn on the software.
“Using cyber forensics and appropriate big data, we are now able to see that pattern, and the moment of insight was highlighted by the visualisation [in the form of a spike]. This can then be turned into an algorithm to find this specific kind of criminal activity that would otherwise be lost in the noise," said Hurd.
Microsoft then uses this intelligence to work with its partners to secure infrastructure and anticipate future criminal activity.
Pluck insight from an ocean of data
As business intelligence technology has matured into providing an analytics capability, Hurd believes organisations are finally able to find small, but valuable insights in big data.
“We are increasingly able to find indicators of good or bad in the ocean of data by interacting with the data in a way that goes beyond static visualisation to ask and answer questions much faster. The more we allow the data to speak for itself, the better our chances of finding insight in it,” said Hurd.
And that is why he believes it is important to design analytical tools that can deliver insight with any big data set rather than designing them specifically for one purpose.
Spotting trends in criminal activity
Another important source of big data that Microsoft can analyse using its advanced business intelligence architecture is the billions of check-in calls from hijacked computers.
Microsoft typically uses civil legal actions to take over botnet domains and redirect calls by infected computers from command and control servers to servers in Microsoft’s Cybercrime Center malware lab.
Analysis of this data helps identify trends in the activity of criminal groups involved in conducting global billion-dollar operations.
“Big data analytics is aimed at delivering actionable insights, such as identifying illegal counterfeit software download sites among around 600 million legitimate websites worldwide,” said Hurd.
In this regard, big data analytics is enabling Microsoft to remove two million illegal software files – which are often laced with malware – and close more than 100 online trading accounts each month.
“While it is easy for counterfeiters to set up websites, it is far more difficult to set up merchant accounts linked to bank accounts into which ill-gotten gains are paid,” said Hurd. “Advanced link analysis has, for example, enabled us to link hundreds of websites dealing in counterfeit software with as few as 12 merchant accounts.”
By closing down these merchant accounts, which allow the criminals to process online credit card payments, Microsoft is disrupting their processes and increasing their cost of doing business.
“We are also analysing the data collected from infected computers to alert ISPs that their customers are infected and to identify where criminal groups are working and anticipate their next move,” said Hurd.