The Rich Untapped Seam in Digital Communications

The blog postings have been a bit thin these past few days as I’ve been head-down, writing up a lengthy feasibility study report on the potential for analysing security behavior in digital communications. It might sound very ambitious, but there’s a lot of interesting things you can do to detect various forms of behaviour and misuse. And there’s a surprising breadth and depth of prior research in this area. Remember that IDS technology has been around for more than a decade and anti-malware scanning for twice as long. Unfortunately the trickle of innovative products in this area has not kept pace with the potential being mapped out by blue sky researchers. So you can’t yet exploit the most promising techniques.

So what can you find from communications analysis? Quite a bit if you put your mind to capturing, analysing, profiling, mining and fusing message content, traffic patterns and IT activity. And even more if you apply modern visualisation techniques to high-speed graphical user interfaces. Psychological and linguistic profiling is still in its infancy but it offers huge potential for the future. Data fusion and mining have already achieved many spectacular successes. And neural networks are an established tool in the fraud detection armoury.

Privacy is clearly an overriding issue, but effective security solutions exist or can be conceived to contain the risks for many applications. Ignorance of privacy considerations is a bigger problem, as demonstrated by the recent decision by US Homeland Security to scrap an ambitious $42 million anti-terrorism data-mining tool after investigators found it was being tested with information about real people without adequate privacy safeguards. Of course it might sound like Big Brother, perhaps something to be resisted. But you can’t reinvent the science behind digital communications analysis. The best approach is to take it forward and develop the necessary safeguards.