Interview: Teradata CTO Stephen Brobst on the challenges of unstructured data

Teradata CTO Stephen Brobst remains untempted by the appeal of analysing unstructured data such as text and voice.

Teradata CTO Stephen Brobst remains untempted by the appeal of analysing unstructured data such as text and voice.

"That is not our core competency," he says. "We are good at very high-speed processing of structured data."

But Teradata is "definitely interested" in processing the data content of voice and text messages once its meaning has been converted into structured formats.

"We are not interested in owning the algorithms that do the conversion, and we work with partners such as universities and start-ups to integrate their algorithms with Teradata's technology," Brobst told Computer Weekly at a Teradata user conference in Berlin.

Among those Teradata has worked with on this technology are SAS and Attensity, a start-up that can extract the sentiments expressed in public and private e-mail conversations. The theory is that such information can signal early warnings of problems or changes in user behaviour long before it becomes overt.

This is why intelligence services around the globe are among the most interested potential customers for the technology, Brobst says. The kind of processing Teradata is good at means it is quicker at identifying who might be worth following more closely.

"You have to look at it all," he says, adding some governments do not face the same sanctions.

Commercial users such as insurance companies may benefit from experimental technology that can tell when a person on the phone is lying and mark them for closer scrutiny, thus reducing the number of fraudulent claims they have to process.

Such applications remain very high-end, but they are creeping into the market, Brobst says. Teradata is helping that shift by developing new "form factors" that allow firms to access high-speed business analytics tools in both public and private clouds.

Brobst says the tools, such as tie-ups with Amazon's EC2 for public access and VMware for private cloud, are very suitable for non-production, proof-of-concept applications.

This may be all small to mid-sized companies need.

Brobst says initial loading of the datawarehouse is best done via optical storage media, but all updates and access are then via the web.

This can give SMEs the same or better competitive capacity as bigger firms, especially as the quality of the decisions prompted by business analytics depends very much on the validity and quality of the data collected and the questions asked.

Read more on Business intelligence software