Fujitsu innovation tackles intractable IT problems

Computing history is littered with the corpses of solutions overtaken by technology.

Computing history is littered with the corpses of solutions overtaken by technology. However, the problems they were set up to address have not gone away, says Nic Holt, technical architect with Fujitsu.

Take database searches, for example. In the 1970s ICL developed the contents-addressable file store, or Cafs. In an age where mainframes ran with 128kb of memory, Cafs was a database appliance, essentially disc storage with built-in search facilities. This removed the onerous task of finding exact matches to wild-card queries from the mainframe. The result was about a 20 times improvement in throughput, Holt says.

However, the cost of memory dropped to the point where it was affordable to run the entire database in main memory. "Today we are running systems with 138Gb, big enough for many databases," Holt says.

Much of the functionality of Cafs migrated into software that ran on later generation mainframes. Given their extra memory capacity, it made little sense to have a separate device just for database queries. Instead, Holt says, the search is on for better algorithms to speed up the finding and matching of search items.

However, Cafs-like devices with solid-state rather than spinning memory may soon be more common. Teradata, for instance, is making a good living off the need to process massive amounts of data in close to real time. With the volume of data to be processed rising constantly, and as the sensor-based "internet of things" comes to life, processing it all will require big leaps in capacity.

"But there will always be a break-even point at which the old solution is superseded by a new one," Holt says.

According to Holt, scheduling (data for processing) is an intractable problem with current technology. "A lot of people are waiting for quantum computing to see if it is a way forward," he says. But they may have to wait. Holt says commercial quantum computers are unlikely to hit showrooms for a decade or two.

Holt maintains that a lot of IT is driven by a product life-cycle management schedule rather than genuine advances. But sometimes genuine solutions exist but can't get to market.

This is the case with privacy, Holt says. This is a "technology versus society" subject, he says. He believes the IT specialists have a good grasp of the problem and its solutions, but emotion fires the political and social discourse. This heat, rather than light, is holding up the implementation of applications that could make a real and positive difference to peoples' lives.

"Unfortunately we are creating a society that is increasingly technologically illiterate," Holt says. "Even though everyone uses Facebook and Twitter, most don't understand the implications. We urgently need a better informed debate on this so that we can develop sensible solutions that can hold the executive authorities to account."

This will become increasingly important as information processing disappears into the cloud, where the source, collection and application of that processing becomes opaque to outsiders and to data subjects, he says.

Read more on IT innovation, research and development