Teradata analytics guru: one algorithm to rival them all?

This is a guest post for the Computer Weekly Developer Network written by Yasmeen Ahmad in her role as practice partner for analytic business consulting at database and data services firm Teradata

Ahmad writes as follows… 

Just as decision making in humans is improved by collective groups of people who individually contribute with diversity in knowledge and experience, machines also benefit from the notion of a collective mindset.

The idea and study of collective intelligence dates back many years – pooling together different information and perspectives can provide a better picture of the overall challenge or problem to be solved.

The collective computer brain

Similarly, individual algorithms operating on overlapping or the same datasets will make predictions of varying quality.

The strengths of one algorithm may help it outperform other algorithms given a specific set of data related to a situation. Leveraging this diversity in strengths of different algorithms means we can use multiple algorithms all working in concert with one another.

This technique is known as ‘ensemble modelling‘.

In fact more advanced Artificial Intelligence algorithms, such as neural networks, make use of this idea of collective intelligence.

Collective intelligence should not be limited to within the human and machine populations, but can be leveraged between human and machine. It is known that medical imaging and analytics is better at detecting cancer in a patient than a human pathologist. However, if we are able to take the expert opinion of a pathologist of how advanced a cancer is, this can augment the image analytics.

The power of automation

The ability of machines to be on par with human decision making in terms of accuracy is great. However, the game changer is the ability to automate in machines, which allows businesses to make millions of decisions that would otherwise be impossible. The speed of execution is a huge benefit of machine learning techniques that becomes a key differentiator.

KPMG predicts that relying on machine learning to partly automate the insurance claims process could cut processing time down from a number of months to just a matter of minutes. Similarly, in the oil industry, what could take eight weeks using human inspectors takes only five days using SkyFuture’s oil rig inspecting drones working with a single drone operator and engineer.

Parallel (decision) power

The power of automation allows tens or hundreds of tests to be run in parallel to compare decisions. This challenger methodology means we can assess whether decisions are good or bad.

Did we make the best move? A decision, when taken, inherently changes everything – the environment, market, customer opinion etc. To really know if it is was the best decision over all others, multiple decisions need to be executed and evaluated in parallel – a challenger methodology allows for this.

An assessment of multiple decisions also helps the machine learning algorithms. They can learn from the positives and negatives of multiple decisions and learn how to mitigate or enhance certain outcomes.

For example in sports science, sporting analytics companies analyse individual player performance and make recommendations that can be used by the coach. If a player was to follow only one strategy, they would become predictable and beatable, hence having multiple options to choose from allows a player flexibility to try different techniques and approaches which are continuously optimising his or her performance.

Intelligent Machines 2.0

This is just the beginning for machine learning in business.

Today, there is still a lot of effort for the human in creating and feeding algorithms to operate with precision and accuracy.

Tomorrow, Artificial Intelligence will enable a two way interaction. Machines will help to challenge our biases by asking questions that require additional or more precise data. Machines today are restricted to learn from data a human decides is relevant, this next wave will supercharge machine learning by providing machine’s the ability to navigate their learning. This human-machine partnership will also create benefits for leaders who will be free from biases to enable creative and insightful decisions.

Machine learning is becoming available to the masses. Through technological advances, such as the rise of cloud, computing power, scale and sophistication is available to all. The real challenge for executives to maximise on the opportunity of data driven decisioning is the internal culture.

The potential for many of our decisions, predictions and diagnoses to be informed by algorithms is here… 

… the human and machine collaboration is the key to unlocking that intelligence. 

About the author

Teradata‘s Ahmad speaks regularly at international conferences and events. After an undergraduate in computing, she followed a tangent into the world of life sciences, completing a PhD in data management, mining and visualisation at the Wellcome Trust Centre for Gene Regulation and Expression. Following this, she has also worked as a data scientist, building analytical pipelines for complex, multi-dimensional data types.