In the election the candidates for social media sentiment failed

Given the way the polls failed to predict the recent election result Nick Booth wonders why bookmakers still call everything better than sentiment analysts

We’re constantly told that big data is all about gauging sentiment and helping companies to be agile.

I’ve never really bought the myth of company agility anyway. Most enterprise employees can’t tell you what time it is without having a meeting.

The idea of mood monitoring is interesting. But there’s never been a way to gauge the effectiveness of technology in capturing the national zeitgeist.

Until now, that is. The last general election gave us a good insight. And the results were pretty disappointing.

Of the three methods for predicting public tastes for each party and their reaction to televised debates, the most successful was the oldest. The Bookmakers, such as PaddyPower, were much more successful at predicting the outcome of the election than their peers The Pollsters and The Social Media analysers.

Why would this be? Do bookmakers know people much better?

Only 28% of Britons declare opinions or political allegiances on social media, according to a study by NetNames. It’s a taboo subject, says its CEO Gary McIlraith. Though this is changing, and 40% of young people are happy to do a bit of ‘value signalling’ who is to say these youngsters won’t change when they get older and wiser.

The NetNames/YouGov poll during the 2015 General Election found liking (16%), posting (15%) or commenting (12%) on posts as the best way to engage with politics publicly. 

Arguably, there are all kinds of questions Netnames should have asked.

For a start, who would be mad enough to declare their politics in public? It could affect their employment. What if the person about to hand you a new contract was a rabid supporter of one party and they discovered you vote for the enemy?

Many social media analysts and pollsters don’t seem to take into account one very important human factor. There are thousands of people who lose their marbles over politics. For some simpletons, party allegiance is a form of personal branding. It’s a way of signaling how good they are, without ever having to donate taxes, charity funds or their own time. Never trust narcissism or sentiment, I would say.

Sentiment tracking via social media is alluring but can be inaccurate, admits Andy Cotgreave, senior technical evangelist at Tableau Software. Many organisations (including Tableau) measured social media during the election campaign – and look where it got them.

While social media might accurately gauge consumer trends and opinions, elections are more complex. Don’t avoid sentiment tracking, but use it with caution, warns Cotgreave.

Those lessons, surely, should apply to the ‘office politics’ of big corporations too. All those elaborate visualisations and models of opinion polls take huge amounts of time and developer resource. But when data doesn’t change in real-time and polls are static (as they were throughout the election) those dashboards became dead ends, says Cotgreave.

“The real value lies in hunting down answers to fresh questions,” says Cotgreave. It’s not the tools that are lacking, he seems to be saying, it’s the human variables. For example, most of the questions set by social media and pollsters are in desperate need of improvement.

Finding the right context in social media is a work in progress, says Jason Smith, CEO and founder of Blurrt.

Blurrt uses linguists to analyst twitter sentiment and uses their knowledge to help develop algorithms. By a mixture of constant testing, fine tuning and machine learning, they are getting a more sophisticated reading of public moods. If someone Tweeted sarcastically about, say, Nigel Farage’s on-off resignation, the system would be able to tell the difference between whether “Oh great” meant Good or Bad.

But it’s a constant battle. Twitter users, being younger and more slavish to IT trends, are much more likely to adopt each new US slang term.  The British language is becoming increasingly Americanised. I have a theory that use of Americanisms, by British people, are a clear indication of an person that’s gullible and easily influenced. Now, these simpletons would be a great category for retailers to concentrate on, as they’ll buy anything as long as it’s trendy.

“We’ve never studied which types of people are most likely to use American terms,” says Smith.  That’s a shame, because online herding of these sheeple could become hugely profitable.

Now there’s an idea for a social media start up. Identifying the stupidest people on social media, and selling them crap. I daresay it’s already been done.

Read more on Business Intelligence Tools and Services

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.