What's in a number?

I was amused to read about the latest estimates of the number of intelligent alien civilisations, recently reported in the International Journal of Astrobiology. Apparently the discovery of more than 330 planets outside our solar system in recent years has helped “refine” the number of life forms that are likely to exist. The new research claims that there might be as little as 361 intelligent civilisations in our Galaxy and possibly as many as 37,964.

You have to admire such breathtaking precision. It brings to mind those heavily-flawed estimates we make of risk probabilities: the ones that suggest the likelihood of a risk might be, say, 40%, but without mentioning that the accuracy of the estimate is plus or minus 90%. Such estimates are obviously worthless as a means of prediction, though they’re often useful for building business cases for investment appraisal, or, ironically, to demonstrate prudent corporate governance to an auditor.

More interestingly, numbers can convey subtle degrees of spin, depending on their precision and context. A number with one or two decimal points comes across as well-measured. A round number sounds suspiciously like a guess. The exception is the 80/20 rule which is strangely compelling and plausible, even though most examples quoted are not based on any sound research. 

Donn Parker always used to quote a made-up, precise number when discussing security risks. Many people took him seriously, though he was actually making the point that such statistics are nonsense and should not be relied upon. He was absolutely right. Taking figures from external sources is potentially dangerous. Many assumptions do not apply outside their original context. That’s why “Assume context at your peril” is a key Jericho Forum principle.

The other problem is that figures tend to get distorted as they’re passed on from person to person. For example, 37% might quickly become “one in three” or “over 30%” or “around 40%” after just a few exchanges. In fact, research has long indicated that around 70% of the details of a story passed on by word of mouth are lost in the first five or six exchanges.

Douglas Adams hit the nail on the head when he suggested that the answer to life, the universe and everything was 42. Because in security, it’s the question that really counts, not the answer. 

Join the conversation

1 comment

Send me notifications when other members comment.

Please create a username to comment.

Dave I came across Donn Parker the first time in an excellent (and hilarious) article he wrote for the ISSA journal in December 2008 (The Expert's Number for Security Risk Assessment). Apparently this 'magic number' that he uses is 84.6, and it works for everything, as it is believable, and then he works backwards, to see how he got there. OK, he is joking a little, but it does hold an element of truth, estimating risk accurately is quite impossible. We can only 'estimate', although the need of managers to feel comfortable tht they have this definitive number, e.g. 84.6 as mentioned by Donn, means they can make decisions, hence sleep at night. People inherently don't like making decisions based on uncertainty, and if we were to wait for that 'definitive' number, maybe many people would make no decisions at all. Facts help to accelerate decision making. My most famous quote is that 'to make a decision is quite often better than to make no decision'. We need to ask ourselves what is the cost of NOT making that decision? If a decision has to be made my next famous quote is 'you don't make bad decisions if you make them into the right decision!'. Afterall instead of saying "if only" after a decision is made because it was felt to be wrong, we need to execute on the decision effectvely and then adapt to suit the business environment that is in effect continually changing, surviving in a somewhate chaotic world. And yes Douglas Adams got it spot on, 42 is the meaning of life. Computers have come a long way since he wrote the book, and I did my own calculations, and its true....lol :-)