Monday, 28 April 2014

We can all be above average (almost)

I was reading a very interesting article on behavioural psychology today and its effect with relation to Peak oil, climate change etc. I really liked the article, it does a pretty good job of explaining skepticism towards change, it is a must read for certain. I could not help but notice a common enough error with statistics.
To illustrate a point about human optimism particularly with regards to our own abilities, the author quotes research showing how people think that they are better drivers than average (88%) or that professors think that they are better than average (94%), which is obviously impossible as "By definition, almost half of those surveyed are 'overly optimistic'" or in other words 38% of drivers surveyed and 94% of professors must be wrong. Interestingly, it is possible for the surveyed people to be right, although very unlikely and here is why.

The arithmetic mean or average of a distribution is simply the sum of all values of the distribution divided by the number of items in the distribution. Say that we quantify driver ability on a scale of 0 to 1000, where 1000 is the driving skills of Michael Schummacher coupled with the law abiding ethos of a 50 year old German protestant pastor and 0 is the law abiding ethos of an Italian teenager coupled with the driving skills of a 100 year old arthritic, half blind grandmother.

We take a random selection of 10 people and measure their driving skills. These are their results:

  1. 501
  2. 500
  3. 504
  4. 502
  5. 503
  6. 508
  7. 516
  8. 490
  9. 491
  10. 532

The average of the above distribution is 504.7 and thus we have 80% above average. what is going on here, I hear you ask?

Nothing much, just good old fashioned confusion, here is another distribution, this one is the annual salaries of ten friends who have met for a dinner party.
  1. £50100
  2. £50000
  3. £50400
  4. £50200
  5. £50300
  6. £50800
  7. £51600
  8. £49000
  9. £49100
  10. £532000

On this distribution 90% are below average, the average is close to £95k.

The author confused arithmetic mean with the median, which is described in Wikipedia as: a median is described as the numeric value separating the higher half of a sample, a population, or a probability distribution, from the lower half.

As I have shown it is possible to have almost everybody above or below average it is impossible to have more than 50% of the drivers above the median, by definition.

This post was just a little bit of facetiousness, his point still stands even if he conveyed it poorly. People think that they are above average and above the median.

Incidentally, the median for the first distribution is: 503 and for the second: £50300.

No comments:

Post a Comment