Statistics can prove anything. Did you know that you can draw a correlation between a declining divorce rate and a fall in margarine consumption? Or link a rise in the number of people who’ve died after becoming entangled in their bed sheets, with a rise in cheese consumption?
Well you can. To find out how, check out the aptly named ‘spurious correlations’ webpage, a blog run by Tyler Vigen, a doctoral candidate at Harvard Law School.
The misuse of statistics is not new. Politicians do it. So do advertisers – ‘80 per cent of dentists use a certain toothpaste’. So do protagonists in public debates. Sometimes it’s based on ignorance, when people confuse statistical correlation with causation. Journalists can do it, too, when they cite statistics to highlight a trend or chart a development.
Sometimes it’s not just ignorance. Sometimes it’s prejudice, like when a journalist uses selective numbers to prove a point he has already arrived at.
This appears to be the case on a number of recent media articles. (I won’t link to them, as I don’t want to boost the reporter’s ‘click rate’ and provide more spurious statistics to prove the ‘success’ of his work.)
The articles were based on official data on criminal offences reported at all Victorian schools.
The first of the articles asserted that crime was ‘rife’ in Independent schools, and insinuated that they were unsafe for students. The story gave the number of reported offences, but gave no indication of a trend – were the numbers up, or down, on previous years?
Nor was he able to say if the numbers, assuming they were up, were based on greater reporting by victims or more efficient compilation of data by statistics agencies.
Buried in the story were some vague cautionary notes: It was ‘unclear how many’ of the offences were committed by students and staff. The reporter could have said that it was ‘unclear if students and staff committed any of the offences’, but this would not have suited his purpose.
Nor did the reporter mention that other factors could influence any interpretation of the data. Assuming students and staff committed some of the offences, was the number of offences disproportionate to the number of students and staff? How did the number of offences compare to the general crime rate in the wider community?
Other cautionary facts were buried, including this: more than half of the reported offences involved thefts and break-ins, with the figures including offences committed on holidays and out-of-school hours. In fact, the data shows offences against the schools, not within the schools. Pointing this out appears not to have suited the reporter’s aim, either.
The journalist returned to the issue a couple of weeks later, using new data to confirm the ‘peril at private colleges’.
This time he sought to pit Independent and Catholic schools against government schools, asserting the latter were less prone to crime.
He sought to compare 11 ‘elite’ Independent and Catholic schools to 11 government schools. On any reading, his methodology (if I can call it that) was flawed. Were the numbers up or down? He doesn’t say. Did staff and students commit the offences? He can’t say.
He tells us 15,000 students attend the 11 government schools. He doesn’t tell us how many attend the 11 non-government schools. Had he asked, we could have told him: for the record, it’s about 22,000.
Nor does he say if the figure of 11 non-government schools includes the number of campuses run by the schools, which might have some bearing on the number of offences. For the record, the 11 schools run 30 campuses.
We could have told the reporter some of these things, if he wanted to prepare a balanced report. But he didn’t call.
If you think I’m being unkind to the journalist, consider the gratuitous slur in the final paragraph of his first report: ‘Former students of the elite schools have gone on to commit murders, sex offences, violent armed robberies rampages (sic), assaults and run major drug trafficking operations’.
Yes, and some have gone on to become teachers, nurses, police officers, aid workers, artists, academics, carpenters, technicians, even prime ministers. Some have become honest journalists, too.