'Alcohol is more harmful than heroin.' Really? Sample before you speak
Professor David Nutt was back in the headlines this week. In October 2009 he was sacked from his position as UK government's chief drug adviser after suggesting that LSD and heroin were less harmful than alcohol and tobacco, and that taking ecstasy was less dangerous than riding a horse. Now, almost exactly a year on, he has done it again. This time his claim is that alcohol is more harmful than heroin. Professor Nutt’s statement is statistically fuelled. However, it is also statistically flawed.
More dangerous than recreational drugs? The Big Raspers At Aintree, 1927 painting of an incident at the Grand National. Copyright of the Lordprice Collection, with permission.
In their study, Professor Nutt and his colleagues rated the harmfulness of 20 drugs on 16 measures using a scale of 0 – 100, with 0 meaning not harmful and 100 meaning the most harmful. This was a ratio scale where the distance between each point on the scale is equal.
For example, a drug scoring 50 on such a scale is twice as harmful as a drug scoring 25. Using this scale, all the drugs included within the study were rated on all 16 measures in relation to both harm to the user and harm to others. Recognising that some measures are more important than others, Nutt weighted the scores for the measures to reflect the importance of each measure.
Based on the above, Nutt’s statement, which initially sounded farfetched, appears to have a strong statistical grounding. However, once we evaluate Nutt’s sampling strategy, flaws in his methodology are highlighted. The drugs in Nutt’s study were scored by a small group of experts. Such a sampling frame has several disadvantages:
• The experts scoring the drugs, whilst having extensive academic backgrounds, presumably are not drugs users themselves. This means that the measurement of harm to users is really the expert’s perceptions of the harm drugs cause users
• Again, when measuring harm to others, the experts are only offering their perceptions of how drugs harm others as they are unlikely to have direct, mass scale experience of how using each of the 20 drugs can affect others
• The size of the sample of experts is not large enough to make statistical inferences. In commercial research, the findings from any sample lower than n=50 are seen as directional and not significant
Despite employing robust scaling and weighting methods, Nutt’s work appears to be let down by a poorly targeted sampling strategy and a small base size, meaning that the methodology behind the highly impactful headline “Alcohol is More Harmful Than Heroin” is “Nutt quite” as robust as it initially appears. Resultantly, the findings of this study are based more on academic perception than social reality and the findings, at best, are directional.
No comments:
Post a Comment