Before Getting Behind the Headlines

The Behind the Headlines project has been going for a while now with aim to tell the truth about stories that are in the news. The Science Media Center (SMC) are starting a new initiative to involve statisticians before a science news story is release to help journalist get the story straight. This is called Before the Headlines.

I tried a first attempt at critiquing a sample paper. I then had a read of some of the other reviews on the SMC webpage to compare. For all of the papers the general responses seem very similar. They talk about the most obvious statistical things to be careful about, as I have. These are:

• What are the assumptions? Are we assuming that things are similar to one another or that something things don't change?
• How are things split-up and why? Are things stratified by age or time or something else?
• What are the confounders? Have they been adjusted for? This is an opportunity to think a little creatively about the problem.
• How big is the data set? Does it take multiple readings?
• Is this hypothesis generation? Is this a case of multiple testing so bound to find something significant.
• Does it just make common sense?? Would people really behave in a certain way? Do the results actually mean anything useful?
• Pull-out some headline statistics. Are these sensible?

Serum Vaccine Antibody Concentrations In Children Exposed to Perfluorinated Compounds JAMA, January 25, 2012—Vol 307, No. 4

Claim supported by evidence?

This paper does claim that there is an association between elevated exposures to PFCs in Faroese children aged 5 and 7 years and reduced humoral immune response to routine childhood immunizations. The paper does not claim that there is a causal relationship It does suggest that this could be the case however.

Summary

The paper does not over-stretch the results of the study by claiming causality [strength] This is for a rather specific cohort in the Faroe Islands and only considers 2 ages (3 measurements) (5 (x2) and 7 years). It says PFC has a half-life of 4 years so is this long enough. Is the study investigating the PFC passed from mother to child or over directly to the child time e.g. 0-5 or 5-7 years old? [limitation]

Study Conclusions

Conclusions are not exaggerated. As ever, more work is needed…

Strengths/Limitations

Well recorded what the details of the study were. Tried to take into account certain covariates and likelihood ratios too. Study uses the fishing community of the Faroe Islands because “frequent intake of marine food is associated with increased exposures to PFCs.” What other characteristics distinguish them from other populations? Perhaps high exposure is a problem but lower exposure is not e.g. does not scale (down) linearly.          The assumption is that all of the mothers and kids are equivalent so that the exposure over time and biological impact and effect are comparable. What about other covariates like profession or diet?          How do these results translate to adult populations?

Glossary

Any specific expertise relevant to studied paper (beyond statistical)?

I am a statistician but do not work as a trials statistician so have mainly applied common sense in this case.

Kids and Probability

The Nuffield Foundation have recently released the result of their Childrens understanding of probability: an intervention study. When I was doing the Widening Participation Award at Manchester University I developed and gave a probability workshop to lots of primary school kids. I was interested to read what the Nuffield lot had found out.

They talk about 4 "cognitive demands": Understanding randomness, working out the sample space, comparing and quantifying probability, understanding correlation.

They believe that fairness, which kids have a better grasp of, could be used to teach randomness.

They highlight the common mistake of thinking that the opposite of the previous event will occur next, which they call `negative recency’. I've never heard this phrase before. They also called the opposite of this- believing the same outcome will occur, the hot-hand fallacy/positive recency. Adults and kids make these 2 types of mistakes in different proportions.

They refer to "aggregation" with the example of suming 2 dice throws. Generally this is a case of a function of multiple events which modfies the probabilities and is often not intuitive.

They think about how a child considers all of the sample space as all of the possible outcomes imaging all future states. Of course, there can only actually be one so these are hyperthetic, counterfactual, "other worlds". That is, this is in essence a philosophic thought experiment and not necessarily a trivial exercise.

How to bet on climate change

The latest S Word article in the guardian. The idea was to introduce the idea of false positives and negatives but this seems to have got lost in the contentiousness of the example. Perhaps this is a good thing!

Pascal's Wager is more useful for deciding which way to go on climate change than on, say, religion.

French mathematician Blaise Pascal whose 'wager' offers a way to view the argument about climate change. Photograph: Alamy

 

If a trusted expert advised you to do something or else face grave consequences, I imagine you'd probably listen. If that advice was backed up by hard evidence then it should be a done deal. But in the case of global warming, some people just don't want to listen.
The anti-science movement growing through the US and the rest of the western world denies that rising carbon dioxide levels in the atmosphere are changing the climate. The recent Republican debates in the US have brought to the fore some rightwing politicians' opinions on the threat posed by global warming. In short, Republican candidates claim to have cottoned on to the "hoax" of global warming and powerful lobby groups are not just denying the science behind climate change but aggressively rubbishing and discrediting it.
This is a risky hand to play. I'll explain why by equating the pros and cons of this position to a famous "wager" posed by the 17th century mathematician Blaise Pascal.
Pascal's Wager is to do with whether to believe in God or not. It says that since we can't prove or disprove the existence of God we should wager that he does exist, because there's a lot to gain if it turns out he does and not much lost if he doesn't.
Perhaps we could frame the arguments about global warming as a similar wager. If we wager that global warming is a serious problem and we need to act urgently then, in a similar way to Pascal's Wager, there would be relatively little lost if it turns out not to be such a problem and plenty to gain if it is.
In a more formal statistical way, we can say that our null hypothesis is that there is no God or that there is no serious danger from global warming.
We can classify the situation as one of four alternatives. The first two are the situations where our conclusions about the null hypothesis are spot-on. So, that is the null hypothesis is actually true and we think that it is true or the null hypothesis is false and we think that it's false too. So, we call it correctly in either situation.
The other two situations are when we call it wrongly whatever the truth is. So, that is when the null hypothesis is true and we think it is false or the null hypothesis is false and we think that it is true. In our global warming example this would be when we stock up on suntan lotion and think that global warming exists but it doesn't, or global warming exists but we back the wrong team and there's no problem.
These are called false positives and false negatives, respectively. In terms of Pascal's Wager, the false positive (thinking there's a problem when there isn't) is not so bad, but the false negative (thinking there's no problem when there is) could be catastrophic.
But of course, although we can think about the global warming issues in terms of a Pascal-type wager there is an important distinction. In the case of the existence of God this is more than a little tricky to prove, but the science and evidence behind the questions on global warming appear to be rather more conclusive. Unfortunately, it seems it'll take God himself to pipe up about the dangers of global warming to convince some Republicans.

Podcasting hardware

I'm thinking about getting hold of some microphones to record some podcasts with. Maybe it would be a good idea to record some first with the laptop microphone and then see. I want to do some interviews so I think having 2 microphones would be a good idea.

I was reading this. From this I think the simplest and cheapest option is to buy a personal audio mixer like this and feed in all the microphones. Maybe one standing on the desk or a clip/head microphone.