Dead Salmon Social Perspective Taking

I read a paper today in the British Medical Journal presenting a compelling case for a randomized controlled trial to measure the effectiveness of parachute use as a medical intervention to prevent injury when jumping from an aeroplane (Smith & Pell, 2003).

Their reasoning was as follows:

  • No such trial has been done before
  • The perception of the success of the parachute is based solely on anecdote
  • Natural history studies show that free fall without does not always result in an adverse outcome, therefore there is not a 100% chance of mortality when not using a parachute
  • There is also not a 100% chance of survival when using a parachute
  • In all other medical interventions not having a randomized controlled trial would be unacceptable
  • Current industry trials are subject to selection and reporting bias, as well as conflicts of interest.

All in all, there seems to be plenty of reasons why a randomised controlled trial should be carried out, so as to determine more appropriately the efficacy of the parachute… Except for the fact it would be stupid. And that, leads me on to the post-mortem Atlantic salmon.

Bennet, Baird, Miller and Wolford (2009) compared fMRI results of a dead Atlantic Salmon to a group of normal participants in a social perspective taking task.

image
Image Credit: Bennet, Baird, Miller and Wolford, 2009

The blood-oxygen-level-dependent (BOLD) signal experienced a significant change during the presentation of stimuli compared to at rest (t(131) > 3.15, p(uncorrected) < 0.001). An 81mm3 cluster of active voxels was identified on the Salmons brain cavity, and another smaller cluster was identified in the Salmons dorsal spinal column.

After the same t-tests were done using Benjamimi-Hochberg and family-wise corrections, both with a type 1 error rating of 0.05, no significant results were found. With those error ratings adjusted to 0.25, there was still no brain activity in Mr Salmon.

So, we have two daft studies here, both of which tickled me. It shows the importance of the correct use of advanced (and less advanced) statistics. It’s important to have a firm grasp of the statistical techniques you are working with, or you can very easily end up with errors. This is especially so in the case of the Salmon where such a large number of fine readings are recorded that without an element of realism being considered during the analysis, obvious statistical errors pop up.

And of course, the same level of realism is required at the opposite end of the spectrum, when you ask if we really need experimental control at all. I am impressed with how statistics allow researchers in the social sciences to measure population variables with precision. I don’t even pretend for one minute that I have an in depth understanding of all the methods covered even at undergraduate level, but I have learnt the value of getting stats right. It is important to properly reflect on what statistical tools are genuinely necessary to answer the research question, and of the ones you choose to use, it is important to understand the intricate details.

Advertisements

A 2:1 in..?

I got my grades back from this year, and I’m very interested to look not at the scores, but at the standard deviation of those scores. Out of 100 percent, 3.279 is a very narrow variance, which would show that my weighted mean is a fair representation of my scores.
Grade Analysis

Key:
Blue: Subjects where I scores 63 & 64%
Green: The most fantastic module where we made giant posters and presented them in a poster conference, then had online discussions about the content of said posters, which was graded. Followed by a multiple choice exam.
Orange: A coursework assignment, followed by a final exam, with multiple choice questions and short essay questions.
Yellow: Completely unimaginative final exam, with multiple choice questions and short essay questions.

And it falls into the 2:1 classification category. Good.

This summer, 70% of students across the nation (and my sister is one of them) will graduate with a 2:1 degree or higher. Those who don’t will either be sent to an old poly to do a masters degree, or to do a PGCE, or to work in the HR department at Tesco. Because we’ve created a stigma to go along with our degree classifications, that if you don’t get your 2:1 you’ve basically failed, the universities are doing everything they can to ensure that the 70% statistic holds.

My solid 65.25% could mean that the grading system is an accurate measure, confirmed by test-retest reliability across time. But what exactly is 62.25% is a score of? Recall? How much of a neuro-psychologist I am? Or a developmental psychologist I am? Or a psycholinguist? Or a researcher? Or a computer scientist?

Wait?! Computer scientist? That has very little to do with psychology. Yet my different modules are also quite unrelated to one another, they talk about different paradigms, different topics, different research methods, some more exciting, some less. Yet even so, my grades are reliably steady. I reckon I could get a 2:1 in absolutely anything. I believe in me!

That’s why I’ve used colour coding in the table above, to try connect the different teaching methods and spot trends in the data. And again, there’s a very random spread among the different levels of the independent variable. For the ‘coursework and typical final exam’ subjects, I got 70%, with subsequent scores regressing to the mean, and for the very ordinary ‘typical exam’ subjects I got my lowest score, as well as one in line with the mean. Most interestingly, the subject which I enjoyed the most, Personality and Individual Differences* was right on the mode average and just one percent above the median. Also very interesting is that the two research methods modules, (RMIII & RMIV), both measured identically, scored a time consistent 60%, despite me thinking I was improving.

This tells us that these results do not conclusively argue for the class results being an accurate measure merely of memory under exam conditions. However they could argue that the system is designed to produce lots and lots of 2:1’s, since across a whole array of different subjects, teaching methods and assessment methods, the results are steady.

I don’t think that the current system allows for the genuine expression of student’s novel thought and intellectual talents at undergraduate level. There’s enough hoop jumping and invalid measures to ensure that students who are at least obedient will get their ordinary degree, without demonstrating much beyond content memorisation.

*Personality and Individual Differences was assessed very uniquely, with class groups making posters, and then subsequent graded online discussion using social media.