Another dot in the blogosphere?

P-hacking, false positives, and good data

Posted on: September 14, 2019

Today I combine a quick reflection on a video I just watched and a book that I am reading.


Video source

If you want to know what a dead fish, an MRI machine, and statistics have in common, watch the video above.

The salmon experiment was a red herring. If you focus on the living results coming from dead fish in an MRI machine, you miss the point of the video: Research needs to be based on good design, not led by a foregone conclusion.

That should seem like a given, but the fact that the point needed to be proven and made is evidence that people and scientists need constant reminders.

Here is another reminder and it comes from page 109 of Charles Wheelan’s book, Naked Statistics.

Our ability to analyse data has grown far more sophisticated than our thinking about what to do with the results. — Charles Wheelan

This quote was from Wheelan’s chapter about starting with good data. He was trying to make the point that no amount of elaborate or sophisticated analysis was going to make bad data any better.

For example, we might need a representative sample but select a biased one instead. There is no analysis process that is going to improve a bad sample. That data and the research is tainted from the start.

So the next time someone declares “Research says…” we know better than to take what follows at face value.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

http://edublogawards.com/files/2012/11/finalistlifetime-1lds82x.png
http://edublogawards.com/2010awards/best-elearning-corporate-education-edublog-2010/

Click to see all the nominees!

QR code


Get a mobile QR code app to figure out what this means!

My tweets

Archives

Usage policy

%d bloggers like this: