We are honored to bring Hope to the lives
of our nation's heroes and their families:
"no sacrifice forgotten, nor need unmet"
Hope For The Warriors ®
It’s a stat! For the Forty-Force episode, the fellows welcome guest panelist, Paul to discuss the statistical particulars that you can’t learn from a Jedi… Plus the Smart Suggestions abound, as always.
This is a great collection of the various ways you can edit (or depending on your political perspective, fudge) your charts to get them to suggest something they don’t actually suggest.
My favorite example is changing the order of magnitude of the x value, which to me is flat out fudging. Here’s the example from the post:
The trend line makes it look like the numbers have gone up and then slightly down but have never gone under the original mark from the beginning of 2011. So the conclusion you are supposed to draw is that things haven’t gotten any better, for a time they were worse, and are probably now just as bad or worse as when we started the year.
However, if you ignore the trend line and instead look at the numbers they tell a different story. To see what I mean, compare the number and point from Jan (first point) to the number and point for Nov (last point). It’s like an optical illusion. For a bigger effect compare the number and point for Mar (third point) to the number and point for Nov. Whoa! 8.6% is higher on the graph than 8.8%. That’s pretty magical. Here’s what it looks like graphed out with the proper units across all data points (again from the post):
This graph still suggests that things got worse for a while but if you want to start comparing differences in magnitude the highest the rate had been in this time period was .2 percentage points above the baseline %. At the end things are .4 percentage points better than the baseline.
Two different graphs, two very different stories.* Check out the full article for more.
Also, further reading (a classic!).
*My personal take on this — and I am a bleeding heart liberal — is that these numbers don’t represent much of anything but random fluctuation. That is, there might be some evidence of a slight downward trend but it’s really only driven by the last data point (i.e., more data points, probably a year’s worth are needed). So what puzzles me is that if they were going to fudge the numbers anyway why not just leave the last data point off and keep the x values the same rather than changing the magnitude of the x values?
A Student emailed me today and I spent so much time on the response I thought I should post it somewhere. Below is a modified version of the email as well as a somewhat more modified version of my response. Enjoy! (ed note: PSY 236 is an introductory personality psychology course).
Hi Dr. Witt,
What was the name of the band you showed us in class today? There were like 30 people in it all in one room doing different things (popping balloons, sweeping etc). they were awesome!
Also, random question - my friends & I were talking about testing taking. In high school, we were always the first ones to finish tests in class and did well on them. Now in college, we still perform well on the tests/exams but we’re often the last ones to finish. Presumably our Intelligence has increased, but why do we take longer to finish tests? Not sure if you’d know, but I thought I would ask.
PSY 236 Student
Hi PSY 236 Student,
Glad to hear that you liked the band. Their name is Efterklang.
As far as the test question is concerned. I think the simplest solution is that you are being made aware of something called “restriction of range”. Be prepared for me to nerd out a bit here as I attempt to illustrate:
Not all the people who attended your High school went to college (based on some estimates I’d say about 25% of them did). So now that you are in college you aren’t surrounded by the same people that you were in high school. You’re surrounded by the top 25% of those people (from your high school and from other high schools). Thus the range of ability (intelligence, test performance) has been “restricted”.
Let’s do a quick simulation to demonstrate what I mean.
Imagine that you had 100 random people (all high school students). You know each person’s average test score across all of the tests they took in high school (expressed as a percentage) and you also know the average time it took them to complete all the tests they took in high school. If you plotted this (see graph below) you would see that there is a strong negative association between the two variables in high school students (~ r = -.75). That is, the less time it takes a person to take a test on average, the better their test scores tend to be (this is what you observed when you were in high school).
Okay, now let’s focus on the top 25 scorers (just as only the top 25% of HS students get into college). Without doing anything (e.g., collecting information about their performance in college courses) we can look at the relationship between their test scores and time it takes to take a test (see graph below). You can see that the relationship between test scores and test time is much weaker in these high performers (~ r = -.35). In fact, you and your friends might be those two dots in the top right corner. That is, people who score well on the exams but take longer. Does this make sense?
The range gets restricted more and more as you move through your education. Many people go to grade/elementary school. Most of them go to High school. Some of those people don’t make it through high school. Then a smaller number of people who complete high school go to college. Some of those people don’t make it through college. Then a much smaller percentage of college graduates go to graduate or professional school. The range gets smaller and smaller and as a result so does the association to the point where it might be no correlation or it might actually change from a negative to a positive association.
In graduate school many of my colleagues (myself included) had trouble with the transition because they were no longer the smartest kids in the class. This was so common that I coined a term for it. I called it “the superman effect.” When superman was on earth he was superman (our sun gave him special powers), but when we was on Krypton (his home planet) he was just like everyone else (their sun was red and kept their powers at bay). So to go from Earth, where he spent most of his life and had super powers. to a planet that had a red sun (such as Krypton, though he couldn’t do that because it had exploded) would have been a humbling experience.
My first experience with a graded assignment in graduate school was quite the wake-up call. I received a 92% on a statistics exam. I was pretty happy that I had done so well. However, I later found out the class average was 94%. I definitely wasn’t on Earth anymore…
Probably more of an answer than you wanted, aren’t you glad you asked?